class_docstr: A portfolio allocation environment for OpenAI gym.
  
  This environment simulates the interactions between an agent and the financial market
  based on data provided by a dataframe. The dataframe contains the time series of
  features defined by the user (such as closing, high and low prices) and must have
  a time and a tic column with a list of datetimes and ticker symbols respectively.
  An example of dataframe is shown below::
  
          date        high            low             close           tic
      0   2020-12-23  0.157414        0.127420        0.136394        ADA-USD
      1   2020-12-23  34.381519       30.074295       31.097898       BNB-USD
      2   2020-12-23  24024.490234    22802.646484    23241.345703    BTC-USD
      3   2020-12-23  0.004735        0.003640        0.003768        DOGE-USD
      4   2020-12-23  637.122803      560.364258      583.714600      ETH-USD
      ... ...         ...             ...             ...             ...
  
  Based on this dataframe, the environment will create an observation space that can
  be a Dict or a Box. The Box observation space is a three-dimensional array of shape
  (f, n, t), where f is the number of features, n is the number of stocks in the
  portfolio and t is the user-defined time window. If the environment is created with
  the parameter return_last_action set to True, the observation space is a Dict with
  the following keys::
  
      {
      "state": three-dimensional Box (f, n, t) representing the time series,
      "last_action": one-dimensional Box (n+1,) representing the portfolio weights
      }
  
  Note that the action space of this environment is an one-dimensional Box with size
  n + 1 because the portfolio weights must contains the weights related to all the
  stocks in the portfolio and to the remaining cash.
  
  Attributes:
      action_space: Action space.
      observation_space: Observation space.
      episode_length: Number of timesteps of an episode.
      portfolio_size: Number of stocks in the portfolio.
class_docstr_tok: ["A","portfolio","allocation","environment","for","OpenAI","gym",".","This","environment","simulates","the","interactions","between","an","agent","and","the","financial","market","based","on","data","provided","by","a","dataframe",".","The","dataframe","contains","the","time","series","of","features","defined","by","the","user","(","such","as","closing",",","high","and","low","prices",")","and","must","have","a","time","and","a","tic","column","with","a","list","of","datetimes","and","ticker","symbols","respectively",".","An","example","of","dataframe","is","shown","below",":",":","date","high","low","close","tic","0","2020-12-23","0.157414","0.127420","0.136394","ADA-USD","1","2020-12-23","34.381519","30.074295","31.097898","BNB-USD","2","2020-12-23","24024.490234","22802.646484","23241.345703","BTC-USD","3","2020-12-23","0.004735","0.003640","0.003768","DOGE-USD","4","2020-12-23","637.122803","560.364258","583.714600","ETH-USD","...","...","...","...","...","...","Based","on","this","dataframe",",","the","environment","will","create","an","observation","space","that","can","be","a","Dict","or","a","Box",".","The","Box","observation","space","is","a","three-dimensional","array","of","shape","(","f",",","n",",","t",")",",","where","f","is","the","number","of","features",",","n","is","the","number","of","stocks","in","the","portfolio","and","t","is","the","user-defined","time","window",".","If","the","environment","is","created","with","the","parameter","return_last_action","set","to","True",",","the","observation","space","is","a","Dict","with","the","following","keys",":",":","{","``","state","''",":","three-dimensional","Box","(","f",",","n",",","t",")","representing","the","time","series",",","``","last_action","''",":","one-dimensional","Box","(","n+1",",",")","representing","the","portfolio","weights","}","Note","that","the","action","space","of","this","environment","is","an","one-dimensional","Box","with","size","n","+","1","because","the","portfolio","weights","must","contains","the","weights","related","to","all","the","stocks","in","the","portfolio","and","to","the","remaining","cash",".","Attributes",":","action_space",":","Action","space",".","observation_space",":","Observation","space",".","episode_length",":","Number","of","timesteps","of","an","episode",".","portfolio_size",":","Number","of","stocks","in","the","portfolio","."]
docstr: Performs a simulation step.
  
  Args:
      actions: An unidimensional array containing the new portfolio
          weights.
  
  Note:
      If the environment was created with "return_last_action" set to
      True, the next state returned will be a Dict. If it's set to False,
      the next state will be a Box. You can check the observation state
      through the attribute "observation_space".
  
  Returns:
      If "new_gym_api" is set to True, the following tuple is returned:
      (state, reward, terminal, truncated, info). If it's set to False,
      the following tuple is returned: (state, reward, terminal, info).
  
      state: Next simulation state.
      reward: Reward related to the last performed action.
      terminal: If True, the environment is in a terminal state.
      truncated: If True, the environment has passed it's simulation
          time limit. Currently, it's always False.
      info: A dictionary containing informations about the last state.
docstr_tok: ["Performs","a","simulation","step",".","Args",":","actions",":","An","unidimensional","array","containing","the","new","portfolio","weights",".","Note",":","If","the","environment","was","created","with","``","return_last_action","''","set","to","True",",","the","next","state","returned","will","be","a","Dict",".","If","it","'s","set","to","False",",","the","next","state","will","be","a","Box",".","You","can","check","the","observation","state","through","the","attribute","``","observation_space","''",".","Returns",":","If","``","new_gym_api","''","is","set","to","True",",","the","following","tuple","is","returned",":","(","state",",","reward",",","terminal",",","truncated",",","info",")",".","If","it","'s","set","to","False",",","the","following","tuple","is","returned",":","(","state",",","reward",",","terminal",",","info",")",".","state",":","Next","simulation","state",".","reward",":","Reward","related","to","the","last","performed","action",".","terminal",":","If","True",",","the","environment","is","in","a","terminal","state",".","truncated",":","If","True",",","the","environment","has","passed","it","'s","simulation","time","limit",".","Currently",",","it","'s","always","False",".","info",":","A","dictionary","containing","informations","about","the","last","state","."]
code: def step(self, actions):
      """Performs a simulation step.
  
      Args:
          actions: An unidimensional array containing the new portfolio
              weights.
  
      Note:
          If the environment was created with "return_last_action" set to
          True, the next state returned will be a Dict. If it's set to False,
          the next state will be a Box. You can check the observation state
          through the attribute "observation_space".
  
      Returns:
          If "new_gym_api" is set to True, the following tuple is returned:
          (state, reward, terminal, truncated, info). If it's set to False,
          the following tuple is returned: (state, reward, terminal, info).
  
          state: Next simulation state.
          reward: Reward related to the last performed action.
          terminal: If True, the environment is in a terminal state.
          truncated: If True, the environment has passed it's simulation
              time limit. Currently, it's always False.
          info: A dictionary containing informations about the last state.
      """
      self._terminal = self._time_index >= len(self._sorted_times) - 1
  
      if self._terminal:
          metrics_df = pd.DataFrame(
              {
                  "date": self._date_memory,
                  "returns": self._portfolio_return_memory,
                  "rewards": self._portfolio_reward_memory,
                  "portfolio_values": self._asset_memory["final"],
              }
          )
          metrics_df.set_index("date", inplace=True)
  
          plt.plot(metrics_df["portfolio_values"], "r")
          plt.title("Portfolio Value Over Time")
          plt.xlabel("Time")
          plt.ylabel("Portfolio value")
          plt.savefig(self._results_file / "portfolio_value.png")
          plt.close()
  
          plt.plot(self._portfolio_reward_memory, "r")
          plt.title("Reward Over Time")
          plt.xlabel("Time")
          plt.ylabel("Reward")
          plt.savefig(self._results_file / "reward.png")
          plt.close()
  
          plt.plot(self._actions_memory)
          plt.title("Actions performed")
          plt.xlabel("Time")
          plt.ylabel("Weight")
          plt.savefig(self._results_file / "actions.png")
          plt.close()
  
          print("=================================")
          print(
              "Initial portfolio value:{}".format(
                  self._asset_memory["final"][0]
              )
          )
          print(f"Final portfolio value: {self._portfolio_value}")
          print(
              "Final accumulative portfolio value: {}".format(
                  self._portfolio_value / self._asset_memory["final"][0]
              )
          )
          print(
              "Maximum DrawDown: {}".format(
                  qs.stats.max_drawdown(metrics_df["portfolio_values"])
              )
          )
          print(
              "Sharpe ratio: {}".format(
                  qs.stats.sharpe(metrics_df["returns"])
              )
          )
          print("=================================")
  
          qs.plots.snapshot(
              metrics_df["returns"],
              show=False,
              savefig=self._results_file / "portfolio_summary.png",
          )
  
          if self._new_gym_api:
              return (
                  self._state,
                  self._reward,
                  self._terminal,
                  False,
                  self._info,
              )
          return self._state, self._reward, self._terminal, self._info
  
      else:
          # transform action to numpy array (if it's a list)
          actions = np.array(actions, dtype=np.float32)
  
          # if necessary, normalize weights
          if (
              math.isclose(np.sum(actions), 1, abs_tol=1e-6)
              and np.min(actions) >= 0
          ):
              weights = actions
          else:
              weights = self._softmax_normalization(actions)
  
          # save initial portfolio weights for this time step
          self._actions_memory.append(weights)
  
          # get last step final weights and portfolio_value
          last_weights = self._final_weights[-1]
  
          # load next state
          self._time_index += 1
          (
              self._state,
              self._info,
          ) = self._get_state_and_info_from_time_index(self._time_index)
  
          # if using weights vector modifier, we need to modify weights vector
          if self._comission_fee_model == "wvm":
              delta_weights = weights - last_weights
              delta_assets = delta_weights[1:]  # disconsider
              # calculate fees considering weights modification
              fees = np.sum(
                  np.abs(delta_assets * self._portfolio_value)
              )
              if fees > weights[0] * self._portfolio_value:
                  weights = last_weights
                  # maybe add negative reward
              else:
                  portfolio = weights * self._portfolio_value
                  portfolio[0] -= fees
                  self._portfolio_value = np.sum(
                      portfolio
                  )  # new portfolio value
                  weights = (
                      portfolio / self._portfolio_value
                  )  # new weights
          elif self._comission_fee_model == "trf":
              last_mu = 1
              mu = (
                  1
                  - 2 * self._comission_fee_pct
                  + self._comission_fee_pct**2
              )
              while abs(mu - last_mu) > 1e-10:
                  last_mu = mu
                  mu = (
                      1
                      - self._comission_fee_pct * weights[0]
                      - (
                          2 * self._comission_fee_pct
                          - self._comission_fee_pct**2
                      )
                      * np.sum(
                          np.maximum(
                              last_weights[1:] - mu * weights[1:], 0
                          )
                      )
                  ) / (1 - self._comission_fee_pct * weights[0])
              self._info["trf_mu"] = mu
              self._portfolio_value = mu * self._portfolio_value
  
          # save initial portfolio value of this time step
          self._asset_memory["initial"].append(self._portfolio_value)
  
          # time passes and time variation changes the portfolio distribution
          portfolio = self._portfolio_value * (
              weights * self._price_variation
          )
  
          # calculate new portfolio value and weights
          self._portfolio_value = np.sum(portfolio)
          weights = portfolio / self._portfolio_value
  
          # save final portfolio value and weights of this time step
          self._asset_memory["final"].append(self._portfolio_value)
          self._final_weights.append(weights)
  
          # save date memory
          self._date_memory.append(self._info["end_time"])
  
          # define portfolio return
          rate_of_return = (
              self._asset_memory["final"][-1]
              / self._asset_memory["final"][-2]
          )
          portfolio_return = rate_of_return - 1
          portfolio_reward = np.log(rate_of_return)
  
          # save portfolio return memory
          self._portfolio_return_memory.append(portfolio_return)
          self._portfolio_reward_memory.append(portfolio_reward)
  
          # Define portfolio return
          self._reward = portfolio_reward
          self._reward = self._reward * self._reward_scaling
  
      if self._new_gym_api:
          return (
              self._state,
              self._reward,
              self._terminal,
              False,
              self._info,
          )
      return self._state, self._reward, self._terminal, self._info
  
imports: from __future__ import annotations
  import math
  import gym
  import matplotlib
  import numpy
  import pandas
  from gym import spaces
  from gym.utils import seeding
  import matplotlib.pyplot
  from stable_baselines3.common.vec_env import DummyVecEnv
  from pathlib import Path
label_desc: Use image node_id 2 for calling the PortfolioOptimizationEnv obj's underlying member method code with example usage: obj.step(actions) and returns: self,  self,  self,  self, self,  self,  self,  self, self,  self,  self,  False,  self, self,  self,  self,  False,  self