utils.py
def full_bptt_batch_generator(input_seqs, target_seqs, max_batch_size, num_epochs=np.inf,
                              shuffle=True, random_roll=False):
  """ Generate batches for full backpropagation through time.

  Each input sequence must have the same length as its corresponding target
  sequence. (If there are fewer targets, insert nans to align the existing
  targets with their corresponding time steps.)

  All sequences in a batch will be cast to floats. To make sequence lengths
  the same for an individual batch, inputs are padded with 0.0 and targets
  are padded with nan.

  TODO: Make this more efficient. This isn't important for now because the time
  spent here is typically negligible compared to the time spent in a single
  forward, backward pass.

  Args:
    input_seqs: A list of 2-D NumPy arrays, each with shape
      `[seq_length, input_size]`.
    target_seqs: A list of 2-D NumPy arrays, each with shape
      `[seq_length, target_size]`.
    max_batch_size: An integer. Smaller batch sizes are possible when
      `max_batch_size` doesn't divide the total number of sequences that we'll
      end up visiting in `num_epochs` epochs.
    num_epochs: An integer or float. The number of epochs to visit.
    shuffle: A boolean. If True, shuffle sequences epoch by epoch.
    random_roll: A boolean. If True, randomly roll each input, target
      sequence along the time axis.

  Yields:
    A tuple `(batch_inputs, batch_targets)`, each a 3-D float NumPy array with
    shape `[batch_size, batch_length, size]`.
  """
  num_seqs = len(input_seqs)
  num_seqs_to_visit = num_seqs * num_epochs
  seq_ind_gen = seq_ind_generator(num_seqs, shuffle=shuffle)
  num_seqs_visited = 0
  while num_seqs_visited < num_seqs_to_visit:
    batch_size = min(max_batch_size, num_seqs_to_visit - num_seqs_visited)
    batch_inds = [next(seq_ind_gen) for _ in range(batch_size)]
    batch_seq_lengths = [len(input_seqs[i]) for i in batch_inds]
    batch_length = np.max(batch_seq_lengths)
    batch_inputs, batch_targets = [], []
    for i in batch_inds:
      input_seq, target_seq = input_seqs[i].astype(np.float), target_seqs[i].astype(np.float)
      if len(input_seq) != len(target_seq):
        raise ValueError('Input sequence length does not match target sequence length.')
      if random_roll:
        shift = np.random.randint(0, len(input_seq))
        input_seq, target_seq = np.roll(input_seq, shift, axis=0), np.roll(target_seq, shift, axis=0)
      input_seq, target_seq = pad_seq(input_seq, batch_length, 0.0), pad_seq(target_seq, batch_length, np.nan)
      batch_inputs.append(input_seq)
      batch_targets.append(target_seq)
    num_seqs_visited += batch_size
    yield np.asarray(batch_inputs), np.asarray(batch_targets)
 Generate batches for full backpropagation through time.

  Each input sequence must have the same length as its corresponding target
  sequence. (If there are fewer targets, insert nans to align the existing
  targets with their corresponding time steps.)

  All sequences in a batch will be cast to floats. To make sequence lengths
  the same for an individual batch, inputs are padded with 0.0 and targets
  are padded with nan.

  TODO: Make this more efficient. This isn't important for now because the time
  spent here is typically negligible compared to the time spent in a single
  forward, backward pass.

  Args:
    input_seqs: A list of 2-D NumPy arrays, each with shape
      `[seq_length, input_size]`.
    target_seqs: A list of 2-D NumPy arrays, each with shape
      `[seq_length, target_size]`.
    max_batch_size: An integer. Smaller batch sizes are possible when
      `max_batch_size` doesn't divide the total number of sequences that we'll
      end up visiting in `num_epochs` epochs.
    num_epochs: An integer or float. The number of epochs to visit.
    shuffle: A boolean. If True, shuffle sequences epoch by epoch.
    random_roll: A boolean. If True, randomly roll each input, target
      sequence along the time axis.

  Yields:
    A tuple `(batch_inputs, batch_targets)`, each a 3-D float NumPy array with
    shape `[batch_size, batch_length, size]`.
  
Generate batches for full backpropagation through time.
Each input sequence must have the same length as its corresponding target
sequence. (If there are fewer targets, insert nans to align the existing
targets with their corresponding time steps.)

All sequences in a batch will be cast to floats. To make sequence lengths
the same for an individual batch, inputs are padded with 0.0 and targets
are padded with nan.

Make this more efficient. This isn't important for now because the time
spent here is typically negligible compared to the time spent in a single
forward, backward pass.
Generate batches for full backpropagation through time . Each input sequence must have the same length as its corresponding target sequence . ( If there are fewer targets insert nans to align the existing targets with their corresponding time steps . ) All sequences in a batch will be cast to floats . To make sequence lengths the same for an individual batch inputs are padded with 0 . 0 and targets are padded with nan . Make this more efficient . This isn ' t important for now because the time spent here is typically negligible compared to the time spent in a single forward backward pass .
def full_bptt_batch_generator(input_seqs, target_seqs, max_batch_size, num_epochs=np.inf,
                              shuffle=True, random_roll=False):
  num_seqs = len(input_seqs)
  num_seqs_to_visit = num_seqs * num_epochs
  seq_ind_gen = seq_ind_generator(num_seqs, shuffle=shuffle)
  num_seqs_visited = 0
  while num_seqs_visited < num_seqs_to_visit:
    batch_size = min(max_batch_size, num_seqs_to_visit - num_seqs_visited)
    batch_inds = [next(seq_ind_gen) for _ in range(batch_size)]
    batch_seq_lengths = [len(input_seqs[i]) for i in batch_inds]
    batch_length = np.max(batch_seq_lengths)
    batch_inputs, batch_targets = [], []
    for i in batch_inds:
      input_seq, target_seq = input_seqs[i].astype(np.float), target_seqs[i].astype(np.float)
      if len(input_seq) != len(target_seq):
        raise ValueError('Input sequence length does not match target sequence length.')
      if random_roll:
        shift = np.random.randint(0, len(input_seq))
        input_seq, target_seq = np.roll(input_seq, shift, axis=0), np.roll(target_seq, shift, axis=0)
      input_seq, target_seq = pad_seq(input_seq, batch_length, 0.0), pad_seq(target_seq, batch_length, np.nan)
      batch_inputs.append(input_seq)
      batch_targets.append(target_seq)
    num_seqs_visited += batch_size
    yield np.asarray(batch_inputs), np.asarray(batch_targets)
def full_bptt_batch_generator ( input_seqs , target_seqs , max_batch_size , num_epochs = np . inf , shuffle = True , random_roll = False ) : num_seqs = len ( input_seqs ) num_seqs_to_visit = num_seqs * num_epochs seq_ind_gen = seq_ind_generator ( num_seqs , shuffle = shuffle ) num_seqs_visited = 0 while num_seqs_visited < num_seqs_to_visit : batch_size = min ( max_batch_size , num_seqs_to_visit - num_seqs_visited ) batch_inds = [ next ( seq_ind_gen ) for _ in range ( batch_size ) ] batch_seq_lengths = [ len ( input_seqs [ i ] ) for i in batch_inds ] batch_length = np . max ( batch_seq_lengths ) batch_inputs , batch_targets = [ ] , [ ] for i in batch_inds : input_seq , target_seq = input_seqs [ i ] . astype ( np . float ) , target_seqs [ i ] . astype ( np . float ) if len ( input_seq ) != len ( target_seq ) : raise ValueError ( 'Input sequence length does not match target sequence length.' ) if random_roll : shift = np . random . randint ( 0 , len ( input_seq ) ) input_seq , target_seq = np . roll ( input_seq , shift , axis = 0 ) , np . roll ( target_seq , shift , axis = 0 ) input_seq , target_seq = pad_seq ( input_seq , batch_length , 0.0 ) , pad_seq ( target_seq , batch_length , np . nan ) batch_inputs . append ( input_seq ) batch_targets . append ( target_seq ) num_seqs_visited += batch_size yield np . asarray ( batch_inputs ) , np . asarray ( batch_targets )
Generate batches for full backpropagation through time.
Generate batches for full backpropagation through time .
""" Generate batches for full backpropagation through time.

  Each input sequence must have the same length as its corresponding target
  sequence. (If there are fewer targets, insert nans to align the existing
  targets with their corresponding time steps.)

  All sequences in a batch will be cast to floats. To make sequence lengths
  the same for an individual batch, inputs are padded with 0.0 and targets
  are padded with nan.

  TODO: Make this more efficient. This isn't important for now because the time
  spent here is typically negligible compared to the time spent in a single
  forward, backward pass.

  Args:
    input_seqs: A list of 2-D NumPy arrays, each with shape
      `[seq_length, input_size]`.
    target_seqs: A list of 2-D NumPy arrays, each with shape
      `[seq_length, target_size]`.
    max_batch_size: An integer. Smaller batch sizes are possible when
      `max_batch_size` doesn't divide the total number of sequences that we'll
      end up visiting in `num_epochs` epochs.
    num_epochs: An integer or float. The number of epochs to visit.
    shuffle: A boolean. If True, shuffle sequences epoch by epoch.
    random_roll: A boolean. If True, randomly roll each input, target
      sequence along the time axis.

  Yields:
    A tuple `(batch_inputs, batch_targets)`, each a 3-D float NumPy array with
    shape `[batch_size, batch_length, size]`.
  """
A tuple `(batch_inputs, batch_targets)`, each a 3-D float NumPy array with
shape `[batch_size, batch_length, size]`.
A tuple ` ( batch_inputs batch_targets ) ` each a 3 - D float NumPy array with shape ` [ batch_size batch_length size ] ` .
input_seqs
A list of 2-D NumPy arrays, each with shape
`[seq_length, input_size]`.
A list of 2 - D NumPy arrays each with shape ` [ seq_length input_size ] ` .
target_seqs
A list of 2-D NumPy arrays, each with shape
`[seq_length, target_size]`.
A list of 2 - D NumPy arrays each with shape ` [ seq_length target_size ] ` .
max_batch_size
An integer. Smaller batch sizes are possible when
`max_batch_size` doesn't divide the total number of sequences that we'll
end up visiting in `num_epochs` epochs.
An integer . Smaller batch sizes are possible when ` max_batch_size ` doesn ' t divide the total number of sequences that we ' ll end up visiting in ` num_epochs ` epochs .
num_epochs
An integer or float. The number of epochs to visit.
An integer or float . The number of epochs to visit .
shuffle
A boolean. If True, shuffle sequences epoch by epoch.
A boolean . If True shuffle sequences epoch by epoch .
random_roll
A boolean. If True, randomly roll each input, target
sequence along the time axis.
A boolean . If True randomly roll each input target sequence along the time axis .