parikh-et-al-and-bilstm-max-pool/modules/Utils.py
def create_dataset(self, pairs, word_dict, label_dict=None,
                       max_len1=None, max_len2=None):
        """
        Generate and return a NLIDataset object for storing the data in numpy
        format.
        :param pairs: list of tokenized tuples (sent1, sent2, label)
        :param word_dict: a dictionary mapping words to indices
        :param label_dict: a dictionary mapping labels to numbers. If None,
            labels are ignored.
        :param max_len1: the maximum length that arrays for sentence 1
            should have (i.e., time steps for an LSTM). If None, it
            is computed from the data.
        :param max_len2: same as max_len1 for sentence 2
        :return: RTEDataset
        """
        tokens1 = [pair[0] for pair in pairs]
        tokens2 = [pair[1] for pair in pairs]
        sentences1, sizes1 = self._convert_pairs_to_indices(tokens1, word_dict,
                                                            max_len1)
        sentences2, sizes2 = self._convert_pairs_to_indices(tokens2, word_dict,
                                                            max_len2)
        if label_dict is not None:
            labels = self.convert_labels(pairs, label_dict)
        else:
            labels = None

        return NLIDataset(sentences1, sentences2, sizes1, sizes2, labels)

        Generate and return a NLIDataset object for storing the data in numpy
        format.
        :param pairs: list of tokenized tuples (sent1, sent2, label)
        :param word_dict: a dictionary mapping words to indices
        :param label_dict: a dictionary mapping labels to numbers. If None,
            labels are ignored.
        :param max_len1: the maximum length that arrays for sentence 1
            should have (i.e., time steps for an LSTM). If None, it
            is computed from the data.
        :param max_len2: same as max_len1 for sentence 2
        :return: RTEDataset
        
Generate and return a NLIDataset object for storing the data in numpy
format.
Generate and return a NLIDataset object for storing the data in numpy format .
def create_dataset(self, pairs, word_dict, label_dict=None,
                       max_len1=None, max_len2=None):
        tokens1 = [pair[0] for pair in pairs]
        tokens2 = [pair[1] for pair in pairs]
        sentences1, sizes1 = self._convert_pairs_to_indices(tokens1, word_dict,
                                                            max_len1)
        sentences2, sizes2 = self._convert_pairs_to_indices(tokens2, word_dict,
                                                            max_len2)
        if label_dict is not None:
            labels = self.convert_labels(pairs, label_dict)
        else:
            labels = None
        return NLIDataset(sentences1, sentences2, sizes1, sizes2, labels)
def create_dataset ( self , pairs , word_dict , label_dict = None , max_len1 = None , max_len2 = None ) : tokens1 = [ pair [ 0 ] for pair in pairs ] tokens2 = [ pair [ 1 ] for pair in pairs ] sentences1 , sizes1 = self . _convert_pairs_to_indices ( tokens1 , word_dict , max_len1 ) sentences2 , sizes2 = self . _convert_pairs_to_indices ( tokens2 , word_dict , max_len2 ) if label_dict is not None : labels = self . convert_labels ( pairs , label_dict ) else : labels = None return NLIDataset ( sentences1 , sentences2 , sizes1 , sizes2 , labels )
Generate and return a NLIDataset object for storing the data in numpy
format.
Generate and return a NLIDataset object for storing the data in numpy format .
"""
        Generate and return a NLIDataset object for storing the data in numpy
        format.
        :param pairs: list of tokenized tuples (sent1, sent2, label)
        :param word_dict: a dictionary mapping words to indices
        :param label_dict: a dictionary mapping labels to numbers. If None,
            labels are ignored.
        :param max_len1: the maximum length that arrays for sentence 1
            should have (i.e., time steps for an LSTM). If None, it
            is computed from the data.
        :param max_len2: same as max_len1 for sentence 2
        :return: RTEDataset
        """
None
self
pairs
list of tokenized tuples (sent1, sent2, label)
list of tokenized tuples ( sent1 sent2 label )
word_dict
a dictionary mapping words to indices
a dictionary mapping words to indices
label_dict
a dictionary mapping labels to numbers. If None,
labels are ignored.
a dictionary mapping labels to numbers . If None labels are ignored .
max_len1
the maximum length that arrays for sentence 1
should have . If None, it
is computed from the data.
the maximum length that arrays for sentence 1 should have . If None it is computed from the data .
max_len2
same as max_len1 for sentence 2
same as max_len1 for sentence 2