text:
Why would it be easy to evaluate a probability, when it is hard to sample from for importance sampling?
Suppose we want to perform importance sampling where we have trajectories from some behavioral policy $b$, but we want to perform off-policy evaluation.  From these prior questions, I understand that importance sampling can be useful because we can easily sample from and evaluate the target policy $\pi$, while it is hard (or no longer possible) to sample from the original policy $b$.  This is given by:
$$E_{x \sim \pi}[f(x)] \approx \frac{1}{n} \sum_{i=1}^n f(x_i) \frac{\pi(x_i)}{b(x_i)} $$
where the original trajectories $x_i$ were from $b$, but have now been reweighted as if they came from $\pi$.
However, if we no longer have access to sample from policy $b$, then how would we hope to easily evaluate the probability of some $x$ using policy $b$?  Conversely, if we are able to easily evaluate $b(x_i)$, how come we can't sample from it?  I'm especially interested in real-life examples where the probability distribution is not Poisson, Gaussian or some simple Bayes Net.  For example, the probability distribution could be related to recognizing cats in images, dialogue generation or question answering.

You might be a bit confused. In IS, we always assume access to some distribution $b$ that is e.g. easier to sample from than the target $\pi$. This is the whole point of IS. In RL it also has the use case of using a highly exploratory policy to generate actions, but again this only works if we have access to the distribution, so in your question we would never ‘lose access’ to such a policy.

I updated the text for clarity, did I mix up the behavioral and target policy?  I know that at least one of them is not available for sampling from during test time.

You seem to be confusing why IS is useful. If you can easily evaluate $\pi$ then the main reason (in RL context) to use IS is for more exploration. To do this you’d use exploratory policy $b$ as you’ve mentioned, but you would never lose access to this. The IS sampling policy is always user chosen so you would never lose access to it.


text:
Datasets input at model.fit produce unexpected results of training loss vs validation loss
Im trying to train a neural network (VAE) using tensorflow and Im getting different results based on the type of input in the model.fit.
When I input arrays I get normal difference between the validation loss and the total loss.
When I input a dataset based on the same input I get a normal total loss and a really small validation loss.
I havent changed the model. The only things that changes is the input format.
The code for when I input an array.
train slices is (2627,138,138,1) and define the batch size in the model.fit
train_slices = preprocess_data(CropTumor, file_array[train_dataset])
val_slices = preprocess_data(CropTumor, file_array[val_dataset])

# reset model weights before training
VAE.set_weights(initial_weights)

# fit model
fit_results = VAE.fit(train_slices,train_slices,
                      epochs=1000,
                      validation_data=(val_slices,val_slices),
                      callbacks=[early_stopping_kfold, tensorboard_callback],
                      batch_size=batch_sz,
                      verbose=2
                      )

The output
Epoch 1/1000
2022-08-01 11:56:35.683852: I tensorflow/stream_executor/cuda/cuda_dnn.cc:384] Loaded cuDNN version 8401
2022-08-01 11:56:36.371780: I tensorflow/core/platform/default/subprocess.cc:304] Start cannot spawn child process: No such file or directory
2022-08-01 11:56:36.461054: I tensorflow/stream_executor/cuda/cuda_blas.cc:1786] TensorFloat-32 will be used for the matrix multiplication. This will only be logged once.
672/672 - 7s - loss: 537.2896 - val_loss: 213.7070 - 7s/epoch - 11ms/step
Epoch 2/1000
672/672 - 5s - loss: 248.5211 - val_loss: 161.9758 - 5s/epoch - 7ms/step
Epoch 3/1000
672/672 - 4s - loss: 192.8771 - val_loss: 125.9349 - 4s/epoch - 6ms/step
Epoch 4/1000
672/672 - 4s - loss: 153.1647 - val_loss: 99.4395 - 4s/epoch - 6ms/step
Epoch 5/1000
672/672 - 5s - loss: 132.0143 - val_loss: 88.9975 - 5s/epoch - 7ms/step
Epoch 6/1000
672/672 - 4s - loss: 118.5642 - val_loss: 81.1653 - 4s/epoch - 6ms/step
Epoch 7/1000
672/672 - 5s - loss: 108.6678 - val_loss: 76.6315 - 5s/epoch - 7ms/step
Epoch 8/1000
672/672 - 4s - loss: 100.9759 - val_loss: 73.8963 - 4s/epoch - 6ms/step

When on the other hand I use the same data in the form of dataset I get a really small validation loss
    train_dset =

tf.keras.preprocessing.image_dataset_from_directory(directory="./Data/09_TrainingSet_VAE1",
                                                                     labels=None,
                                                                     label_mode=None,
                                                                     image_size=(138, 138),
                                                                     color_mode="grayscale",
                                                                     batch_size=None,
                                                                     shuffle=True)

    val_dset = tf.data.Dataset.from_tensor_slices(val_slices)
    train_dset = (train_dset.map(preprocess_dataset).batch(batch_sz).shuffle(1))
    val_dset = (val_dset.map(preprocess_dataset).batch(batch_sz).shuffle(1))
    # reset model weights before training
    VAE.set_weights(initial_weights)

    # fit model
    fit_results = VAE.fit(train_dset,
                          epochs=10,
                          validation_data=val_dset,
                          callbacks=[early_stopping_kfold, tensorboard_callback],
                          verbose=2
                          )

And my output is
 Epoch 1/10
2022-08-01 12:04:08.656012: I tensorflow/stream_executor/cuda/cuda_dnn.cc:384] Loaded cuDNN version 8401
2022-08-01 12:04:09.335957: I tensorflow/core/platform/default/subprocess.cc:304] Start cannot spawn child process: No such file or directory
2022-08-01 12:04:09.431082: I tensorflow/stream_executor/cuda/cuda_blas.cc:1786] TensorFloat-32 will be used for the matrix multiplication. This will only be logged once.
613/613 - 7s - loss: 466.5601 - val_loss: 17.3872 - 7s/epoch - 12ms/step
Epoch 2/10
613/613 - 5s - loss: 217.4277 - val_loss: 7.7309 - 5s/epoch - 8ms/step
Epoch 3/10
613/613 - 5s - loss: 167.2855 - val_loss: 6.0742 - 5s/epoch - 9ms/step
Epoch 4/10
613/613 - 6s - loss: 130.8230 - val_loss: 1.9557 - 6s/epoch - 10ms/step
Epoch 5/10
613/613 - 6s - loss: 112.1165 - val_loss: 1.1561 - 6s/epoch - 10ms/step
Epoch 6/10
613/613 - 5s - loss: 101.3152 - val_loss: 0.6442 - 5s/epoch - 8ms/step
Epoch 7/10
613/613 - 5s - loss: 93.3648 - val_loss: 0.4150 - 5s/epoch - 8ms/step
Epoch 8/10
613/613 - 5s - loss: 87.1542 - val_loss: 0.2232 - 5s/epoch - 8ms/step

The loss function for both is
def loss_func(encoder_mu, encoder_log_variance):
def vae_reconstruction_loss(y_true, y_predict):

    reconstruction_loss = tf.math.reduce_sum(tf.math.square(y_true-y_predict), axis=[1, 2, 3])
    return reconstruction_loss

def vae_kl_loss(encoder_mu, encoder_log_variance):
    kl_loss = -0.5 * tf.math.reduce_sum(1.0 + encoder_log_variance - tf.math.square(encoder_mu) - tf.math.exp(encoder_log_variance),
                              axis=1)
    return kl_loss

def vae_loss(y_true, y_predict):
    reconstruction_loss = vae_reconstruction_loss(y_true, y_predict)
    kl_loss = vae_kl_loss(y_true, y_predict)
    loss = reconstruction_weight*reconstruction_loss + kl_weight*kl_loss
    return loss

return vae_loss

and model is compiled with
VAE.compile(optimizer=tfk.optimizers.Adam(learning_rate=learning_rate),
        loss=loss_func(encoder_mu_layer, encoder_log_variance_layer))

Solved it. It was a bug in my code after all.
preprocess_dataset function divide the data by 255 and I used the same for both the training dataset which was coming from a directory (0-255 range) and the validation dataset which came from a loaded array already divide it by 255

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
Does this kind of attention exist?
As someone who is new to deep learning, I am only familiar with self-attention.
I'm designing a model. Imagine there are n data, which the $i_{th}$ data can be represented as a vector $x_i$. And the data has an attribute $a$, the attribute of the $i_{th}$ data is $a_i$. now I only use one query vector $q$ to multiply each data to calculate the result as the weight $w_i$ ($w_i=q^t x_i$) , and use the weight of each data to multiply each data's attribute $a_i$ as the final result. ($A=\sum w_i a_i$)
Does this kind of attention exist? if so what its name is?
For example, $x_i$ is a 768-dimensional vector representing each sentence, and $s_i$ is a 3-dimensional vector representing each sentence's sentiment. $q$ is a 768-dimensional vector multiplied with each $x_i$ to produce each sentence's weight $w_i$. And the weighted sum $S=\sum w_i s_i$ is the overall sentiment of all sentences.

I think this description needs a bit more clarity. Here are the things I don't yet get: 1) Is $*$ the dot- or the elementwise product? 2) You first have $A = \sum w_i * a_i$, where $a_i$ denote attributes (i.e. vector dimensions), but in the bottom you have $S = \sum w_i s_i$ where of $s_i$ you have 3 and of $a_i$ you have 768, so $A$ and $S$ are not the same thing, right? 3) Is the $\sum$ taken over attributes $i$? In that case you will get per-sentence sentiment and not overall sentiment of all sentences, right?

@Chillston no it simply means matrix multiplication, $q^t$ is a 1768 dimensional matrix (a row vector) and $x_i$ is a 7681 matrix (a column vector), so $w_i$ is a scalar. $s_i$ is $a_i$ and $S$ is $A$ described above, because it is sentiment score so I used the letter s. I never said $a_i$ is 768-dimensional, it is $x_i$ with 768 dimensions. For example, there are n sentences, the $i_{th}$ sentence can be vectorized as $x_i$, $s_i$ is the sentiment 3-dimensional vector of this sentence, which represents [positive probability, neutral probability, negative probability]

Okay, thanks for clarifying. Ah so you are not trying to do sentiment analysis, but use the sentiment as additional input to your model?

@Chillston it is not taken over attributes $i$, it is taken over the whole dataset (the sentences set), and $i$ means the $i_{th}$ sentence. imagine it is an article or something like it. $w_i$ is the weight I give to the $i_{th}$ sentence and $s_i$ is the $i_{th}$ sentence's sentiment vector. there is only one attribute which is the sentiment vector of each sentence, and $W = \sum w_i s_i$ is the sum over all sentences' sentiment and yes it is 3-dimensional as well.

@Chillston yes I simply used pretrained-bert models to produce the sentiment vector for each sentence.

Okay, I get now how you create the attention weights $w_i$, but the $A = \sum w_ia_i$ is a sum over the whole dataset? Isn't it rather the sum over the input sentences to the model?

@Chillston  yes by whole dataset I mean all the inputs. for example there are n news headlines, it is all fed into this model to calculate weight for each headline. So it is the weighted sum over all headlines. it is later fed into a larger model as input.

Sorry, I really try to understand this: Given an input $X$ to your model. Then $X$ are a number of sentences and $X$ is a subset of your full dataset $D$, right? Is the sum computed over $X$ or over $D$?

@Chillston For example, there are 10 days' data, each day has 50 news. I calculated a weighted sum of each day's sentiment,  so each day's weighted sentiment is calculated using this day's 50 news. and there are 10 3-dimensional vectors because there are 10 days. and I put it into a RNN or something. I really dont know what makes the confusion. the whole aim is to train the query vector q. you dont have to understand my model, it is just an example. I just want to know if there is already any kind of attention like this. is it naive attention? plain attention or something like this?

Probably what causes my confusion is that, effectively, your attention for the $i-th$ element is independent of the other elements in the sentence, right? I am thinking that inside a GRU/LSTM you have the update gates that seem to be a more expressive version of what you are doing, but I could be wrong. To answer your question: I don't know of any paper that proposes something like this, but something like the update-gates in RNNs or the gated activation seems be very related.

I found my idea quite similar to this picture https://pic4.zhimg.com/v2-a5b5df8902420bb10688fcf6ed1dc38d_r.jpg?source=1940ef5c, where $q$ is the query, $k_i$ is the key vector of the $i_{th}$ data and $v_i$ is the value vector of the $i_{th}$ data. But I still can't find which paper it comes from.

It looks pretty silimar, do you have a link to a paper or something where that figure is from?

@Chillston https://www.zhihu.com/question/68482809/answer/1876764572 It is written in chinese though. It is not from the first paper in his reference. Sukhbaatar, Sainbayar, et al. "End-to-end memory networks." arXiv preprint arXiv:1503.08895 (2015) so I don't know where it is from.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
How to determine whether this situation belongs to data leakage or not
Suppose that I use three features (x1, x2, x3) to predict the value of y. After hyperparameters tuning, the r2 score on train/valid/test set is 0.92, 0.54, 0.55 respectively, it's not so good.(It is called "original" below)
There are some relationships between x1 and y, so I change the prediction target from y to y/x1, the situation becomes using x1, x2, x3 to predict y/x1, the r2 score on train/valid/test set becomes 0.91, 0.72, 0.73, the performance of model is improved than origin, but is this data leakage?
Similarly, if use x1, x2, x3 to predict y/x2, the r2 score on train/valid/test set is 0.90, 0.64, 0.64 (slightly increase than original); if use x1, x2, x3 to predict y/x3, the r2 score on train/valid/test set is 0.85, 0.45, 0.56 (almost same as original). If introduce feature into target belongs to data leakage, why doesn't the introduction of different features into target lead to a significant improvment in r2 score?

Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking.

Can you explain your rationale for changing the target from y to y/x1?


text:
What to say about time complexity of SVM?
I've read it in some literature now that the training speed of SVM (in general) is very low. Why is that the case? What is to say about time complexity of SVM?

The slow training of support vector machines comes from the Sequential minimal optimization algorithm, which is proven to have time complexity of roughly $O(n^3)$ (mostly because it involves matrix inversion operations).


text:
Why DQN model select same action during the training
Now i try to create the DQN model. During the training process, the action value of each step is different, but most of the time, the same action is always selected. How can i solve it?
Replay memory is 1000,
batch size is 32,
The learning rate is 0.0025,
eplison is 1.0
epsilon decay is 0.98
The discount fatcor is set to 0.98.
activation function for hidden layer is ReLu, and for output layer is linear

What sort of action selection strategy are you using? (EGreedy, EGreedy Linear, EGreedy Exp, etc)

Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking.


text:
Bouding the state using the reward in RL
I'm wondering what the common approaches are bounding out state $s\in\mathbf{R}$ to some values $\in[s_0,s_1]$ is required. So in my case, for example, the state represents an angle of rotation, that should Ideally be bounded between $0$ to $\pi$ radians. My naive approach to enforce that was to check the state in every step, and "punish" when the state is not contained in this range. With respect to this approach - how the "punishment" should actually be implemented? Two options I thought of are:

if $s_t\notin[0,\pi]$: $r_t \leftarrow 0.1$ - This, in my opinion, may not be optimal as there is no indication of how far are we from $[0,\pi]$. Also, there is no incremental aspect - Even if we diverged from $[0,\pi]$ once or 1000 times, the reward is always the same.
if $s_t\notin[0,\pi]$: $r_t \leftarrow 0.9\cdot r_t$ - the longer we stay outside the needed range, the reward tends to zero, but still It doesnt count for distances

Any other common ideas?

Since you are working with an angle, did you consider simply unwrapping the angle? For example, we know that a rotation of $3\pi$ radians is equivalent to a rotation of $\pi$.

My desired goal would be to learn a reciprocating motion, so I don't think that unwrapping is good here. If the angle is $3\pi$ that means full rotations which I'm trying to avoid. (Think of a robot with wings that should learn how to fly where $s=\phi$ is the angle of wing rotation)

I don't really understand the question. The state is not an output of your agent, but comes from the environment, which is not something you train. The action, the state value (a function of the state), or the action state value (a function of the state and the action) are, so these could be constrained in some bounds by penalising the agent, but not the value of the state itself.

@RaphaelLopezKaufman I can define the reward to penalize unwanted states so the agent will try to avoid them as they do not contribute to the maximization of the reward

Ah I see, is there a particular reason why you don't want to use a negative reward? Your examples are both positive rewards.

Hmm not really - do you have any suggestion on how this should be implemented? I did try, for example $r_t \leftarrow r_t -1$, with no luck


text:
Reward Function for Reinforcement Learning model
I am trying to create a reinforcement learning model to control the acceleration of a car. I am designing the model such that initially the acceleration is provided and then deceleration is provided so that the the car covers a particular distance and the end velocity comes out to be zero.
The reward function I have come up till now looks like this:
$$R = (f(x_t) - f(x_{t-1}))*K - \frac{v}{|f(x_t)|} - r_\text{penalty}$$
where $f(x_t) = -|x_t - x_d|$ is the error function.
Based on this reward function, the model learns to provide acceleration. However, it doesn't provide any deceleration causing the velocity to be very high at the end.
The objective of the model is to find the right time when it should change from acceleration to deceleration.

Could you please elaborate a bit more on the other variables ? What are $K$, $v$ and $r_{penalty}$ ?

@nathanraynal K is a constant, v is velocity and $r_{penalty}$ is penalty when constraints are violated.


text:
Why GANs trained on QM9 dataset produces drug-like molecules?
Why the GANs (Generative adversarial Network) trained on QM9 dataset (contains 134K molecules but none of them is complete to be eligible for drug-like molecule) produces drug-like molecules.
sine GANs trained on cats images will generate fake images of cats only then why in-complete drug-like  sample of QM9 fed into GANs can produce complete drug-like molecules?

I dispute the premise of this question, have you seen: https://chemistry.stackexchange.com/q/135122/73620 https://paperswithcode.com/sota/drug-discovery-on-qm9  https://www.researchgate.net/publication/342093683_GEOM_Energy-annotated_molecular_conformations_for_property_prediction_and_molecular_generation  https://academic.oup.com/bib/article/21/3/919/5498046 - the QM9 Dataset contains molecules which are sufficiently complete.

@Rob https://chemistry.stackexchange.com/questions/167632/are-the-133-885-molecules-in-the-qm9-database-drug-like

Vinay, Karsten's answer: "... No, probably not a single one because they are too small. ...". --- Geoff's answer: "... Almost none of these are "drug-like" by the usual rules (e.g., Lipinski Rule-of-Five).".  --- See also: https://www.researchgate.net/publication/274256239_Quantum_chemistry_structures_and_properties_of_134_kilo_molecules and  section 4.2.4 of https://www.frontiersin.org/articles/10.3389/fphar.2021.827606/full --- PS: You should include the link to your question on Chem.SE in the above question, so people know that you have asked on more than one site.


text:
Is the neural network 100% accurate on training data if epoch loss is minimized to 0?
This seems like a silly, trivial question, but I just want to confirm it in case I'm missing something.
I'm trying to train a ReLU neural network, which is supposed to be a function that satisfies some conditions.
For example, I want the network $\mathcal{N}(x)$ to satisfy $\mathcal{N}(x) \leq \eta, \ \forall x \in D_1$, and $\mathcal{N}(x) \geq \eta, \ \forall x \in D_2$, where $D_1 \cup D_2$ is the training data set.
I incorporate these conditions as a custom loss function as follows:
$$L := \cup_{x \in D_1} \mathrm{ReLU}( \mathcal{N}(x) - \eta) + \cup_{x \in D_2} \mathrm{ReLU}(-\mathcal{N}(x) + \eta) $$
The idea is that if $L = 0$, then the conditions are satisfied for $\mathcal{N}(x)$. So when I train the network, I ensure that the epoch loss is 0 (that's when the algorithm converges). If it matters, I use an SGD optimization technique for training.
So it is safe to say that after training, the network for sure satisfies those conditions with a 100% guarantee over the training data set, right?
I understand that this may not be the case for random unseen data, but I just want to make sure that the guarantee holds for the training data set, and it will not be required to validate this a posteriori.

Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer.

Interesting loss term you've got there. Regardless, are you asking whether this is the case without any further verification? Otherwise, you can just calculate $L$ after the training on $\mathcal{N}$ was completed. Also, I think that if you're using batches, the loss term is in fact a union over $x\in B$, which may result in some non-zero values over all $D_1$

@HadarSharvit Can you explain how it may result in non-zero values? I do in fact use batches. So my data set $D_1$ is divided into let's say batches $B_1$, $B_2$ and $B_3$ (randomly shuffled in each epoch), epoch loss is 0 only if the batch losses $B_1, B_2$ and $B_3$ are 0, so that means it should be 0 over $D_1$. Am I missing something?

@HadarSharvit and yes I want to avoid further verification!

There is NO guarantee that the neural network will be 100% accurate on the training data set. Even if the epoch loss is minimized to $0$, There is always a margin of error when training a neural network, there is still a possibility that the neural network could make errors when applied to new data.

Yes, If the epoch loss is minimized to $0$, then the conditions are satisfied for $N(x)$ &  it is safe to say that after training, the network for sure satisfies those conditions with a 100% guarantee over the training data set. However, this does not mean that the network will be 100% accurate on new, unseen data.


text:
Can NotA (none of the above) class be combined with object detection / segmentation models?
So there's this way of classifying an object as NotA (none of the above) class as described in https://arxiv.org/abs/1910.02830
So mostly this uses a sort of confidence thresholding with different approaches.
Can a classifier with such a NotA class be used as part of an object detection framework?
I'm assuming if the classifier isn't trained on a background class then NotA would "light up" also on background, so that class might need to be added as well to make some sense.

Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking.

so a bot is asking me for clarifications? We live in the future, don't we


text:
How to classify images which are slightly different from each other?
(I hope I'm in the right place to ask such a question.)
A robot has a fixed camera which takes images before extending its sticks to grab a box from two sides and pull it back. But if the box is not in straight position and is rotated more than ~15% from its original straight position (a), robot's sticks cannot retrieve the box because the box will have extended its length (h+α) close to its diagonal length. I tried to visualize the case below:

Here, I want to collect hundreds of such images and classify them as straight, rotated, and if the image of rotated (either side) box is detected I should be able to send a notification to control unit to increase the robot's arms accordingly.
I couldn't think of any computer vision/image processing technique to address this issue, so
I wonder if a simple CNN can classify such small rotations.
What would be the optimal approach to this, in general?
Thanks.

To simplify your explanation - can we refer to this task as a binary classification given both regular and slanted images? Furthermore, have you tried some classical approaches like looking for edges and their angles w.r.t some fixed point?

@HadarSharvit yes, it could be thought as a binary classification task. I didn't try any classical approach, because if the box's longer edge isn't visible due to the occlusion of its near shorter edge (as in image b) the method would fail. Besides, although the camera is fixed into robot, the robot itself is supposed to move to various locations, and box may be in different shapes, so I tend to think that there is no one-fits-all solution in classical approaches.


text:
Which language model technique should be used with a low-resource language?
I am working with a language for which the amount of text available is relatively small (less than a billion words), what techniques exist? In particular, is it worth using a transformer or is it better to use more traditional techniques?
The aim is to generate texts in continuation of a given prompt.

have you taken a look to the spacy pretrained models documentation? They support quite a lot of languages already, see left bar in the linked page


text:
Do Support Vector Machines have the ability to learn while in use?
I've read in some literature,that SVMs are characterized by their adaptivity. Does that mean they can learn while in use?

Support vector machines are a supervised machine learning method that can be used for both classification and regression problems. They are based on finding a  Maximum Margin hyperplane that best separates the data into different classes or predicts the output value.
Support vector machines are not designed to learn online while in use. They are trained on a fixed set of data points and then used to classify new data points.
However, they can learn online, if they are implemented with online learning algorithms. Online learning algorithms can update the SVM model with new data that arrives in batches or streams, without retraining the model from scratch every time. This can improve the performance and efficiency of the SVM model in dynamic or large-scale applications.
There are some extensions of support vector machines that can enable online learning, such as incremental SVM, kernel recursive least squares, and budgeted SVM.

Incremental SVM helps in online learning by updating the SVM model with new data that arrives in batches or streams. It does not need to retrain the model from scratch every time new data comes in, which saves time and resources. It also can handle concept drift, which means that the data distribution may change over time. For example, if you are using SVM to classify spam emails, the features and patterns of spam emails may change over time as spammers use different techniques to avoid detection. Incremental SVM can adapt to these changes by adding or removing support vectors as needed.

Kernel recursive least squares helps the SVM to learn online by performing online regression with nonlinear kernel functions. It updates the model parameters and the kernel matrix with new data, using a recursive algorithm that is similar to the Kalman filter. It does not need to store all the data or solve a large optimization problem, which reduces the memory and computational requirements. It can also handle noisy data and outliers, by using a forgetting factor or a sparsity criterion. For example, if you are using SVM to predict the stock price of a company, the data may be noisy and contain outliers due to market fluctuations or external events. Kernel recursive least squares can filter out the noise and outliers and update the model with the most relevant data.

Budgeted SVM helps in online learning by performing online classification or regression with a fixed number of support vectors. It updates the model by adding new support vectors and pruning the least relevant ones when the budget is exceeded. It uses a strategy that balances the trade-off between accuracy and sparsity. It can also handle nonlinear kernel functions and different loss functions. For example, if you are using SVM to classify images of animals, the data may be large and diverse, and you may have a limited memory or computational capacity. Budgeted SVM can reduce the size of the model and the complexity of the calculations by keeping only the most informative support vectors and discarding the redundant or noisy ones.

Does "kernel trick" help with online learning?

The kernel trick does not directly help the SVM to learn online, but it helps the SVM to solve nonlinear problems by mapping the data into a higher dimensional space where it may be easier to find a linear hyperplane. The kernel trick also avoids the explicit computation of the mapping function and the coordinates of the data in the higher dimensional space, which can be costly and inefficient. Instead, the kernel trick uses a kernel function that can replace the inner product of the mapping function. This makes the SVM more flexible and scalable.

SVMs are characterized by their adaptivity, Does that mean they can learn while in use? $\rightarrow$ I think you may have misunderstood the meaning of adaptivity in the context of SVMs. Adaptivity does not mean that they can learn while in use, but rather that they can handle different types of data and problems, such as linear or nonlinear, classification or regression, and various kernel functions. SVMs are characterized by their adaptivity because they can find the optimal hyperplane that separates the data with the maximum margin, regardless of the shape or dimensionality of the data. However, once the SVM is trained, it does not change its parameters based on new inputs.

References:

Support Vector Machine — Introduction to Machine Learning Algorithms.

Incremental learning algorithm of least squares support vector machines based on Renyi entropy

An incremental LS-SVM learning algorithm ILS-SVM

A novel incremental Kernel Nonparametric SVM model (iKN-SVM) for data classification: An application to face detection

How exactly does the "kernel trick" help with online learning?

@Faizy imprecise. The ability to perform online learning is a property of SVM itself; the kernel trick has nothing to do with it. In fact, due to the computational expense of kernel, it is more difficult to learn online with a kernel than without.

Check out this answer.

The maximum margin property does not imply online trainability, as this answer claims.  What matters is that the maximum margin property is maintained as new data arrives.  This depends entirely on the employed optimization algorithm and the data generating process’ properties.  (Aside: the kernel trick is only relevant if it affects the interaction of those things.)


text:
Very high ACC (ca. 95%) with 1DConvNet for Time Series
Does this sound legit, for people working with CNN and Time Series?
I have a Framework that applies Dynamic Tim Warping (DTW) on time series, using the DTW distance matrix, I  cluster my data and create labels for each time series and then apply a 1DConvNet to make classification.
I end up with a very high validation accuracy, ca. 95%, and it also does not really look like overfitting or underfitting to me. But a 95% accuracy is too high in my eyes. I have a dataset that has 150 time series and each has 200 time steps. For clustering, I use KMeans Cluster with 4 clusters.

Just a quick comment: I'd check the size of your validation split and eventually do a k-fold cross validation to be sure that you didn't randomly sample a 'too simple' validation set. What looks a bit weird is that the curve reaches 0.95 and after 1k steps then kind of stops there - that might be due to the selection of validation set. From what's visible, everything else looks legit to me though.

@Chillston you mean this clean cut of the blue curve? Yeah that what I saw too. I'll try the k-fold validation. Thank you.


text:
how to handle different objectives in Atari games in reinforcement learning
My impression on DeepMind's Deep-Q RL for learning Atari games paper is that it uses the same model to learn to play multiple different games at the same time. I wonder how did the RL agent learn in such a setting. In particular, when the game environment changed, how does the agent know the corresponding objective to achieve in that specific game? Is there a prompt to tell the agent what game it is currently in before training/evaluation? Or the agent can infer what is the goal simply by observing the feature from the environment (e.g. by observing blocks of bricks it can infer it needs to break all of them, can by observing there is a controllable vehicle its knows it is probably a racing game)?

Does this answer the question? https://ai.stackexchange.com/questions/2190/was-deepminds-dqn-learning-simultaneously-all-the-atari-games

yes. thank you so much!


text:
Does a constant background in training data affect image classification model's performance?
For a client I needed to figure out if a machine learning model can classify plastic container loading patterns. These containers are delivered on pallets in varying patterns.
So I installed a camera setup that takes images of incoming pallets and saves these images together with the correct label. The region of interest is always in the exact same place in the image and ~40% of the image is background.
After training a model with ML.NET's modelbuilder on a dataset of 7000 images divided over six classes I get a model that's 97% accurate, not bad.
I made a test setup using the code generated by ML.NET in the garage with a pallet and said containers. When testing the model in this setup, the results are disappointing, always predicting the same (wrong) class and with scores as high as 0.999. The test setup is almost identical to the setup that made the training data, except for the background and maybe the ambient lichting.
Is the background the problem here?
Can I solve this with data augmentation? (vertical mirroring, trans, rotate and scale a bit)
Should I write a script to crop the images so mostly the ROI is left?
Or is the data I generated useless?
Now you're thinking why didn't he include some images? Well, my client doen't allow it. Blacking out sensitive areas would leave the image 95% black. But here is a paint doodle of the situation if that helps.
So to clarify the only thing that varies in the 7000 images is the content on the pallet, it varies in pattern and height. The background of the training data never varies and has lots of features.

EDIT:
What I can show is some of the cropped and high-pass filtered data that I used to train the model, one closed example and one open.

If the background and the ambient light are the problem, I suggest you to agument your training data by introducing that kind of variance there. Just mirroring and rotating might not be sufficient, since they have little effect on the background's texture etc.
Alternatively you'll need to collect a more realistic dataset for training.
Cropping is a great idea, that way there is less chance for the network to be mislead by irrelevant changes in the input.
An other idea is to try filtering the image before passing it to the network, for example with a high-pass filter. This way the large-scale gradients caused by ambient light sources don't show up as strongly. The AI should focus on sharp edges of the boxes anyway, not their color. Your example image shows this point very well :)
And more thing to try is to make your network much simpler, to avoid overfitting.
Now that you have more images, you can add them to your validation set and immediately see whether you are making progress.

Far fetched idea: Would it be possible to train a model to semantically segment the ROI's (which are easily distinguishable), and with that removing the background? This could be applied to the full dataset to generate a dataset only containing ROI.. To prevent the black background from becoming a feature I could generate random backgrounds from images from other unrelated datasets available on the internet.

That is a good pre-processing step, and great if you can automate that by training a model to do it. But I don't quite understand why would you add random backgrounds, after having just removed it? Is this for data augmentation, and the final network would run without the ROI identification? Here assume the whole palled would be the ROI, not individual boxes.

And looking at your illustration from a fresh perspective, it might be worthwhile to try identifying the borders which separate different boxes from each other. If you can get this to work, it isn't too much effort to determine how many connected components (boxes) there are. The output is also easier to debug, rather than a plain classification. Here it is worth noting that down and right facing walls must be ignored, we are interested only of box faces facing up! Hey maybe that could be the ROI. And this should be a totally separate answer, to allow up & downvoting.

I thought adding a non static background would make it generalize better.. Detecting the individual boxes is an interesting idea, maybe using instance segmentation, unfortunately the containers can differ very much and can even be open on top containing random parts.
I trained a model with my previous plans (no background, color/high pass filters, skew, scale, rotate, flip), it performs horribly on test set as well as original images, I'm so very stuck now.

Since you cannot share any of the original images, I cannot really try out any of my ideas either. But actually if your network is performing poorly on for example high-pass filtered images, I see it as a sign of progress! If a human can still solve the problem with augmented images, so should a neural network. And if you are lucky, once you get it to work on the training set it will also work on the test set. Remember to apply the filter on all images, not just some of them, to be consistent.

I added some imagery to my question that I can show!


text:
Is ANN architecture mesh topology exist?
I'm just wondering if there's ANN architecture that looks like mesh topology at context of computer networking.
If exist or possible, is layer notion still applied?

I don't know if that's what you're looking for, but there are graph neural networks that model arbitrary network structures

Generally we need data to flow from input to output - for obvious reasons. You might make a graph-shaped network but then which way is input to output? Maybe you say it flows both ways and you run this two-way network for a few steps but then if we unpack that, it turns out each step is like a layer.

Can you give me example the graph you mean? And how it's eventually same (still have layers).


text:
openVINO human pose
Can someone give me a tip on what to research for predicting human pose with computer vision with the open VINO toolkit? Not a lot of wisdom here so any tips appreciated even at a high level on what I need to learn/research...
I preprocess an image with:
def preprocessing(input_image, height, width):
    '''
    Given an input image, height and width:
    - Resize to width and height
    - Transpose the final "channel" dimension to be first
    - Reshape the image to add a "batch" of 1 at the start
    '''
    image = np.copy(input_image)
    image = cv2.resize(image, (width, height))
    image = image.transpose((2,0,1))
    image = image.reshape(1, 3, height, width)

    return image

and run inference with the open VINO toolkit models:
# Run inference.
predicted_poses = compiled_pose_model([image])[compiled_pose_model.output(0)]

Which returns a numpy array but then get lost on what the next steps are to turn an image:

Into this:

This is the code I am working. Any tips appreciated.
EDIT
Im working with the open Model Zoo (part of Intel's Open VINO project) and the model I am working with is the human-pose-estimation-0001.

It helps to know what exactly your model is. Specifically, what does it produce as output? Is it a probability map, is it a list of bounding boxes, what is it?

I added an EDIT and am trying to follow this example: https://github.com/openvinotoolkit/openvino_notebooks/blob/main/notebooks/402-pose-estimation-webcam/402-pose-estimation.ipynb


text:
How to keep the best shot of an object with an Object detection model over a video
I’m pretty new to Machine learning and stuff. I’m going to start to work on a project, I have a background with code. My goal is to collect data from my city roads and build a strong dataset for people to use it.
At the moment, i have a kinda working solution, my ML model (using yolov7 https://github.com/WongKinYiu/yolov7 + Python) is able to detect cars and license-plate. I implemented a way to crop cars from from a video stream BUT it’s cropping every single frame where there’s a “car” detected.

I found a website of a company showing how their solution works “https://www.sighthound.com/” and we can see on the video presentation (~0.6s into the video)
It’s exactly what i want to acheive, which is to keep the best result of all my cropped images from a batch and use it to create a dataset.
Do i have to put some kind of “ID” over the “car” label (like an object tracking) to identify each unique cars passing in my stream, and after that, use some kind of machine learning algorithm to select the best looking cropped images for each batch of images of a unique car ? Or there’s better solution?
Ps : I'm not doing OCR or surveillance, My only goal with this project it to create an open and free dataset for my community to use.
Update :
I want this dataset to be useable in a large quantity of sub-project and maybe inspired people to create their own dataset and contribute to the community.
So by "best" i mean : Less noise, less blur, the object "car" can be seen in its fullest, a clear view of the object "lience-plate".

For context, the country where i live is very unique (multiple gouvernment making the laws, we're a hub a population), we have 5~6 different type license plate in circulation, and little to no information about the what's going on the road.
I want to provide a high quality dataset to the community and help people/students/whoever want to work on a solution to fix road problems :

Analyse who's using the road (country 1: 50%, country 2: 25%, country 3: 25%), ...

Maybe it would help if you further defined what you mean by "best". Is "best" defined as the most complete picture of the vehicle? So any vehicle with the bottom or top cutoff is worse than the entire picture, or is "best" defined as the car having more pixels?

I think that @juicedatom is exactly right.  There are an infinite number of ways to contrive "best" and many of them contradict.  A "best" for shortest time between start and stop is the opposite of "least energy" or "least max force" between stops.  If you can get exactingly precise, the definition can lead straight to the formulation you wish you had.

I updated the post, but basically, I want to build a high quality dataset from a video-stream of a camera on the road. So i want to keep only 1 picture of every car that i capture in the video. By "Best" i mean, the image where the object "car" and/or object "license-plate" is visible in their fullest with mininum blur and noise (as clear as possible).
PS: by "High quality" i don't mean 4K, but highly reusable. Maybe OCR on license-plate, licence-plate recognition for parking (we have many different plate), car statistics (want kind of car are on the road), and so on..


text:
For which problem sizes is Deep Q-Learning suitable and why?
I am wondering for which problem sizes a Deep Q-Learning algorithm is most appropriate. For example, whether it is particularly suited for low complexity problems or not for high complexity problems. And if that is the case, why?

Typically DQN works best with actions that have a continuous state space and a discrete, not too large action space. The complexity of the problem is more down to how expressive your model is. If you model is capable of learning a complex value manifold then DQN is likely able to obtain a good estimate of the values and thus provide a good policy. There are many updates to vanilla DQN to help learning too, such as a prioritised replay buffer.

Deep Q-learning is most suitable for problems with a large or infinite number of potential states. This is because the more states there are, the more information the Q-learning algorithm has to work with, and the more accurate its predictions about future states and rewards will be, with a large number of potential states the algorithm is able to learn from more data. In problems with fewer potential states, the algorithm may not have enough data to work with in order to make accurate predictions about future states and rewards.
Hence a large number of potential states is that the algorithm is able to generalize better. In problems with fewer potential states, the algorithm may not be able to learn the underlying structure of the problem as well, and thus may not be able to generalize as well to new states.

This is not quite right. If there are a small number of states then tabular Q-learning can be used in which it is guaranteed to converge to an optimal value function. Deep Q-learning is used, as you say, when state space is large or continuous, in which case it is impossible to store a tabular representation of the Q-values.

Yes, but in the Deep Q-learning case still the action is selected based on the max value of the Q-function, so either we get the optimal action always (under the assumption that the agent spends an infinity amount of time learning), or we do not know if the action chosen is the optimal action.
But yes, in the tabular case it is guaranteed that the agent will eventually converge to the optimal action-value function.

Sorry, I re-read again and see that you’re saying if the deep Q-learning had a small amount of data then it wouldn’t learn. I guess I would edit your answer to highlight you’re talking about deep Q-learning here (currently it is just Q-learning in bold).


text:
Do genetic algorithms "learn"?
I am currently working my way into Genetic Algorithms (GA). I think I have understood the basic principles. I wonder if the time a GA takes to go through the iterations to determine the fittest individual is called learning time ?

It really depends on what the GA is being used for. The prototypical use case is function optimization. Suppose you have a Traveling Salesperson Problem to solve. You have N cities, and you need to find the shortest route that visits each city once. You can attack that problem with a GA, and it will run for some period of time trying to find successively better and better solutions until whatever stopping criteria is reached. At that point, you have your answer. There's no remaining computation that needs to be done that would equate to something like "running time" versus "training time". As well, it's slightly odd to describe this is as "training" since there's no generalization available. You haven't trained a model that can solve any other TSP instance using what was learned in solving the first one. You can run the same code on a new problem and evolve a solution for that, but that's closer to what we'd consider a whole new training pass than just executing a previously trained model. In short, optimization just isn't really like ML problems where it makes sense to have training and running times. You just have the computation time needed for the search algorithm to find a solution.
However, many ML models require some sort of optimization as part of learning. Neural nets require fitting a set of weights to minimize a loss function. Support Vector Machines involve finding the optimal solution to a quadratic programming problem. We often have special-purpose techniques to solve those problems, like backpropagation for NNs, but you could also use a GA to solve the optimization problem, and then the GA time is equal to the training time for whatever that model was.

I believe genetic algorithms DO NOT learn, because they're a search and optimization algorithms. They keep on filtering the better solutions in each iteration, but they can easily "forget" what had "found" earlier, if mutation or crossover happens.


text:
Surveys, Papers, Hand on Tutorials about training data generation for anomaly detection
I am searching for anything related to supervised, semi supervised or unsupervised anomaly detection w.r.t training data generation.
I am looking toward reading any work that tackles the issue how to generate a training/test dataset for anomaly detection.
If someone knows any work related to the subject above, please share it with me.
Regards

Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer.

Like how? I mean, I've already found one but only one paper that exactaly is about my question.


text:
How to speed up the learning process
I have built a network that performans pretty well on my data. The issue I have is that for a larger number of epochs at the start of the training process the val/train acc/loss are stagnating (for some train/val splits the val_losses increase for a certain number of epochs) and then dropping dramatically.
What can be the reason for this, and how can I make it drop faster?

You should add more info about the optimizer you're using, the hyper parameters (especially learning rate), if you're using also a scheduler and the loss you're optimizing.

I agree with @Edoardo Guerriero. More training details are needed to help diagnose this issue.  Some details about the data, task, and model would also be helpful.   Finally, even though we can surmise from your text which curves are validation curves and which are training curves, a legend should always be included with your plots when plotting multiple curves as a matter of good practice and professionalism.  This is an interesting problem.  Help us help you.

@EdoardoGuerriero Yeah I know. I was more searching for a general overview. I do not think it's realistic to see my problem "solved" here (because the causes may be very varied). Eventually, I was looking for a starting point where I can research by myself to find out the cause.A list with most common reasons for such behavior.

Upvoted this is a genuine question. Quite essential generic research topic too.


text:
agent based DNN with a loopback
I have a data problem with no direct reward mechanism,(test/train) good and fault solutions.
Though over a long time period good decisions might be made.
I've been searching for days now for an agent solver, and some example code.
As most neural networks go from input to output on static data sets (train/test).
Essentially one problem, with known results or classifications.
Since there is no direct good/bad reward.  I'm looking for some code example perhaps using multiple evolving agents over time to compete against eachother over previous runs, with a reward that's based on multiple cycli..  Though I'm not after numeric pattern estimators  (ea not RNN or LSTM's) ... what else to use?

as I don't have direct rewards straight backpropagation is not something I can use.

Though there do exist more ways to solve things.
For example:

simple Fuzzy logic   >>though easy thermostat alike rules don't exist for the data.
Q-Learning a form logic that dynamically builds a logic table.
It's mostly used in games, where there is a kinda binary logic (wall, enemy, ..)
And the output are a few choices.

however my data isn't in a common game environment (wall, enemy, .. ).

Genetic-based neural networks, start with randomized weights.
Train and mix and add noise, and move weights towards the best result for each generation.

comes close though I don't have a good/fault set, it's still a hard problem.

Agent-based Genetic neural network, I've seen some evo sims around the idea

The evo sims still have quick rewards, though they come close to it.
But my rewards happen randomly and rarely, not at a fixed interval.
But maybe i'm wrong and i've seen the wrong samples of it.
A good python example of this would still be great.

So far I'm a bit unlucky and I wonder, maybe a differeent NN design like :
another type of network exists, one that's a like a normal DNN, but can have X outputs re-used as inputs besides its data input. This sounds a lot alike LSTM but i'm thinking of a small neural net in which 1 output is for action, the other outputs get a delayed or averaged backpropagation after x cycles, also those redirection to input might have some delay on their feedback loop. (a genetic net still can solve such circulair networks).
(Oh and its not a straight LSTM i'm after here).

How are such networks called?
Is it possible in pytorch / kerras ?.
python examples ??

I was reminded of the neural cellular automata a bit here (might be way off of your problem), but you don't provide enough information (at least for me) to get ideas. Can you provide any kind of objective, what is the learning task here? You say you neither have a loss nor a reward. Essentially, what do you expect the model to do?

I do have rewards but they happen later, its about personal medical date.

not date ...data.

Sorry, I still don't understand what you are after.
So far what you have given us is that you are looking for a specific ANN architecture and that you found some already, but the important part is missing - what do you want the network to do? If you don't want to disclose it all, could you abstractly describe what the input and the output of the ANN are going to be?

Well, there needs to be found a medication 'moments'  (over a longer period) so that I stay within the limits, while also using minimal and best guess, the medication has directly an impact on the next measurements since they affect my body. inputs as heart rate, electric skin resistance, blood oxygen level and a few more.
The output would be 'now is the best moment to take medicine, but it might also not advice to take medicine (its not life treathning medication)

It will learn to discriminate certain patterns / time / trained on data i have, though its not patterns as like in LSTM;s more alike Qlearning with a neural net. or a Genetic algo.
Perhaps training without a good/fault set is possible as well afterall it needs to recognize tendencies in patterns (but again not predict them like a lstm or rnn ).

Okay I think get what you are after. You could record a lot of training data from your own measurements - this would allow you to train very simple (non-deep) models (you could start with an SVM or shallow MLPs). If you have the possibility to get more training data from more patients you can go a bit more sophisticated (like LSTMs/GRUs). However, I'd phrase it as a supervised learning problem, RL/Q-Learning is not feasible in your case (research simply isn't there yet).
So your main problem is training data - if you could get some from more patients this would help you greatly.

Feel free to follow up on that if you have any further questions

The training data isnt the problem (but LSTM's and RNN) also the modern GRU doesnt solve it, those are pattern 'repeaters', At some point I will test this on myself, let the algorythm decide, the best balancing moment. (like the blancing stick ( https://www.youtube.com/watch?v=P1LmYVvnEJM ). kinda like that .

@Peter Can you please edit this post to include the information that you shared in the comments directly there, so that people don't have to read the comments to actually understand your problem? Comments are temporary.

this can be closed i found the answer to qlearning using openai chatbot


text:
How to evaluate the performance of off-line & model-free reinforcement leaning?
I'm currently studying on off-line reinforcement learning (RL) and trying to utilize it for medical data. Because it seemed hard to develop well-performing environment model, I decided to adopt model-free RL algorithms. Then I faced somehow ridiculous issues which cannot be ignored. "How to evaluate this RL model?".
My ideal model will yield the (estimated) best action at the specific state, and there must be a paired real action at that state. We know the reward of the real action, but the reward of the (estimated) best action is totally unknown. All I can describe is the difference between the real actions and best actions. How can I know this RL model outperforms real actions? (according to reward)
Of course I can change to model-based RL algorithm and get predicted rewards from trained environment model, but I'm not sure this 'predicted rewards' are reliable (Anyway, it is also  predicted).
Is there any smart method for calculate estimated rewards in off-line RL model? (e.g. mean rewards of real actions -> 0.5, mean rewards of best actions -> 0.8)

Your title and question do not match, how to evaluate is not the same as how to get the best reward. We evaluate RL models currently using just the accumulated reward. It all depends on what you want to evaluate...

Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking.


text:
Feasibility of NNs in solving NP-hard problems
Can neural networks be used to solve NP-hard problems? I'm assuming they learn to approximate a polynomial function and hence cannot solve NP-hard unless P=NP.
In case they aren't, what are the general methods that are followed to approach these problems?

There are a lot of NP-hard problems that become P if you restrict them to some subset of inputs. And there are lot of NP-hard problems for which there exist polynomial approximate algorithms. Furthermore, you seem to have some confusion about what it is that is polynomial in a P problem and what it is that is not polynomial in a NP-hard problem.

Also, neural networks can learn to approximate a lot of different functions, not just polynomial functions. If neural networks could only approximate polynomial functions, we'd probably use neural networks a lot less and linear regression a lot more. Indeed, finding a best approximating polynomial can be done with a simple linear regression.


text:
How to find the impactness between the features?
I need some guidance, source, and articles where to look in my search. I am kinda lost about where to look and what to look for. So basically I want to find the synergistic effect or interconnections effect of one source on the other source. For Eg: the attribution of TV is let's say 10% of overall sales, but TV includes some interconnectedness for other media-driven sales within like (x%) of Digital driven TV sales, or (y%) on Print driven TV sales, including (z%) of TV base driven TV sales. We need to find the values of x, y, and z by studying the interconnectedness between the media variables. How to find those x, y, and z? I am looking at integrated gradients and Shapley values by which we can find the attribution of the model.
But is there any way to find how much a feature impacted to another feature maybe during finding attribution?

Have you thought of Principal Components Analysis (PCA)?

@OliverMason How PCA will help me solve this? Can you explain little?

PCA is used to identify the most important factors in combinations, so it might help you to identify those factors which are most relevant.


text:
What is the theory behind rejecting too good heuristics in search problems?
Currently I have found that there is an article in which a search problem is posed and to solve it a heuristic is proposed which, in essence, is the solution of the problem itself. I seem to remember that in search problems based on heuristics (e.g. A*) there was an impediment by which you should not pose too good heuristics, since you generated a kind of dependence on solving the problem before using the algorithm.
Does this phenomenon have any formal theory behind it, and does it have any specific name?

Which article are you referring to exactly? It would be great that you provide more context (e.g. quotes from articles where this is claimed).

@nbro It is a Google article called Data Valuation using Reinforcement Learning. They use the score of training data with the model trained on validation, which can be easily proved to be the target of the search problem itself. Anyway, do you know how can I get to some theorem/statement which explains this phenomenon?

It's not fully clear to me what you mean by "you generated a kind of dependence on solving the problem". Can you clarify that? In A*, there are heuristics $h(n)$ that overestimate or underestimate the weight/length of the path from a node $n$ to the goal. A heuristic that never overestimates (so it underestimates or estimates correctly) the path from $n$ to the goal is called admissible, and A* with admissible heuristics is guaranteed to find an optimal solution. I don't know if this is related to what you're asking or not.

@nbro By generating a dependency on solving the problem I mean that if your heuristic is directly the solution, it makes no sense to pose a search problem, and even if you pose it you would need to solve the problem to obtain the heuristic that would allow you to "solve" the problem with the search algorithm. I am indeed looking for a term like "admissible" as you comment, but just in the case where the heuristic is the perfect solution to the problem.

Creating an admissible heuristic doesn't require you to solve the problem (e.g. you can just have an non-informed heuristic, i.e. $h(n) = 0$, for all $n$ - it's admissible). However, creating a perfect heuristic, i.e. the heuristic that tells you the exact length from $n$ to the goal, probably does.


text:
FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models
I am trying to understand this paper and the github code given with it, specifically the CNF script: in the loss function (compute_bits_per_dim) the Normalizing Flow is run forward to retrieve the base point z:
z, delta_logp = model(x, zero)
However, in the paper it is stated that the Normalizing flow must run backwards, taking the observed point x as input and running from t1 to t0 in order to retrieve the base point z. Does anyone have an explanation

From your reference for equation (6): This approach is a continuous-time analog to the backpropgation algorithm (Rumelhart et al., 1986; Andersson, 2013) and can be combined with gradient-based optimization methods to fit the parameters θ.

Not equation (6) but equation (4). z0 is obtained by integrating backwards (pay attention to the integral bounds they are reversed t1 to t0). But the code and the Algorithm 1 seem to miss this


text:
How can validation accuracy be more than test accuracy?
I have been trying to implement DenseNet on small dataset using k-fold cross validation. Training accuracy is 94% ,validation accuracy is 73% whereas test accuracy is 90%.I have taken 10% of my total dataset as test set. I know some overfitting is present, but how can  validation accuracy be greater than test accuracy?

Can edit your post to include the total number of samples, the number of samples in your training and test sets, the value of k in k-fold, and finally the standard deviations of your training and validation accuracy?

Also, your title and question imply val > test but your numbers indicates val < test.  Please verify that your numbers are correct and clarify your question.


text:
Adjusting weight updates in a generative model
Suppose I am training a generative model G to produce vectors z in R^d, where d is fixed. The objective of G is to produce realistic vectors, which I am calling the "reality objective". Further suppose that G produces "plausible" and "implausible" vectors during training and I have a second model E, which can tell whether the vector is plausible or not. E is trained separately under what I call the "plausibility obective". In this context, plausibility is a weaker constraint than being realistic, i.e. everything that is realistic is plausible, but not everything that is plausible is realistic. Moreover, training E is far easier than training G.
How do I best combine G and E ? Can I let E output a score r which I then use to reweight the weight update for G toward more plausible outputs? The issue is that, sometimes, weight updates through backprop will lead G into regions of the space R^d which are implausible. This is easier to tell for E than for G.
I would like to prevent G from venturing into implausible space through the involvement of E. Is there a name to what I am trying to do and do you know of a feasible method?

How do you train G to produce realistic vectors? And then what is the difference between plausible and realistic? Do you have a mathematical definition for that? And Im sure you heard of GANs which combine a generator and discriminator module?

G is a GAN in my case and it is learning in a latent space z provided by a variational autoencoder (VAE). VAEs are known to suffer from the prior hole problem (some latent vectors are not reconstructed to valid outputs). I want to guide the GAN's optimisation s.t. it only considers latent vectors z which are valid. In other words, all my data are transformed into latent space, and learning as well as backprop for the GAN also occurs in latent space. I want to prevent gradient updates from venturing into regions of "implausible" vectors z of the VAE.

To answer your other question, realistic means the discriminator assigns the vectors a high score. Plausible means that any vector generated by G can be reconstructed from the latent space z to a valid output. Unfortunately, I do not have a mathematical definition for plausiblility. In this paper (https://arxiv.org/abs/2211.00321), they are tryingt to gauge the prior hole problem by counting the consistent units in the bottleneck of the VAE. However, they do not provide an explanation for this.

Let me know if I can improve my question or whether I should ask it differently.


text:
Which mathematical properties do PSNR and MSE hold?
We know the Structural Similarity Index (SSIM) holds the following properties:

Unique maximum:  S(x, y) = 1 if and only if x = y
Boundedness:  S(x, y) ≤ 1
Symmetry:  S(x, y) = S(y, x)

My question is: do we know simple some properties like these for Peak signal-to-noise ratio (PSNR)? And for MSE? Do some of the above properties hold for PSNR and MSE? For instance, I have never obtained a PSNR value higher than 80 and, in microscopy super-resolution, never higher than 30.
PS: Do you know any other interesting property of any metric?

It would be great if you split this post into 2, one for each metric, so that people can focus on one metric at time. Can you please do that?

Of course! Do you mean one for properties of PSNR and another one for properties of MSE? I put them together because PSNR is highly determined by MSE. Thanks!


text:
Does the number of epochs measure a correlation?
i have built a two-layers neural network (1000 => 1000) to predict a dynamical system driven by two real-world parameters.
When using the first parameter as input to the first layer, training the model takes 10 epochs, while the second parameters takes 1000 epochs. Does this mean that the first parameters is highly correlated with the dynamical system?

This could simply mean either of your input parameters for the same dynamical system contains much error or noise There's nothing unusual to take 1000 epochs for a two layer NN with 1000 units in each layer.

Im not quite understanding the specific question you are asking. Could you please provide a more detailed description of your network, your training procedures and what you are asking?

@RobinvanHoorn  It is just about the model's performance. I have an irregular time series(real-data) for which i am looking for a  correlation with the two environmental parameters.


text:
Doesn't every single machine learning classifier use conditional probability/Bayes in its underlying assumptions?
I'm reading about how Conditional Probability/ Bayes Theorem is used in Naive Bayes in Intro to Statistical Learning, but it seems like it isn't that "groundbreaking" as it is described?
If I'm not mistaken doesn't every single ML classifier use conditional probability/Bayes in its underlying assumptions, not just Naive Bayes? We are always trying to find the most likely class/label, given a set of features. And we can only deduce that using Bayes rule since we are (usually) solving for P(class|features) with P(features|class)?

Conditional probability and Bayes rule are related but they are not the same thing, you can predict conditional probabilities without using Bayes rule.
So no, not all machine learning classifiers use Bayes rule, standard neural networks do not use Bayes rule at all, SVMs and linear classifiers neither.
A better counterexample is Bayesian Neural Networks, which have a probability distribution over the weights, and Bayes rule is used during learning and inference, these are not the same as standard neural networks.
As reference for this statement, I leave the following quote from Section 3.1 of the the paper Uncertainty Quantification for Deep Neural Networks: An Empirical Comparison and Usage Guidelines:

BNNs are neural networks with probabilistic weights, instead of scalar
weights as in PPNN, and are represented as probability density
functions. To train a BNN, first, a prior distribution p(θ) over
weights θ has to be defined. Then, given some data D, the posterior
distribution p(θ|D), i.e., the trained BNN is inferred using Bayes
rule:

In practice, BNNs (that I most familiar with) do not directly use Bayes rules to learn the posterior. They use variational inference (i.e. formulate the inference problem as an optimization problem). So, it's not correct to say that the Bayes rule is used during inference and training of (all) BNNs.

@nbro What you are saying is basically, there is one kind of BNN that uses VI, which means all BNNs use VI, which is incorrect. Even with VI you use the predictive posterior distribution to make prediction which implicitly uses Bayes rule.

I'm not saying there's one kind of BNNs and that all BNNs use VI. I said "BNNs (that I most familiar with)". Maybe the parentheses were not necessary. In other words, I am saying that some BNNs do not use Bayes rule. Your last paragraph seems to suggest that all BNNs use the Bayes rule, which is incorrect. With VI, you do not really use any Bayes rule. You're solving an optimization problem, which is equivalent (up to a constant) to solving the inference problem using the Bayes rule by finding the integrals (with Monte Carlo methods or in closed-form)

@nbro No, that is not correct, even with VI there are priors and posteriors, what you learn is the transformation from one to another, its an approximation, but still it is Bayesian.

Yes, there are priors and posteriors, but that doesn't make it a Bayes rule. You do not apply the Bayes rule. It's Bayesian, yes, because you have priors and posterios, but you do not apply the Bayes rule $p(y \mid x) = \frac{p(x \mid y) p(y)}{ p(x)}$.

@nbro Its in the formulation, you might not see it, but its still there, they are still Bayesian NNs. Maybe ask yourself, how to obtain the posterior given the prior and the model, without any use of Bayes rule.

Note: I'm not saying they are not Bayesian. They are! We put priors on the weights and find the posteriors. That's what usually people refer to as "Bayesian". So, yes, I agree with you. However, we do not (directly) apply the Bayes rule to find the posterior.

@nbro Priors, evidence, and posteriors, related with Bayes rule, is what actually means to be Bayesian.

@nbro What you are implying is that you can be Bayesian without using Bayes rule and I do not think that makes sense.

If you think that doesn't make sense, then you must think that variational BNNs are not Bayesian, because they do not directly multiply the likelihood times the prior divided by the evidence to get the posterior (Bayes rule). Instead, they solve an optimization problem that is equivalent (up to a constant) to minimizing the KL between a variational distribution and the posterior. In practice, when you maximize the ELBO, you max the evidence of your data. I don't see how maximizing the ELBO is equal to the Bayes rule and it's not. The Bayes rule does not even have a variational distribution.

@nbro Consider from where the ELBO comes from, you are just looking at the final  derivation and not starting from the beginning, from Bayes rule.

The ELBO comes from an approximation of the problem: KL divergence between the variational distribution and the posterior, which you cannot solve, because you don't know the posterior, which is what you want to find in the first place. That's why people invented the ELBO. The original optimization problem is just a distance between a variational distribution (e.g. Gaussians) and the unknown posterior. How is this equal to the Bayes rule?

I think the most one may say is that, because we're trying to find a posterior, given some prior and likelihood, we're trying to do something that is somehow equivalent to the Bayes rule.

@nbro No, again incorrect, you seem to forget that posterior, prior, likelihood, and evidence, are all terms in Bayes rule. And you brought up VI, my answer does not mention VI, there is also the theoretical concept of a Bayesian Neural Network which by definition uses Bayes rule to propagate inputs to outputs.

What is incorrect? I did my master's thesis on Bayesian neural networks, I did extensive review of the literature (not just VI), I derived many equations (including the ELBO), and I am very familiar with VAEs, as can be confirmed by the fact that I am the top answerer on this site on the topic. Moreover, I didn't say that all BNNs don't use the Bayes rule. In fact, I said that some BNNs (read above) don't use the Bayes rule (directly), which is true. You can go on forever defending your thesis, but ok, at this point, this conversation is useless.

Yes, it is useless, I teach this stuff, I write papers, its my main research topic at the post-PhD level. And you only fixated on the ELBO, without mentioning the predictive posterior distribution, which is literally Bayes rule in integral form, and used to make predictions with BNNs.

Yes, I know, I read your profile. You're teaching it incorrectly in my view and you don't fully understand everything. So, to conclude: 1) The Bayes rule is: $p(y \mid x) = \frac{p(x \mid y) p(y)}{p(x)}$, 2) If you know it, variational BNNs usually optimize the ELBO, so they do not directly apply the Bayes rule: optimizing the ELBO is not the same thing as applying the Bayes rule. 3) Variational BNNs make use of a variational distribution, which is not even a notion in Bayes rule. 4) I am not saying that all BNNs are variational, so I am not saying that all BNNs don't use the Bayes rule.

I am not saying your answer is really incorrect, but just a bit misleading. 6) I agree that one can consider variational BNNs Bayesian because you learn a posterior from a prior and likelihood. However, variational BNNs don't use the Bayes rule directly (I'm just saying what is completely obvious from the definition).

You keep on saying that I am incorrect, while you don't fully explain why. You keep saying only because there are priors, likelihoods and posteriors that automatically means we're using the Bayes rule, but to use the Bayes rule, you need to multiply the likelihood by the prior (divided by the evidence). How do you explain the existence of a variational distribution? Where is the variational distribution in the Bayes rule? Nowhere.

Probability is one way to solve classification problems. Still, there are other ways like clustering and K nearest neighbor approach where we tend to analyze the position of the current data point and its neighboring points to classify it. Also, in the decision tree classifier, information gain is the core concept used to classify.


text:
Do Neural Networks tend to have Zero Mean Errors in each Output?
My NN (a few linear layers with ReLUs + batch normalization, no activation in the last layer) learns to approximate vector-valued labels $y_z$ from data $z\sim\rho_z$ in a supervised way, i.e. net$(z)=y'_z\approx y_z$. I was wondering if there exists any literature that investigates the distribution of the error in each component of the output, i.e. the distribution of $(y'_z)_i-(y_z)_i$. Ideally, I want this to have zero mean, i.e.
$$
\mathbb{E}_{z\sim\rho_z}[(y'_z)_i-(y_z)_i]=0.
$$
Do any such results exist, or probably also results that suggest this is not typically the case?

The formula you show is exactly the loss function that you optimize in training, so ideally this should be zero mean in any case. It sounds like you might be interested in the field of optimization theory. There are results but not much that you can easily apply in practice - that's why it's theory^^ Besides that I don't really understand what you mean by 'distribution of the error in each component of the output'. What are 'components' here? If you mean dimensions, then you won't find a concrete answer, as this is task-specific and you cannot make universal assumptions about that.

Yes, I'm talking about dimensions (i.e. first entry of the vector, second, etc.). And yes, ideally, it should not just be zero mean, but actually zero :D but as there will always be some error, I was wondering if there are results on how it usually behaves. I've checked the literature, but only found results on the absolute error, not on it's expecatition w.r.t. the data distribution...


text:
Are there any books that teach text-to-image generation?
I read some of the research papers about text-to-image generation using Imagen, DALL-E 2, etc. but they are heavily scientific and I don't understand a lot of their concepts, so I was wondering are there any books that teach text-to-image generation for beginner ?

As far as I know, no books have been released specifically on Diffusion models. Newer versions of books on generative models are planned for release in 2023, which probably will include diffusion models. One of the reasons is that the research on these models is still very new. If you want to get into these models, I suggest you start with some blog posts on diffusion models. These will most likely be quite doable and give you a basic overview.

Also suggest starting with some colab scripts - there are a variety of them, and they allow you to start with a working version and pick it apart to understand how the top level components fit together


text:
Is there any different evaluation metrics(Performance Metrics) for Deep learning ,Machine, learning and NLP?
I'm a little confused about machine learning. I know we can use accuracy, and precision-recall when it comes to a classification problem, and when it comes to regression problems, we usually go with the MSE, RSME etc. For unsupervised learning, we go with a silhouette score ( For machine learning problems) is it the same for Deep learning and NLP can you guys please elaborate . Thank-you

What do you mean by "is it the same"? As long as the variables have the same nature (e.g. both are categorial in DL/NLP/whatever), the loss function/metric has the same meaning. This does not mean that the same metric/loss is equally suitable for all problems where the variables have the same nature. In fact, in some cases, e.g., the accuracy is not a "good" metric, but the precision or recall are.

Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking.


text:
How to tokenize compound sentences based on the conjunctions?
I am trying to tokenize sentences of a document for aspect-based sentiment analysis. There are some sentences that consist of more than one topic. For example,

The touch screen is good but the battery is weak

or

Their smartphones are great and their TVs are perfect.

I want to tokenize sentences based on these "conjunctions". To be more specific, for example, in the first case, I want to get these tokens:

[The touch screen is good, the battery is weak]

Existing tokenizers split documents based on punctuation. Even if I spit based on words such as but, and, and so on, it is possible to get splits that are words. For example

the phone is handy and cool

will be

[the phone is handy, cool].

but instead it should remain the same, that is

[the phone is handy and cool].

Is there any pre-trained model for this task? Are there any other solutions?

Can you clarify what you mean by "I want to tokenize sentences based on these conjunctions". Can you provide an example of how you would like the given example sentences to be tokenized? Can you also explain why existing solutions don't meet your requirements? Please, edit your post to clarify all of this.

Thank you so much. I added more explanation. I really appreciate it if you help me.


text:
How to relate the definition for entailment, with soundness and completeness?
Is it fair enough to say for a language model, φ, which makes certain variable A true, and if φ also makes another variable B true, then we can conclude:

A ⊨ B

And for a certain inference calculus c, from which B can derive A,

A ⊢ cB

The logic is sound if A ⊢ cB is true, which also makes A ⊨ B true.
The logic is complete if A ⊨ B is true, which also makes A ⊢ cB true.

Can you please put your question in the title? A question is something like this "What is the color of the sky?"

Changes made to the title. @nbro

Can you please define those symbols ⊨ and ⊢? How are they related to "soundness", "entailment" and "completeness"? Given that you tagged the post with [tag:norvig-russell] I suppose you're reading their book. Which section and page of that book?

⊨ refers to entailment, ⊢ refers to inference; please refer to Pg: 241- 243 for the proper illustration of these symbols and definitions( I replaced α, β with A, B).@nbro


text:
Which of these 3 mutation rates is the best in terms of performance?
I am need some comments since I am conducting experiments with 3 different mutation rates and hesitate to choose the best one.
I am working on a minimization problem where the optimal value is 1.039. Using 3 different mutation rates, I am able to to get optimal value. However, I am not sure which curve (shown in Fig.) is the best representation of a good mutation operator.
Output:

Which curve is the best representation of a mutation operator? I am in the process of testing other hyperparameters (number of generations, sample size) to see results.

Wouldn't it be better to compare the fitness with the number of iterations/generations (rather than time in seconds)? The time it takes may depend also on your CPU workload, etc.

@nbro I already did a comparison based on the fitness with the number of iterations. Besides, I am comparing multiple sample sizes to see the execution time it takes to process candidates solution.

How do the graphs looks when you run seven times with the same mutation rate?


text:
Does fixing an action in the state transition function of an MDP yield a transition matrix?
I stumbled across this seemingly elementary question while studying the theory of Markov Decision Processes. Suppose $\mathcal{M} = (\mathcal{S}, \mathcal{A}, \mathcal{P}, \mathcal{R})$ is an MDP and fix an action $a \in \mathcal{A}$. Does $(i, j) \mapsto \mathcal{P}(i, a, j)$ define a transition matrix? If that is so, how to accommodate the possibility that only a strict subset of $\mathcal{A}$ may be available from a given state, i.e. what is $\mathcal{P}(i, a, j)$ for $a \notin A(i)$?

the question can be clearer. For example what is (i,j)?

@user58136 these are two states, I believe this is standard notation


text:
Are autoencoders computationally cheaper than MLPs with the same number of neurons?
Are autoencoders computationally cheaper than other neural networks such as MLP with the same number of neurons?
I have read in some papers that autoencoders train the network faster, and I could check it in python, the training part is faster than a MLP. However, all these papers mention it briefly. However, I could not find an accurate explanation for this. Do you know why this happened?

Could you link the papers that you are talking about? I do not have enough information/context to understand the question at the moment

Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking.

The papers that I have read are https://www.sciencedirect.com/science/article/pii/S002002552100462X (page 13) and https://www.sciencedirect.com/science/article/pii/S0020025517309659 (page 2)


text:
Seq2seq with attention model only predicts index 0 in a text generation task
I am trying to build a model which summarizes some brief input text (X) and generates a headline (Y).
The problem is that the model always outputs 0 as the vocabulary index, which I think that might be due to sentence padding, I have tried to use mask_zero parameter in the embedding layer but results are the same.
I also have considered that the problem might be that I have very little data to train the network but I supposed it would fail in some manner but outputing random words, not always 0.
The network architecture I am trying to use is a seq2seq Bi-LSTM with attention mechanism and I am using Glove 50D embeddings as input layer.
Basically the model is based in three code functions that comprehend the previously explained model.
A good visual representation of the model is the following from Andrew Ng just substituting one hot vector inputs with embeddings in the input layer:

Model layers definition at global scope to share weights

# Attention layers globally defined
repeator = tf.keras.layers.RepeatVector(Tx)
concatenator = tf.keras.layers.Concatenate(axis=-1)
dense_1 = tf.keras.layers.Dense(10, activation='tanh')
dense_2 = tf.keras.layers.Dense(1, activation='relu')
activator = tf.keras.layers.Activation('softmax', name="attention_weights")
dotor = tf.keras.layers.Dot(axes=1)

# Post-attention LSTM globally defined to share weights
post_attention_LSTM = tf.keras.layers.LSTM(units=n_s, return_state=True)
output_layer = tf.keras.layers.Dense(vocab_size, activation='softmax')

Full model structure:

def seq2seq_clickbait(Tx, Ty, n_a, n_s, vocab_size, emb_size):

    # Define Model Inputs
    sentence_indices = tf.keras.layers.Input(shape=(Tx,), dtype='int32')

    emb_matrix = build_embeddings_matrix(word_to_vec, vocab_size, word_to_idx, emb_size)

    pretrained_embedding = pretrained_embedding_layer(emb_matrix, vocab_size, emb_size)

    embeddings = pretrained_embedding(sentence_indices)

    s0 = tf.keras.layers.Input(shape=(n_s,), name='s0')
    c0 = tf.keras.layers.Input(shape=(n_s,), name='c0')

    s = s0
    c = c0

    outputs = []

    a = tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(units=n_a, return_sequences=True))(embeddings)

    for t in range(Ty):

        # Once we have a, we can get the context to pass it to the post attention lstm time step
        context = one_step_attention(a, s)

        # Get current states from post-attention lstm
        s, _, c = post_attention_LSTM(inputs=context, initial_state=[s, c])

        # Apply dense (softmax vocab_size)
        out = output_layer(inputs=s)
        outputs.append(out)

    model = tf.keras.models.Model(inputs=[sentence_indices, s0, c0], outputs=outputs)

    return model

A good visual representation of what does one_step_attention(a, s)

context

def one_step_attention(a, s_prev):

    # Repeat the s_prev vector Tx times to expand the vector
    s_prev = repeator(s_prev)

    # Concatenate a and s_prev
    concat = concatenator([a, s_prev])

    # Forward prop to a dense layer
    e = dense_1(concat)

    energies = dense_2(e)

    # Apply softmax to energies to get the alphas
    alphas = activator(energies)

    # Get context from the alphas
    context = dotor([alphas, a])

    return context

I present the outputs to the model as a one hot vector so if the ouput sentence is length (Ty) 20 the output tensor is (20, m, vocab_size). My guess is that many of the labeled sentences are not length 20 and therefore they are padded, so the network is learning to output 0's since it is a pretty good way of improving accuracy / decrease loss in short senteces widely padded.
If half of the output steps of the network are filled with a [1, 0, 0,..., 0] one hot vector which represents the 0 index (pad) it dense layer will learn to output probabilities distributions like [0.8, 0.002, 0.00003, ....] selecting the 0 index.
Any ideas on how to solve this? Thank you in advance
EDIT
Some additional info about the model.
loss function is categorical_crossentropy, I have not added extra tokens to the vocabulary given by GloVe embeddings, I removed 'PAD' as a vocabulary token since TF embeddings docs says so when mask_zero=True. m is the number of examples, Tx and Ty is the sentence max length of X and Y, n_s is the number of post-attention LSTM units and n_a is Bi-LSTM units. I think indeed is a masking problem but mask_zero=True is not solving it. It only works with return_sequences=True and post att LSTM has return_squences=False because it loops through steps.

What loss function are you using, can you post it? What kind of extra tokens are you using? PAD, EOS, UNK, ... ? Can you name your variables (m, Tx, Ty, n_a, n_s)?
It sounds most likely like a masking issue, because your model thinks that 0 (i.e. PAD) is the most likely token. Remember that you have to mask twice: in the model, and in the loss. Padding is only necessary for under-the-hood time-wise optimisation when passing mini-batches. The neural network should never actually "compute" on the padding tokens, unless it is important to predict them.

loss function is categorical_crossentropy, I have not added extra tokens to the vocabulary given by GloVe embeddings, I removed 'PAD' as a vocabulary token since TF embeddings docs says so when mask_zero=True. m is the number of examples, Tx and Ty is the sentence max length of X and Y, n_s is the number of post-attention LSTM units and n_a is Bi-LSTM units. I think indeed is a masking problem but mask_zero=True is not solving it. It only works with return_sequences=True and post att LSTM has return_squences=False because it loops through steps. Thanks for your answer!

So what is PAD then encoded as if it is not in the vocabulary? I don't get it. You also should have a GO token and an EOS token for the decoder of the seq2seq. Sequences go from GO + sentence -> sentence + EOS. You need GO to tell the model the sentence starts (see here for a visual: https://doi.org/10.1145/3419106).
And the way you talk about masking makes me think you rely too much on keras taking care of it for you. To my knowledge, you only need to mask the RNN computation and attention.

The fact that the model predicts the padding token as the most likely one means that there is also a mistake in your loss. With crossentropy, you will do sth like loss(logits, targets), but you should only do that for the non-padding tokens in your target. Hence, when you accumulate the loss, make sure to multiply it by zero once you reach the padding tokens in your mini-batch. Also make sure you understand what this means for averaging the loss over the sequence length.
If your problem persists, I advise you to post a link to your whole code.

Yes, it looks like the loss function is taking into account Y label`s padding since last steps of the dense layer's output (padding) get high accuracy / low loss but first steps ar low accuracy / high loss and therefore the network learns to ouput 0 as the most probable index for every step. I will look deeper into masking the Y label's padding to the loss function and probably also acquire more data. Thank you for your advice!

You're welcome! I would postpone looking for more data, however. Do you have a script to overfit? That's usually the first sanity test, to see if you can just learn 1 or 2 sequences. It's fast, and you can spot errors much easier.


text:
Document Processing using AI
Deal all
I want to Build a Document Processing AI Model
1-that Identify Document Elements
Like table, Text , List , Heading ,  etc.
2-Sort this element in correct way if I wan to dump this data as XML
so I attached two images the first one the original image
the second one show me the expectation output recognize Document Elements and the sort Index for each item
you can see the Sort not follow a regular sort
I already build a model using some Tensor flow and train my Model with a lot of documents Classes elements and model recognize me the Boundary box + Class Name as Shown in the third image ,
so my missing part how to predict the correctly order for this elements
I've a good dataset that I can extract the sort data to use while training the model
but I don't know which thing will match my needs
any help

I still don't understand why you want to rank the elements that are detected and how.

this very important to help blind people to read document in the correct orders using some application that read tag and it's attribute and convert this to sound
any web page contain some HTML tags that the blind people can access it through some accessibility roles and reader use the HTML Tags to interpreter the page
H1 read with high volume , H2 read will a smaller , if there is image and there is alt text so it read the alt text , tables , charts ,..etc.
so it should listen to the content in the correct order


text:
Why do training and fixing a reservoir yield very similar results (in an echo state network)?
Disclaimer: I asked this question 2 days ago in Cross Validated, but it has been left unanswered.

I am trying to better understand how echo state networks work. To see, how fixing the weights of the reservoir of an echo state network impacts the prediction quality of an echo state network, I have conducted a very simple experiment using ESN layer of the tensorflow-implemented Keras having the two models below:
model_untrainable = tf.keras.models.Sequential([
    tfa.layers.ESN(units= 1000, spectral_radius=0.99, trainable=False),
    tf.keras.layers.Dense(1, kernel_initializer="lecun_normal")
])

model_trainable = tf.keras.models.Sequential([
    tfa.layers.ESN(units= 1000, spectral_radius=0.99),
    tf.keras.layers.Dense(1, kernel_initializer="lecun_normal")
])

So, as one sees, the only difference is that in the model_trainable, the reservoir's weights are allowed to be updated during training, but in the model_untrainable, they are just fixed (by setting trainable=False). My hypothesis was that the model_trainable should be way better because its settable parameters are more than those of the model_untrainable. So, using identical setting for optimizers, loss functions, and regularization and taking monthly sunspots dataset into account, here are the predictivity results of the cited models.
model_trainable:

model_untrainable:

It seems that the untrainable_model is almost as good as the model_trainable. Why is that the case? In other words, shouldn't the trainable_model significantly better than the untrainable_model?

trainable = False is a hardcoded setting for each ESN layer in esn_cell.py. So, regardless of what one sets in any usage of ESN, its corresponding reservoir cannot be trained.


text:
Why is it called a Seq2Seq model if the output is just a number?
Why is it called a Seq2Seq model if the output is just a number?
For example, if you are trying to predict a movie's recommendation, and you are inputting a sequence of users and their ratings, shouldn't it be a Seq2Number model since you're only predicting 1 rating at a time?

The premise of your question is wrong. A model that goes from a sequence to a single prediction is simply NOT called a sequence to sequence to model.
The model type you are describing is called a sequence encoder.
An example would be sentiment prediction, where we input a sequence of text and output a number.
Similarly, a model that goes from a fixed size value to a sequence is called a sequence decoder. An example would be image captioning.
If a model inputs one sequence and outputs another, as in machine translation, it is called a sequence to sequence model, and consists of both an encoder and a decoder.
If you saw different terminology, it was either mislabeled or misunderstood.

Thank you for the clarification! I definitely misunderstood bc I couldn't find definitions as clear as this one.


text:
Should we forget last weight update in neural network?
When training a neural network, the general process could be something like this:
While error < min_error

Forward pass
Compute error and cost funcion
Back propagation
Update weights

But when we get out of the loop because the error is small enaugh, if we save the neural network as it is, we are saving something that we have not tested yet. Wouldn't it be better to keep the previous weight values, as those are the ones for which we know the error?

if you are doing the loop this way, then quite possibly yes

If you're not going to save the last set of weights, then there's no need to compute them in the first place.