text:
Can RNNs be used to classify these time series into two classes?
My task is to classify into two classes the time series like these shown in the figure.

The figure shows one class on the left sub-figure and second one on the right. The series are shown in pairs for more clarity, but each series on the left (and right) belongs to one of the respective classes. The scale of the right panel is reduced to show all series, but these series are of the same amplitude as the left ones.
Is it possible to apply RNN (or other methods) to classify the series of this kind into two classes?
I have never used neural networks, but I am just looking for an adequate method for this problem.

Yes this is definitely possible, considering you have enough curve with labels.
You should maybe go through a dummy example of RNN or LSTM (in Keras or Pytorch) to familiarize yourself with this.

@Romain Thanks a lot. I will familiarize with these method. Could you estimate a number of cases that I should have for teaching?

Impossible to say as this is problem dependent, but from my personal experience I would say in the 100s or 1000s. Maybe someone else has a better idea than me on that part.

This quantity seems to be available in my case (but not readily :). Thank you very much for you advice.


text:
Are hill climbing variations always optimal and complete?
Are hill climbing variations (like steepest ascent hill climbing, stochastic hill climbing, random restart hill climbing, local beam search) always optimal and complete?

No, they are prone to get stuck in local maxima, unless the whole search space is investigated.
A simple algorithm will only ever move upwards; if you imagine you're in a mountain range, this will not get you very far, as you will need to go down before going up higher. You can see that going down a bit will have a net benefit, but the search algorithm will not be able to see that.
Random restart (and similar variations) allow you to do that, up to a point. Imagine you have ten people that you parachute over your mountain range, but they can only go upwards. Now you've got a better chance of finding a higher peak, but there's still no guarantee that any of them will reach the highest one.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
What are the math theorems regarding the Multilayer Perceptron?
I've come across a theorem "Convergence theorem
Simple Perceptron" for the first time, here-> https://zaguan.unizar.es/record/69205/files/TAZ-TFG-2018-148.pdf, page 27, (is in Spanish)
Are there others like this one but for the Multilayer Perceptron?
Could someone please point me out to them?
Thank you in advance.

Hi. Maybe you should at least provide the link to the source where you found this theorem "Convergence theorem Simple Perceptron". Moreover, in the title, you're mentioning MLPs, which are not exactly the same thing as "simple perceptrons", so you may want to clarify what you're really interested in.

Hi moderator @nbro. Ok, the document is in Spanish, doesn't matter? In the later, MLP, I'll edit.

To have more context, a document in Spanish is probably better than no document at all. In any case, I think this question is somehow a duplicate of this one. Let me know if that's the case.

@nbro I don't think it's a duplicate, it's related. In any case, I think from there one can conclude a theorem about MLP.

I've read somewhere (maybe an ML course from Ng on Coursera) that the main reason for convergence is stochastic gradient descent.

@ArayKarjauv "stochastic gradient descent" .but that's not a theorem, or well, at least it's not how they named it on the book Neural Network Design

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
Why object detection algorithms are poor in optical character recognition?
OCR is still a very hard problem. We don't have universal powerful solutions. We use the CTC loss function
An Intuitive Explanation of Connectionist Temporal Classification | Towards Data Science
Sequence Modeling
With CTC | Distill
which is very popular, but it's still not enough.
The simple solution would be to use object detection algorithms for recognizing every single character and combine them to form words and sentences.
We already have really powerful object detection algorithms like Faster-RCNN, YOLO, SSD. They can detect even very complicated objects that are not fully visible.
But I read that these object detection algorithms are very poor if you use them for recognizing characters. It's very strange since these are very simple objects, just a few lines and circles. And mainly grayscale images. I know that we use object detection algorithms to detect the regions of text on big images. And then we recognize this text. Why can't we just use object detection algorithms (small versions of popular neural networks) for recognizing single characters?
Why we use CTC or other approaches (besides the fact that it would require much more labeling)?
Why not object detection?

Good question! Using Yolo to recognise characters would be a good experiment to try. It may be because of the density of characters on a page -- systems like Yolo are very good at detecting a small number e.g. 2,3 or 10, objects, but don't work so well when the number of objects  is the hundreds as you might have with OCR. A better approach might be to try face detection methods that work well with large crowds.


text:
Backpropagation - what does rate of change calculated from the partial derivatives actually relate to?
I understand conceptually how backpropagation works according to the chain rule, and I understand that partial derivatives calculate the rate of change of a function containing multiple variables with respect to one of those variables, the rest being fixed.
What I'm struggling with is what the value from these partial derivatives actually relates to. I found this https://activecalculus.org/multi/S-10-2-First-Order-Partial-Derivatives.html which gives some good examples. But with a NN I'm not sure what units the results of the derivatives relate to.
One of the examples on the website used z = f(x,y) z horizontal distance travelled of a projectile, x initial speed in feet per second, and y was the angle. So if taking the partial derivative with respect to x the results tell us how much the distance travelled changes with respect to the change in speed. So it might be that for every one foot per second increase of the initial speed, we get an increase of 8 feet horizontal travel if using a fixed value for y.
But when calculating the derivatives for backpropagation, does this mean that if we get an answer of (random value) 0.08, this means that for every change of 1 to the non-static variable we would get a change of 0.08 to our output? And what units (if any) do these values relate to?

that's the right idea, $\frac{\partial f}{\partial \theta_i}$ just represents how much a unit change in $\theta_i$ affects the output f  assuming all other variables remain constant and linearity, the chain rule just breaks this up by calculating how $\theta_i$ affects future nodes and then how those nodes affect the function. The units are just $\frac{\text{units of output}}{\text{units of parameter}}$

It's similar to the way iteratively reweighted least squares (IRLS) regression works, where a one-step regression is first run, and then the residuals (i.e. error) are regressed iteratively on the input features, and each time, the least squares coeffs represent the delta error, which are added together over iterations to arrive at a solution to the coeffs.

@quest-ions That makes sense, and this was kind of the conclusion I came to yesterday after thinking about it for a while. I think I was struggling with not knowing what the "units" were but thinking about them as just units of output/units of parameter makes sense.


text:
How to design training loop in RPN?
I have a short question. I understand the concept of RPN but one small details keeps me from implementing it. How should I design the training loop given that I have to use only a subset of anchor boxes (128 positive and 128 negative). In other words, if the ouput of reg layer is a map of every bounding box, how do I update only the bounding boxes that match my 128 positive/128 negative samples

Hello. Could you please put your specific question in the title? "region proposal network training" is not a question and it's very general/vague. Moreover, what do you mean by "reg layer"?

Ok,i will update the title. Reg layer is regressive layer for bounding box coordinates


text:
Is there a bias-variance equivalent in unsupervised learning?
In supervised learning, bias, variance are pretty easy to calculate with labeled data. I was wondering if there's something equivalent in unsupervised learning, or like a way to estimate such things?
If not, how do we calculate loss functions in unsupervised learning?

Yes, the concept applies but it is not really formalized.  Consider unsupervised learning as a form of density estimation or a type of statistical estimate of the density.
Variance: You will train on a finite sample of data selected from this probability distribution and get a model, but if you select a different random sample from this distribution you will get a slightly different unsupervised model.  This variation caused by the selection process of a particular data sample is the variance.
Bias: This is a little more fuzzy depending on the error metric used in the supervised learning.  An unsupervised learning algorithm has parameters that control the flexibility of the model to 'fit' the data.  For example,  k means clustering you control the number of clusters.  Simple example is k means clustering with k=1.  You could imagine a distribution where there are two 'clumps' of data far apart.  The mean would land in the middle where there is no data.  This model is biased to assuming a certain distribution.  For a higher k value, you can imagine other distributions with k+1 clumps that cause the cluster centers to fall in low density areas.  For a low value of parameters, you would also expect to get the same model, even for very different density distributions. This is also a form of bias. This unsupervised model is biased to better 'fit' certain distributions and also can not distinguish between certain distributions.
You can see that because unsupervised models usually don't have a goal directly specified by an error metric, the concept is not as formalized and more conceptual.


text:
Reinforcement learning for rearranging the mobile home screen icon layout: what inputs/states do I need to pass into the algorithm?
I have a problem where I need to rearrange a particular user's mobile home screen icon layout. Let's say that the social media app usage of a user is high compared to other app usage. So I need the reinforcement algorithm to process this information and send back the instructions to the android operating system as to how the icons needs to be arranged. To address this problem, I have chosen three algorithms:

Q-learning.
State-Action-Reward-State-Action.
Deep Deterministic Policy Gradient.

I have decided to first consider only Q-learning, so I am trying to understand the states, rewards, and the actions I need to pass in order to make this algorithm work.
The principles I have considered are:
– The environment is the mobile device operating system.
– Moving the apps up in the list depending on their usage can be the action where the app can be moved left, right, up or down.
– The reward can be a periodic reward, check if the user has rearranged an app which was given prominence in the list by the algorithm and receive a negative feedback if the user has rearranged the icon position or if it is in the same position receive a positive reward.
The initial challenge I am facing is to understand what inputs/states I need to pass into the algorithm and is there any reinforcement learning library I can use to mimic such an environment?
Are there any resources or papers I can use to solve this problem I am facing?

That's a lot of questions. We generally want a question to address a specific subject, rather than multiple subjects. Please edit your post accordingly.

Do I need to edit the question more ?, I actually have edited the question to ask 1 question

That seems ok. I have tried to clean it up further. What does "While the reward can be based" mean? The sentence seems incomplete.

Thank I completed it as well


text:
What is the difference between multi-head and normal output?
Let's say that I have a neural network with 2 heads. The first consists of X neurons. The second consists of Y neurons. I have these 2 heads because I want to predict 2 different variables. And I can see the loss for each head during training.
Now, let's say that I have only one head that consists of X+Y neurons. I can interpret the output because I know that the first X neurons describe some variable and the latter Y neurons describe the second variable.
I want to know if there is any difference between these 2 methods (maybe in performance or something). What are the pros and cons? Are there any advantages of one method over another for some particular tasks?

It depends on what your outputs are. For example, if both outputs are similar then you can use one output branch. However, what if the two outputs are different? With two output branches you can used two different loss functions. Now your model will optimize the two branches separately.
Imagine if you have a model that has to output a class label for the input and a real value describing something in the input. One is a classification task and the other is a regression task. And the two  will require different loss functions, so you would use two branches.


text:
Speech diarization for a conversation detector: A good idea or not?
I am trying to write a program in which an ai can detect whether a conversation is occurring or not. The ai does not need to transcribe words or have any meaning about the conversation, simply if one is occurring. A conversation can then simply be defined as having more than one speaker.
Anyways, while searching for past research on the subject, I came across the field of speech diarization, which is where an AI is trained to distinguish the numbers of speakers in a conversation. This seems perfect to me, however, while implementing I came across a few troubles. First of all, it wasn't good. I used this tutorial: https://medium.com/saarthi-ai/who-spoke-when-build-your-own-speaker-diarization-module-from-scratch-e7d725ee279 to write a simple program for this task, but I found it wasn't good at finding if there was a single or two speakers. Also the times where it was distinguishing speakers were all off.
It occurred to me that perhaps speech diarization may not be the best approach for this problem, so I decided to ask here about if this is the best solution, or if there are better ones out there. If it is the best solution, I would love some insight into why this wasn't working for me. Is the tutorial simply not good enough? I used 45 second - 1 minute long clips of just myself speaking or other people speaking with me and it did not work well at all like I said.

I don't think you've provided enough details here to form an answerable question.

Yes, some more details would be good. Also, in a conversation you often have people speaking at the same time, that might be an additional factor degrading the performance.


text:
Do the training and test datasets need to be equally preprocessed as one whole dataset?
I have developed, trained and tested an NLP model. It is persisted in a pickle file. The model contains the data preprocessing function that includes text cleaning and new features engineered with word2vec.
With the trained model, I want to make predictions on a new text. The new text data, after preprocessing, won't contain the same engineered features of the training dataset.
Therefore my question is, how can the trained model make predictions on the new dataset as it has different engineered features (different numbers of columns and different columns)?
Should I preprocess the new text data and the training dataset as one dataset?

Your question/problem is not entirely clear to me. You say that, after preprocessing the test data, you get the test dataset, which has different features than the features of the training dataset. So, why would preprocessing both the training and test datasets, as a whole, fix this issue? Are you currently using a different preprocessing procedure for the training and test datasets? If that's the case, why? If that's not the case, then why do you get different features for the training and test datasets?

@nbro I trained and tested my model and preprocessed train and test as a whole. So both having the same features. My issue rises when I want to make predictions on a new data set different from train and test. My question is, do I need to append the new data set to train and test, preprocess them as a whole dataset and then make predictions on the new dataset (without training the model again)?


text:
Regression for a discrete variable
I'm building a model (neural net) that would predict a quality score for images.
Ground truth is given by a 4-level discrete variable (0%, 33%, 67%, 100%), and I would like to build a model that would give something that looks like a continuous result over the 0-100% scale.
What should I pay attention to?
What I'm afraid of is that the model might stick to ground-truth levels and prefer them over any value in between.

This sounds like you'd be better of treating this as a classification problem. Maybe try having 2 outputs, one for "good" and one for "bad", then treat them as a probability (ie, softmax). This will ensure you are bounded in the 0-100% range, and would also likely mean (assuming you don't over-train) it will predict continuously

I'm afraid that softmax would push the predictions towards the edges (0 and 100) and the distribution will come out very non-uniform.

It doesn't do this if used properly, take a look at some image classifiers, you'll notice that with ambiguous images the network will often output low-confidence scores. If you train the network properly it shouldn't approach extremes. Ultimately, you'll find the best luck with experimenting though, so just start trying a bunch of things until it works


text:
CartPoleV0 model is not getting trained in even after 1500+ episodes using deep Q-learning
I am new to deep Q learning and trying to train the open AI cartpole_V0 game using deep Q learning. Here is my code:
import gym
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
import tensorflow as tf
from collections import deque
import numpy as np
import random
import matplotlib
matplotlib.use('tkagg')
import matplotlib.pyplot as plt

EPISODES = 5000
output_dir = "/home/ug2018/mst/18114017/ML/"
EPSILON = 1
REPLAY_MEMORY = deque(maxlen=800)
MIN_EPSILON = 0.01
DECAY_RATE = 0.995
MINIBATCH = 750
GAMMA = 0.99

env = gym.make('CartPole-v0')
state_size = env.observation_space.shape[0]
action_size = env.action_space.n

class DQNagent:

    def __init__(self):

        self.fit_model = self.create_model()

        self.predict_model = self.create_model()
        self.predict_model.set_weights(self.fit_model.get_weights())

        self.targets = []
        self.states = []

    def create_model(self):

        model = tf.keras.models.Sequential()
        model.add(tf.keras.layers.Dense(64, activation ="relu",input_dim = state_size))
        model.add(tf.keras.layers.Dense(128, activation ="relu"))
        model.add(tf.keras.layers.Dense(256, activation ="relu"))
        model.add(tf.keras.layers.Dense(128, activation ="relu"))
        model.add(tf.keras.layers.Dense(64, activation ="relu"))
        model.add(tf.keras.layers.Dense(32, activation ="relu"))
        model.add(tf.keras.layers.Dense(action_size, activation="linear"))
        model.compile(loss="mse", optimizer=tf.keras.optimizers.Adam(learning_rate=0.001), metrics=['accuracy'])
        return model

    def model_summary(self,model):
        return model.summary()

    def get_q(self, state):
        return self.predict_model.predict(state)

    def train(self,batch_size):
        minibatch = random.sample(REPLAY_MEMORY, batch_size)
        for state, reward, action, new_state, done in minibatch:
            if done :
                target = reward
            else:
                target = reward + (GAMMA * np.amax(self.get_q(new_state)[0]))
            target_f = self.get_q(state)
            target_f[0][action] = target

            self.states.append(state[0])
            self.targets.append(target_f[0])

        self.fit_weights(self.states,self.targets)

    def fit_weights(self, states, targets):
        self.fit_model.fit(np.array(states), np.array(targets), batch_size = MINIBATCH, epochs = 1 ,verbose=0)

    def predict_save(self, name):
        self.predict_model.save_weights(name)
    def fit_save(self, name):
        self.fit_model.save_weights(name)

agent = DQNagent()
print(agent.fit_model.summary())

x=[]
y=[]
z=[]

def update_graph(z,y):
    plt.xlabel("Episodes")
    plt.ylabel("Score")
    plt.plot(z,y)
    plt.pause(0.5)
plt.show()

for eps in range(EPISODES):
    env.reset()
    done = False
    state = env.reset()
    state = np.reshape(state, [1,state_size])
    time = 0
    exp=0
    elp=0
    while not done:

        if EPSILON >= np.random.rand():
            exp +=1
            action = random.randrange(action_size)
        else:
            elp +=1
            action = np.argmax(agent.get_q(state)[0])
        new_state, reward, done, _ = env.step(action)
        new_state = np.reshape(new_state,[1, state_size])
        if not done:
            reward = -10
        else:
            reward = 10
        REPLAY_MEMORY.append((state,reward,action,new_state,done))
        state = new_state
        time += 1
    x.append([eps,exp,elp,time,EPSILON])
    y.append(time)
    z.append(eps)
    update_graph(z,y)
    if (len(REPLAY_MEMORY)) >= MINIBATCH:
        agent.train(MINIBATCH)
        if EPSILON > MIN_EPSILON:
            EPSILON *= DECAY_RATE
    if eps % 50 == 0:
        agent.predict_save(output_dir + "predict_weights_" + '{:04d}'.format(eps) + ".hdf5")
        agent.fit_save(output_dir + "fit_weights_" + '{:04d}'.format(eps) + ".hdf5")
    with open("score_vs_eps.txt", "w") as output:
        output.write("Episodes"+"   "+"Exploration"+"   " + "Exploitation" + "  "+ "Score" + "  " + "Epsilon"+"\n")
        for eps,exp,elp,time,epsilon in x:
            output.write("      "+str(eps)+"        "+str(exp)+"        "+str(elp)+"        "+str(time)+"       "+"{:.4f}".format(epsilon) +"\n")

agent.predict_model.save('CartPole_predict_model')
agent.predict_model.save('CartPole_fit_model')

Code is running perfectly but the model is taking too many episodes to get trained and even after it scores 200, there is no continuity of it. Could please help me with how can I train the model in fewer episodes and maintain the continuity of the 200 scores?
Here are some of the steps listed:
Episodes Exploration Exploitation Score    Epsilon
  0        18        0        18       1.0000
  1        32        0        32       1.0000
  2        43        0        43       1.0000
  3        17        0        17       1.0000
  4        17        0        17       1.0000
  5        16        0        16       1.0000
  6        13        0        13       1.0000
  7        21        0        21       1.0000
  8        16        0        16       1.0000
  9        20        0        20       1.0000
  10        35       0        35       1.0000
  11        14       0        14       1.0000
  12        13       0        13       1.0000
  13        12       0        12       1.0000
  14        16       0        16       1.0000
  15        17       0        17       1.0000
  16        27       0        27       1.0000
  17        24       0        24       1.0000
  18        14       0        14       1.0000
  19        28       0        28       1.0000
  20        20       0        20       1.0000
  21        13       0        13       1.0000
  22        12       0        12       1.0000
  23        23       0        23       1.0000
  24        17       0        17       1.0000
  25        43       0        43       1.0000
  26        61       0        61       1.0000
  27        29       0        29       1.0000
  28        21       0        21       1.0000
  29        17       0        17       1.0000
  30        41       0        41       1.0000
  31        9        0        9        1.0000
  32        18       0        18       1.0000
  33        23       0        23       1.0000
  34        28       0        28       0.9950
  35        24       0        24       0.9900
  36        25       0        25       0.9851
  37        28       1        29       0.9801
  38        26       1        27       0.9752
  39        35       2        37       0.9704
  40        19       0        19       0.9655
  41        48       0        48       0.9607
  42        25       2        27       0.9559
  43        13       0        13       0.9511
  44        20       2        22       0.9464
  45        21       0        21       0.9416
  46        13       0        13       0.9369
  47        28       5        33       0.9322
  48        23       3        26       0.9276
  49        24       1        25       0.9229
  50        20       2        22       0.9183
  51        13       0        13       0.9137
  52        19       1        20       0.9092
  53        13       1        14       0.9046
  54        18       1        19       0.9001
  55        12       1        13       0.8956
  56        29       7        36       0.8911
  57        28       2        30       0.8867
  58        16       1        17       0.8822
  59        28       6        34       0.8778
  60        13       3        16       0.8734
  61        15       4        19       0.8691
  62        19       2        21       0.8647
  63        27       4        31       0.8604
  64        19       4        23       0.8561
  65        16       1        17       0.8518
  66        60       9        69       0.8475
  67        24       1        25       0.8433
  68        21       7        28       0.8391
  69        14       0        14       0.8349
  70        31       4        35       0.8307
  71        64       13       77       0.8266
  72        58       13       71       0.8224
  73        32       9        41       0.8183
  74        15       1        16       0.8142
  75        23       6        29       0.8102
  76        27       5        32       0.8061
  77        66       6        82       0.8021
  78        30       6        36       0.7981
  79        74       22       96       0.7941
  80        14       1        15       0.7901
  81        18       1        19       0.7862
  82        28       7        35       0.7822
  83        28       4        32       0.7783
  84        12       2        14       0.7744
  85        10       2        12       0.7705
  86        21       4        25       0.7667
  87        13       6        19       0.7629
  88        19       6        25       0.7590
  89        16       4        20       0.7553
  90        46       16       62       0.7515
  91        12       1        13       0.7477
  92        30       15       45       0.7440
  93        38       9        47       0.7403
  94        14       7        21       0.7366
  95        10       1        11       0.7329
  96        16       8        24       0.7292
  97        10       2        12       0.7256
  98        20       5        25       0.7219
  99        19       7        26       0.7183
  100       31       9        40       0.7147
  .
  .
  .
  1522        0        104       104      0.0100
  1523        0        35        35       0.0100
  1524        0        27        27       0.0100
  1525        0        52        52       0.0100
  1526        0        25        25       0.0100
  1527        1        199       200      0.0100
  1528        0        30        30       0.0100
  1529        0        57        57       0.0100
  1530        0        35        35       0.0100
  1531        0        25        25       0.0100
  1532        0        22        22       0.0100
  1533        0        24        24       0.0100
  1534        1        199       200      0.0100
  1535        0        68        68       0.0100
  1536        0        200       200      0.0100
  1537        0        22        22       0.0100
  1538        2        42        44       0.0100
  1539        1        111       112      0.0100
  1540        0        91        91       0.0100
  1541        0        45        45       0.0100
  1542        2        108       110      0.0100
  1543        1        181       182      0.0100
  1544        0        30        30       0.0100
  1545        0        21        21       0.0100
  1546        1        25        26       0.0100
  1547        4        196       200      0.0100
  1548        0        95        95       0.0100
  1549        0        53        53       0.0100
  1550        0        55        55       0.0100
  1551        0        29        29       0.0100
  1552        0        40        40       0.0100
  1553        0        25        25       0.0100
  1554        0        33        33       0.0100
  1555        0        63        63       0.0100
  1556        0        23        23       0.0100
  1557        0        45        45       0.0100
  1558        0        25        25       0.0100
  1559        0        36        36       0.0100
  1560        0        24        24       0.0100
  1561        1        31        32       0.0100
  1562        0        30        30       0.0100
  1563        1        56        57       0.0100
  1564        0        22        22       0.0100
  1565        0        20        20       0.0100
  1566        1        22        23       0.0100
  1567        0        45        45       0.0100
  1568        1        50        51       0.0100
  1569        0        25        25       0.0100
  1570        0        30        30       0.0100
  1571        2        198       200      0.0100
  1572        2        198       200      0.0100
  1573        1        185       186      0.0100
  1574        0        26        26       0.0100
  1575        4        196       200      0.0100
  1576        3        197       200      0.0100
  1577        1        29        30       0.0100
  1578        0        25        25       0.0100
  1579        0        32        32       0.0100
  1580        3        197       200      0.0100
  1581        1        23        24       0.0100
  1582        0        25        25       0.0100
  1583        0        66        66       0.0100
  1584        1        27        28       0.0100
  1585        0        32        32       0.0100
  1586        0        21        21       0.0100
  1587        0        23        23       0.0100
  1588        1        47        48       0.0100
  1589        0        42        42       0.0100
  1590        0        26        26       0.0100
  1591        0        47        47       0.0100
  1592        0        200       200      0.0100
  1593        2        52        54       0.0100
  1594        1        19        20       0.0100
  1595        0        33        33       0.0100
  1596        0        27        27       0.0100
  1597        1        79        80       0.0100
  1598        0        54        54       0.0100
  1599        0        50        50       0.0100
  1600        0        25        25       0.0100

I have initiated the epsilon with 1 and it has already reached its minimum possible value. Still, the result is very fluctuating from the score's point of view. Why is this happening? How can I maintain the continuity?

Please help me with this, how can I train this model to score 200 scores continuously. Am I doing anything wrong in applying the concept?

You make the reward 10 for it being done and -10 for it staying alive. Reversing those should help. Right now, it’s being trained to fall as quickly as possible. Another problem is that your neural network is way larger than it needs to be for this simple environment.

@S2673 , Thank you for pointing out the error. Could you please tell me how to decide how many layers will be good for optimal performance?

You can make a guess based on how complex the environment is and your experience with similar environments, but there is no one way I know of to easily decide the number of layers. From my experience with CartPole, you need very few layers or none at all. Around 2 should probably work for you.

@S2673 , As I understand the value stored in the done variable is false until the episode gets terminated. From that point of view, I have written the code rightly. It would have not been reached the score of 200 if I had implemented it wrongly.

Right, if done is false when the episode is still going then if not done will return True and the reward will be -10, meaning it is bad for the cart to balance the pole longer. I’m not sure why you got the 200 rewards, but it could be random. If you think the code is right, then you might want to try changing the parameters like learning rate and hidden layer sizes.


text:
How can I compare the results of AC1 with the results of A3C (on the CartPole environment)?
I am implementing A3C for the CartPole environment. I want to compare the results I got from A3C with the ones I got from AC1. The problem is I don't know which process to look at. If I use, let's say, 11 processes, should I take the first one which got to average 495 points (over the last 100 episodes), last one, or should I take mean of all?
I don't want to take the first one that got to 495 since it is using a model that was already updated by the first few processes and it looks like cheating. Does some norm exist I can follow for valid results?

Hi Leon, and welcome to AI Stack Exchange! Wow, interesting question! Right now, I'm not 100% sure what your question is asking, and I want to be sure before I try to give an answer. For starters, could you please explain the chart in a little more detail? For instance, what exactly do each of the lines represent? Also, I'm used to Cartpole only yielding rewards up to 200. Could you please explain how your mean_reward dependent variable allows for values over 200? Thank you!

Hello, thank you! So I think you are using Cartpole v0 which is 200 maximum reward, i am using Cartpole v1 which is 500 maximum reward. Chart represents: x axis - mean reward (last 100 episodes) from 0 to 500, y axis - number of episodes. So lower the number of episodes better the algorithm.

Thank helps; thank you! I have two more questions. What algorithm is AC1 - I haven't seen that initialism before? Could you spell out the name or provide a link to a source? Also, I'm still trying to exactly determine what the A3C lines in the graph represent. From my understanding, A3C uses multiple processes to gather data in separate environments. Is each line the average reward from the corresponding process?

AC1 is Vanilla Actor Critic, i added n-step, entropy and advantage but it is still single process Actor Critic. Yes, each line is one process and its mean reward over last 100 games.


text:
Number of classes vs number of parameters/layers?
How to estimate the number of parameters in CNN for object detection?
I know that there are some well-known architectures that was trained on a lot of data (AlexNet, ResNet, VGG, GoogleLeNet). But they were trained for example for classifying 1000 classes. Or they were used as backbones in the algorithms like YOLO to localize 80 classes of objects.
Now let's say that I want to classify only 5 classes. Or I want to perform object detection and I am interested only in cars and people. I want to detect/classify this small number of objects. So the network must learn only the features of cars and people (instead of learning the features of hundreds of objects).
So my intuition is that I can use smaller network with fewer number of parameters. Correct me if I am wrong. And my second intuition is that the number of layers should not have a big impact. I mean, you shouldn't decrease the number of layers only because you have less classes. Because the network learns more and more sophisticated features in deeper layers. And it wouldn't be able to detect advanced features of cars (or other objects) if you don't have enough layers.
Recently I tried to use CenterNet https://arxiv.org/abs/1904.07850 to detect digits on 64x64 grayscale images and I achieved success having quite simple 900k convnet. Then I tried to use slightly modified GoogLeNet to detect cars using 224x224, 448x448, and 512x512 images. I trained it on 450 images. After a lot of trials and errors I still cannot train a good model. GoogLeNet is quite small network in compare to other well-known architectures, but I heard that it's very good. It was carefully designed to be very powerful despite being small (7M parameters).
So to be clear. My question is about the dependencies between the number of classes and the number of layers and parameters.

You are correct about your first assumption, but not about your second assumption.
More layers does not always mean better pattern detection. The analogy that in the deeper layers the network learns more complex features is an oversimplified one. It is true to some extent, although it is not enough to explain very complex architectures like GoogLeNet.
Moreover, using more layers than neccessary increases the risk of observing vanishing gradients. Because the network is too deep i.e. has too many layers, the gradient will be vanishingly small when backpropagating so many layers in the beginning will not get any significant update during training.

Ok, I understand the answer to my second assumption. But I also ask about something else. I have GoogLeNet (22 layers deep) which is great for complicated tasks (like classifying 1000 classes). But I want to classify let's say 4 classes (and I use only few hundreds/thousands images instead of millions). Can I decrease the number of layers (for example delete half of them) without being worried about performance? Can I decrease the number of layers because I have less classes?

@Kacper777 You are correct in that decreasing the number of classes will also mean a decrease in the number of layers. Personally I would try with something less complex, like AlexNet. I believe that changing the last two layers of AlexNet to output 5 classes should be enough. Nevertheless, I believe that you can simplify AlexNet by cutting out some layers or increasing the strides and as long as you output 5 classes in the end, you should get good results.


text:
Why don't we use this intialization with SGD rather than random?
Suppose I have a loss function as a polynomial with its variables being the weights of a network I wish to tune. Now, we want to find the minima of the loss function - so basically argmin.
In ML, we simple use SGD with any initialization. But consider this: we take a few $n$ random combinations of weights and plot a visual graph (not to be confused with computation graph) where we find the local minimas of the graph (basically any point surrounded by larger point values would be minima). We store the weights (value of the variables in the polynomial) used for each point in the graph in a data structure.
Theoretically, if $n$ is big enough to be computationally efficient while being quite descriptive, we can simply take the weights of a random minima as initialization to the network and then perform SGD on it to converge to a global minima (hopefully).
This method would be quite faster since the initialization is better, and we don't need to compute for large values of $n$ - simply having a decent enough estimate. SGD would finally be used with a low learning rate to give the final push and we can be done with easier and faster.

So why don't we do this instead of having random initialization? is there theoretical basis on which this can't work?

AFAIK, Just Random Initialization (not your method) is not used, because it leads to either saturation or the variance in activations and gradients keeps decreasing. For reference

Are you suggesting that you do some kind of up-front random parameter search of a limited number of parameters before training the model?  It's an interesting idea, but I suspect the trade-off of how large $n$ needs to be would prevent this from being useful.

@DavidHoelzer possibly yeah, but I expect the $n$ would have to be tuned as to be computationally viable - I see that it's much better to have a larger value of $n$ because substitution is much faster than backpropogating with SGD. Maybe, it might actually offset the time + computation because then SGD would converge extremely fast - but I guess it has to be tested.
Maybe a good research topic, eh? I might try pitching it to professors to see what their opinions are about this.

I don't want to discourage you from it.  It's interesting, but it feels like a monte carlo approach to possibly saving some time, but there's no guarantee that you're not still stuck in local minima.


text:
Weird KL divergence behaviour
I'm training a complex model for motion prediction using a VAE, however the KL divergence has a very strange behavior.
A scheleton of the network is the following:

At the end my network compute the MSE loss of the trajectory and the Kullback Leibler loss (with gaussian prior with mean 0 and std equal to 1) given as:
kld_loss = -0.5*torch.sum(1 + sigma - mu.pow(2) - sigma.exp())

Any idea of the possible causes? Do you need further details?

Yes, do need further details.

Added more details @Kostya


text:
Text to image GANs and failure
My knowledge of GANs is relatively basic at the moment but I seem to remember reading somewhere that GANs that generate images from a text prompt, when they fail to understand some of the text/words render those text/words in the image itself instead of interpreting and rendering what they refer to - this is apparently a known bug or failure.
Can somebody confirm that this is true or false? And if possible provide a link to writing about it that will allow me to verify the details e.g. if its specific to a type of GAN, version etc. Many thanks in advance.

do you mean DALL-E https://openai.com/blog/dall-e/ ?

Yes but I'm using VQGAN + CLIP at the moment and I can see what seems to corresponds to the behaviour I've described above. I just need to to find details that this is a known bug/behaviour.


text:
Document clustering from ordered pages list
I have a series of ordered pdf pages which own to different documents. Let me give you an example:
Pages: 1 2 3 4 5 6
True Pages: 1 2 | 1 2 3 4
So I have like six ordered pages, two of which from document A, and the remaining from document B. I do not have documents labels so the grouping should be done in an unsupervised way.
Which could be a reasonable approach? Using only CNN to detect border pages shouldn't be enough to discern documents, so I was thinking to something like RNN->CNN or CNN->RNN but I don't know how it would practically work because it is the first time I don't use labels in my TF model.
Do you think it would be a reasonable idea?

Hello. Welcome to AI SE. To give an idea of what your actual question is, you could please put your main specific question in the title?

I tried but I don't know whether this problem has a specific name


text:
Evaluating a convolutional neural network on an imbalanced (academic) dataset
I have trained a posture analysis network to classify in a video of humans recorded in public places if there is a) shake-hand between two humans, b) Standing close together that their hands touch each other but not shake hand and c) No interaction at all.
There are multiple labels to identify different parts of a human. The labels are done to train the network to spot hand-shaking in a large dataset of videos of humans recorded in public.
As you can guess, this leads to an imbalanced dataset.
To train, I sampled data such that 60% of my input contained handshaking images and the rest contained different images than hand-shaking.In this network, we are not looking at just labels but also the relative position of individual labels wrt to one another. We have an algorithm that can then classify them into the three classes.
I am stuck on how to evaluate the performance of this network. I have a large dataset and it is not labeled. So I have decided to pick 25 from class A) and B) and 50 from class (C) to create a small dataset to show the performance of the network.
And to run the network on the large dataset without labels, but because classes A and B are quite rare events, I would be able to individually access the accuracy of the network prediction of True positive and false-positive cases.
Is this a sound way to evaluate ? Can anyone having experience or opinion share their input on this? How else can I evaluate this?

Can you please help me understand what the final labels are for your optimization problem? Are they 3 labels 1) shake hands, 2) standing close, 3) no interaction; or are they human body keypoints such as in COCO. If the labels are a combination such as in a multitask setting, please share that as well.

They are not three labels, but by thresholding a set of labels and by assumptions from data we categorize each video into one of these classes. All the three classes are made of the same set of labels, it's just their combinations and likelihood values that decides which of the three classes they fall into.


text:
AI model to predict/generate person's image
I want to make a model that predicts person's shape depending on his son's image.
My plan is to create a dataset and each data point in it consists of two images; One for the father or mother and one for the son. Then make a model and train it with this dataset.
So when I give the model an image of a son, it predicts / generates / draws the father's image.

Is that is possible ?
If yes, How can I make it ? ML ? Deep Learning ? Something else ?

I searched a lot but didn't find something helpful; So any ideas or opinions are welcomed.

Take a look at generative adversarial networks (GANs), in particular at conditional GANs and image-to-image translation (Pix2Pix, CycleGAN), as well as style transfer (StyleGAN2).

Thanks, I will have a look at them.


text:
What are the standard ways to measure the quality of a set of numerical predictions that include uncertainties?
I have a radial basis function that supplies uncertainties (standard deviations) with its predictions, which are numerical values.
This function is computed for a particular point by computing its relative distance to a large set of other reference points in high dimensional space, and compositing a prediction from them.
Over the training set I can compute R to get a correlation between prediction and actual. Weights are assigned to each dimension and optimized to maximize R.
Over a validation set, it seems I'd want to calculate something other than R to measure the model's predictive power, since its predictions are not single values, but ranges.

Hello. What do you mean by "agreement between a validation data set, and predictions for that set that include uncertainty"? Please, explain what you mean by "agreement" here and "set that include uncertainty". Also, what is your ultimate goal? What are you trying to achieve?

Maybe you mean Accuracy, Precision, Recall & F1 Score?

Huh. You're saying I could make a binary classification (false positives / false negatives) from whether or not the prediction error falls outside a certain multiple of the predicted standard deviation? Then I pick the confidence (say, two sigma) and evaluate the usefulness of the model that way? That sounds cool. Think I'll try that.

@PJ7 When you talk about uncertainties, it sounds like Bayesian neural networks to me, which are different from regular neural networks.

We're using radial basis functions.

@PJ7 It might make sense if you add more details about it in your question. ps: don't forget to add @ to your comments to notify users

Can you please explain how you're using "radial basis function" as a machine learning algorithm/model/function that provides uncertainties? Could you provide a reference to read more about the topic? I've never seen "radial basis functions" being used for this purpose, so I am trying to understand how they would be used for that.

@nbro I have to walk the line between trying to seek help for stuff out of my expertise, and respecting an NDA. Supposedly it's a novel AI design though nobody really publishes the valuable stuff. I'll try to add more to the description while honoring my commitments.

Ok, but given a prediction for a point with the RBF, how exactly do you compute an "uncertainty" for that prediction?

@nrbo It's a standard deviation.


text:
Loss function to Push response value towards extremes
I have a feature map whose values are in the range of [0,1]. I want to push these values either towards extreme 0 or 1 using some loss function. Since I don't have any target value so it had to be in unsupervised way. I want to visualize this feature map in a way that either pixels values are approaching 1 or 0. One possible technique is to use entropy loss. What are other possible techniques used in Loss function to get extreme pixel values?

I am not sure I understand your problem. Can you explain why you want to "push" the values of the feature map to 0 or 1?  So, is your goal only to see some kind of animation? It's also not clear how the entropy loss function would be used in your case. Can you provide more details about this?

So my goal is a sort of heat map where i could only highlight those pixels whose value is almost 1. Like GRAD-CAM shows the heatmap about which pixels are important in classification. Similarly i want the same heatmap for classification but only show either red(1) and blue(0) color so any pixel which is red would really contribute in the classification task


text:
How to choose the reward in reinforcement learning?
I am solving a combinatorial optimization problem, where I do not have a global optimum, so the goal is to improve the objective function as much as possible. So, to do this, I was inspired by this article Reactive Search strategies using Reinforcement Learning, local search algorithms and Variable Neighborhood Search, I apply during several iterations, heuristics to improve the solution, that is to say, that at each iteration I must choose a heuristic and apply it on the current solution.
In this article, they have defined the state space as the set of heuristics to apply and the action space is the choice of a heuristic among these heuristics.
Regarding the reward, they gave +1 if the solution is improved and -1 if the solution is not improved.
Sincerely, I did not understand how we define the reward for example here -1 and 1, and according to which criteria we choose the reward to use?

Welcome to AISE. Please add a reference to the article you talk about

This answer might be useful : How to make a reward function in reinforcement learning


text:
Why the collection of background/negative image dataset is not taught in object detection tutorials and books?
While I was doing an object detection project, I have encountered the problem of getting FALSE POSITIVES and FALSE NEGATIVES. After days of research on StackOverflow, I figured out that I need to collect more negative images or background images.I decided to document this process so other people could easily solve this issue and the result of documentation is this. After training the model with Negative/Background images, my FP/FN rates were normalized so that in video frames I started getting fewer FPs. All of us, machine learning developers get experience by getting hands dirty - this is clear to all of us. But I haven't seen(probably missed) any video tutorials or examples on books showing how to collect background images and why we need them at all.
So here is the question: Okay, so every experienced ML engineer knows what is the FP/FNs are, and their prevention methods. But why this topic is less known and taught within popular object detection tutorials and books? Or am I missing something?

Normally you select from the background image.

@FourierFlux what does that mean?


text:
How do I quantify the difference in sample efficiency for two almost similar methods?
I am comparing my coded TD3 (Twin-Delayed DDPG) and the same TD3 (same hyperparameters) but with Priority Replay Buffer instead of a normal Replay Buffer.
From what I have read, PER (Priority Experience Replay, Priority Replay Buffer) aims to improve sample efficiency. But how do I measure or quantify sample efficiency on these two? Is it who gets the highest average reward in a given number of episodes? Does it have something to do with the batch size?

What is TD3 and PER? Please take the time to make your question clearer by providing further context.

@ThePointer I have already elaborated it


text:
To denote a training example should I use row vector or column vector?
This code accesses the first 3 examples in the iris data set,
from sklearn.datasets import load_iris
iris = load_iris()
print(iris.data[:3])

and gives
[[5.1 3.5 1.4 0.2]
 [4.9 3.  1.4 0.2]
 [4.7 3.2 1.3 0.2]]

To denote the first example, $x_1$, should I use a column vector like
\begin{bmatrix}
5.1\\3.5\\1.4\\0.2
\end{bmatrix}
or a row vector like the following?
$$[5.1 \ 3.5 \ 1.4 \ 0.2]$$
Andrew Ng suggests putting examples in columns

while typical relational databases putting examples in rows.
I'd just like to know the pros and cons of different notations so that I can decide which one I would follow.

Hello. What do you mean by "to denote a training example..."? Are you writing e.g. a paper or a homework and you don't know which notation you should use or maybe something else?

@nbro Thank you. I'd just like to know the pros and cons of different notations so that I can decide which one I would follow.


text:
How to change a single object detection network to a multiple object detection network?
I have trained a CNN network to detect a circle and approximate its centre and radius in an image. What I want to do now is detect the centre and radius of all the circles if there are multiple circles present in an image.
How do I proceed to go on about it? Do I have to make changes to my dataset to be able to do so? I tried to look at different architectures that do multiple object detection, but I couldn't understand what changes I could make to my architecture.

I won't be able to give you the exact answer but I will direct you to anchor boxes in object detection. The idea is to create multiple bounding boxes with different shapes centered around each pixel (or pixel groups) such that you can optimize the network parameters based on loss functions such as Intersection-over-Union (IoU) over many ground truth bounding boxes instead of just one. Hope this helps :)

It does help a little, it would be great if you could share the link of an example for the same so that I can understand it better.

One of the best content for anchor boxes is here: https://d2l.ai/chapter_computer-vision/anchor.html
You can also look at http://ronny.rest/tutorials/module/localization_001/iou/ and https://stackoverflow.com/questions/25349178/calculating-percentage-of-bounding-box-overlap-for-image-detector-evaluation to read about IoU.


text:
Does this code mean the model trains 10 epochs?
Here is an implementation for Perceptron
class Perceptron:
    def __init__(self, eta=.1, n_iter=10, model_w=[.0, .0], model_b=.0):
        self.eta = eta
        self.n_iter = n_iter
        self.model_w = model_w
        self.model_b = model_b

    def predict(self, x):
        if np.dot(self.model_w, x) + self.model_b >= 0:
            return 1
        else:
            return -1

    def update_weights(self, idx, model_w, model_b):
        w = model_w
        b = model_b
        w += self.eta * y_train[idx] * x_train[idx]
        b += self.eta * y_train[idx]
        return w, b

    def fit(self, x, y):
        if len(x) != len(y):
            print('error')
            return False
        for i in range(self.n_iter):
            for idx in range(len(x)):
                if y[idx] != self.predict(x[idx]):
                    self.model_w, self.model_b = self.update_weights(idx,
                                            self.model_w, self.model_b)

Does this code
Perceptron(eta=.1, n_iter=10)

mean the model trains 10 epochs?

Questions about programming issues are generally off-topic here. It seems to me that this is one of those questions. Please, take a look at https://ai.stackexchange.com/help/on-topic for more details about our scope.

@nbro Thanks for the comments. This is not about programming issues. The code works well. This is about terminology. The code is an easy-to-understand example of the algorithm.

yes that's 10 epochs, for idx makes 1 epoch, for i makes 10 epochs


text:
Model not learning anything, what can be the problem?
I've trained a model for heart sound classification with transfer learning (MobileNet) on Physionet dataset, and it works fine.
However, when I train it on my own dataset, it seems that it can not learn anything: more specifically, the loss is not decreasing and the accuracy is not going up. I've checked my labels and they seem to be correct. What other things should I check?

Can you clarify what you mean by "it seems that it cannot learn anything". How did you measure the learning rate?

@Jean-MarcVolle it's loss is not decreasing and accuracy is not going up, unfortunately I don't have access to learning curves right now, otherwise I would have shared them.

Go through these dot points and try each of them to see if they help: https://ai.stackexchange.com/a/26987/26726

@Recessive I think those points help if it was overfitting, the problem is it not overfitting, it is not learning at all! Accuracy and loss on the training data, itself, does not change! BTW, thanks.

@SepehrGolestanian I've had similar issues with training and have had success trying those techniques. Honestly the biggest mistake most people make is not normalizing their data. Without any code or more details though, there is no way for us to help because it could literally be anything that's causing the problem.

I always normalize my data, but I don't know what is the best way to normalize sound spectrum images.

Could you please give more info about your dataset? What are the inputs and outputs? What is its size? Moreover, how exactly are you training the MobileNet with transfer learning on your dataset? Are you freezing the initial layers and only letting the last layers be learnable, i.e. fine-tuning MobileNet on your dataset? Please, edit your post to include these details directly in it.

@nbro Thanks for your comment. I will update the question later.


text:
Why don't I get the same results of Q-Learning as in Aurélion Géron's Hands-on Machine Learning book?
I noticed something rather intriguing while testing the Deep Q-Network implementation from Aurélion Géron's book Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow, 2nd Edition; I copy-pasted the code exactly as it is but added some lines to get the graph on figure 18-10 presenting the sum of total rewards gained during each episode.
So everything is the same as the book except the training part where I added the lines related to rewards lists and the plotting:
all_rewards = []
for episode in range(600):
  obs = env.reset()
  episode_rewards = []
  for step in range(200):
    epsilon = max(1 - episode / 500, 0.01)
    obs, reward, done, info = play_one_step(env, obs, epsilon)
    episode_rewards.append(reward)
    if done:
      break
  all_rewards.append(episode_rewards)
  if episode > 50:
    training_step(batch_size)

sum_rewards = []
for i in range(len(all_rewards)):
  sum_rewards.append(sum(all_rewards[i]))

import matplotlib.pyplot as plt
episodes = range(1,601)
plt.plot(episodes, sum_rewards)

At my surprise I didn't get the same graph as the one the author presents in its book, so I reran the code again and got a totally different graph from what I had for the first time. Please find below two graphs that I obtained. I'm plotting the total sum obtained during each episode with respect to the episode.

I'd like to ask you if there is some intrinsic to the algorithm that makes it so random and in that case I'd like some references (if there are any) that prove that or I'm just doing something wrong. Thank you.

I’d wager you’re using an $\epsilon$-greedy policy which is random, you’d need to set the same seed as the book to get the same results.

@DavidIreland that's true. There is an $\varepsilon$-greedy policy in play. And I saw that the notebook provided on the Github of the author setting the seed at 42, but I find it rather intriguing that the results depend so much on that. Or is it considered as a hyperparameter?

it is a hyper-parameter, but the results should not wildly depend on setting a seed. It is likely the author tuned a seed to have good results from the book as DRL can be quite sensitive to hyper parameter selection. Have you tried training for a longer amount of time? you might need to play around with some of the other hyper parameters to get a similar performance to the author on an arbitrary seed.

What is the environment? Is it perhaps something that fails/ends early with a mistake? In which case, I don't think the seed is acting as a meaningful hyperparameter here, nor is 42 "tuned" - you are likely to be seeing the learning acting as intended. What is probably wrong is your expectations on your current plots - if you want to see true progress on the learning challenge you have set then you need to measure something else

@DavidIreland I have tried a training a longer amount of time but it's of no use. It's still a bit random. For the environment it's CartPole-v0 from gym library. Each episodes is played 200 steps if it's not ended because of the pole falling or else. I agree that we have to examine other metrics but I find it weird how sometimes in 600 episodes the algorithms reaches that 200 reward (eventhough it may fall after) and sometimes even in 2000 episodes it is not the case. I wonder if there are any "proofs" of why it's intrinsically so random.

you should be able to train a DQN that can perform well on cartpole (I have done it myself) independent of any seed. Maybe the architecture is too/not complex enough. There are walkthroughs online that will show you how to write the code from scratch to solve cart pole though.

@DavidIreland is it normal DQN or some variants like dueling DQN? I only tried the code for a simple DQN from the book I stated, as since it's considered by many as a good book I think its code must work and it's more a question of DQN's learning property. Should I put in the post the whole code presented in the book?


text:
Does randomly adding hand-engineered features increase the CNN's sample efficiency/performance?
It is a known fact that preprocessing images using CV techniques will improve CNN performance (see this answer).
But what happens when you feed in the entire image and the filtered image randomly to the network? Would the Neural Network learn to focus on the relevant aspects of an unfiltered image?
If yes, please explain how randomly processed images improve the CNN's performance/sample efficiency.

The question in the title seems to be different and more general than the question in the body of your post. Could you please clarify what your main question is here and how the two questions are related?

@nbro Thank you. Made the corrections.

Ok, it's still not clear to me whether you're talking about "data augmentation", i.e. where you artificially generate data by applying some transformations to your original data so that to increase the size of your dataset. Then you would feed either the original images or some transformed image. So, would your question be: "Why is data augmentation useful?". If yes, please, edit your post to reformulate the question by mentioning "data augmentation" specifically.

To me, what's still confusing in your post is this sentence: "when you feed in the entire image and the filtered image randomly to the network". It's not clear whether you're considering the case where you feed 2 images to the neural network each time: one is the original one and the other is the transformed one (chosen randomly??).


text:
How to apply the formatting of one json file to another. Coding style transfer for JSON
Time after time I need to merge two large JSON files, or more precisely add a json fragment to another file.
The too pieces are often written by different people and have different formatting (spacing), so margining them mechanically result in an ugly code with ragtag spacing, and formatting one of them manually or semi-manually takes a lot of time and effort.
How I can reformat the fragment in the same style the first file is formatted. I do not know what IDE, formatter, or style guide was used
I know that I can use automated tool to reformat the whole document into whatever is supported, but prefer to keep the original style.
I have heard that coding style can be imitated e.g. in the context of adversary authorship recognition by AI and imagine that for simple cases such as JSON that should be easy. I am interested in merely spacing, indentation, brackets placing, not, say property naming or nesting convention (not sure of ordering, probably not so interesting).
I am software developer trying to automate my tasks, rather than AI researcher, so please be patient.
I posted to stackoverflow, yet the task might be to challenging and open ended for traditional programming methods https://stackoverflow.com/questions/68396829

Hello. You say "I have heard that coding style can be imitated by AI". In case you still remember where you read this (if you read this), it may be a good idea to provide the link to the reference.

I cannot remember but added the first reference I was able to google


text:
End-to-end learning using LSTM-AE
I want to use prediction models like LSTM-AE to predict time-series data. The feature that the neural network should learn is in frequency between 40-60Hz. So, in order to learn the feature more effectively and removing the noises, the signal will be filtered using a bandpass filter and the result will be then passed to the network.
The problem is that, if I want to develop an end-to-end (i.e. omitting the bandpass filtering) solution, how can I do that?

It's not fully clear what your problem really is. Are you trying to model your problem so that the LSTM-AE would also perform some kind of "bandpass filtering", or maybe your problem is another (i.e. why would end-to-end learning in this case be a problem for you)?

I do a pre-processing step before sending the data to the network. This pre-processing step is just a bandpass filter (40-60Hz). Now I don't want to do this step and want the network to extract the corresponding feature from signal which is in this frequency band. I hope it clear now.


text:
How can I compute a mathematical formula for my CNN?
Let's say, for example, I have built the following CNN model using Keras:
model = Sequential()
model.add(Conv2D(32, (3,3), activation='relu', input_shape=(32,32,3)))
model.add(MaxPooling2D((2, 2)))
model.add(Conv2D(32, (3,3), activation='relu'))
model.add(MaxPooling2D((2, 2)))
model.add(Flatten())
model.add(Dense(512))
model.add(Dense(10, activation='softmax'))

I wish to be able to transform the above model into a mathematical formula.

I understand the basic structure of a CNN as follows:

where

However, I do not know how to go from the above recursive formula to something like this (the first two summations are weights and the second two are adjustable biases):

Note: The formula above is just an example and not representative of the code given above.

Do I need to trace each weight, each bias and each connection of every neuron? If so, how?
Furthermore, I would highly appreciate it if someone could provide a generalized strategy for tackling such a problem (like finding a math formula to suit a different kind of classifier).
Lastly, is this an easy task and is it a worthwhile one?

Note: This question was originally posted on Stack Overflow. Unfortunately, I received no responses even after offering a bounty. Hence, I am uploading the question here. Link to the original post here.

You can unroll the recursion, but it won't simplify much (if at all), as it is based on non-linear function composition, e.g. $y=f(g(h(i(j(x)))))$ where each of $f()$, $g()$ etc represents the function of a single layer. That's one of the main points for having layers - if there was something simpler possible, then deeper neural networks would not be as useful.

@NeilSlater That makes sense, thanks!


text:
How can I take continuous video input into my model?
Let's say I have designed an ML model that can take video input of a dog running around and give the breed of the dog as output. However, I do not want to wait for the video to finish before it is input into my model. I want something like the following to happen:
I am casually taking a video of my backyard when mid-way through a dog runs past the camera. Immediately, my model should identify (a) a dog has appeared within view and (b) the dog is a Labrador Retriever.
In an attempt to achieve the above, I have the following questions:

Do I need to train a new model that detects when a dog has appeared within view?
How can I make my model such that the input is continuous and that the model keeps running providing instant output?

Note: This question was originally posted on Stack Overflow. It was closed as a consequence of it being unfitting to the site. Hence, I am uploading the question here. Link to the original post here.

Sorry to push you from pillar to post, but I think the topic here is best suited to Data Science Stack Exchange. It is related to data engineering and machine learning engineer skillsets. The answer will depend on lots of specifics - where you are running the detector, how that is connected to your camera (have you written something embedded on the camera, or are you sending a feed to a nearby computer over wifi?), and what frameworks you are using (Tensorflow? OpenCV?) There won't be a satisfactory "generic" answer. So please add those details if you decide to ask again.

@NeilSlater Thanks! I will add those details and ask my question on Data Science Stack Exchange.


text:
Questions about a research paper on salient region detection and segmentation
I am reading this paper in an attempt to recreate the salient region detection and segmentation model employed. I have the following questions pertaining to section 3 of the paper and I would highly appreciate it if someone could provide clarity on them.

The word "scales" is used at multiple points in the section, for example, line 4 of the section states "saliency maps are created at different scales". I do not exactly understand what the authors mean by the word scales. Moreover, is there a mathematical way to think about it?

I understand that a saliency value  is computed for each pixel at () using the equation

However, there is no mention of  in the equation. Hence, I am confused as to what pixel the saliency value is being computed for. Is it ?

I did not understand what the authors meant by the term "bin" in section 3.2 line 5 where it is stated, "The hill-climbing algorithm can be seen as a search window being run across the space of the d-dimensional histogram to find the largest bin within that window."

Note 1: This question was originally posted on Stack Overflow. I was advised to post it on another platform as a consequence of it being unfitting to the site. Hence, I am uploading the question here. Link to the original post here.
Note 2: In case you are unable to access the link to the research paper, the following citation may help: Achanta, R., Estrada, F., Wils, P., & Süsstrunk, S. (2008, May). Salient region detection and segmentation.

I will offer an additional bounty of 50 (Total = 50 + 50 = 100) if someone provides a "good" answer to this question. I will provide an additional +50 bounty (along with the 100) in any community you like (other than Math Stack Exchange and Stack Overflow) if the answer is "excellent"!

I feel you man, I also used to post here paper questions and nobody answered me :'( I will try to help

Please, next time, ask only one question per post.

Very interesting paper, I did not know you could get such results using traditional image processing.
Question 1
From the paper:

Since only average feature vector values of $R_1$ and $R_2$ need to be
found, we use the integral image approach as used in [14] for
computational efficiency. A change in scale is affected by scaling the
region $R_2$ instead of scaling the image. Scaling the filter instead of
the image allows the generation of saliency maps of the same size and
resolution as the input image

So the saliency maps at different scales are just saliency maps with different $R_2$ filter size. So they vary the sizes as they say in:

For an image of width w pixels and height h pixels, the width of region R2,
namely wR2
is varied as:
$w/2 \geq (w_{R_2}) w/8$

So basically you run the same algorithm for different values of $w_{R_2}$ that will give you different saliency maps of different scales ($R_2$ scales).
Question 2
From the paper:

At a given scale, the contrast based saliency value $c_{i,j}$ for a
pixel at position $(i, j)$ in the image is determined as the distance
D between the average vectors of pixel features of the inner region
$R_1$ and that of the outer region $R_2$

So the coordinates $(i, j)$ are referenced to the whole image, it is almost a  convention, everybody uses those indexes to refer the whole image, I do not know why, maybe it was inherited from matrix notation.
So for each pixel in the image you overlap on top $R_1$ and then $R_2$ on top of $R_1$, then you compute the distance $D$ for those 2 regions to get the saliency value of that pixel, then slide the $R_1$ and $R_2$ regions in a sliding window manner (which is basically telling you to implement it with convolution operation)
Question 3
"Bin" is just one of the groups you divide an histogram into. The authors say to compute one histogram (it is used to approximate probability density functions) and then select the value of the biggest bin (the range of values with more occurrences.
So if you compute 1 histogram (search how to do it in google, there are plenty of implementations I use the openCV one) per saliency map, you could say you are computing d-dimensional histogram (one dimension per saliency maps)

Thank you so much for your answer! It is precise, simple and demonstrative of the effort you took to read and understand the research paper! I couldn't have asked for more. I will award you an additional +50 bounty after this bounty expires. Also, let me know what community you want me to offer you another +50 bounty in and how I should do that (through upvotes/bounty on an answer/something else). Once again, thank you :)

No problem man, happy to help, as I told you, I was just in your same position when I started :)) Also, I love mathy questions ^.^

Hey! How can I give you an additional bounty?

no need, you just keep learning ;)

You are amazing

Hey, I was revisiting this answer and had a doubt I wanted to clarify - if the region $R_{2}$ moves in a sliding window manner, won't the saliency map have a smaller size than the original image (like in convolution the output image is smaller)? If this is so, wouldn't it be impossible to add the saliency maps at different scales since they each have different sizes?


text:
What type of neural network do I need?
I am working on protein structure prediction.
Suppose, I am solving a problem using Neural Networks. I know how many inputs and outputs there will be in the model, as it directly depends on the problem statement.
However, how do I know:

What type of neural network do I need to apply to optimally solve my specific problem?

I think here are some decision flowcharts available that can help you navigate this set of possible solutions. Is that the kind of thing you are looking for, or do you have a specific problem that you want help analysing to decide a good fit?

You should bear in mind that many problems at the top level can be solved in multiple ways, and some of the ways of solving the same problem could point at different neural network architectures (for simplicity we can count "don't use a NN" as one of the architectures here too). So a good direct answer to your question may leave you with further questions, because you have not made some key analysis or decisions about your problem

@NeilSlater, I am actually working on protein structure prediction. However, I also want to know the general process of selecting a neural network.

I don't think any of the architectures in your diagram are that great for protein structure. You probably want to look into graph convolutional networks (GCNs). See e.g. https://www.youtube.com/watch?v=2aGgNas9IA4 (about protein function, so not identical architecture to what you want, but shows that your diagram is not complete in the area you want to work in)

Also of course this: https://www.nature.com/articles/d41586-020-03348-4

please state the problem your trying to solve.   it depends on whether the problem is linear or nonliner, text or timeseries, or binary or multiple classification.   You can use cnn or lstm or dense layer networks to solve many of the problems.


text:
Regress values inside the bounding boxes to predict a value in Object Detection
I am currently working on an object detection task. I have a dataset of Grayscale and Depth Images.  The annotation format is x1, y1, x2, y2, class, depth. I have calculated this depth (of each object/bounding box) using a clustering algorithm and depth image.
My plan is to use the Grayscale images to detect the bounding boxes and the class labels using a pre-trained CNN.
Furthermore, I want to use the depth images to predict the depth (ground truth values in the dataset as mentioned above). For this task, my plan is to build a Regression-based Neural Network that regresses the depth values inside the bounding boxes of the depth image and compares them to the ground truth values. An RMSE loss function can be used to keep track of the predictions.
How do I go about making this NN and is there a better alternative?

Hello. Could you please put your main specific question in the title to immediately give an idea of what this post is really about?

Hey @nbro , I am struggling to find the title. Any suggestions?

@nbro I have changed the title. Hopefully, I get some help now.


text:
What approaches are there to generate complex structures like syntactic trees?
What approaches are there to generate and evaluate complex structures like, let's say, syntactically correct code? I know the approach of Genetic Programming (GP) as a type of Evolutionary Algorithm, but I wonder if there are any other techniques that are being used to produce complex structures more efficiently.
Note that the syntactically correct code example is just that, an example. The code wouldn't be generated to solve a specific task, although it could try to maximize a fitness function. We could be talking about 3D models, music compositions, etc. What interests me about this issue is if there are Computational Creativity techniques being used or researched in the last years, apart from the mentioned GP.

Hello. It seems that you are aware of "generative models", so I guess you have already heard of GANs and VAEs. Is that the case? If so, what exactly are you looking for that would be different than these techniques? GP can be used to evolve programs and you're not just interested in programs, but in other structures. GANs can be used to generate content that is similar to the traning data. Maybe that's not what you are looking for and you may need to clarify exactly what you're looking for then. So, what is your ultimate goal? Do you want to generate just "random complex structures"?

I was aware of GANs, but you just discovered me VAEs. I didn't mention GANs because, as you say, they require training data, while GP does not, and that's what interests me (not requiring data, only simple rules).
An example goal would be to generate puzzles solvable by code: for example, generate a layout of objects and the code to make an agent take them all (something similar to what you find in CodeMonkey)


text:
What are the existing AI methods to approach 3D volumes of computed tomography?
I have a dataset which consists of computed tomography images (CT scans) of parts that contain pores and cracks. The sets for each part are of about 1100 * 1100 * 3000-ish resolution. Currently, I use a method of thresholding and calculations to find the volumes and locations of these defects, and I would like to reproduce those results with a machine learning approach.
What are the methods known for this type of problem, and what are your general recommendations?
Edit:

Here is the current method I am using :

And this is what I aim to achieve :

Hello. To give a more concrete idea of your problem, you could post an example of an image that you have and what you want to find in the image.

I will edit post to include more details.

I added images from the data set and outputs.


text:
Can people set loss function of neural network by themselves instead of choosing cross entropy or mean square error?
I found people used deep neural network to get optimal policy by solving a nonconvex optimization problem. Moreover, they didn't use any set of training data and claimed that it's the difference between their approach and the supervised learning. I wonder can people set loss function of neural network by themselves instead of choosing cross entropy or mean square error?
My experience in machine learning is very limited. I audited two machine learning courses offered by applied math department in my school. I read twenty or more papers on the application of machine learning. I began to use Keras very recently.

I think it'd be better if you ask these as separate questions with more context..

Hello. Please, follow the suggestion above. Edit this post to ask only one question and provide more context/background, i.e. what do you know about neural networks and/or machine/deep learning so far?

Thanks very much. I will ask them as separate questions with more context.

This Towards Data Science post on deep learning to do quantile regression gets into how to implement your own loss function.


text:
How to Study Improving In-depth Reading Comprehension?
There are multiple datasets for machine comprehension tasks such as SQuAD. However, most of the questions are straightforward. One can find the answers easily by using the find feature of the browser to look for the question keywords in the passage.
I'd appreciate it if you let us know about standardized in-depth reading comprehension tests for either human or machine that are generalizable. By generalizable, I mean they include a broad range of disciplines and academic levels and are not specifically designed for a target population.
I thought of GRE reading comprehension but was not able to find any study indicating that GRE reading comprehension questions are standardized or generalizable.

Hello. It's not fully clear to me what you're asking here. Are you looking for datasets that somehow would be more general than SQuAD? Please, edit your post to clarify what your question really is (i.e. explicitly ask a question) and how it is related to AI. It's not fully clear to me whether this question is about AI or not.

SQuAD is already generalizable with many questions from Wikipedia articles on a large diversity of topics. My problem with SQuAD, as explained in the question, is the difficulty level of the questions. Most of them are very straightforward to answer. One can easily find the answers by searching the keywords of the question in the passage. We're looking for a dataset of more analytical questions. That's why we were thinking of using the GRE verbal reasoning questions but were not able to find any study indicating that GRE reading comprehension questions are standardized or generalizable.

So, you're looking for a dataset where the answers to the questions would be so easy to find?

The opposite, difficult. We're looking for more of analytical questions than straightforward ones.


text:
How to calculate cosine similarity for classification when you have say 10000 samples belonging to two classes have a bunch of samples
Does anyone have experience with using Cosine Similarity for text classification?  I see a number of articles on how to find cosine similarity between documents using Doc2Vec, Gensim, etc.
I have a classification problem (binary) where I want to try out the cosine similarity. I do know how to calculate it, but all the articles that I see only explain until the point of calculating it between two documents.
Right now, I am planning to do this.

Calculate the cosine similarity of 'my paragraph' (the one that I want to classify) with all samples in classi (their class is known). Then take the average (call that avgi)

Calculate the cosine similarity of my paragraph (the one that I want to classify) with all samples in classo (their class is known). Then take the average (call that avgo)

Compare avgi and avgo and then predict the class for 'my paragraph'

That sounds like a very manual way of doing it. Is there some better/widely used way of doing it?

What are the samples in (1), (2)? paragraphs or docs?

Each sample is a line of text in excel row.

@Sanny28 It seems that you want to use the cosine similarity to build a model that decides which class your paragraph belongs to? Why do you want to use this? Why don't you want to train a model that takes as input the paragraphs and produces the class (if you have a labelled dataset)? It's also not clear what your problem really is. Are you asking how to efficiently compute the cosine similarity when you have too many paragraphs?

Yes, what I want is a model that takes a paragraph and produces a class. I have done that successfully using other algorithms like RF, logistic regression and ANN. Someone suggested I try out cosine similarity too. I tried to get sample code from internet. All the articles that I see only explain how to calculate cosine similarity between two paragraphs. No one seems to explain how to use it do classification (compare with a set of paragraphs in both classes which are already classified). That is why i thought of the average method I explained in the question.


text:
Time series forecasting for multiple objects with common features
I know the title of this question may raise an eyebrow, but I can't find the technical terms to define or investigate the actual problem.
To demonstrate my problem with a simple hypothetical scenario: Let's say you have dataset pretraining to fruits!

The dataset contains $N$ fruits

Each fruit has properties ${\{p}\}$, for example, $p_1$ is type, $p_2$ is color, $p_3$ flavour. It is important to note that (i) these properties are communal across all fruits (all fruits have the properties above) and (ii) these properties are constant for each individual fruit (for a fruit,${\{p}\}_n$ stays constant over time).

Each fruit has a time series $\{W_t\}_n$ which relate to, for example, measured weight over time. It is important to note that the fruits aren't measured at regular, or the same intervals. Therefore, each fruit in the dataset will have a different  weight time series.

Therefore the aggregated dataset will have $\sum_{n=1}^{N} dim(\{W_t\}_n)$ observations

Let's assume there is some hidden correlation between the weight for a fruit $\{W_t\}_n$ over time and the fruit properties ${\{p}\}$.

So the problem is: What model(s) can we use that it is able to predict the next weight values $\{W_{(t+1)}\}$? More formally stated $f(\{p\}_n,\{W_t\}_n) = \{W_{(t+1)}\}_n$ ?
The challenges here is:

We want to maintain the 'uniqueness' of each fruit, that is, we can't simply say if two fruits have the same properties ${\{p}\}$ then they will have the same weight changes over time. To conceptualize this, imagine things happen to certain fruits during their life time, the model is supposed to remember this has happened to those specific fruits and incorporate that into the prediction.
Our measurement device was bought at IKEA and sometimes it provides inaccurate readings, so we can't expect a linear or smooth weight time series per fruit.
We don't have a lot of weight measurements, let's say 10 on average, but we have a lot of fruits, let's say 100 000.

I have some experience with vanilla and stacked LSTM's. However, I struggle to consolidate my understanding of LSTM's in the abovementioned scenario.
Thank you for reading. I hope this will get the creative juices flowing, or give you a fun mental challenge at least.

Hello. You can use latex/mathjax on this site, so I would suggest that you edit your post to format your math symbols with it.

Oh thank you, I was unaware of that.


text:
How will actual labels be matched with predicted labels when LSTM discards data even from current time stamp input data?
I read the tutorial of LSTM from here. However, I have certain doubts that I need to address.

Since we use true labels and do not remove anything from the original data, then how is it possible for the LSTM model's predicted output to match the real labels as it throws data?

And how do we determine the number of output neurons?

According to my understanding, in word-to-word prediction, one cell's outputs are the number of words (exiting in vocabulary).

What data does LSTM throws away?

I think your doubt is related to self supervised learning. Suppose there is a text on which next word prediction LSTM is trained, then label is the next word itself for the window of words. Suppose your LSTM takes 4 words as input and predicts next word as ouput, then the window of words i am a good girl. Then word girl will be the true label for the input I am a good.

@hanugm thank you for the comment. No it's not self-supervision but the actual procedure of LSTM how it works. Somehow I got the concept but still have confused.

@hanugm Please see my this Question https://ai.stackexchange.com/questions/28843/how-long-short-term-memory-lstm-work-for-time-series-classification

Okay.............


text:
Non-locally Electrically Programmable Logic Gates - Technological Advances Progress
Preface: I’d like to clarify that I understand what a relay is and that a PLC uses a fairly conventional microprocessor that only digitally establishes logical logic gate configuration as a digitally programmable alternative to relay banks for analog and/or (depending on the PLC) digital signals. My question is based on the understanding that to date actual logic gates (as far as I know) aren’t non-locally programmable (“re-wirable”) without a person manually rewiring truly programmable actual (not logical programming of a statically wired microprocessor) logic gates.
Rectenna work interests me specifically around any potential relevance of varying transmission wavelengths and material resistances (if this is not possible with MoS2, generally as a concept for other potential materials) to making possible remote switch activation of logically chosen switches along an array. Essentially I am curious about if this or other research has potential for constructing truly physically reprogrammable (externally and maybe wirelessly) logic gates.
In general any information on advances towards this capability would be appreciated as right now it seems like the only rudimentary build I could manage for my project is a 64 gate one. That’s not great because anything less than 512 gates would be very hard to make useful for my proof of concept project, and I know there’s no way I could get to a more ideal 262,144 gates.
One example would be any publication which covers if the kind of uses of phase-engineered low-resistance contacts for ultrathin MoS2 transistors covered in the articles below would be able to be produced with varying resistance in a band usable for varying activation via radio waves for switches.
https://doi.org/10.1038/nmat4080
https://www.ece.cmu.edu/news-and-events/story/2019/05/rectennas-converting-radio-waves-into-electricity.html
I’m not picky if someone knows about other technological advances approaching this capability such as biochemical non-locally programmable switch activation equivalent processes. Thanks everyone.
Update 1: My specific question is: Have there been any significant technological advances towards non-locally electrically programmable logic gates?
Update 2: After further review I’ve found that FPGAs are not what I am asking about. Their reprogramming like PLCs is digital not analog. They seem to just be a more generalized similar thing to PLCs rather than being factory equipment. I might incorporate one or more in my project, but they aren’t what I am referring to which is true analog reprogramming. Why does analog matter? Analog means more efficient at the surface level, but it also allows structured logic similar to ladder logic at the hardware level which enables significantly different uses in structuring and restructuring logic execution.
Update 3: This is for an efficiency proof of concept project trying to prove it is possible to structure logic in a certain way to increase efficiency of certain specific processes. This is a project involving programming and/or design at every single level of development (transistors, machine code, assembly, mid level (such as C/C++), and high level (Python/Tensorflow). I will be creating custom NAND gate structures, writing the instructions to execute on them, writing in assembly, writing in a mid level language, and writing in Python and TensorFlow for different parts of this overall project’s functionality.
In conclusion the straightforward version of this question is: What are the current capabilities for or research done towards creating physically rewired logic gates using non-local digital instructions?

Hello. Welcome to AI SE. Can you clarify how is this question on-topic on this site according to https://ai.stackexchange.com/help/on-topic? So, please, before answering, take the time to read our on-topic page that I just linked you to.

“Questions related to specific software, hardware, or datasets are off-topic, including…“ This would be on topic because it is about developing a machine learning proof of concept proving that the number of operations required to perform a task using in part physically reprogrammable logic gates is possible. Specifically I am trying to train a high level model that can select NAND gate structures to integrate processing into a ladder logic-esque process based on external (non-local to this process) processing inputs and outputs across a few systems. I can clarify further if needed.

Sorry my first comment had a mistake. I meant proving the number of operations required to perform a specific task can be reduced using…. logic gates. (Is possible isn’t part of the properly rephrased sentence)
Basically I’m trying to prove that a specific machine learning task can be made more efficient using in part what I’m asking about.

You might be interested to learn about FPGAs. What do you think is the difference between "analog reprogramming" and "digital reprogramming"? You want a robot that physically moves wires around? Why?

I thought I mentioned both PLCs and FPGAs aren’t what I’m referring to. My bad. I want the equivalent of a person moving wires around because there are things you can do with that that you cannot do with programming NAND gates in an FPGA. You can’t integrate data flows based on programmed I/O (think PLC-like outputs) into the structure of an FPGA in a way that suits machine learning the way I plan to do it without it being inefficient which defeats the whole point. You can’t make 3 dimensional gate structures, and you can’t make sets of gate that conditionally are or aren’t connected.

In other words you can’t do physical design and are limited to however a fixed FPGA is set up as a fixed 2D NAND gate array limiting you to writing compatible machine and assembly up rather than designing at every level a (software or hardware) design that is dynamic, trainable, and reconfigurable.
It’s not about what I can do. It’s about creating a proof of concept for demonstrating efficiency which can only be done this way.


text:
Why does one-step TD strengthen only the last action of the sequence of actions that led to the high reward, while n-step TD the last n actions?
In the caption of figure 7.4 (p. 147) of Sutton & Barto's book (2nd edition), it's written

The one-step method strengthens only the last action of the sequence of actions that led to the high reward, whereas the n-step method strengthens the last n actions of the sequence, so that much more is learned from the one episode.

Why does one-step TD strengthen only the last action of the sequence of actions that led to the high reward, while n-step TD the last n actions?
Here's a screenshot of the figure.

Please provide a link if possible.

I've edited your post to include the link to the book and to clarify your question, but, please, next time, provide more details. In particular, you should explain what you don't understand, and what is your background in RL. Feel free to edit your post to add more details.