text:
is it possible to make use of classes of coco-pretrained weights on custom dataset just training on custom dataset?
I am trying to use detcetron2 panoptic_FPN and panoptic_deeplab models for optimization on custom dataset.
You might already now that coco has around 133 classes (both thing and stuff). And my custom dataset contains 7 classes ( both thing and stuff), out of which 5 classes are already present in coco and 2 classes are new.
I tried to train panoptic_FPN on custom dataset with using coco-pretrained weights and my custom classes which are mapped to the categories id's of coco-classes. but when i tried to test the newly obtained weights on test data, i could only see the predictions of my custom classes. The predicitons of the coco-classes are available. what is the mistake i am doing in trainig ?.
yes, I did change the following values in config file:
MODEL.ROI_HEADS.NUM_CLASSES = 135 (133 coco + 2 new custom classes)
MODEL.SEM_SEG_HEAD.NUM_CLASSES = 136 (N+1)
I would like to know whether, i need to retrain the model combinig both the coco and custom dataset. to make use of both the model classes


text:
Continuous Pathfinding for 3D Complex Environments
I'm the creator of an autonomous bot referred to as SwarmBot for Minecraft. One of SwarmBot's core features is its parkouring ability. Despite being relatively effective, there are instances where the current algorithm (block-discretized A*) fails to ensure that the bot operates as intended. For certain complex jumps, such as the neo-jump that is shown below, the bot lacks the required finesse to pull off.

Neo jumps pose a unique hurdle as the jumping sequence does not entail the bot looking towards the next target block. Instead, it requires the bot to jump around a block by facing away from the next intended block before changing directions.
Now, I am trying another approach—discretizing based on game ticks, with consequent nodes based on the player-pressed keys. The image below shows A* without a heuristic trying to find a path over a course where the bot can turn/jump when it is on the ground. Each green circle is a block, and each black pixel is an expanded node.

Clearly, this is not feasible. Turns are tried at lowest g-scores (since there is no heuristic), and I cannot think of a good consistent & admissible heuristic to try turning furthest down the chain. Perhaps I do not need a consistent nor admissible heuristic. I am only trying to find good paths, not optimal ones. I have also tried negative costs for certain nodes instead of a heuristic, which seems better, but this could cause issues as I could get a path of infinite length if conditions are just right.
What are some recommendations for this problem? It looks like I want to do something like is demonstrated in this Udacity video on A* for their cars, but I have a hard time finding why theirs looks much more like a sparse tree than mine. Perhaps they are using something more akin to Rapidly exploring random tree. Is this a path that makes sense to go down? Would you recommend any other papers?

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
Is it possible to generate new text matching a vector embedding with an LLM
I'm interested in generating variations of text using an LLM - is it possible to take a text embedding, move it in different directions in vector space, and generate new text from the resulting vectors?

Let's say you have a function $f: \mathcal{X} \rightarrow \mathbb{R}$ and you have  $\mathcal{X} \subset \mathbb{R}$ and $x, y \in \mathcal{X}$ and $y = x + 1$.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
Is this a valid application of Autoencodeers/VAE?
I am trying to predict a spectrum (1D vector) from various scalar inputs which are known to be correlated. As the spectrum vector is very long (4000 points) it was suggested that I use dimensionality reduction, rather than trying to produce a NN which has a 4000 output size. So, train an autoencoder on the spectra, encode the spectra, then use a neural network to predict the encoded spectra from the various inputs, and then reconstruct the spectra by decoding the predicted encoded spectra.
It was further suggested to me that
A VAE would be a better solution here than an autoencoder, as the structure of the encoded spectra may make it easier for the neural network to learn.
I should use a KL-Divergence term on the output of the neural network predicting the encoded spectra. Not sure on the justification of this one.
Are either of these points valid? Or, indeed, is the use of an autoencoder the way to go here at all? I should ay we are also data limited (around 700 points total).
I have already given it a bit of a go with a little bit of success, but I wanted to check if this is a good way forward before investing too much time/resource

You've probably answered this question yourself by now, but I'm responding in case others will find it useful... These are all sensible ideas, indeed they were used by Wu and Peek for predicting galaxy spectra in this preprint: https://arxiv.org/abs/2009.12318. The use of a KL divergence term is a fundamental part of VAEs -- it arises from the theory behind VAEs, though some flavours of VAE do modify the KL divergence term (which undermines its theoretical basis but can improve performance). The dataset is quite small for the proposed approach but it might be competitive with other options.


text:
Policy Value network predictions in Alpha Zero with ranked rewards
So I have been trying to implement the ranked rewards (R2) algorithm from the paper "Ranked Reward: Enabling Self-Play Reinforcement Learning for Combinatorial Optimization" http://arxiv.org/abs/1807.01672 The idea is to leverage the very successful 2 player setting from Alpha Zero to the single player setting for combinatorial optimization.
Instead of getting positive/negative reward for winning/losing the game against the opponent, the (single) agent here gets positive/negative reward for performing better/worse than itself in a certain percentage of previous games.
This is achieved by replacing the final_reward with a ranked_reward in the training example triplets (state, mcts_action_probabilities, final_reward) -> (state, mcts_action_probabilities, ranked_reward) where ranked_reward = 1 if final_reward is larger than a certain percentile of past rewards, and -1 otherwise.
Now here is where I am stuck: replacing the final_reward with the ranked_reward in the training examples means that the neural network learns to predict the ranked_reward from a current state, rather than the actual game reward. On the other hand, inside the neural network guided MCTS, upon leaf expansion, the neural network is used to estimate the final reward from the leaf. But if that neural network has learned to predict only if it is going to be better or worse than itself in previous games, there is a mismatch between neural network predicted rewards and actual state rewards you get from the states during MCTS simulation (e.g. when reaching a terminal state), which - I would think - messes up the tree exploration based on the estimated rewards.
How should this be dealt with? In the paper they just state in Algorithm 1: Perform a Monte Carlo tree search consisting of S simulations guided by fθk but they don't mention the problem of two kinds of rewards at all. Only for the final reward at the end of the actual game (not MCTS simulation for one step) they say to convert to ranked reward.
My guess is that MCTS should be done totally in ranked_reward units, i.e. every reward returned by any state should be converted to ranked_reward inside the MCTS simulation. Only at the end of the actual played game, the final obtained state's actual final reward should be added to the buffer. In other words, evaluating the final state of a game is the only time we ever consider real final rewards rather than ranked_rewards. Is this correct?


text:
How to fine-tune a pre-trained model for customer service like tasks chatbot?
I want to make a trainable conversational bot that can respond to customer service questions. The bot should be able to adapt to different domains based on the dataset provided by the user (a company data science engineer).
I have a general idea of how to approach this problem, but I need some guidance and feedback. My plan is to start from a pre-trained model like Flan T5, GPT, etc. Then, I would do an intermediate fine-tuning on a general conversational dataset like PersonaChat or DailyDialog. Finally, I would do a final fine-tuning on the customer service conversation dataset from the user.
Is this a reasonable approach? What are some potential challenges or pitfalls that I should be aware of? What are some best practices or resources for fine-tuning conversational models? Any help or suggestions would be appreciated.
Thank you in advance.

Your approach to building a trainable conversational bot sounds reasonable. Starting with a pre-trained model and then fine-tuning it on conversational datasets is a common and effective approach. Remember that building an effective conversational bot is an iterative process. Continuous monitoring, evaluation, and improvement based on user feedback are key to refining the performance and usability of your bot.


text:
Filter distribution of Latent variable models
In this paper https://arxiv.org/pdf/1907.00953.pdf, about stochastic latent variable models, the paper says "We use the
reparameterization trick to sample from the filtering distribution". I read the paper, and looked up what it was. I'm a self-learning reading the paper. Can anyone tell me or give me a reference on what a filtering distribution is?


text:
ChatGPT4all to create chatbot to answer questions on your own docs without external calls
So, I came across this Tutorial (Apologies, if you cannot access it, it is a member's only story) and I gave it a shot. Technically, it "works". However, it seems to be a bit poor in the sense that I only fed it 5-600 PDF files and even if I ask a question copying the title of the file, it gives some other answers. I played around with the "template" variable and this seems to be the best to me. Basically, I just want it to answer questions from the "context" which is basically an index of my docs. Any suggestions on how to improve this?
import os
from langchain import PromptTemplate, LLMChain
from langchain.llms import GPT4All
from langchain.callbacks.base import CallbackManager
from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler
from langchain.document_loaders import TextLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.document_loaders import UnstructuredPDFLoader
from langchain.document_loaders import PyPDFLoader
from langchain.document_loaders import DirectoryLoader
from langchain.indexes import VectorstoreIndexCreator
from langchain.embeddings import LlamaCppEmbeddings
from langchain.vectorstores.faiss import FAISS
from langchain.embeddings import HuggingFaceEmbeddings

# Assign the path for the GPT4All model
gpt4all_path = './models/gpt4all-converted.bin'

# Callback manager for handling calls with the model
callback_manager = CallbackManager([StreamingStdOutCallbackHandler()])

# Create the HuggingFace embeddings object
embeddings = HuggingFaceEmbeddings(model_name='sentence-transformers/all-MiniLM-L6-v2')

# Create the GPT4All LLM object
llm = GPT4All(model=gpt4all_path, callback_manager=callback_manager, verbose=True)

# Load our local index vector db
index = FAISS.load_local("my_faiss_index", embeddings)

# Create the prompt template
template = """Using only the information provided: {context}
Please provide an answer to the following question: {question}
Answer:
"""

# Function to handle similarity search and return the best answer
def get_best_answer(question):
    matched_docs, sources = similarity_search(question, index, n=1)
    context = "\n".join([doc.page_content for doc in matched_docs])
    prompt = PromptTemplate(template=template, input_variables=["context", "question"]).partial(context=context)
    llm_chain = LLMChain(prompt=prompt, llm=llm)
    answer = llm_chain.run(question)
    return answer

# Function to handle similarity search
def similarity_search(query, index, n=4):
    matched_docs = index.similarity_search(query, k=n)
    sources = []
    for doc in matched_docs:
        sources.append(
            {
                "page_content": doc.page_content,
                "metadata": doc.metadata,
            }
        )
    return matched_docs, sources

# Main loop for continuous question-answering
while True:
    # User input for the question
    question = input("Please enter your question (or type 'exit' to close the program): ")

    # Check if the user wants to exit the program
    if question.lower() == "exit":
        break

    # Get the best answer
    answer = get_best_answer(question)

    # Print the answer
    print("Answer:", answer)

# End of the program

One very irritating thing about this is also that it prints the whole "template" variable, I cannot seem to get rid of it, because I must use the "context", and even if it gets the right context 95% of the time, it still gives a wrong answer, not sure why?
EDIT:- I see that there are LLMs you can download and feed your docs and they start answering questions about your docs right away. So, I think  steering the GPT4All to my index for the answer consistently is probably something I do not understand. It should not need fine-tuning or any training as neither do other LLMs. So, my guess is that I am lacking in the "template" area? maybe and perhaps tempereture, top_p etc. :(


text:
Can I combined the trained model between different source but same model structure?
Here I got two different deep-learning models that were trained by LSTM and time-series data.
The data is the usage percentage of CPU from two different computers.
Each computer job was the same. It was a simple stress test with different stress power and period.
Basically, the target source is the same as CPU. but data is not the same.
Based on two time-series data, I trained the model using LSTM and finally, I got two '*h5' saved models.
As a step of data preprocessing, I applied 'Differencing' which is the removing step in order to erase trends and periodical conditions. And also the scaling by 0 to 1.
Due to the 'Differencing', the model can train the data's potential variation.
And the model will use a future stress test reference and will provide some safety guidelines.
I think the data characteristics are basically the same because it is CPU.
My question is, Can I combine two trained models and believe in the performance?
I didn't validate and test this issue but MY current opinion is,

Because of Differencing, Each model might train the CPU's potential variation. So It is already independent of the trend or periodical behavior. Both data can use for the single model.
Basically, Each data represented the CPU conditions. Even if one data had a dramatic variation, It can be treated as a special case from long CPU data. big data is always better than small data. so Both data can use for this.

My thought are shown above. I Would like to hear a different or additional opinion from experts.
Thank you for your help!


text:
What is the main difference between class label prediction through Machine Learning and Conditional Label generation that can be done on raw data?
I have understood that machine learning helps us to classify various instances using the labeled data, but why do we need machine learning when we obtain the same results through generating labels by directly asserting some parameters on the feature.
For example, let us use Air Quality dataset which has various features such as CO(GT), O$_3$, CO gases which are associated with certain units, in case of unknown or unidentified class label I can set certain parameters based on general assumptions of air quality index on raw data using excel operations and generate a class label but why do I need to train any kind of classifier for the exact purpose of predicting class label which can be done in raw format.
I would really like to know the difference between them and also different procedures that I can follow.


text:
DDQN doesn't learn my modified version of Taxi-v3 from OpenAI
I tried to enhance the Taxi-v3 environment from OpenAI (true credit goes of course to https://arxiv.org/abs/cs/9905014) to be capable of handling multiple taxis/passengers. Don't worry, I'm not starting with multi-agent RL for debugging, I decided for a simple 3x3 grid with one passenger, one taxi, and no borders.
Here is some random starting state:
+-----+
|A: : |
|a:0: |
| : : |
+-----+

a is the current location of a passenger, A its destination location, and 0 the location of the taxi. The taxi will get a positive reward when first picking up the passenger and then dropping him/her off at A; everything else is punished.
After spending quite some time debugging it, I decided to ask for help ... my DDQN agent does not learn!
You can find my code here: https://github.com/SorenSc/multi-taxi-drl
The DRL agent is build via the library tianshou.
I tried various things:

Try hyperparameter tuning (learning rate, gamma, ...)
Looked at weights and biases (the network seems to learn something)
Used logging (didn't see anything out of the ordinary)
Tracked various metrics like the reward.

But even after 5000 epochs of training, the DRL agent selects the same action all the time. The action selected might vary with different start states, but it doesn't vary in one epoch.
Do you have any idea what I'm doing wrong?
My best guess is that I'm not seeing a bug in the environment...
Learning of a DDQN agent in my environment (blue) in comparison to one learned in the original Taxi-v3 environment in orange.

Weights/biases during training:


text:
Relation between the number of parameters and the features in Gradient descent algorithm

My book describes this as an equation for minimizing the $\theta$ value, but I have a few questions regarding the intuition behind this equation:

The book describes $j$ as the number of features. If we have to compute the $\theta$ value for every $j$, does this mean that the number of features $\left(x_1,\:x_2,...\right)$ is equal to the number of parameters $\left(\theta _1,\:\theta _2,...\right)$?

How are the initial $\theta$ and $\alpha$ values selected? What if the initial values selected are too low/ too high?

If anyone could clear up my confusion, that would be great. Thanks.

Which book are you using (please, also specify the page/chapter/section)? It might make sense to learn one parameter for each feature but we probably need more context

@nbro Course notes here: https://cs229.stanford.edu/lectures-spring2022/main_notes.pdf
please refer to page 10


text:
2D layout problem with RL - questions about state representation
I want to solve the following sort of problem with RL, specifically Q-learning (using the tensorflow-agents library):
The input is a polygon $\mathcal{P}$ and a list of rectangle sizes $\mathcal{L}=\{s_{1}=[h_1,w_1],s_2=[h_2,w_2],\ldots,s_k=[h_k,w_k]\}$. It is required to place polygons of the sizes given in $\mathcal{L}$ inside $\mathcal{P}$ without overlap. Each configuration is scored at the end of the episode according to the placed rectangles' total area and some other criteria which are immaterial at the moment.
My problem is how to represent the problem in a way that allows for generalization. My first approach was to discretize the polygon $\mathcal{P}$ and to define an action = (rectangle placement) as a triple $(x_{TL}, y_{TL}, i \in \{1,\ldots,k\})$ and to train a DQN agent. But this approach forces me to train a separate agent for each instance $(\mathcal{P},\mathcal{L})$ which is woefully inefficient and very unsatisfying.
So what I am looking for is a way to somehow:
(1) encode the state/observation in a fixed-length representation
(2) encode the action space in a fixed-lenght representation
while
(3) taking into account the fact that part of the action is choosing a placement point in $\mathcal{P}$
I thought about handling (1) by training an autoencoder that takes as input a, say, $299x299$ image of $\mathcal{P}$ with a number of polygons drawn inside it and learns an internal code of length, say, $100$. But I fail to see how to handle (2) and (3) with this approach.
Any suggestions or pointers will be greatly appreciated.


text:
Why use masked attention in the second to 6th decoder blocks during training
For the first block we have access to all the output tokens, so we will have to use masked self-attention. After that in the first decoder block, we have encoder-decoder attention, which will bring some information about the future tokens, but this information is not cheating because model is only seeing future tokens in input language (let's say the transformer is translating from one to another language). So the final output of the first decoder block might have some features from the future token but they will be from the input language and this can help the model decode the current word better. So is there a reason to mask from second to 6th decoder block that I am missing?

Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking.


text:
How are groups created in maxout units when dividing the set of inputs  into groups of  values?
I don't get $G^(i)$the set of indices into the inputs for group $i$, $\{(i −1)k+ 1, . . . , ik\}$ when creating a maxout units/function, these thing that outputs the maximum element of groups: $$g(z)_i= max_{[j∈G^{(i)}]}z_j$$

Maxout units(Goodfellow et al., 2013a) generalize rectiﬁed linear
units. Instead of applying an element-wise function $g(z)$,
maxout units divide $z$ into groups of $k$ values. Each maxout unit
then outputs the maximum element of one of these groups:
$$g(z)_i= max_{[j∈G^{(i)}]}z_j$$
where $G^{(i)}$is the set of indices into the inputs for group
$i$,$\{(i −1)k+ 1, . . . , ik\}$.
This provides a way of learning a piecewise linear function that
responds to multiple directions in the input $x$ space. A maxout unit
can learn a piecewise linear, convex function with up to $k$ pieces.
Maxout units can thus be seen as learning the activation function
itself rather than just the relationship between units. With large
enough $k$, a maxout unit can learn to approximate any convex function
with arbitrary ﬁdelity. In particular, a maxout layer with two pieces
can learn to implement the same function of the input $x$ as a
traditional layer using the rectiﬁed linear activation function, the
absolute value rectiﬁcation function, or the leaky or parametric ReLU,
or it can learn to implement a totally diﬀerent function altogether.
The maxout layer will, of course, be parametrized diﬀerently from any
of these other layer types, so the learning dynamics will be diﬀerent
even in the cases where maxout learns to implement the same function
of $x$ as one of the other layer types.
Deep Learning, Bengio and al. page 190

I tried to work it out with a concrete example:
Let's take a concrete example with k = 3. In this case, the groups would be as follows:
$$G^{(1)} = {(1 - 1)3 + 1, ..., 1 * 3} = \{1, 2, 3\}$$
$$G^{(2)} = {(2 - 1)3 + 1, ..., 2 * 3} = \{4, 5, 6\}$$
$$G^{(3)} = {(3 - 1)3 + 1, ..., 3 * 3} = \{7, 8, 9\}$$
And so on...
So, in this example, if we have 9 inputs, we'd have 3 maxout units, and each maxout unit would use 3 inputs to calculate its output, choosing the maximum from these 3 values.
Suppose we want to model the cost of producing a certain number of units of military weapons as a function of the quantity produced. We can use a piecewise linear function for this.
So let's imagine that the production cost is defined by a convex function with three pieces (k = 3):

For the first 100 units produced, the cost is \$10,000 per unit.
For 101 to 200 units, the cost is \$8,000 per unit.
For 201 units and above, the cost is \$7,000 per unit.

So here, I guess each maxout unit would take as input the quantity produced, and each group $G^(i)$ would be associated with one of the pieces of the function.
For example, if we wanted to calculate the cost of production for 150 units, I'm not sure but I guess we'd have :
$G^{(1)} = \{1, 2, 3\}$ corresponding to the first part of the function (\$10,000 per unit).
$G^{(2)} = \{4, 5, 6\}$ corresponding to the second part of the function (\$8,000 per unit).
I don't get what are $\{1, 2, 3\}$ and $\{4, 5, 6\}$ standing for, concretely
If I apply the maxout function for each group, I guess it means that maxout unit 1 will select the maximum of the values associated with $G^{(1)}$, and maxout unit 2 will select the maximum of the values associated with $G^{(2)}$, and so on ... but then it doesn't make any sens
Indeed, for 150 units, I guess maxout 1 would give a value of \$10,000 per unit, and maxout 2 would give a value of \$10,000 per unit, ... I don't know how I am expected to calculate the total cost of production then ...
I am a slow but passionate learner in math and deep learning. Don't hesitate to explain it to me as if I was a teenager.


text:
Large value predictions using Physics informed neural network (PINN) for inverse problems
So i am stuck forever in this problem where we have to predict two variables values in an ODE equation using the given data points. The problem is everything works perfectly when the values of the variables to be predicted is very small (upto range of 10). But as soon as the values to be predicted is large (assume in 100s) it breaks. It predicts everything as 0. But the loss (data loss + PDE loss) is still very less. I tried adaptive weight method, curriculum learning, sequential learning and everything in between. Still no results. I know the landscape becomes very sophisticated at large values but i want to know the solution for it?

Can u precise the ODE you want to solve, your Neural Network architecture and loss to be minimized?


text:
Choosing the most appropriate neural network type
I want to use deep learning to improve cache hit rate. Let me explain a few basic principles of the cache.
Each cache entry can have one of two different priorities. The priorities decide when that cache line is evicted(lower priority first). In general, a cache should aim to have a hit rate as high as possible.
Currently the priorities are assigned statically according to the traffic type. And all entries belonging to the same traffic type would have the same priority.
Based on experiments I have conducted, this is not the best algorithm for highest hit rate. Cache hit rate improves if the prioroties are set based on the access pattern.
For example, suppose address 100 belonging to traffic type A is accessed for a certain period of time, and then other addresses belonging to traffic type A are accessed, and 100 is accessed again. Due to the intervening accesses to other addresses between the 2 two periods of accesses to 100, the entry corresponding to 100 gets evicted and needs to be refetched again. However, if 100 was assigned a higher priority, it would not be evicted and cache hit rate would be higher.
This is a naive example just to give an idea of what the problem statement is. The actual traffic pattern is very complex and it is generally very hard to manually pick which addresses to give a high priority. I hope to achieve this with a deep learning network.
For the moment, we can assume that the access pattern is known before hand. The goal is to initially feed the access pattern to the deep learning network. And then when the accesses are actually made, the network assigns the most appropriate priorities to the incoming requests and this should improve the cache hit rate.
What kind of of a neural network best suites this requirement? For training the network, I don't have any cues except the cache hit rate - a hit rate higher than baseline indicates that the network is performing well.
I don't have much experience in neural networks. Any leads related to the class of neural networks that can solve such a problem and details related to how it will be trained are much appreciated.


text:
Are the dimensions in embedding vectors ordered (similar to PCA)?
I am getting started with the vector embeddings. I have a general question about the embedding vectors generated by popular algorithms.
In PCA, usually, there is an implicit order of importance in the dimensions, with the most informative (by largest Eigen value) dimension first and the least at the end. Is there a similar property with vector embeddings generated by various models like Sentence transformers, OpenAI Embedding API, Google PaLM models etc?


text:
Difference between using transformer for multi-class classification and clustering using last hidden layer
My data is a collection of URLs where I am interested in categorizing them into multiple groups.
At the moment, I am using a pre-trained transformer model and fine-tuning it according to my data with manual labeling.
My dataset can have undiscovered classes that I did not capture while labeling and also the future data also can have new categories. I want to identify the these different categories.
I am confused on what is the difference between "fine tuning a transformer model for multi-class classification and use it with a classification header for multi-class classification on new data" and "fine tuning a transformer model for multi-class classification and then extract the last hidden layer embeddings and perform clustering on them".
At the moment I am fine-tuning the model with 6 classes. Then pass through the new data on the trained model and extract the last hidden layer embeddings and cluster them.
The elbow for clustering always show 6 clusters.
Is that because there is actually only 6 categories, or am I interfering the clustering process by using a fine-tuned model for 6 classes?
Thanks
Matheesha


text:
Understanding Test Output Calculation in DropConnect
I've been studying the DropConnect regularization technique for neural networks and I'm trying to understand how the test output is calculated. I understand that during training, DropConnect randomly sets a subset of weights to zero. However, I'm unclear on how this translates to the test phase.
Could someone please explain the process of calculating the test output in DropConnect? I would appreciate if you could provide a detailed explanation and, if possible, refer to any relevant sources or literature for further reading.
Thank you in advance for your help.

Can you provide a link to the paper or resource you're referring to? Thanks

Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking.


text:
Consider an AI Image generation system AI IMAGe. How hard is it to remove specific images from the training dataset and all traces of the images?
Consider an AI image generation system called AI IMAGE. Suppose someone wanted to remove 100 of their photos that had been used to train AI IMAGE for a year in generating images. Would it be easy to remove all traces of the photos from the AI system?
How hard or difficult might this be? Links to academic papers or case studies on this would be great! Thank you!

Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer.


text:
MADDPG: reward drops fast after updating ac network
I'm using MADDPG to do computation offloading. But the reward drops fast once I start sampling from the replay buffer.

class critic(abstract_agent):

    def __init__(self, obs_shape, act_shape):
        super().__init__()
        self.LRelu = nn.LeakyReLU(0.01)
        self.linear_c1 = nn.Linear(act_shape + obs_shape, 64)
        self.linear_c2 = nn.Linear(64, 64)
        self.linear_c = nn.Linear(64, 1)

    def forward(self, obs_input, act_input):
        x_cat = self.LRelu(self.linear_c1(torch.cat([obs_input, act_input], dim=1)))
        x = self.LRelu(self.linear_c2(x_cat))
        x = self.linear_c(x)

        return x

class actor(abstract_agent):

    def __init__(self, num_input, action_size):
        super().__init__()
        self.tanh = nn.Tanh()
        # self.LRelu = nn.LeakyReLU(0.01)
        self.LRelu = nn.ReLU6()
        self.linear_a1 = nn.Linear(num_input, 64)
        self.linear_a2 = nn.Linear(64, 64)
        self.linear_a = nn.Linear(64, action_size)

        self.action_bound = 1.0

    def forward(self, x, model_original_out=False):
        x = self.LRelu(self.linear_a1(x))
        x = self.LRelu(self.linear_a2(x))
        model_out = self.linear_a(x)

Below is the network update code:
    def agents_train(self, game_step, update_cnt, memory, obs_size, action_size,
                     actors_cur, actors_tar, critics_cur, critics_tar, optimizers_a, optimizers_c, write):
        if (game_step > learning_start_step) and (game_step % learning_fre == 0):
            if update_cnt == 0: print('\r=start training...' + ''*100)
            update_cnt += 1

            for agent_idx, (actor_c, actor_t, critic_c, critic_t, opt_a, opt_c) in enumerate(zip(actors_cur, actors_tar, critics_cur, critics_tar, optimizers_a, optimizers_c)):
                if opt_c == None: continue

                rew = []
                obs, action, reward, obs_ = memory.sample(batch_size, agent_idx)

                for i in range(batch_size):
                    r = reward[i]
                    ar = sum(r)/len(r)
                    rew.append(ar)
                # update critic
                # rew = torch.tensor(reward, dtype=torch.float)
                rew = torch.tensor(rew, dtype=torch.float)
                action_cur = torch.from_numpy(action).to(torch.float)
                obs_n = torch.from_numpy(obs).to(torch.float)
                obs_n_ = torch.from_numpy(obs_).to(torch.float)
                action_tar = torch.cat([a_t(obs_n_[:, obs_size[idx][0]:obs_size[idx][1]]).detach() for idx, a_t in enumerate(actors_tar)], dim=1)
                q = critic_c(obs_n, action_cur).reshape(-1)     # q
                q_ = critic_t(obs_n_, action_tar).reshape(-1)   # q_
                tar_value = q_ * gamma + rew
                loss_c = torch.nn.MSELoss()(q, tar_value)
                opt_c.zero_grad()
                loss_c.backward()
                nn.utils.clip_grad_norm_(critic_c.parameters(), max_grad_norm)
                opt_c.step()

                # update Actor
                # There is no need to cal other agent's action
                model_out, policy_c_new = actor_c(
                    obs_n[:, obs_size[agent_idx][0]:obs_size[agent_idx][1]], model_original_out=True)
                # update the action of this agent
                action_cur[:, action_size[agent_idx][0]:action_size[agent_idx][1]] = policy_c_new
                # loss_pse = torch.mean(torch.pow(model_out, 2))
                loss_a = torch.mul(-1, torch.mean(critic_c(obs_n, action_cur)))

                opt_a.zero_grad()
                # loss_t = 1e-3 * loss_pse + loss_a
                loss_t = loss_a
                loss_t.backward()
                nn.utils.clip_grad_norm_(actor_c.parameters(), max_grad_norm)
                opt_a.step()

Here are my train parameters:
batch_size = 64
gamma = 0.9
lr = 0.01
memory_size = 1024
EPOCH = 1000
STEP = 100

Besides, every time I try to increase the batch_size or lr, the actions all change to 'nan'.
Does anybody face the same issue or know the reason?

reward too big making the gradient too big, too? i just skimmed the codes. try printing the grad before optimizer step, for each epoch. check if grad become Nan too while you're at it. If so, maybe try different reward function with smaller scale?

Since I need to minimize time consumption, the reward is set as allowed:
            if energy_cost > ED_energy:
                energy_penealty = -1
            else:
                energy_penealty = 1
            if time_cost > time_limit:
                time_penalty = -2
            else:
                time_penalty = 1
            reward = min(1/(time_cost + 1), time_penalty, energy_penealty)

And I try to print the grad, the result is none.

wait, why does the loss just blowing up then? to the scale of 1e+30? make sure nothings wrong with the calculation..
have you backward() before printing the grad?

sorry, I checked the grad log again, this is part of the print: grad=tensor([[ 1.4188e-06,  1.5014e-06, -1.9357e-04, -1.0162e-04,  1.5669e-06,
-5.2142e-05,  6.3059e-07,  6.9160e-07, -2.0823e-04, -1.3216e-04

check what happens to the grads when the scale of the loss gets really big


text:
How to calculate CIoU or DIoU loss only for certain unmasked boxes in tensor and ignore the masked values?
# bbox loss
bbox_labels = batch['bbox'][:, 1:]
bbox_masks = batch['bbox_mask'][:, 1:]

masked_bbox_preds = bbox_preds*bbox_masks
masked_bbox_labels = bbox_labels*bbox_masks

if self.config.bbox_loss == "smoothl1":
    box_loss = self.bbox_loss(masked_bbox_preds, masked_bbox_labels)
elif self.config.bbox_loss == "diou":
    masked_bbox_preds_xyxy = ops.box_convert(masked_bbox_preds, 'cxcywh', 'xyxy')
    masked_bbox_labels_xyxy = ops.box_convert(masked_bbox_labels, 'cxcywh', 'xyxy')
    box_loss = ops.distance_box_iou_loss(masked_bbox_preds_xyxy, masked_bbox_labels_xyxy, reduction='sum')
    bbox_masks_inv = (bbox_masks + 1) % 2
    box_loss -= bbox_masks_inv.sum()

box_loss = box_loss / (bbox_masks.sum() + self.eps)

This is a Transformer Decoder model in which one branch of the Decoder predicts the HTML structure sequence for a table image and the other predicts the bbox for corresponding cell tokens ('< td >', '< td >< /td >', '<td'). So, for all the other tokens, I have to mask those predictions and calculate loss only for the td tokens.
In this, DIoU gives a value of 1.0 for boxes which are [0.0,0.0,0.0,0.0]. So, I subtract the number of masked boxes from the total loss to keep it consistent however, the model is predicting random boxes in places where it should predict properly. I'm not sure if this approach is correct in terms of weights/gradient updates. Is there any other way to use DIoU/CIoU with masks?
CIoU straight away gives nan for [0.0,0.0,0.0,0.0] because it considers the aspect ratio also, so I couldn't figure out how to fix that.


text:
How far is AI from making movies just by using script or description like we use for generating images in midjourney or DALL-E etc
I'm curious about the current state of AI technology when it comes to generating movies from textual descriptions. I'm aware of impressive advancements in generating images using models like Midjourney and DALL-E, but I'm wondering if similar progress has been made in the domain of movie production.
Specifically, I'm interested to know if there are existing technologies that can take a script or a detailed description and generate a full-length movie, including visuals, audio, and potentially even dialogues. If such technology exists, I would appreciate any insights into its capabilities and limitations.
Alternatively, if there are no known solutions that can accomplish this task, I would like to understand the current state of research and development in this area. Are there any ongoing projects or promising approaches that are bringing us closer to the goal of generating movies solely from text inputs?

Very very far. Even image generation requires a lot of try or tunes to get a good result when you expect something specific (less now with the use of ControlNet however). In order to make a movie, it's more realistic that some AI would be able to make really small videos of a few seconds or minutes and you will have to design hundreds of them and concatenate them to make your film. You would also need different AI models for the audio etc.


text:
Computing confidence score for transformer output
I'm sampling multiple generations from a transformer model and I would like to have a confidence score for each generation. The Hugging Face library's generate() method can return per-token logits if output_scores=True is set. What is the right way to combine these per-token logits to get an overall score?
My first thought was to simply add them up. Since they are negative numbers, this causes the confidence score to favor shorter outputs. Is there a score that is not biased by output length?

probably softmax and then perplexity?


text:
How to identify location from a given input in a semantic way
I have a sample use case, where user will give us a xls file with some location data. Where may be location information is given in different ways , sometime its IATA code, sometimes its standard abbreviation, sometimes they use own abbreviation or sometime just a short name of that location too and it can be in any format.
Now job of my application will take XLS as input and convert it my standard. By “My Standard” I mean to say I have a complete data set with almost all location in my database.
So, In simple words it will map user given input to my standard location name, which stored in my DB.
My model should have that capability also , where may be , if for any location it not able to map with any data in my location DB , I will train the model with that data. My "train" I mean to say I will map that user input with any location of master data.
Next time onward, if model receive same input, it should be able to map it with my master data, as per my last training.
Sample User Input and inference for location:

Can any one suggest any suitable solution this use case , Just a outline of though would be very helpful and will be appreciated.
I tried with fuzzywuzz to solve this problem, But that did not help me much to solve the use case completely.

If something is not in your database, how can you then know what the nearest known place is?

This sounds more like a database lookup operation or data cleaning to me than ML.

@emely_pi , Thanks for your reply ...  I presume no , May be I am not able to articulate the issue. In database search how will you map Pimpri to pune or HCM City to Ho Chi Minh City? every time it would be same , for other user it can be Ho C M City also , as I mentioned in the post "sometime its IATA code, sometimes its standard abbreviation, sometimes they use own abbreviation or sometime just a short name of that location too and it can be in any format"

@OliverMason , by DB I mean say my application master DB.

@sujoy If a place is not in your DB, you do not know where it is, and thus you don't know what the nearest place in your DB is. In your example it is Pimpri, but what about Ghljhwkdjgj? To a computer that is equally unknown.

@OliverMason , by database in this case I mean to , application limited scope database. Not the database used for location Search process. Obviously some where some how we have to map this near location of Pimpri is pune.


text:
Justification for the MCTS backpropagation rule?
In Monte Carlo Tree Search, we back-propagate the results of a simulation with reward $R$ to a parent node N like this
$$
V_N \leftarrow \frac{\text{visits}(N)\cdot V_N + R}{\text{visits}(N)+1}
$$
and $V_N$ is our first term in a selector like UCT.
I was looking at alternative MCTS selectors and realised this update rule is not so straightforward as I thought. With it, what we end up with is (for non leaf nodes)
$$
V_N = \sum_{N' \in \text{Children}(N)} \frac{V(N')\cdot\text{visits}(N')}{\text{visits}(N)}
$$
So $V_N$ seems to be the expectation of reward from $N$'s state, if the probability of us ending up at a child node is proportional to the number of times we visited it during tree expansion.
This is completely true if for instance, in the future we acted based on MCTS and did a random number of iterations $\leq \text{visits}(N)$ with $N$ as a root. With a fixed number of iterations though, we would always being doing at least $\geq \text{visits}(N)$ iterations. This implies to me that the child with the most visits (in the present) is overwhelmingly more likely to end up as the actual selected action, than is suggested by our $V_N$ formula.
Sorry if that last part wasn't clear. My point is that the $V_N$ estimate seems "off-policy". So it assumes we're going to act in a non-MCTS way. This is somewhat at odds with the case in MCTS where all children are leaves, and so their values are the actual simulation reward expectations.
Since final move selection is based on visit counts and not Q value, this probably doesn't affect the correctness of MCTS much. I wanted to ask if anyone knows if there is a better description of what $V(N)$ actually is, or if there is a good mathematical justification I am not seeing?


text:
Is regression method the best for my case?
newbie here. I'm starting to work on a custom model for a very specific task, so I found no pre-trained models for this task so far.
After checking (un)supervised learning approaches I believe that regression will work the best, but could you please take a look and say if there's something better around?
The data is like that:
input1(int) input2(float) input3(int) => output(int)

The set of [inp1, inp2, inp3] is unique and can generate only one output. There are always two and only two such sets that generate the same output (like [2, 5.0, 3] => 0 and [12, -3.0, 1] =>0 too). It's always three inputs in a set (no less, no more), I see no reason to consider there's anything like a label, output varies from +100million to -100 million or even more in the future.
There will be no texts, the data is not time-correlated. The full dataset size is a few hundred million of such sets, if it matters.
So, is regression the best choice here? Any advice on this task for beginner?
Thank you.

What is the meaning of the output ? Is there some ordinal order ? Like is is better to predict $2$ or $58$ when you expect $3$  ? If it is classification both are usually as bad (like predicting horse or dog instead of pandas, both errors are as bad) while regression is sensible to numerics.

Classification models predict a label. In your case that may be a whole number (e.g. 0, 1, 2, ...). Regression models predict a value. An important question is: How many possible outputs do you expect to have? If it is limited and the values are whole numbers you could treat the problem as a classification problem.

@Lelouch, Brian, it is hardly classification because output varies from +100million to -100 million or even more in the future. And yes, getting 58 is much worse than 2 if we expect 3.

So its probably regression then. Try the simplest model and increase the complexity if needed.

@BrianO'Donnell What you said makes no sense, classification predicts discrete values, regression predicts continuous values, value and label have no interesting meaning in your comment.

Predicting integers is classification, not regression, as integers are discrete values.

@Dr Snoopy you confirmed what I said: Predicting integers is classification. I said that the labels can be the whole numbers, i.e. integers.

@BrianO'Donnell Labels can be continuous or discrete, being a label does not imply classification.


text:
Why was Tacotron trained on <1000h of speech data?
Tacotron TTS models (e.g. Tacotron 2 and Parallel Tacotron 2) were trained on 25h and 405h of speech data respectively. By comparison, more recent TTS systems are trained on >50,000h of speech data. Why were Tacotron models trained on such a relatively small volume of data?

Could the answer just be when they were trained? Bigger models and input data sets require more resources. So this may follow a pattern analogous to Moore's law.


text:
Should the norm of samples from an untrained denoising diffusion probabilistic model scale with number of diffusion steps?
I have implemented a diffusion probabilistic model for the first time, and I am finding some of the model behavior undesirable, but I am not sure if it is normal or the result of an incorrect implementation.
Specifically, when I draw samples from an untrained reverse diffusion process with 20 denoising steps (by sampling from a standard normal distribution and passing the sample sequentially through the denoising kernel), I find that the elements of my final denoised samples have an average absolute value of around 400 (this number increases into the 1000s if I reverse diffuse for more steps). My training data is more like order 1 magnitude (although I am training the model to predict the noise not the noiseless training data), and so this seems problematic for efficiently training a model to me.
Even though the behaviour seems undesirable, it also seems expected from the math. For example, $x_t = \sqrt{\bar{\alpha}_t}x_0 + \sqrt{(1-\bar{\alpha}_t)}\epsilon$, and so $x_0 = \frac{1}{\sqrt{\bar{\alpha}_t}}(x_t - \sqrt{(1-\bar{\alpha}_t)}\epsilon)$, where $x_0$ is the end result of denoising, $\bar{\alpha}_t = \prod_{t=1}^{T}\alpha_t$, $\alpha_t = 1 - \beta_t$, and I am using a cosine noise schedule for the betas (see code at the bottom). Therefore, $x_0$ scales with $\frac{1}{\sqrt{\bar{\alpha}_t}}$, which is equal to 406 when the number of diffusion steps (and $t$) is 20 and equal to 20291 when the number of diffusion steps (and $t$) is 1000.
Is this behavior normal? It seems problematic for efficiently training a model.
def cosine_beta_schedule(T, s = 0.008):
    """
    cosine schedule as proposed in https://proceedings.mlr.press/v139/nichol21a/nichol21a.pdf
    """
    t = np.linspace(0, T, T + 1)
    alphas_cumprod = np.cos((t / T + s) / (1 + s) * np.pi / 2) ** 2
    alphas_cumprod = alphas_cumprod / alphas_cumprod[0]
    betas = 1 - (alphas_cumprod[1:] / alphas_cumprod[:-1])
    betas_clipped = np.clip(betas, a_min = 0, a_max = 0.999)
    return betas_clipped

import numpy as np
n_diffusion_steps = 20
betas = cosine_beta_schedule(n_diffusion_steps)
alphas = 1. - betas
alphas_cumprod = np.cumprod(alphas)


text:
model versioning and validation of online models
While training an online model, usually we use progressive validation https://maxhalford.github.io/blog/online-learning-evaluation/. My question is what we should do when we detected performance regression. In a traditional offline training setting, we won't promote a model unless it surpasses previous models in validation which is not possible in online setting.
Related to this question, is there a way to version models in online setting? Maybe we just keep a snapshot of the model on a daily basis?


text:
What model should I use to find coordinate locations of objects on an image (not bounding boxes)
I am developing a neural network to count pits on an etched crystal wafer, and I created a dataset by placing points on each pit.
Each line in my annotation file looks like this:
Id, Filepath, Coordinates
965,822.jpg,"[{""x"": 2, ""y"": 439}, {""x"": 50, ""y"": 384}, {""x"": 130, ""y"": 454}, {""x"": 177, ""y"": 396}, {""x"": 261, ""y"": 429}, {""x"": 251, ""y"": 529}, {""x"": 272, ""y"": 351}, {""x"": 475, ""y"": 407}, {""x"": 481, ""y"": 313}, {""x"": 492, ""y"": 231}, {""x"": 511, ""y"": 138}, {""x"": 427, ""y"": 111}, {""x"": 335, ""y"": 98}, {""x"": 489, ""y"": 82}, {""x"": 436, ""y"": 71}, {""x"": 399, ""y"": 56}, {""x"": 366, ""y"": 55}, {""x"": 325, ""y"": 49}, {""x"": 278, ""y"": 44}, {""x"": 237, ""y"": 28}, {""x"": 190, ""y"": 15}, {""x"": 398, ""y"": 15}, {""x"": 421, ""y"": 29}, {""x"": 513, ""y"": 31}, {""x"": 131, ""y"": 17}, {""x"": 54, ""y"": 5}]"
All of the models I've tried using only work with bounding boxes, and I'm not experienced enough to adapt them to work with points.
Is there a model that I can use that works with points by default? Or just an easy way to do this. I'm planning on using pytorch, but I would be open to other implementations.

You can try segmentation and ask to predict a heatmap. Or if you have a fixed number of points on each image, you can try regression of the coordinates directly.


text:
If the agent is at the same state but at different times and receives a different reward, wouldn't this be violating somehow the MDP assumption?
I've been trying to train an agent, I've received and read suggestions to improve its speed to reach the goal. The suggestion is to use a time penalty, for example, adding $-0.1$ to the reward each time step.
However, at a first glance, it seems weird to use it because in the core of all RL algorithms we assume that it is a Markov decision process, so it should be able to choose the best action based on the current observation of the environment alone.
If the agent is at the same state but at different times and receives a different reward, wouldn't this be violating somehow the MDP assumption? Or at least preventing it from learning? Because it has no time parameter to learn anything related to time (i.e this state yielded worst expected return because it is in a later time step)

By "MDP assumption", do you mean the "Markov assumption"? I had asked a related question here. This post that I wrote later should clarify that there can be many reward functions that lead to the same optimal policy for continuing tasks.

Yes with MDP assumption I mean Markov assumption, because I think the agent would need more than the current observations to know the right action to take. My doubt arises because I read a post in towards data science where they trained multiple pistons to move a ball from one end to another, they used this reward $( \Delta X/X_{e}) \cdot 100 + \tau t$. Where $\tau$ is a time penalty of -0.1. To me this seems like the value function would change with time.


text:
Correction of value update in off policy TD(0)
In this question (inherited from slides coming from D. Silver) they are arguing that the value update in off policy TD(0) should be:
$$
V(S_t) = V(S_t) + \alpha \left( \frac{ \pi(A_t|S_t)}{\mu (A_t|S_t)} (R_{t+1} + \gamma V(S_{t+1})) - V(S_t) \right)
$$
And the answer is saying that if $\pi(a|s) = 0$  then this formula will counterbalance the "overestimation"
However, I really don't see this overestimation in the derivation.
Moreover, in the RL book ok Sutton and Barto, in the off-policy td(0) (section 11.1 "Semi-gradient Methods"), they clearly states:

Here the whole error is weighted by the importance sampling ratio, which makes sense in the case $\pi(a|s)=0$, because "if i'll never do that action, my value should not consider that branch of the tree"
Any idea?


text:
neural network not training correctly
i'm trying to learn how a neural network works. i'm writing the neural network in C for handwritten digit recognition and training it on MNIST dataset. the neural network has an input layer with 28*28 neurons, an hidden layer with 50 neurons, and an output layer with 10 neurons. the activation function is sigmoid. weights and biases are initialized randomly between -0.5 and +0.5. i tried implementing forward propagation and back propagation but i think i'm doing something wrong. the training seems not to work well because the accuracy is not that good and sometimes it decrease while training. the learning rate is set to 0.01 and reducing it to 0.001 didn't change much (it had almost the same behavior)
Example:
EPOCH: 0        ACCURACY: 63.180000     CORRECT ANSWERS: 6318
EPOCH: 1        ACCURACY: 63.640000     CORRECT ANSWERS: 6364
EPOCH: 2        ACCURACY: 67.730000     CORRECT ANSWERS: 6773
EPOCH: 3        ACCURACY: 68.530000     CORRECT ANSWERS: 6853
EPOCH: 4        ACCURACY: 71.160000     CORRECT ANSWERS: 7116
EPOCH: 5        ACCURACY: 76.120000     CORRECT ANSWERS: 7612
EPOCH: 6        ACCURACY: 81.080000     CORRECT ANSWERS: 8108
EPOCH: 7        ACCURACY: 81.410000     CORRECT ANSWERS: 8141
EPOCH: 8        ACCURACY: 78.440000     CORRECT ANSWERS: 7844
EPOCH: 9        ACCURACY: 77.910000     CORRECT ANSWERS: 7791
EPOCH: 10       ACCURACY: 82.070000     CORRECT ANSWERS: 8207
Here is the back propagation function:
void BackPropagation(double *input,double *output){
double hidden_gradients[NUMHIDDEN];
double output_gradients[NUMOUTPUTS];

    for(int i=0;i<NUMOUTPUTS;i++)
        output_gradients[i]=(output[i]-output_layer[i])*dSigmoid(output_layer[i]);

    for(int i=0;i<NUMOUTPUTS;i++){
        for(int j=0;j<NUMHIDDEN;j++)
            output_layer_weights[i][j]+=LR*output_gradients[i]*hidden_layer[j];
        output_layer_bias[i]+=LR*output_gradients[i];
    }

    for(int i=0;i<NUMHIDDEN;i++){
        double sum=0.0;
        for(int j=0;j<NUMOUTPUTS;j++)
            sum+=output_gradients[j]*output_layer_weights[j][i];
        hidden_gradients[i]=dSigmoid(hidden_layer[i])*sum;
    }

    for(int i=0;i<NUMHIDDEN;i++){
        for(int j=0;j<NUMINPUTS;j++)
            hidden_layer_weights[i][j]+=LR*hidden_gradients[i]*input[j];
        hidden_layer_bias[i]+=LR*hidden_gradients[i];
    }

}

i'll leave here also the forward propagation function:
void ForwardPropagation(double *input){

    for(int i=0;i<NUMHIDDEN;i++){
        hidden_layer[i]=hidden_layer_bias[i];
        for(int j=0;j<NUMINPUTS;j++)
            hidden_layer[i]+=input[j]*hidden_layer_weights[i][j];
        hidden_layer[i]=sigmoid(hidden_layer[i]);
    }

    for(int i=0;i<NUMOUTPUTS;i++){
        output_layer[i]=output_layer_bias[i];
        for(int j=0;j<NUMHIDDEN;j++)
            output_layer[i]+=hidden_layer[j]*output_layer_weights[i][j];
        output_layer[i]=sigmoid(output_layer[i]);
    }
}

I would be grateful if someone could help me.
Thanks

82% seems a good accuracy, what is the issue ? Just do more epochs .

is it normal that the accuracy decrease sometimes? because I was wondering if my implementation was wrong, because I tried also a version i found online and the accuracy was never decreasing

It can happen as the training is usually done in a stochastic way, and you are not optimizing on a smooth surface either. However it should increase on average otherwise you probably have an error or a learning rate which lead to divergence... Also here you are only given the accuracy, not the loss. The network is only aware of the loss, not accuracy.

ok thanks, I will try with more epochs and I will try to compute the loss

You already have the loss otherwise how could you train it ?

oh yes, it's target_output[i]-output_layer[i] right? that should be only for one input image, the total loss is the sum of the loss of every input?

Usually we look at the mean loss , so that the loss value does not depend on batch size or epoch size

i reviewed the code again and after finding some errors, (for example computing the activations instantly in the forward propagation, instead of when i needed them), i was able to get ~95% accuracy on test inputs and ~96% accuracy on training inputs. thanks again for the help


text:
What do I need to learn to tackle the following problem: make a program that optimizes decisions in the game PlateUp!
So, recently my friends and I have been hooked on a videogame called PlateUp!. The game is kind of a management game where the objective is to succesfully run a restaurant. The game can be roughly described as an endless loop of:

A passive phase, in which you organize your kitchen, tools and other stuff. In this phase you also get to buy, upgrade or duplicate items, and sometimes you're forced to choose a "card" that changes some behaviour in the game.

An active phase, in which customers show up and you must tend to them in real time and make sure they don't get tired of waiting and leave (if a single customer exceeds their impatience meter, it's game over)

My friends and I have been having fun coming up with strategies and optimizations for the game, and thought it'd be fun to try to come up with a program that plays the game and comes up with its own strategies to see what it does (mostly in the planning phase, since the active phase is more concerned with flawless execution of tasks)
So we'd like to make an AI that plays the game that makes no mistakes in the active phase (this sounds like the easy part) but also I'd like to make one that at each turn decides which debuffs to pick, which items to buy, how to organize the kitchen, etc. Its objective would then be to have the longest possible run in the game.
I've never done anything remotely close to this (most I've done related to AI is program a simple MLP and a GHA for some simple learning tasks) but I don't know how these kind of optimization problems can or should be tackled
What should I learn in order to do something like this? Specifically, the optimization part (decisions made in the planning phase) is what I'd like to tackle

reinforcement learning, and if the game is big (many possible states), then deep learning +  reinforcement learning, if instead is relatively small (no idea) you can use evolutionary algorithms


text:
What would be an appropriate way of making an AI for Cell Machine?
The game Cell Machine is absolutely by itself not suitable for AI usage, but a certain niche within a mod of the game (Mystic Mod) known as "vaults" does seem like a suitable use case for AI. It involves making and solving vaults out of cells. It's entirely deterministic. These cells move in predetermined fashions each tick. The creator of a vault's goal is to to protect an enemy cell, and the solver wants to destroy this enemy cell. As the dataset for vaults and their solves is extremely small, this would have to be dealt with using self play. Could a framework like AlphaZero work, or would I have to build something from scratch? Note: Large Language Models surprisingly show some success at creating these vaults, but that would not be an improvable way.


text:
Handling Feature Selection Discrepancy in Image Classification Model
I have developed an image classification model that categorizes images into two classes (we'll say good and bad for the sake of example) based on a set of tags. To improve the model's performance, I performed feature selection, which resulted in a subset of informative features. However, I encountered a challenge when dealing with a new batch of images.
The issue is that some features used in the initial feature selection process are present only in the first dataset, while other features are exclusive to the new batch. To address this discrepancy, I decided to select the features that are common to both sets (the intersection of the two feature sets) and retrain the model using these shared features.
While this approach seems logical to me, I would appreciate additional perspectives and insights on how to handle this situation more effectively. Is selecting the intersection of features a valid approach, or are there alternative methods or considerations that I should be aware of? I am open to suggestions and recommendations to improve the feature selection process and ensure robust classification performance for the new batch of images.
Thank you in advance for your insights and advice!


text:
KL divergence and sign
In the Auto-Encoding Variational Bayes paper, the formula for KL divergence is
$$ \frac{1}{2} \sum \bigl (1 + \log(σ^2) - μ^2 - σ^2 \bigr) \space\space\space\space...(10)$$
, but the equation is
$$- D_{KL}(q(z|x)||p(z)) \space\space\space\space...(7)$$
with a minus sign. why is that?

Does this answer your question? How is this Pytorch expression equivalent to the KL divergence? In particular, see my answer there ;)

https://en.wikipedia.org/wiki/Kullback%E2%80%93Leibler_divergence#Definition as you can see, there is a very simple way to get rid of the minus

I see. I understood. thank you.


text:
What to do with not used variables when optimising a fitness function?
I'm optimising a fitness function using a GA. DNN is used to compute the fitness function using 4 input variables. The original data has 8 variables but to optimise DNN accuracy I have dropped 4 variables. Now the best solution exists of only the 4 parameters used. The complete solution should contain 8. The difference in DNN accuracy is significant, the MSE dropped from around 150 to around 20. I can't use all variables. Should I simply ignore the others? Or use some other method? Please let me know if you have any ideas/suggestions.

it's unlikely that dropping features leads to such an improvement with DNN, you are probably missing something else

@AlbertoSinigaglia why not? Those features may be junks. Otherwise, why do we do feature selection?

You do that with ML models, because they add degrees of freedom... neural network will just ignore them

@AlbertoSinigaglia I have never heard that a vanilla neural network does this, not without dropout/regularization at least. Please quote your source. Neural network is also ML model.

Some past posts on this topic: 1, 2

I really struggle to find anything backed by theory/paper/literature online that shows that FS in NN actually improves performance, but ok (btw, you linked the same question twice)


text:
How to improve CRNN model(CTC loss) accuracy for OCR task?
I take this as baseline model. The main difference on RNN part,
class BidirectionalLSTM(nn.Module):
    # Inputs hidden units Out
    def __init__(self, nIn, nHidden, nOut):
        super(BidirectionalLSTM, self).__init__()
        self.rnn = nn.LSTM(nIn, nHidden, bidirectional=True)
        self.embedding = nn.Linear(nHidden * 2, nOut)

    def forward(self, input):
        recurrent, _ = self.rnn(input)
        T, b, h = recurrent.size()
        t_rec = recurrent.view(T * b, h)
        output = self.embedding(t_rec)  # [T * b, nOut]
        output = output.view(T, b, -1)
        return output

class CRNN(nn.Module):
    ...
    self.cnn = cnn #line 61
    self.rnn = nn.Sequential(
        BidirectionalLSTM(512, nh, nh),
        BidirectionalLSTM(nh, nh, nclass))
    ...

I change it to GRU not LSTM,
self.gru = nn.GRU(512, 256, num_layers=2, batch_first=True,dropout=0.5, bidirectional=True)

After training his dataset, I got 0.96 in accuracy. However, when I traing my dataset, both fine-tuning and retraining achieved an accuracy of only 0.83-0.85.
both of them are consist of Chinese chars, however in his dataset, there are only
only about 10 chars , and all shape is (280, 32), like that,

but there are over 40 chars in my images, the height is between 45 and 55, and the width is between 1700 and 1900.  For instance,

(BTW,there are also some images with a width of 300)
To ensure consistency in size, I first scale them to a height of 32, and then add 0 to 1504 in width behind each image (this is the maximum width of the image)
scale = self.ori_h / 32
img = cv2.resize(img, (math.ceil(img_w*scale), self.ori_h))
img_h, img_w = img.shape
border_w_r = 1504 - img_w
img = cv2.copyMakeBorder(img, 0, 0, 0, border_w_r, cv2.BORDER_CONSTANT, value=255) #add 0 on the right of image
img_h, img_w = img.shape

P.S. learning rate is 1e-4, optimizer is adam, batch size is 64, and dataset 71000 for training while 1000 for testing. Reaching 80 epoch didn't change, actually 0.85 is overfitting.
The problem is how to change the structure of model or other hyperparameters that can improve the accuracy to 0.95?


text:
Batch Normalization Layer is not learning the data semantics of a dataset comprised of datasets from different sources
I have built a dataset for image segmentation that is comprised of datasets from several different sources.
Almost all of my models have problems with learning the correct parameters of the batchnormalization layer, the networks are very deep and it helps a lot keeping them, but if I put them into evaluation mode (i.e. not compute the batch normalization of the input but the learned one) there is a huge drop of performance.
I presume the statistics of the different datasets source images are very different.
The labels might also contain different amount of noise.
I have tried normalizing the images to standard channel mean and standard deviation, centered around 0 and 0.5.
What else could I try?
Many thanks in advance for your advice and insight!
requested snippet:
class ContractingBlock(nn.Module):
    '''
    ContractingBlock Class
    Performs two convolutions followed by a max pool operation.
    Values:
        input_channels: the number of channels to expect from a given input
    '''
    def __init__(self, input_channels, use_dropout=False, use_bn=True):
        super(ContractingBlock, self).__init__()
        self.conv1 = nn.Conv2d(input_channels, input_channels * 2, kernel_size=3, padding=1)
        self.conv2 = nn.Conv2d(input_channels * 2, input_channels * 2, kernel_size=3, padding=1)
        self.activation = nn.LeakyReLU(0.2)
        self.maxpool = nn.MaxPool2d(kernel_size=2, stride=2)
        if use_bn:
            self.batchnorm = nn.BatchNorm2d(input_channels * 2)
        self.use_bn = use_bn
        if use_dropout:
            self.dropout = nn.Dropout()
        self.use_dropout = use_dropout

    def forward(self, x):
        '''
        Function for completing a forward pass of ContractingBlock:
        Given an image tensor, completes a contracting block and returns the transformed tensor.
        Parameters:
            x: image tensor of shape (batch size, channels, height, width)
        '''
        x = self.conv1(x)
        if self.use_bn:
            x = self.batchnorm(x)
        if self.use_dropout:
            x = self.dropout(x)
        x = self.activation(x)
        x = self.conv2(x)
        if self.use_bn:
            x = self.batchnorm(x)
        if self.use_dropout:
            x = self.dropout(x)
        x = self.activation(x)
        x = self.maxpool(x)
        return x

class ExpandingBlock(nn.Module):
    '''
    ExpandingBlock Class:
    Performs an upsampling, a convolution, a concatenation of its two inputs,
    followed by two more convolutions with optional dropout
    Values:
        input_channels: the number of channels to expect from a given input
    '''
    def __init__(self, input_channels, use_dropout=False, use_bn=True):
        super(ExpandingBlock, self).__init__()
        self.upsample = nn.Upsample(scale_factor=2, mode='bilinear', align_corners=True)
        self.conv1 = nn.Conv2d(input_channels, input_channels // 2, kernel_size=2)
        self.conv2 = nn.Conv2d(input_channels, input_channels // 2, kernel_size=3, padding=1)
        self.conv3 = nn.Conv2d(input_channels // 2, input_channels // 2, kernel_size=2, padding=1)
        if use_bn:
            self.batchnorm = nn.BatchNorm2d(input_channels // 2)
        self.use_bn = use_bn
        self.activation = nn.ReLU()
        if use_dropout:
            self.dropout = nn.Dropout()
        self.use_dropout = use_dropout

    def forward(self, x, skip_con_x):
        '''
        Function for completing a forward pass of ExpandingBlock:
        Given an image tensor, completes an expanding block and returns the transformed tensor.
        Parameters:
            x: image tensor of shape (batch size, channels, height, width)
            skip_con_x: the image tensor from the contracting path (from the opposing block of x)
                    for the skip connection
        '''
        x = self.upsample(x)
        x = self.conv1(x)
        skip_con_x = crop(skip_con_x, x.shape)
        x = torch.cat([x, skip_con_x], axis=1)
        x = self.conv2(x)
        if self.use_bn:
            x = self.batchnorm(x)
        if self.use_dropout:
            x = self.dropout(x)
        x = self.activation(x)
        x = self.conv3(x)
        if self.use_bn:
            x = self.batchnorm(x)
        if self.use_dropout:
            x = self.dropout(x)
        x = self.activation(x)
        return x

```

Have you tried a different normalization layer? Also the model is a standard one (e.g. taken from literature) or is just made by you?

Thanks for commentin.
Well it's a UNet architecture, it's made by me, I've tested it on different datasets on similar tasks (segmentation) and it works.
I've tried layernormalization, which works, as in it is consistent, however the performance is much worse than the batchnormalization in training mode.

Alright. So, can you check the statistics of the train and test sets?

@LucaAnzalone yes I can, they are quite different, so I tried standardizing them, are there other better techniques?

I understood you have a dataset made of different datasets, and that standardization is fine. What I mean is that if the overall statistics (i.e., mean and variance over all datasets) of the train are different from the test set, then BN will cause the model to perform poorly. So, if you keep updating the BN's parameters during test, the BN layers would adapt to that statistics improving performance although introducing a dependency on the test, which is unwanted. Therefore, do you split each dataset randomly to determine the test and train?

@LucaAnzalone I, see what you mean.
Yes this makes sense.
So, no, the training dataset is a collection of different datasets, and my test and evaluation dataset are a small subset of the small dataset I want to perform on.
So if I have Datasets $D_1, D_2, D*$, I split $D*$ into train, test and eval.
And I create a training set: $D_1 + D_2 + D*_{train}$ and I evaluate and test respectively on the $D*$ splits.
If the parameters are fixed from the training set, they basically predict all 0's on the eval/test of D*, if in train mode, i.e. normalize to the batch of test, it works very well.

Is $D_*$ much smaller than $D_1 + D_2$? And also, just to check, when you standardize the data do you consider the overall stats of $D_1+D_2+D_{*train}$ or standardize each $D_i$ individually?

@LucaAnzalone thanks. I standardize just to either zero mean and standard deviation or (0.5) mean and standard deviation for all across each channel.
$D_*$ is much much smaller about 200x so.

Ok, so it can be an imbalance problem since $D_$ is much smaller. Have you tried data augmentation only on $D_$ and/or oversampling it?

Thanks @LucaAnzalone yes, I'm oversampling it such that $D_*$ is about as large as D_1 + D_2, I also tried smaller multiples.
I use augmentation as well.

It seems that you're doing right.. maybe there is a bug in your code. Could you post some relevant snippets?

@LucaAnzalone Sure, I've edited my post to include the snippet.

Thanks. I notice that you use the same BN layer multiple times in the forward() functions. Can you try to define multiple BN layers and use each of them only once? I think the same layer to receive different statistics and may get confused during training.

sure I'll try that thanks, this makes sense.


text:
Does anyone recognize this formula to quantify the likelihood that a transformer will generate the same response twice?
The idea is simple enough. Just multiply the likelihood of filling in the blank with the same result as the original response.
$$\prod_{s:substring}^{t:string}P(t|masked(t,s))$$
Motivation:
Rather than estimating P(response|prompt) directly, it may also be useful to estimate confidence values for masked substrings. This helps identify regions of high confidence "The color of the sky is [?]." vs regions of low confidence "The color of the [?] is blue."


text:
Is there any available assessment of relative capabilities of Claude-2 and GPT-4?
My first experience with Claude-2 indicates that it has capabilities well comparable to GPT-4 and noticably above those of GPT-3.5.
There is an AI models ladder but it currently does not include Claude-2. In the ladder Claude-1 is rated above GPT-3.5 but below GPT-4.


text:
Does anyone know about a reference where someone has aggregated the cost optimization strategies for deploying LLMs?
I am looking for a source where someone has mentioned the most commonly used strategies and optimisations to deploy LLMs on consumer hardware. I have read about layer offloading, quantisation methods using libraries like GPTQ and GGML, model pruning and distillation. I would love to know if people know about good blogs for this type of information. Lillian weng's blog seems to be one of the good one's that I know of.

Could you please add a link to Lillian Weng's blog for ease of access?


text:
What are the advantages of using RL over classical approaches in problems such as collision avoidance and path planning?
What are the advantages of going for RL based approaches than classical approaches (geometric for example) in problems such as collision avoidance and path planning?
Are RL based solutions better for adhoc replanning due to its faster execution than classical planning algorithms?
Also, if RL is supposed to be generalized solution, isn't it still based on underlying environment dynamics/map and cannot be applied to a different albeit somewhat similar environment?


text:
How to use diffusion models to draw on top of image?
I am thinking about a pipeline that involves multiple diffusion networks. The first one generates an image from a prompt, a use case that is well-documented in literature. The following networks however should draw on top of the output image from step 1, without altering that image (except for the overdrawings of course), so they would be img2img diffusion networks, conditioned on the input image and with the hard restriction not to alter that input image.
A simple example use case would be a first diffusion model that draws an anime character from a prompt, and a second one that draws the outline of the anime character a lot thicker, and a third one that draws evil-looking eyes on top of the real eyes.
Is there any research as to how to achieve this?

You have a problem defining "without altering that image" as a "hard restriction", whilst simultaneously describing an alteration to the image. What you seem to be asking for is something that does alter an image, but only in a specific way that you intended. Perhaps you could expand on how those "only change this aspect" intentions will be supplied to the model?


text:
Less mutation rate is performing better in bigger neural network
I have a genetic AI neural network that evolves every generation and can add or remove neurons and change weights.
It evolves good in first generations with mutation rate probability of e.g. 0.005 per gene (with total 100 genes). The problem is when the neural network gets bigger (200 genes), I must decrease mutation rate. Otherwise it will be like a random search. (By gene I mean a connection between two neurons)
My question is how can I formulate a dynamic mutation rate?
Why does it become random search when genes are increased? Is having a probability based mutation rate wrong? For example should it be an static number instead of a probability or any other way?


text:
What are the *non-cost-related* reasons RNN+Attention underperform Transformers?
There are obvious trainability and performance challenges with RNNs, such as having to process in serial and BPTT. But let's say we magically had an "optimal" set of weights for the RNN + Attention, as well as for the Transformer. Assume as many things about the architecture as possible are held equal between the two models. Would the inference of the RNN + Attention still be worse than that of the Transformer?
This post hints at yes; in the original "Attention Is All You Need" paper, the Transformer outperformed not only in cost but also in BLEU. It seems like they experimented to a degree where one could say it's not the optimization process / BPTT that failed in the RNN + Attention models, but truly when the model was saturated the performance still was worse.
Why is this?
Intuitively I feel like the set of programs that can be expressed as "stateful" programs is a strict superset of the set of programs that can be expressed as "stateless" programs, such that the only disadvantage would be that it's harder to find the optimal "stateful" program.


text:
In DDPM model, why authors subtract the largest value along j dimension of q*k values matrices?
I have a question about a part of DDPM model code.
Below is the Attention class of DDPM model from this website. My questions is at the bottom.
class Attention(nn.Module):
    def __init__(self, dim, heads=4, dim_head=32):
        super().__init__()
        self.scale = dim_head**-0.5
        self.heads = heads
        hidden_dim = dim_head * heads
        self.to_qkv = nn.Conv2d(dim, hidden_dim * 3, 1, bias=False)
        self.to_out = nn.Conv2d(hidden_dim, dim, 1)

    def forward(self, x):
        b, c, h, w = x.shape
        qkv = self.to_qkv(x).chunk(3, dim=1)
        q, k, v = map(
            lambda t: rearrange(t, "b (h c) x y -> b h c (x y)", h=self.heads), qkv
        )
        q = q * self.scale

        sim = einsum("b h d i, b h d j -> b h i j", q, k)
        sim = sim - sim.amax(dim=-1, keepdim=True).detach()
        attn = sim.softmax(dim=-1)

        out = einsum("b h i j, b h d j -> b h i d", attn, v)
        out = rearrange(out, "b h (x y) d -> b (h d) x y", x=h, y=w)
        return self.to_out(out)

class LinearAttention(nn.Module):
    def __init__(self, dim, heads=4, dim_head=32):
        super().__init__()
        self.scale = dim_head**-0.5
        self.heads = heads
        hidden_dim = dim_head * heads
        self.to_qkv = nn.Conv2d(dim, hidden_dim * 3, 1, bias=False)

        self.to_out = nn.Sequential(nn.Conv2d(hidden_dim, dim, 1),
                                    nn.GroupNorm(1, dim))

    def forward(self, x):
        b, c, h, w = x.shape
        qkv = self.to_qkv(x).chunk(3, dim=1)
        q, k, v = map(
            lambda t: rearrange(t, "b (h c) x y -> b h c (x y)", h=self.heads), qkv
        )

        q = q.softmax(dim=-2)
        k = k.softmax(dim=-1)

        q = q * self.scale
        context = torch.einsum("b h d n, b h e n -> b h d e", k, v)

        out = torch.einsum("b h d e, b h d n -> b h e n", context, q)
        out = rearrange(out, "b h c (x y) -> b (h c) x y", h=self.heads, x=h, y=w)
        return self.to_out(out)

In def forward method, I know what this line does but don't understand why they do this:
sim = sim - sim.amax(dim=-1, keepdim=True).detach()

Why do they subtract sim.amax(*) values from sim?
Is this DDPM specific thing? I don't see this in the original Transformer.
I'm also referring to this website that explains Transformer.
Appreciate your help!


text:
RetNet: Why Diagonalize A Square Matrix?
The paragraph before eq. (3) in The RetNet paper says to diagonalize the $R^{d{\times}d}$ matrix $A$. I am confused as to why one would not just use a vector?
One of the arguments that I have heard on why you would diagonalize a matrix is that they are easy for exponentiation, but I am not sure if that really is the reason here.


text:
Can we naively merge source and target datasets to train for the same task instead of performing domain adaptation?
I have seen from literature that models such as DANN or ADDA are typical in the field of Domain Adaptation, a branch of transductive learning. I know that these methods are extremely useful especially when we try to perform the same tasks as for source on the unlabelled targets.
One question that rises to my mind is if we have target and source domains both with labels, other than doing adaptation, is it possible that we naively merge the two dataset and perform the normal supervised learning setup?
My first thought was that DANN's purpose lies on trying to extract a somewhat universal feature that can generalize over the whole scope of domain. If we just naively merge the dataset, the model might be able to classify the data's originating domain to perform the specified task. In other words, it can internally classify the domain and source looking at the feature vector and do a prediction.
However suppose we have an MNIST and USPS dataset being the source and target domain. We can make a DANN network to make a generalized feature vector, which indeed seems logically convincing. But after all, our goal of classifying between the number labels is based on the qualitative assessment of features derived from the image. How can we really tell that classifying one's domain and then performing their task is something different from the methodology just stated. Then why bother performing adversarial training?
I am definitely new to this field, so I may have a lot of non-standard terminology or some errors in my reasoning. Thank you very much in advance.


text:
Train/Validation/Test split in Zero-Shot learning?
Is it recommended to do test/validation epochs in zero shot learning tasks? how can one split the unseen and seen classes in that case?
Can the validation set be a mixture of the two, under the assumption that we do not update the weights in validation steps?
If so, how can it help? would it be OK to use such thing to (for instance) observe overfitting and to stop the training process at a different phase?


text:
Model architecture object detection where entire image context and nearby objects are important to accurate prediction
I'm working on a model to classify multiple extremely similar looking objects in a single image. A simple object detection model works ok but the issue is that I'm classifying similar looking object from aerial imagery. Think houses in a neighborhood that you want to match to an address. So the way a person would classify would be to pull up a satellite image and use it as a reference and learn the spatial relationships between the various houses to label them. I'm wondering if there are better models out there that will take the entire image context and the spatial relationships of the objects in the image in account when making a prediction. I can easily create an object detection model to classify the universe of objects (houses) first and then do a custom image classification model second. I was considering an approach like this: https://arxiv.org/ftp/arxiv/papers/1812/1812.00291.pdf
Are there any other techniques I should be looking at? Since the relationships between these objects is so consistent it seems like a problem that should be very easy to solve with ML.


text:
how to approach this problem (open set classification of diverse folder of images)
suppose we are presented with a folder of images. the task is just detect if any new image should belong in this folder or not.
in this folder, there may be natural groups of images that are similar to each other, but the groups themselves are very different from each other and is probably ineffective to be generalized under one class.
for example maybe we have images of sports cars, then images of dogs. The images in the folder are not divided in any way. there are no labels for any of these images. except that whatever is in the folder is what we want to look for from new images.
how do we approach this problem in the most efficient and effective way?
one approach i thought of was perhaps train a deep clustering model, and also separately train a few shot open set recognition model. divide the folder into N classes, then take K samples from each of the classes in the folder and check if a new image should belong to any of these. having multiple steps like this using separately trained models may accumulate too much error in the end. any other suggestions?


text:
How to make better modelling with less data?
I was working on modelling with less data which is the one of the most problem on DL. So i learned somethings about to deal with it. The model was always overfitting. And i learned Data Augmentation.
Data
train_dir = r"C:\Users\haytr\OneDrive\Masaüstü\yemekler\train"
test_dir = r"C:\Users\haytr\OneDrive\Masaüstü\yemekler\test"

train_data = ImageDataGenerator(rescale = 1./255,
                               rotation_range = 40,
                               width_shift_range = 0.2,
                               height_shift_range = 0.2,
                               shear_range = 0.2,
                               zoom_range = 0.2,
                               horizontal_flip = True,
                               fill_mode = 'nearest')

test_data = ImageDataGenerator(rescale = 1./255)

train_data = train_data.flow_from_directory(train_dir,
                                            target_size = (128, 128),
                                            batch_size = 50,
                                            class_mode = 'categorical')

test_data = test_data.flow_from_directory(test_dir,
                                          target_size = (128, 128),
                                          batch_size = 50,
                                          class_mode = 'categorical')
Found 2500 images belonging to 10 classes.
Found 750 images belonging to 10 classes.

There is 250 images per a file about foods in train_data and 75 images per a file in test_data.
This way is solved my overfitting problem.
Model
model = Sequential()
model.add(layers.Conv2D(filters = 16, kernel_size = (3, 3), activation = 'relu', input_shape = (128, 128, 3)))
model.add(layers.MaxPooling2D((2, 2)))

model.add(layers.Conv2D(filters = 32, kernel_size = (3, 3), activation = 'relu'))
model.add(layers.MaxPooling2D((2, 2)))

model.add(layers.Conv2D(filters = 64, kernel_size = (3, 3), activation = 'relu'))
model.add(layers.MaxPooling2D((2, 2)))

model.add(layers.Conv2D(filters = 128, kernel_size = (3, 3), activation = 'relu'))

model.add(layers.Flatten())

model.add(layers.Dense(128, activation = 'relu'))
model.add(layers.Dense(10, activation = 'softmax'))

model.compile(loss = tf.keras.losses.CategoricalCrossentropy(),
              optimizer = tf.keras.optimizers.Adam(),
              metrics = ["accuracy"])

model.fit(train_data, validation_data = test_data, epochs = 10)

Training
Epoch 1/10
50/50 [==============================] - 14s 261ms/step - loss: 2.2492 - accuracy: 0.1688 - val_loss: 2.1461 - val_accuracy: 0.2133
Epoch 2/10
50/50 [==============================] - 13s 262ms/step - loss: 2.1402 - accuracy: 0.2096 - val_loss: 2.0766 - val_accuracy: 0.2427
Epoch 3/10
50/50 [==============================] - 13s 262ms/step - loss: 2.0728 - accuracy: 0.2488 - val_loss: 2.0440 - val_accuracy: 0.2747
Epoch 4/10
50/50 [==============================] - 13s 264ms/step - loss: 2.0270 - accuracy: 0.2692 - val_loss: 2.0222 - val_accuracy: 0.2707
Epoch 5/10
50/50 [==============================] - 13s 258ms/step - loss: 1.9880 - accuracy: 0.2828 - val_loss: 1.9479 - val_accuracy: 0.3187
Epoch 6/10
50/50 [==============================] - 13s 258ms/step - loss: 1.9160 - accuracy: 0.3332 - val_loss: 1.9857 - val_accuracy: 0.3080
Epoch 7/10
50/50 [==============================] - 13s 264ms/step - loss: 1.8774 - accuracy: 0.3420 - val_loss: 1.9064 - val_accuracy: 0.3253
Epoch 8/10
50/50 [==============================] - 13s 257ms/step - loss: 1.8684 - accuracy: 0.3396 - val_loss: 1.8923 - val_accuracy: 0.3173
Epoch 9/10
50/50 [==============================] - 13s 261ms/step - loss: 1.8192 - accuracy: 0.3564 - val_loss: 1.9205 - val_accuracy: 0.3227
Epoch 10/10
50/50 [==============================] - 13s 260ms/step - loss: 1.7873 - accuracy: 0.3788 - val_loss: 1.8581 - val_accuracy: 0.3493

Model Evaluating
model.evaluate(train_data)
50/50 [==============================] - 12s 228ms/step - loss: 1.7583 - accuracy: 0.4016
[1.7583075761795044, 0.4016000032424927]

model.evaluate(test_data)
15/15 [==============================] - 2s 114ms/step - loss: 1.8581 - accuracy: 0.3493
[1.858059048652649, 0.34933334589004517]

So as you see. I did so many hiperparameters tuning. I tried so many epoch combination but after 5-6 epochs, accuracy passing val_accuracy again and being overfit.
So is there wrong stuff in my modelling or is that all capacity of Neural Networks with small data. How to make a better modelling? (Btw i know that Transfer Learning is powerful solution for small data but i'm just asking for my models.)

Have you tried regularization? Also can you describe a bit the problem you're trying to solve?


text:
Stable Diffusion: Is it possible to merge specific token weights from one model into another?
I have three models: A, B, C
I merged A+C and B+C together.
A+C gives me great results for certain things but is not so good with specific other things. B+C is great with things that A+C is not, but performs less great in other areas.
I was wondering if there is a way to merge specific types of tokens from A+C into B+C so that it performs better with these things.
I read about token merging, but it seems to be something else (about performance).
Is there a way to achieve that only specific things get merged from one model into another?

I hardly think that linear interpolation of solutions on a non convex function might have any meaning


text:
Analysis of the output samples from an autoencoder
I am conducting some experiments on an autoencoder as part of my research project. For our first experiment, we have a feedforward neural network (using pytorch), which is being given an input of dimensions $80 \times 1$ and is giving out an output of dimensions $80 \times 1$. The $80$ input random variables are jointly gaussian. The input samples are being generated from a randomly generated block diagonal covariance matrix of size $80 \times 80$ and $0$ mean vector $80 \times 1$.
So first, we are training the autoencoder on some randomly generated samples from this input distribution using mean squared error as the loss function (we are sending the input column by column) and then we plan to train it on KL divergence. After that we will give it some inputs, get the output and analyze the output distribution with respect to the input distribution.
The problem we are facing is as follows:

During the training, the covariance matrix of the output distribution is sometimes of rank < 80 and we are getting an error stating that the covariance matrix is not positive definite. We are using torch.cov() to generate the covariance matrix

What would be the best way to calculate KL divergence in this particular context? We need to use it as a loss function.

This is the code we are using for it right now:
loss = torch.distributions.kl.kl_divergence
(
 torch.distributions.MultivariateNormal(mu1_tensor, cov1_tensor),
 torch.distributions.MultivariateNormal(mu2, cov2)`
 )

Here is another function that we have come across for it.
kl_loss = torch.nn.functional.kl_div(output_tensor.log(), input_tensor, reduction='batchmean')

Does your AE ouputs only the mean? Also, the KL as loss is mandatory or you can use maximum likelihood estimation for example?


text:
Binary classification using softmax and categorical crossentropy: monitoring validation
I run a binary classification using different CNN versions in Tensorflow.
When I label samples from each class using 0 and 1, I select a sigmoid output in the last layer of the CNN, like
 x = Dense(1, activation="sigmoid")(x)

Then, I set up a series of metrics to follow training progress, as in
metrics = [
  keras.metrics.TruePositives(name='tp'),
  keras.metrics.FalsePositives(name='fp'),
  keras.metrics.TrueNegatives(name='tn'),
  keras.metrics.FalseNegatives(name='fn'),
  keras.metrics.BinaryAccuracy(name='accuracy'),
  keras.metrics.Precision(name='precision'),
  keras.metrics.Recall(name='recall'),
  keras.metrics.AUC(name='auc'),
  keras.metrics.AUC(name='prc', curve='PR'), # precision-recall curve]

Finally, my model is compiled and fit using binary crossentropy loss:
# Compile the model
model.compile(optimizer=Adam(), loss='binary_crossentropy', metrics=metrics)
history = model.fit(X_train, y_train,
                         batch_size=batch_size, epochs=number_of_epochs,
                         verbose=verbosity_mode,
                         validation_data = (X_val, y_val),
                         callbacks=[es, cc])

Everything works fine, as I can see the metrics both for my training and my validation set changing during training:
Epoch 58/60
132/132 - 24s - loss: 0.0205 - tp: 2413.0000 - fp: 9.0000 - tn: 1770.0000 - fn: 7.0000 - accuracy: 0.9962 - precision: 0.9963 - recall: 0.9971 - auc: 0.9999 - prc: 0.9999 - val_loss: 0.3301 - val_tp: 1402.0000 - val_fp: 182.0000 - val_tn: 2039.0000 - val_fn: 178.0000 - val_accuracy: 0.9053 - val_precision: 0.8851 - val_recall: 0.8873 - val_auc: 0.9607 - val_prc: 0.9426 - 24s/epoch - 180ms/step
Epoch 59/60
132/132 - 25s - loss: 0.0217 - tp: 2408.0000 - fp: 11.0000 - tn: 1768.0000 - fn: 12.0000 - accuracy: 0.9945 - precision: 0.9955 - recall: 0.9950 - auc: 0.9999 - prc: 0.9999 - val_loss: 0.3722 - val_tp: 1472.0000 - val_fp: 296.0000 - val_tn: 1925.0000 - val_fn: 108.0000 - val_accuracy: 0.8937 - val_precision: 0.8326 - val_recall: 0.9316 - val_auc: 0.9586 - val_prc: 0.9313 - 25s/epoch - 191ms/step

Now, if my labels are one-hot encoded, a softmax activation is used in the last layer for two classes and categorical crossentropy should be selected in model compilation:
x = Dense(2, activation="softmax")(x)
...
# Compile the model
model.compile(optimizer=Adam(), loss='categorical_crossentropy', metrics=metrics)
history = model.fit(X_train, y_train,
                         batch_size=batch_size, epochs=number_of_epochs,
                         verbose=verbosity_mode,
                         validation_data = (X_val, y_val),
                         callbacks=[es, cc])

But then my metrics are shown in a different way:
Epoch 59/60
132/132 - 25s - loss: 0.0217 - tp: 2408.0000 - fp: 11.0000 - tn: 2408.0000 - fn: 11.0000 - accuracy: 0.9945 - precision: 0.9945 - recall: 0.9945 - auc: 0.9999 - prc: 0.9999 - val_loss: 0.3722 - val_tp: 1472.0000 - val_fp: 296.0000 - val_tn: 1472.0000 - val_fn: 296.0000 - val_accuracy: 0.8937 - val_precision: 0.8937 - val_recall: 0.8937 - val_auc: 0.9586 - val_prc: 0.9313 - 25s/epoch - 191ms/step

It seems that true positives equal true negatives, and false positives equal false negatives, that is, all the metrics are strangely shown.
Does anyone know a solution to this? That is, a way to see metrics during training in the latter setup (one-hot, softmax, categorical crossentropy) as shown in the former setup (int, sigmoid, binary crossentropy).
Thank you!