问:
Fast dataframe normalization in Python/Pandas
In Python/Pandas, I am trying to normalize all numeric columns of a dataframe to counts per million using the following code:

```<code>import pandas as pd
def normalize(df):
    numerics = ['int16', 'int32', 'int64', 'float16', 'float32', 'float64']
    numeric_cols = [col for col in df.columns if df[col].dtype in numerics]
    sums = df.sum(axis=0, skipna = True)
    for col in numeric_cols:
        df.insert(df.columns.get_loc(col)+1, col+'_norm', ((df[col]/sums[col])*1000000)+1)
    return df
dataset = pd.read_csv(csv_file, index_col=['Guide1', 'Guide2'], sep=',')
dataset = normalize(dataset)
</code>```

This is a short example input:

```<code>Guide1,Guide2,Gene1,Gene2,NHT1,NHT2,hart_essential_1,hart_essential_2,lib,RPE1_n1,RPE1_n2,RPE1_n3
Gene_1-KO-3,Non-Human-Target-150-KO-26,Gene_1,Non-Human-Target-150,False,True,False,False,12426.0,10634.0,8701.0,8084.0
Gene_2-KO-3,Non-Human-Target-150-KO-26,Gene_2,Non-Human-Target-150,False,True,False,False,12300.0,12383.0,6252.0,5388.0
Gene_1-KO-3,Gene_4-KO-2,Gene_1,Gene_4,False,False,False,False,11685.0,10006.0,10621.0,7002.0
Gene_1-KO-3,Gene_5-KO-2,Gene_1,Gene_5,False,False,False,False,11347.0,6726.0,3927.0,3943.0
Gene_1-KO-3,Gene_6-KO-1,Gene_1,Gene_6,False,False,False,False,11250.0,12469.0,3552.0,3334.0
</code>```

The code works as intended and normalizes all numeric columns of that dataframe, but it seems to work very slow, also with other normalization methods. I am regularly working with dataframes that contain up to several millions of rows. For ~1 million rows and only 4 numeric columns, normalization takes almost 10 minutes on my mobile computer with an Intel Core i7-6600U @ 2.6GHz, although I precompute the sums for every column.
Is there a way to speed this up, e.g., parallelization or some neat Pandas trick?

Any help and/or pointers are much appreciated!

答:
The normalization in this adaptation of your code is pretty much instantaneous. Most of the time is spent generating those random numbers...</p>

```<code>import pandas as pd
import random

numeric_dtypes = {'int16', 'int32', 'int64', 'float16', 'float32', 'float64'}

# Generate data...
n = 1000000
floats = [random.uniform(0, 100000) for x in range(n)]
ints = [int(i) for i in floats]
df = pd.DataFrame()
for dtype in numeric_dtypes:
    df[dtype] = pd.Series((ints if 'int' in dtype else floats), dtype=dtype)

# Normalize...
sums = df.sum(axis=0, skipna=True)
for col in df.columns:
    if str(df[col].dtype) not in numeric_dtypes:
        continue
    df[col + "_norm"] = df[col] / sums[col] * 1000000 + 1

print(df)
</code>```

Body:
In Python/Pandas, I am trying to normalize all numeric columns of a dataframe to counts per million using the following code:

import pandas as pd
def normalize(df):
    numerics = ['int16', 'int32', 'int64', 'float16', 'float32', 'float64']
    numeric_cols = [col for col in df.columns if df[col].dtype in numerics]
    sums = df.sum(axis=0, skipna = True)
    for col in numeric_cols:
        df.insert(df.columns.get_loc(col)+1, col+'_norm', ((df[col]/sums[col])*1000000)+1)
    return df
dataset = pd.read_csv(csv_file, index_col=['Guide1', 'Guide2'], sep=',')
dataset = normalize(dataset)

This is a short example input:

Guide1,Guide2,Gene1,Gene2,NHT1,NHT2,hart_essential_1,hart_essential_2,lib,RPE1_n1,RPE1_n2,RPE1_n3
Gene_1-KO-3,Non-Human-Target-150-KO-26,Gene_1,Non-Human-Target-150,False,True,False,False,12426.0,10634.0,8701.0,8084.0
Gene_2-KO-3,Non-Human-Target-150-KO-26,Gene_2,Non-Human-Target-150,False,True,False,False,12300.0,12383.0,6252.0,5388.0
Gene_1-KO-3,Gene_4-KO-2,Gene_1,Gene_4,False,False,False,False,11685.0,10006.0,10621.0,7002.0
Gene_1-KO-3,Gene_5-KO-2,Gene_1,Gene_5,False,False,False,False,11347.0,6726.0,3927.0,3943.0
Gene_1-KO-3,Gene_6-KO-1,Gene_1,Gene_6,False,False,False,False,11250.0,12469.0,3552.0,3334.0

The code works as intended and normalizes all numeric columns of that dataframe, but it seems to work very slow, also with other normalization methods. I am regularly working with dataframes that contain up to several millions of rows. For ~1 million rows and only 4 numeric columns, normalization takes almost 10 minutes on my mobile computer with an Intel Core i7-6600U @ 2.6GHz, although I precompute the sums for every column.
Is there a way to speed this up, e.g., parallelization or some neat Pandas trick?

Any help and/or pointers are much appreciated!

Body:
The normalization in this adaptation of your code is pretty much instantaneous. Most of the time is spent generating those random numbers...

import pandas as pd
import random

numeric_dtypes = {'int16', 'int32', 'int64', 'float16', 'float32', 'float64'}

# Generate data...
n = 1000000
floats = [random.uniform(0, 100000) for x in range(n)]
ints = [int(i) for i in floats]
df = pd.DataFrame()
for dtype in numeric_dtypes:
    df[dtype] = pd.Series((ints if 'int' in dtype else floats), dtype=dtype)

# Normalize...
sums = df.sum(axis=0, skipna=True)
for col in df.columns:
    if str(df[col].dtype) not in numeric_dtypes:
        continue
    df[col + "_norm"] = df[col] / sums[col] * 1000000 + 1

print(df)


来源: stackexchange

问:
DateTimePicker Leave / ValueChanged events
I have a simple little dialog that lets user setup a "time block".  The user indicates the specified start time for the block, and then can either enter a duration, or enter the end-time for the block.   For example, they can either say start at 8AM, end at 11AM, or start at 8AM and duration of 180 minutes.  I have both a Text Box for the duration, and a DateTimePicker for the end time.  The DateTimePicker's custom format is "hh:mm tt".  The Text Box works just fine. On the Leave event of the Text Box, I update the DateTimePicker with the correct end time.

However, getting the DateTimePicker to work is not nearly as easy.  The DateTimePicker doesn't always have its value correct when the Leave event fires.

Also, I have a requirement to make sure the duration of the time block is at least 15 minutes, so ideally, when the user "Leaves" either control, I can update the other control, and also validate that its at least 15 minutes.  (If < 15, I set it to 15).

Ok, time for some code:

```<code>private void SetDuration(int Duration)
{
    Duration = Math.Max(15, Duration);
    m_TextBoxDuration.Text = Duration.ToString();
    m_DateTimePickerEndTime.Value = m_DateTimePickerStartTime.Value.AddMinutes(Duration);
}

private void m_TextBoxDuration_Leave(object sender, EventArgs e)
{
    int duration = 0;
    int.TryParse(m_TextBoxDuration.Text, out duration);
    SetDuration(duration);
}

private void m_DateTimePickerEndTime_Leave(object sender, EventArgs e)
{
    SetDuration((int)(m_DateTimePickerEndTime.Value.Subtract(m_DateTimePickerStartTime.Value).TotalMinutes));
}
</code>```

Here's the difference.  Suppose the date time picker has its time set to 07:00 PM. If the user highlights the "07" and types 6 then hits Tab, the Leave event fires before the ValueChanged event.  If the user types 06 and then Tab, the events go in reverse order.   So I cannot rely on either event to be the final event.  I don't want to (can't) update every time I get the ValueChanged event, because then the minimum duration requirement can really mess with the user trying to make edits.

Help?

答:
You need to validate the control first, before leaving. If you tab from m_DateTimePickerEndTime to the textbox, you can use this.Focus() to validate all the controls.

```<code>private void m_DateTimePickerEndTime_Leave(object sender, EventArgs e)
{
  this.Focus();
  SetDuration((int(m_DateTimePickerEndTime.Value.Subtract
          (m_DateTimePickerStartTime.Value).TotalMinutes));
}
</code>```

If your tab order is different, put the this.Focus in the other leave event.

ContentLicense: CC BY-SA 3.0

Body:
I have a simple little dialog that lets user setup a "time block".  The user indicates the specified start time for the block, and then can either enter a duration, or enter the end-time for the block.   For example, they can either say start at 8AM, end at 11AM, or start at 8AM and duration of 180 minutes.  I have both a Text Box for the duration, and a DateTimePicker for the end time.  The DateTimePicker's custom format is "hh:mm tt".  The Text Box works just fine. On the Leave event of the Text Box, I update the DateTimePicker with the correct end time.

However, getting the DateTimePicker to work is not nearly as easy.  The DateTimePicker doesn't always have its value correct when the Leave event fires.

Also, I have a requirement to make sure the duration of the time block is at least 15 minutes, so ideally, when the user "Leaves" either control, I can update the other control, and also validate that its at least 15 minutes.  (If < 15, I set it to 15).

Ok, time for some code:

private void SetDuration(int Duration)
{
    Duration = Math.Max(15, Duration);
    m_TextBoxDuration.Text = Duration.ToString();
    m_DateTimePickerEndTime.Value = m_DateTimePickerStartTime.Value.AddMinutes(Duration);
}

private void m_TextBoxDuration_Leave(object sender, EventArgs e)
{
    int duration = 0;
    int.TryParse(m_TextBoxDuration.Text, out duration);
    SetDuration(duration);
}

private void m_DateTimePickerEndTime_Leave(object sender, EventArgs e)
{
    SetDuration((int)(m_DateTimePickerEndTime.Value.Subtract(m_DateTimePickerStartTime.Value).TotalMinutes));
}

Here's the difference.  Suppose the date time picker has its time set to 07:00 PM. If the user highlights the "07" and types 6 then hits Tab, the Leave event fires before the ValueChanged event.  If the user types 06 and then Tab, the events go in reverse order.   So I cannot rely on either event to be the final event.  I don't want to (can't) update every time I get the ValueChanged event, because then the minimum duration requirement can really mess with the user trying to make edits.

Help?

Body:
You need to validate the control first, before leaving. If you tab from m_DateTimePickerEndTime to the textbox, you can use this.Focus() to validate all the controls.

private void m_DateTimePickerEndTime_Leave(object sender, EventArgs e)
{
  this.Focus();
  SetDuration((int(m_DateTimePickerEndTime.Value.Subtract
          (m_DateTimePickerStartTime.Value).TotalMinutes));
}

If your tab order is different, put the this.Focus in the other leave event.


问:
Java class loader bug: Caused by: java.io.IOException: Stream closed
I'm getting a strange bug regarding I believe class loader issues when I deploy my webapp to Tomcat. The bug doesn't appear when I run my webapp locally using Jetty. It seems like my input streams for my .yml resource files are being closed for some reason when they shouldn't be. This bug first appeared when I tried to convert my single module project into a multi module project. Before that, it was working fine on Tomcat using the exact same code:

```<code>Caused by: org.yaml.snakeyaml.error.YAMLException: java.io.IOException: Stream closed
    at org.yaml.snakeyaml.reader.StreamReader.update(StreamReader.java:200)
    at org.yaml.snakeyaml.reader.StreamReader.<init>(StreamReader.java:60)
    at org.yaml.snakeyaml.Yaml.load(Yaml.java:412)
    at com.config.ConfigProvider.<init>(ConfigProvider.java:20)
    ... 49 more
Caused by: java.io.IOException: Stream closed
    at java.io.PushbackInputStream.ensureOpen(PushbackInputStream.java:57)
    at java.io.PushbackInputStream.read(PushbackInputStream.java:149)
    at org.yaml.snakeyaml.reader.UnicodeReader.init(UnicodeReader.java:90)
    at org.yaml.snakeyaml.reader.UnicodeReader.read(UnicodeReader.java:122)
    at java.io.Reader.read(Reader.java:123)
    at org.yaml.snakeyaml.reader.StreamReader.update(StreamReader.java:184)
    ... 55 more
</code>```

Here's the line that causes the bug:

```<code>String s = ConfigProvider.getConfig().getString("test");
</code>```

Here's the ConfigProvider class. It basically scans for all resource files of regex ^.*\\.config\\.yml$, converts it into a Map<String, Object>, and combines all the obtained Map<String, Object> into a single Map<String, Object>:

```<code>1 public class ConfigProvider {
2     protected static final String CONFIG_PACKAGE = ConfigProvider.class.getPackage().getName();
3     protected static final Pattern CONFIG_PATH_REGEX = Pattern.compile("^.*\\.config\\.yml$");
4
5     private static final ConfigProvider INSTANCE = new ConfigProvider();
6     private Map<String, Object> configMap;
7
8     protected ConfigProvider() {
9         configMap = new HashMap<String, Object>();
10
11        Set<String> configPaths = new Reflections(CONFIG_PACKAGE,
12            new ResourcesScanner()).getResources(CONFIG_PATH_REGEX);
13
14        if (configPaths.isEmpty()) {
15            throw new RuntimeException("no config paths found");
16        }
17
18        for (String path : configPaths) {
19            InputStream inputStream = Thread.currentThread().getContextClassLoader().getResourceAsStream(path);
20            Map<String, Object> fullConfig = new Map<String, Object>((Map) new Yaml().load(inputStream));
21
22            try {
23                inputStream.close();
24            } catch (IOException e) {
25                throw new RuntimeException("error closing stream");
26            }
27
28            MapUtils.merge(configMap, fullConfig);
29        }
30    }
31
32    public static ConfigMap getConfig() {
33        return INSTANCE.configMap;
34    }
35 }
</code>```

Here's my project structure, titled Foo:

```<code>- Foo (this is a module)
    - .idea
    - application (this is a module)
        - src
            - main
                - java
                - resources
                    - application.config.yml
                - webapp
            - test
        - pom.xml
    - client (this is a module)
        - src
            - main
                - java
                - resources
                    - client.config.yml
                - webapp
            - test
        - pom.xml
    - pom.xml
</code>```

ConfigProvider is a class I get from my parent pom file (Foo/pom.xml). I package a WAR file from the application module (Foo/application/target/application.war), and deploy it with Tomcat. Previously, my project was a single module project with just a Foo module being identical to application module. Then I added a client module and converted the project into a multi module project, and the problem has showed up. I think it's because my class loader is getting messed up due to the multiple modules. I've spent a lot of time trying to debug this and still haven't gotten anywhere. Anyone know what could be the cause, or can think of possible things to try?

Please let me know if you need more info.

答: Bean class member variable should match with .yml file keys and second option is there might be chances that you might be giving wrong file path.  </p>

Body:
I'm getting a strange bug regarding I believe class loader issues when I deploy my webapp to Tomcat. The bug doesn't appear when I run my webapp locally using Jetty. It seems like my input streams for my .yml resource files are being closed for some reason when they shouldn't be. This bug first appeared when I tried to convert my single module project into a multi module project. Before that, it was working fine on Tomcat using the exact same code:

Caused by: org.yaml.snakeyaml.error.YAMLException: java.io.IOException: Stream closed
    at org.yaml.snakeyaml.reader.StreamReader.update(StreamReader.java:200)
    at org.yaml.snakeyaml.reader.StreamReader.<init>(StreamReader.java:60)
    at org.yaml.snakeyaml.Yaml.load(Yaml.java:412)
    at com.config.ConfigProvider.<init>(ConfigProvider.java:20)
    ... 49 more
Caused by: java.io.IOException: Stream closed
    at java.io.PushbackInputStream.ensureOpen(PushbackInputStream.java:57)
    at java.io.PushbackInputStream.read(PushbackInputStream.java:149)
    at org.yaml.snakeyaml.reader.UnicodeReader.init(UnicodeReader.java:90)
    at org.yaml.snakeyaml.reader.UnicodeReader.read(UnicodeReader.java:122)
    at java.io.Reader.read(Reader.java:123)
    at org.yaml.snakeyaml.reader.StreamReader.update(StreamReader.java:184)
    ... 55 more

Here's the line that causes the bug:

String s = ConfigProvider.getConfig().getString("test");

Here's the ConfigProvider class. It basically scans for all resource files of regex ^.*\\.config\\.yml$, converts it into a Map<String, Object>, and combines all the obtained Map<String, Object> into a single Map<String, Object>:

1 public class ConfigProvider {
2     protected static final String CONFIG_PACKAGE = ConfigProvider.class.getPackage().getName();
3     protected static final Pattern CONFIG_PATH_REGEX = Pattern.compile("^.*\\.config\\.yml$");
4
5     private static final ConfigProvider INSTANCE = new ConfigProvider();
6     private Map<String, Object> configMap;
7
8     protected ConfigProvider() {
9         configMap = new HashMap<String, Object>();
10
11        Set<String> configPaths = new Reflections(CONFIG_PACKAGE,
12            new ResourcesScanner()).getResources(CONFIG_PATH_REGEX);
13
14        if (configPaths.isEmpty()) {
15            throw new RuntimeException("no config paths found");
16        }
17
18        for (String path : configPaths) {
19            InputStream inputStream = Thread.currentThread().getContextClassLoader().getResourceAsStream(path);
20            Map<String, Object> fullConfig = new Map<String, Object>((Map) new Yaml().load(inputStream));
21
22            try {
23                inputStream.close();
24            } catch (IOException e) {
25                throw new RuntimeException("error closing stream");
26            }
27
28            MapUtils.merge(configMap, fullConfig);
29        }
30    }
31
32    public static ConfigMap getConfig() {
33        return INSTANCE.configMap;
34    }
35 }

Here's my project structure, titled Foo:

- Foo (this is a module)
    - .idea
    - application (this is a module)
        - src
            - main
                - java
                - resources
                    - application.config.yml
                - webapp
            - test
        - pom.xml
    - client (this is a module)
        - src
            - main
                - java
                - resources
                    - client.config.yml
                - webapp
            - test
        - pom.xml
    - pom.xml

ConfigProvider is a class I get from my parent pom file (Foo/pom.xml). I package a WAR file from the application module (Foo/application/target/application.war), and deploy it with Tomcat. Previously, my project was a single module project with just a Foo module being identical to application module. Then I added a client module and converted the project into a multi module project, and the problem has showed up. I think it's because my class loader is getting messed up due to the multiple modules. I've spent a lot of time trying to debug this and still haven't gotten anywhere. Anyone know what could be the cause, or can think of possible things to try?

Please let me know if you need more info.

ContentLicense: CC BY-SA 4.0


问:
Neural Network - Vector Centric Python implementation
Hello I try to use a vector-centric Neural Network with one input node, one output node and two hidden layer with 3 nodes each to fit a very easy x**2 function. - Just to verify its functionality. Therefore I use the code below. As result I get the orange line, the blue line is the true line.

As you can see something is not working. I have tried to change the number of iterations together with the value of the learning rate but no success. If I plot the Loss over the iterations, I get the following picture for 100 Iterations:

I have not added bias yet but I think this easy function should be fit-able without extra bias nodes. Additionally I assume the failure in the code most likely to be in the "Calculate the Gradients with respect to the weights" part of the code...

So in principal I have two questions:

Is there any basic failure in my code such that the code CAN not work

If not, why does my model fail to fit the easy data

Thanks for help in advance!

Here comes the code - It is ready to play:

```<code>class Neural_Net:
    """
    """
    def __init__(self, activation_function, learning_rate, runs):
        self.activation_function = activation_function
        self.X_train = np.linspace(0,1,1000)
        self.y_train = self.X_train**2
        plt.plot(self.X_train, self.y_train)
        self.y_pred = None
        self.W_input = np.random.randn(1, 3)
        self.Partials_W_input = np.random.randn(1, 3)
        self.W_hidden = np.random.randn(3,3)
        self.Partials_W_hidden = np.random.randn(3,3)
        self.W_output = np.random.randn(3,1)
        self.Partials_W_output = np.random.randn(3,1)
        self.Activations = np.ones((3,2))
        self.Partials = np.ones((3,2))
        self.Output_Gradient = None
        self.Loss = 0
        self.learning_rate = learning_rate
        self.runs = runs
        self.Losses = []
        self.i = 0

    def apply_activation_function(self, activation_vector):
            return 1/(1+np.exp(-activation_vector))

    def forward_pass(self, training_instance):
        for layer in range(len(self.Activations[0])):
            # For the first layer between X and the first hidden layer
            pre_activation_first = self.W_input.T @ training_instance.reshape(1,1)
                # print('pre activation: ', pre_activation)
                # Apply the activation function
                self.Activations[:,0] = self.apply_activation_function(pre_activation_first).ravel()
            else:
                pre_activation_hidden = self.W_hidden.T @ self.Activations[:, layer-1]
                self.Activations[:, layer] = self.apply_activation_function(pre_activation_hidden)
                # print('Activations: ', self.Activations)
        output = self.W_output.T @ self.Activations[:, -1]
        # print('output: ', output)
        return output

    def backpropagation(self, y_true, training_instance):
        if self.activation_function == 'linear':
            # Calculate the ouput gradient
            self.Output_Gradient = -(y_true-self.y_pred)
            # print('Output Gradient: ', self.Output_Gradient)

            # Calculate the partial gradients of the Error with respect to the pre acitvation values in the nodes
            self.Partials[:, 1] = self.Activations[:, 1]*(1-self.Activations[:, 1])*(self.W_output @ self.Output_Gradient)
            self.Partials[:, 0] = self.Activations[:, 0]*(1-self.Activations[:, 0])*(self.W_hidden @ self.Partials[:, 1])
            # print('Partials: ', self.Partials)

            # Calculate the Gradients with respect to the weights
            self.Partials_W_output = self.Output_Gradient * self.Activations[:, -1]
            # print('Partials_W_output: ', self.Partials_W_output)
            self.Partials_W_hidden = self.Partials[:, -1].reshape(3,1) * self.Activations[:, 0].reshape(1,3)
            # print('Partials_W_hidden: ',self.Partials_W_hidden)
            self.Partials_W_input = (self.Partials[:, 0].reshape(3,1) * training_instance.T).T
            # print('Partials_W_input: ', self.Partials_W_input)

    def weight_update(self, training_instance, learning_rate):

        # Output Layer weights
        w_output_old = self.W_output.copy()
        self.W_output = w_output_old - learning_rate*self.Output_Gradient

        # Hidden Layer weights
        w_hidden_old = self.W_hidden.copy()
        self.W_hidden = w_hidden_old - learning_rate * self.W_hidden
        # print('W_hidden new: ', self.W_hidden)

        # Input Layer weights
        w_input_old = self.W_input.copy()
        self.W_input = w_input_old - learning_rate * self.W_input
        # print('W_input new: ', self.W_input)

    def train_model(self):
        for _ in range(self.runs):
            for instance in range(len(self.X_train)):
                # forward pass
                self.y_pred = self.forward_pass(self.X_train[instance])

                # Calculate loss
                self.Loss = self.calc_loss(self.y_pred, self.y_train[instance])
                # print('Loss: ', self.Loss)

                # Calculate backpropagation
                self.backpropagation(self.y_train[instance], self.X_train[instance])

                # Update weights
                self.weight_update(self.X_train[instance], self.learning_rate)

        # print(self.Losses)
        # plt.plot(range(len(self.Losses)), self.Losses)
        # plt.show()

        # Make predictions on training data to check if the model is basically able to fit the training data
        predictions = []
        for i in np.linspace(0,1,1000):
            predictions.append(self.make_prediction(i))
        plt.plot(np.linspace(0,1,1000), predictions)

    def make_prediction(self, X_new):
        return self.forward_pass(X_new)

    def calc_loss(self, y_pred, y_true):
        loss = (1/2)*(y_true-y_pred)**2
        self.Losses.append(loss[0])
        return (1/2)*(y_true-y_pred)**2

    def accuracy(self):
        pass

Neural_Net('linear', 0.0001, 10).train_model()
</code>```

答:
As long as your activation function is linear, the entire ANN will provide a simple weighted sum, so a linear output. Practically you are doing linear regression at the moment. It is good for verifying that learning works to some extent (try to teach a number of linear functions), but that is all, for the real thing you need nonlinear.

See https://en.wikipedia.org/wiki/Activation_function#Comparison_of_activation_functions on Wikipedia for ideas. In fact the comparison of functions starts with a recap of desirable features, and the first one is being nonlinear:

Comparison of activation functions

Some desirable properties in an activation function include:

Nonlinear – When the activation function is non-linear, then a two-layer neural network can be proven to be a universal function approximator.[6] The identity activation function does not satisfy this property. When multiple layers use the identity activation function, the entire network is equivalent to a single-layer model.

Body:
Hello I try to use a vector-centric Neural Network with one input node, one output node and two hidden layer with 3 nodes each to fit a very easy x**2 function. - Just to verify its functionality. Therefore I use the code below. As result I get the orange line, the blue line is the true line.

As you can see something is not working. I have tried to change the number of iterations together with the value of the learning rate but no success. If I plot the Loss over the iterations, I get the following picture for 100 Iterations:

I have not added bias yet but I think this easy function should be fit-able without extra bias nodes. Additionally I assume the failure in the code most likely to be in the "Calculate the Gradients with respect to the weights" part of the code...

So in principal I have two questions:

Is there any basic failure in my code such that the code CAN not work

If not, why does my model fail to fit the easy data

Thanks for help in advance!

Here comes the code - It is ready to play:

class Neural_Net:
    """
    """
    def __init__(self, activation_function, learning_rate, runs):
        self.activation_function = activation_function
        self.X_train = np.linspace(0,1,1000)
        self.y_train = self.X_train**2
        plt.plot(self.X_train, self.y_train)
        self.y_pred = None
        self.W_input = np.random.randn(1, 3)
        self.Partials_W_input = np.random.randn(1, 3)
        self.W_hidden = np.random.randn(3,3)
        self.Partials_W_hidden = np.random.randn(3,3)
        self.W_output = np.random.randn(3,1)
        self.Partials_W_output = np.random.randn(3,1)
        self.Activations = np.ones((3,2))
        self.Partials = np.ones((3,2))
        self.Output_Gradient = None
        self.Loss = 0
        self.learning_rate = learning_rate
        self.runs = runs
        self.Losses = []
        self.i = 0

    def apply_activation_function(self, activation_vector):
            return 1/(1+np.exp(-activation_vector))

    def forward_pass(self, training_instance):
        for layer in range(len(self.Activations[0])):
            # For the first layer between X and the first hidden layer
            pre_activation_first = self.W_input.T @ training_instance.reshape(1,1)
                # print('pre activation: ', pre_activation)
                # Apply the activation function
                self.Activations[:,0] = self.apply_activation_function(pre_activation_first).ravel()
            else:
                pre_activation_hidden = self.W_hidden.T @ self.Activations[:, layer-1]
                self.Activations[:, layer] = self.apply_activation_function(pre_activation_hidden)
                # print('Activations: ', self.Activations)
        output = self.W_output.T @ self.Activations[:, -1]
        # print('output: ', output)
        return output

    def backpropagation(self, y_true, training_instance):
        if self.activation_function == 'linear':
            # Calculate the ouput gradient
            self.Output_Gradient = -(y_true-self.y_pred)
            # print('Output Gradient: ', self.Output_Gradient)

            # Calculate the partial gradients of the Error with respect to the pre acitvation values in the nodes
            self.Partials[:, 1] = self.Activations[:, 1]*(1-self.Activations[:, 1])*(self.W_output @ self.Output_Gradient)
            self.Partials[:, 0] = self.Activations[:, 0]*(1-self.Activations[:, 0])*(self.W_hidden @ self.Partials[:, 1])
            # print('Partials: ', self.Partials)

            # Calculate the Gradients with respect to the weights
            self.Partials_W_output = self.Output_Gradient * self.Activations[:, -1]
            # print('Partials_W_output: ', self.Partials_W_output)
            self.Partials_W_hidden = self.Partials[:, -1].reshape(3,1) * self.Activations[:, 0].reshape(1,3)
            # print('Partials_W_hidden: ',self.Partials_W_hidden)
            self.Partials_W_input = (self.Partials[:, 0].reshape(3,1) * training_instance.T).T
            # print('Partials_W_input: ', self.Partials_W_input)

    def weight_update(self, training_instance, learning_rate):

        # Output Layer weights
        w_output_old = self.W_output.copy()
        self.W_output = w_output_old - learning_rate*self.Output_Gradient

        # Hidden Layer weights
        w_hidden_old = self.W_hidden.copy()
        self.W_hidden = w_hidden_old - learning_rate * self.W_hidden
        # print('W_hidden new: ', self.W_hidden)

        # Input Layer weights
        w_input_old = self.W_input.copy()
        self.W_input = w_input_old - learning_rate * self.W_input
        # print('W_input new: ', self.W_input)

    def train_model(self):
        for _ in range(self.runs):
            for instance in range(len(self.X_train)):
                # forward pass
                self.y_pred = self.forward_pass(self.X_train[instance])

                # Calculate loss
                self.Loss = self.calc_loss(self.y_pred, self.y_train[instance])
                # print('Loss: ', self.Loss)

                # Calculate backpropagation
                self.backpropagation(self.y_train[instance], self.X_train[instance])

                # Update weights
                self.weight_update(self.X_train[instance], self.learning_rate)

        # print(self.Losses)
        # plt.plot(range(len(self.Losses)), self.Losses)
        # plt.show()

        # Make predictions on training data to check if the model is basically able to fit the training data
        predictions = []
        for i in np.linspace(0,1,1000):
            predictions.append(self.make_prediction(i))
        plt.plot(np.linspace(0,1,1000), predictions)

    def make_prediction(self, X_new):
        return self.forward_pass(X_new)

    def calc_loss(self, y_pred, y_true):
        loss = (1/2)*(y_true-y_pred)**2
        self.Losses.append(loss[0])
        return (1/2)*(y_true-y_pred)**2

    def accuracy(self):
        pass

Neural_Net('linear', 0.0001, 10).train_model()


问:
What is the type returned by quandl.get?
I'm currently following tutorial in which you get data form quandl. I know the basics of python, but I can't figure out what is the type returned by quandl.get. I have searched for it, but everything I found refers to pandas. Do you know what type is it and where can I read some documentation about it?
Here is the code I don't undestand:

```<code>df = quandl.get('WIKI/GOOGL')

df = df[['Adj. Open','Adj. High','Adj. Low','Adj. Close','Adj. Volume']]
</code>```

I also dont get why there are double brackets in the second line, but I hope it will be clear after I can read something about it.

答: The two sets of square brackets mean the dataframe is a 2 dimensional array ie lots of colums- inner bracket and one row-outer bracket. This reffers to the title row of the dataframe. </p>

Body:
I'm currently following tutorial in which you get data form quandl. I know the basics of python, but I can't figure out what is the type returned by quandl.get. I have searched for it, but everything I found refers to pandas. Do you know what type is it and where can I read some documentation about it?
Here is the code I don't undestand:

df = quandl.get('WIKI/GOOGL')

df = df[['Adj. Open','Adj. High','Adj. Low','Adj. Close','Adj. Volume']]

I also dont get why there are double brackets in the second line, but I hope it will be clear after I can read something about it.


问:
parameters needed for connecting to aws rds instance via ssh
Does anyone know where to find the values needed for the parameters required for an ssh connection involving a bastion host and an AWS RDS instance?

I have listed what I have been able to find so far below:

SSH Hostname - Endpoint given for RDS instance in the AWS console

SSH Username - I'm seeing in tutorials all over the place it's ec2-user, should I also use this?

SSH password - do I need this, since I already have a .pem file for the key pair?
`

SSH Key File` - got this

MySQL Hostname - I'm not sure what to put for this.

MYSQL Server Port - 3306

Username - root

Password - not sure which password this is referring to

答:
There are two ways ,

If you are trying to access the RDS instance from local computer, you need to make sure that public access is enabled from db settings. By this way you can connect using db endpoint and username password.

Another way is through ssh tunneling,  in which database is not needed to be accessible publicly but needs to be accessible from instance inside vpc.. You need to ssh to the machine using instance's credentials like username password or key file. Then, once connected to ec2 instance, you can use db endpoint credentials to connect to RDS instance using SQL connection commands. You will need to configure security groups to allow the connections from RDS and ec2 to do this.

Details of parameters :

SSH Hostname - Ec2 instance external IP address or DNS name. Can be found on ec2  console description tab.

SSH Username - ec2 instance username. Can be found in ec2 console -> ec2 details.

SSH password - ec2 instance password. - not needed if using key file.

SSH Key File` - pem file created or used when ec2 instance was created.

MySQL Hostname - RDS endpoint - you can get from RDS console.

MYSQL Server Port - usually 3306, but if modified during RDS creation, check in RDS console.

Username - database username that was set while creating RDS.

Password - Database password set while creating instance.


问:
AADAuthenticationFilter called only Once
I downloaded the Azure AD with spring boot from https://github.com/microsoft/azure-spring-boot. Its all nice and works well as long as I run theazure-active-directory-spring-boot-sample using

```<code>mvn spring-boot:run
</code>```

as shown in the README file.

I am trying just a basic real-life scenario where the angular/react app is running separately on a separate port using npm. The moment I separated them out the main Filter AADAuthenticationFilter is only invoked once on the first request where it does token verification and never again for subsequent requests.

I went around and configured an nginx proxy so that the domains of these two are now same
localhost:9999 and I redirect based on the next path like web for the npm and app for the Spring boot. This enabled the cookies to be available at every request.

This helps because what I learned is that Spring uses cookies to maintain a session. But ideally, the AAD filter should revalidate the token every request.

But if I add a property of azure.activedirectory.session-stateless=true to my application.properties, as mentioned in the filter's code to make it stateless and validate on every request. It expects roles as a different attribute in AD instead of it being a group of the user.

I also don't understand get the behaviour, when the js files are embedded within the spring app and run all this works perfectly fine and the filter is invoked at every request. But when I separate them out it just doesn't work.

Any help or pointers would be appreciated.

答:
I realize that a lot more info may be required for someone to debug the above problem. But I was just able to solve this - at least found the problem that was causing it. Like I mentioned earlier the problem started when I separated the client and spring APIs. And since the client was not able to send back the cookies set by server, due to change in domains I had to put a proxy server.
What I found out later was that every GET request was always invoking the AAD filter. Even multiple requests. But other requests like POST, PUT were failing. I tried disabling the csrf config in spring WebSecurityConfig.java  -->>  http.csrf().disable();
This solved my problem and now I receive a request and all works as expected. Now I am trying to figure out how not to disable csrf completely.
Thanks

Body:
I downloaded the Azure AD with spring boot from https://github.com/microsoft/azure-spring-boot. Its all nice and works well as long as I run theazure-active-directory-spring-boot-sample using

mvn spring-boot:run

as shown in the README file.

I am trying just a basic real-life scenario where the angular/react app is running separately on a separate port using npm. The moment I separated them out the main Filter AADAuthenticationFilter is only invoked once on the first request where it does token verification and never again for subsequent requests.

I went around and configured an nginx proxy so that the domains of these two are now same
localhost:9999 and I redirect based on the next path like web for the npm and app for the Spring boot. This enabled the cookies to be available at every request.

This helps because what I learned is that Spring uses cookies to maintain a session. But ideally, the AAD filter should revalidate the token every request.

But if I add a property of azure.activedirectory.session-stateless=true to my application.properties, as mentioned in the filter's code to make it stateless and validate on every request. It expects roles as a different attribute in AD instead of it being a group of the user.

I also don't understand get the behaviour, when the js files are embedded within the spring app and run all this works perfectly fine and the filter is invoked at every request. But when I separate them out it just doesn't work.

Any help or pointers would be appreciated.


问:
No module named numpy with pypy
This question is related to Using Numpy with pypy

I tried to use

pip install git+https://bitbucket.org/pypy/numpy.git

The system (Fedora release 24 (Twenty Four)) said

git: 'submodule' is not a git command. See 'git --help'.
  Command "git submodule update --init --recursive -q" failed with error code 1 in /tmp/pip-LD8rrP-build

Still it seems that numpy is not compatible with pypy.

Is there any solution for this problem? P.S. I managed to setup it on ubuntu, but not fedora..

Additional information:

Fedora release 24 (Twenty Four)

pip 9.0.1 from /usr/lib/python2.7/site-packages (python 2.7)

git version 2.7.4

答:
I stumbled over this when I started using pypy with pyenv. I'm used to python

```<code># Set the Python version:
$ echo "pypy3.6-7.3.0" > ".python-version"

# Check if it worked:
$ python --version
Python 3.6.9 (1608da62bfc7, Dec 23 2019, 10:50:04)
[PyPy 7.3.0 with GCC 7.3.1 20180303 (Red Hat 7.3.1-5)]

$ pip --version
pyenv: pip: command not found
</code>```

After that, I was pretty confused and thought that maybe pip does not exist / work for pypy.

The solution was simply:

```<code>$ python -m pip install numpy
</code>```

Title: No module named numpy with pypy

Body:
I stumbled over this when I started using pypy with pyenv. I'm used to python

# Set the Python version:
$ echo "pypy3.6-7.3.0" > ".python-version"

# Check if it worked:
$ python --version
Python 3.6.9 (1608da62bfc7, Dec 23 2019, 10:50:04)
[PyPy 7.3.0 with GCC 7.3.1 20180303 (Red Hat 7.3.1-5)]

$ pip --version
pyenv: pip: command not found

After that, I was pretty confused and thought that maybe pip does not exist / work for pypy.

The solution was simply:

$ python -m pip install numpy


问:
update row after Period of time using laravel
I am developing a site that works for reservation and  i have rows for payment and status .son i need to update row for payment and status automatic after 24 hours How can  I do this

答:
Just adding one column named 'expired_at' to your table. The value of expired_at just adding 24 hours when data saved. You can use carbon library to adding 24 hours. And check expired_at using task scheduling or cron jobs.
If expired_at value is bigger than date now, do function.

See detail at https://laravel.com/docs/5.8/scheduling


问:
Get username using select prepared statement
I wrote this code, but it is just outputting "1". It should output the username of the selected user.  </p>

```<code>$uid = $_GET['id'];
if (empty($uid)) {
    echo "Error: Wrong input of values.";
} else {
    $stmt = $conn->prepare("SELECT username FROM usersinfo WHERE id = ?");
    $stmt->bind_param("s", $uid);
    $stmt->execute();
    $result = $stmt->fetch();
    echo $result;
    $stmt->close();
    $conn->close();
}
</code>```

答:
What about this,

I have used ->fetch_assoc(); so us to fetch the results in an array as opposed to ->fetch(); which has been used in the question

in case username has duplicates:- then

```<code>   $uid = $_GET['id'];
    if (empty($uid)) {
        echo "Error: Wrong input of values.";
    } else {
        $stmt = $conn->prepare("SELECT username FROM usersinfo WHERE id = ?");
        $stmt->bind_param("s", $uid);
        $stmt->execute();
        $result = $stmt->get_result();

         while($row = $result->fetch_assoc())  { //fetch_assoc() function fetches a result row as an associative array - here we are putting it in a loop
          echo $row['username'];
         }

        $stmt->close();
        $conn->close();
    }
</code>```

If we are to retrieve just one result, assuming there are no duplicates we can use below without loop.

```<code>   $uid = $_GET['id'];
    if (empty($uid)) {
        echo "Error: Wrong input of values.";
    } else {
        $stmt = $conn->prepare("SELECT username FROM usersinfo WHERE id = ?");
        $stmt->bind_param("s", $uid);
        $stmt->execute();
        $result = $stmt->get_result();

         $row = $result->fetch_assoc(); //fetch_assoc() function fetches a result row as an associative array
         echo $row['username']; //we can specify just to return specific column i.e. username

        $stmt->close();
        $conn->close();
    }
</code>```

Body:
I wrote this code, but it is just outputting "1". It should output the username of the selected user.

$uid = $_GET['id'];
if (empty($uid)) {
    echo "Error: Wrong input of values.";
} else {
    $stmt = $conn->prepare("SELECT username FROM usersinfo WHERE id = ?");
    $stmt->bind_param("s", $uid);
    $stmt->execute();
    $result = $stmt->fetch();
    echo $result;
    $stmt->close();
    $conn->close();
}

Body:
What about this,

I have used ->fetch_assoc(); so us to fetch the results in an array as opposed to ->fetch(); which has been used in the question

in case username has duplicates:- then

   $uid = $_GET['id'];
    if (empty($uid)) {
        echo "Error: Wrong input of values.";
    } else {
        $stmt = $conn->prepare("SELECT username FROM usersinfo WHERE id = ?");
        $stmt->bind_param("s", $uid);
        $stmt->execute();
        $result = $stmt->get_result();

         while($row = $result->fetch_assoc())  { //fetch_assoc() function fetches a result row as an associative array - here we are putting it in a loop
          echo $row['username'];
         }

        $stmt->close();
        $conn->close();
    }

If we are to retrieve just one result, assuming there are no duplicates we can use below without loop.

   $uid = $_GET['id'];
    if (empty($uid)) {
        echo "Error: Wrong input of values.";
    } else {
        $stmt = $conn->prepare("SELECT username FROM usersinfo WHERE id = ?");
        $stmt->bind_param("s", $uid);
        $stmt->execute();
        $result = $stmt->get_result();

         $row = $result->fetch_assoc(); //fetch_assoc() function fetches a result row as an associative array
         echo $row['username']; //we can specify just to return specific column i.e. username

        $stmt->close();
        $conn->close();
    }


问:
how to scrape all data from this table and store it into php array
how to scrape all data from this table and store it into php array </p>

<p>i have tried this code but it pulls data from other tables also and makes data unusable </p>

```<code><?php
error_reporting(0);
$htmlContent = file_get_contents("https://tools.tracemyip.org/search--ip/list:-v-:gTr=1&gNr=50");

    $DOM = new DOMDocument();
    $DOM->loadHTML($htmlContent);

    $Header = $DOM->getElementsByTagName('th');
    $Detail = $DOM->getElementsByTagName('span');

    //#Get header name of the table
    foreach($Header as $NodeHeader)
    {
        $aDataTableHeaderHTML[] = trim($NodeHeader->textContent);
    }
    //print_r($aDataTableHeaderHTML); die();

    //#Get row data/detail table without header name as key
    $i = 0;
    $j = 0;
    foreach($Detail as $sNodeDetail)
    {
        $aDataTableDetailHTML[$j][] = trim($sNodeDetail->textContent);
        $i = $i + 1;
        $j = $i % count($aDataTableHeaderHTML) == 0 ? $j + 1 : $j;
    }
    //print_r($aDataTableDetailHTML); die();

    //#Get row data/detail table with header name as key and outer array index as row number
    for($i = 0; $i < count($aDataTableDetailHTML); $i++)
    {
        for($j = 0; $j < count($aDataTableHeaderHTML); $j++)
        {
            $aTempData[$i][$aDataTableHeaderHTML[$j]] = $aDataTableDetailHTML[$i][$j];
        }
    }
    $aDataTableDetailHTML = $aTempData; unset($aTempData);
    print_r($aDataTableDetailHTML); die();
?>
</code>```

答:
This code works. I'm not sure if the ID of the table will ever change.

<?php
libxml_use_internal_errors(true);

if (!file_exists('table.html')) {
    file_put_contents('table.html', file_get_contents('https://tools.tracemyip.org/search--ip/list:-v-:gTr=1&gNr=50'));
}

$html = file_get_contents('table.html');

$doc = new DOMDocument();
$doc->loadHTML($html);

$table = $doc->getElementById('tlzRDTIPv4');

$thead = $table->getElementsByTagName('th');
$rows = $table->getElementsByTagName('tr');

$data = [];
$cols = [];

foreach ($thead as $th) {
    $cols[] = $th->textContent;
}

foreach ($rows as $row) {
    $tds = $row->getElementsByTagName('td');
    if (count($tds) == 0) // Ignore "thead > tr"
        continue;
    $row_data = [];
    for ($i = 0; $i < count($tds); $i++) {
        $row_data[$cols[$i]] = $tds[$i]->textContent;
    }
    $data[] = $row_data;
}

print_r($data);
```

```Array
(
    [0] => Array
        (
            [ID] => 1
            [IP Address] => 195.114.148.11
            [Organization / ISP] => Private Joint Stock Company datagroup
            [Country] => Ukraine
            [State] => Kyiv City
            [City] => - - -
            [Timezone] => Europe/Kiev
            [Browser] => Chrome 80.0.3987.99
            [Operating System] => Android, 9
            [Bot/spider] => No
        )

    [1] => Array
        (
            [ID] => 2
            [IP Address] => 102.184.99.182
            [Organization / ISP] => Vodafone Egypt
            [Country] => Egypt
            [State] => Cairo Governorate
            [City] => Cairo
            [Timezone] => Africa/Cairo
            [Browser] => Chrome 80.0.3987.99
            [Operating System] => Android, 9
            [Bot/spider] => No
        )
)
```

Body:
how to scrape all data from this table and store it into php array

i have tried this code but it pulls data from other tables also and makes data unusable

<?php
error_reporting(0);
$htmlContent = file_get_contents("https://tools.tracemyip.org/search--ip/list:-v-:gTr=1&gNr=50");

    $DOM = new DOMDocument();
    $DOM->loadHTML($htmlContent);

    $Header = $DOM->getElementsByTagName('th');
    $Detail = $DOM->getElementsByTagName('span');

    //#Get header name of the table
    foreach($Header as $NodeHeader)
    {
        $aDataTableHeaderHTML[] = trim($NodeHeader->textContent);
    }
    //print_r($aDataTableHeaderHTML); die();

    //#Get row data/detail table without header name as key
    $i = 0;
    $j = 0;
    foreach($Detail as $sNodeDetail)
    {
        $aDataTableDetailHTML[$j][] = trim($sNodeDetail->textContent);
        $i = $i + 1;
        $j = $i % count($aDataTableHeaderHTML) == 0 ? $j + 1 : $j;
    }
    //print_r($aDataTableDetailHTML); die();

    //#Get row data/detail table with header name as key and outer array index as row number
    for($i = 0; $i < count($aDataTableDetailHTML); $i++)
    {
        for($j = 0; $j < count($aDataTableHeaderHTML); $j++)
        {
            $aTempData[$i][$aDataTableHeaderHTML[$j]] = $aDataTableDetailHTML[$i][$j];
        }
    }
    $aDataTableDetailHTML = $aTempData; unset($aTempData);
    print_r($aDataTableDetailHTML); die();
?>

Body:
This code works. I'm not sure if the ID of the table will ever change.

<?php
libxml_use_internal_errors(true);

if (!file_exists('table.html')) {
    file_put_contents('table.html', file_get_contents('https://tools.tracemyip.org/search--ip/list:-v-:gTr=1&gNr=50'));
}

$html = file_get_contents('table.html');

$doc = new DOMDocument();
$doc->loadHTML($html);

$table = $doc->getElementById('tlzRDTIPv4');

$thead = $table->getElementsByTagName('th');
$rows = $table->getElementsByTagName('tr');

$data = [];
$cols = [];

foreach ($thead as $th) {
    $cols[] = $th->textContent;
}

foreach ($rows as $row) {
    $tds = $row->getElementsByTagName('td');
    if (count($tds) == 0) // Ignore "thead > tr"
        continue;
    $row_data = [];
    for ($i = 0; $i < count($tds); $i++) {
        $row_data[$cols[$i]] = $tds[$i]->textContent;
    }
    $data[] = $row_data;
}

print_r($data);

Array
(
    [0] => Array
        (
            [ID] => 1
            [IP Address] => 195.114.148.11
            [Organization / ISP] => Private Joint Stock Company datagroup
            [Country] => Ukraine
            [State] => Kyiv City
            [City] => - - -
            [Timezone] => Europe/Kiev
            [Browser] => Chrome 80.0.3987.99
            [Operating System] => Android, 9
            [Bot/spider] => No
        )

    [1] => Array
        (
            [ID] => 2
            [IP Address] => 102.184.99.182
            [Organization / ISP] => Vodafone Egypt
            [Country] => Egypt
            [State] => Cairo Governorate
            [City] => Cairo
            [Timezone] => Africa/Cairo
            [Browser] => Chrome 80.0.3987.99
            [Operating System] => Android, 9
            [Bot/spider] => No
        )
)


问:
How can I fill down in kusto
I would like my kusto query to remember and return, i.e. fill-down, the last non-null or non-empty value when I parse or extract a field from a log as below.

```<code>datatable (Date:datetime, LogEntry:string) [
  datetime(1910-06-11), "version: 1.0",
  datetime(1930-01-01), "starting foo",
  datetime(1953-01-01), "ending foo",
  datetime(1910-06-11), "version: 2.0",
  datetime(1930-01-01), "starting foo",
  datetime(1953-01-01), "ending foo" ]
| parse LogEntry with 'version: ' Version
| project Date, Version, LogEntry
</code>```

Is there a way to do this?

答:
Sorry for the delay, are you looking for something like this:

```<code>let sampleData = datatable (Date:datetime, LogEntry:string) [
  datetime(1910-06-11), "version: 1.1",
  datetime(1930-01-01), "starting foo",
  datetime(1953-01-01), "ending foo",
  datetime(1910-06-11), "version: 2.1",
  datetime(1930-01-01), "starting foo",
  datetime(1953-01-01), "ending foo",
  datetime(1950-01-01), "version: 3.1"]
| parse LogEntry with 'version: ' Version:double
| order by Date asc
| extend rn = row_number()
| extend rn = iif(isempty(Version),0 , rn)
| extend rn = row_cumsum(rn, rn!=0);
sampleData
| project-away Version
| lookup (sampleData | where isnotempty(Version) ) on rn
| project-away Date1, LogEntry1, rn
</code>```

here is the output:

Title: How can I fill down in kusto

Body:
I would like my kusto query to remember and return, i.e. fill-down, the last non-null or non-empty value when I parse or extract a field from a log as below.

datatable (Date:datetime, LogEntry:string) [
  datetime(1910-06-11), "version: 1.0",
  datetime(1930-01-01), "starting foo",
  datetime(1953-01-01), "ending foo",
  datetime(1910-06-11), "version: 2.0",
  datetime(1930-01-01), "starting foo",
  datetime(1953-01-01), "ending foo" ]
| parse LogEntry with 'version: ' Version
| project Date, Version, LogEntry

Is there a way to do this?

Body:
Sorry for the delay, are you looking for something like this:

let sampleData = datatable (Date:datetime, LogEntry:string) [
  datetime(1910-06-11), "version: 1.1",
  datetime(1930-01-01), "starting foo",
  datetime(1953-01-01), "ending foo",
  datetime(1910-06-11), "version: 2.1",
  datetime(1930-01-01), "starting foo",
  datetime(1953-01-01), "ending foo",
  datetime(1950-01-01), "version: 3.1"]
| parse LogEntry with 'version: ' Version:double
| order by Date asc
| extend rn = row_number()
| extend rn = iif(isempty(Version),0 , rn)
| extend rn = row_cumsum(rn, rn!=0);
sampleData
| project-away Version
| lookup (sampleData | where isnotempty(Version) ) on rn
| project-away Date1, LogEntry1, rn

here is the output:


问:
FirebaseRecyclerAdapter not working or showing any data
I'm a novice in Android programming. I've been able to create a userprofile table in firebase and can show user data from userprofile table in recyclerview but I cannot get my products table named "Arbrg" to show in firebaserecyclerview. Please tell me where I've done wrong in the codes. Your help is highly appreciated.

My codes as follow:

viewARBRG.java ( called on Button pressed )

```<code>package com.example.fb_app2;

import androidx.annotation.NonNull;
import androidx.appcompat.app.AppCompatActivity;
import androidx.recyclerview.widget.LinearLayoutManager;
import androidx.recyclerview.widget.RecyclerView;

import android.content.Intent;
import android.os.Bundle;
import android.view.LayoutInflater;
import android.view.View;
import android.view.ViewGroup;
import android.widget.Toast;

import com.firebase.ui.database.FirebaseRecyclerAdapter;
import com.firebase.ui.database.FirebaseRecyclerOptions;
import com.google.firebase.database.DatabaseReference;
import com.google.firebase.database.FirebaseDatabase;

public class viewARBRG extends AppCompatActivity {
    LinearLayoutManager mLinearLayoutManager;
    RecyclerView mRecyclerView;
    FirebaseDatabase mFirebaseDatabase;
    DatabaseReference mDatabaseReference;
    FirebaseRecyclerAdapter<MasterBarangClass, ViewHolderARBRG> firebaseRecyclerAdapter;
    FirebaseRecyclerOptions<MasterBarangClass> options;

    @Override
    protected void onStart() {
        super.onStart();
        firebaseRecyclerAdapter.startListening();

    }

    @Override
    protected void onStop() {
        super.onStop();
        firebaseRecyclerAdapter.stopListening();
    }

    @Override
    public void onBackPressed() {
    }

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_view_arbrg);

        mLinearLayoutManager = new LinearLayoutManager(this);
        mLinearLayoutManager.setReverseLayout(true);
        mLinearLayoutManager.setStackFromEnd(true);

        mRecyclerView = findViewById(R.id.recyclerView_arbrg);
        mFirebaseDatabase = FirebaseDatabase.getInstance();

        showData();
    }

    private void showData() {
        mDatabaseReference = mFirebaseDatabase.getReference("Arbrg");

        options = new FirebaseRecyclerOptions.Builder<MasterBarangClass>()
                .setQuery(mDatabaseReference, MasterBarangClass.class).build();

        Toast.makeText(getApplicationContext(), "Table : " + mDatabaseReference.toString(), Toast.LENGTH_SHORT).show();

        firebaseRecyclerAdapter = new FirebaseRecyclerAdapter<MasterBarangClass, ViewHolderARBRG>(options){
            @Override
            protected void onBindViewHolder(@NonNull ViewHolderARBRG viewHolderARBRG, int i, @NonNull MasterBarangClass masterBarangClass) {
                final String barcode = masterBarangClass.getBarcode();
                final String divbrg  = masterBarangClass.getDivbrg();
                final String hrgjual = masterBarangClass.getHrgjual();
                final String jnsbrg  = masterBarangClass.getJnsbrg();
                final String kdbrg = masterBarangClass.getKdbrg();
                final String nmbrg = masterBarangClass.getNmbrg();

                viewHolderARBRG.setDetails(viewARBRG.this,
                        masterBarangClass.getBarcode(),
                        masterBarangClass.getDivbrg(),
                        masterBarangClass.getHrgjual(),
                        masterBarangClass.getJnsbrg(),
                        masterBarangClass.getKdbrg(),
                        masterBarangClass.getNmbrg());

                viewHolderARBRG.mView.setOnClickListener(new View.OnClickListener() {
                    @Override
                    public void onClick(View v) {
                        Intent intent = new Intent(getApplicationContext(), showuser2klik.class);
                        intent.putExtra("barcode", barcode);
                        intent.putExtra("divbrg", divbrg);
                        intent.putExtra("hrgjual", hrgjual);
                        intent.putExtra("jnsbrg", jnsbrg);
                        intent.putExtra("kdbrg", kdbrg);
                        intent.putExtra("nmbrg", nmbrg);
                        startActivity(intent);
                    }
                });
            }
            @NonNull
            @Override
            public ViewHolderARBRG onCreateViewHolder(@NonNull ViewGroup parent, int viewType) {
                final View itemView = LayoutInflater.from(parent.getContext()).inflate(R.layout.row_arbrg, parent, false);

                final ViewHolderARBRG viewHolder = new ViewHolderARBRG(itemView);

                viewHolder.setOnClickListener(new ViewHolderARBRG.ClickListener() {
                    @Override
                    public void onItemClick(View view, int position) {
                        Toast.makeText(getApplicationContext(), "clicked on " +position, Toast.LENGTH_SHORT).show();
                    }

                    @Override
                    public void onItemLongClick(View view, int position) {
                        Toast.makeText(getApplicationContext(), "onItem Long click...", Toast.LENGTH_SHORT).show();
                    }
                });
                return viewHolder;
            }
        };
        mRecyclerView.setLayoutManager(mLinearLayoutManager);
        firebaseRecyclerAdapter.startListening();
        mRecyclerView.setAdapter(firebaseRecyclerAdapter);
    }

    public void back2main(View view) {
        finish();
        Intent int1 = new Intent(viewARBRG.this, MenuMasterActivity.class);
        startActivity(int1);
    }
}
</code>```

activity_view_arbrg.xml

```<code><?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    xmlns:tools="http://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:orientation="vertical"
    tools:context=".viewARBRG">

    <TextView
        android:id="@+id/textView3"
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:text="RecyclerView - MASTER BARANG VIEW" />
    <TextView
        android:id="@+id/textViewUserList"
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:textSize="20dp"
        android:text="LIST BARANG (ALL)" />
    <Button
        android:id="@+id/buttonBack"
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:onClick="back2main"
        android:text="BACK" />

    <androidx.recyclerview.widget.RecyclerView
        android:id="@+id/recyclerView_arbrg"
        android:layout_width="match_parent"
        android:layout_height="wrap_content" />
</LinearLayout>

</code>```

MasterBarangClass.java ( Model class )

```<code>package com.example.fb_app2;

public class MasterBarangClass {
    public String barcode, divbrg, hrgjual, jnsbrg,  kdbrg, nmbrg;

    public MasterBarangClass(){}

    public MasterBarangClass(String barcode, String divbrg, String hrgjual, String jnsbrg, String kdbrg, String nmbrg) {
        this.barcode = barcode;
        this.divbrg = divbrg;
        this.hrgjual = hrgjual;
        this.jnsbrg = jnsbrg;
        this.kdbrg = kdbrg;
        this.nmbrg = nmbrg;
    }

    public String getBarcode() {
        return barcode;
    }
    public void setBarcode(String barcode) {
        this.barcode = barcode;
    }

    public String getDivbrg() {
        return divbrg;
    }
    public void setDivbrg(String divbrg) {
        this.divbrg = divbrg;
    }

    public String getHrgjual() {
        return hrgjual;
    }
    public void setHrgjual(String hrgjual) {
        this.hrgjual = hrgjual;
    }

    public String getJnsbrg() {
        return jnsbrg;
    }
    public void setJnsbrg(String jnsbrg) {
        this.jnsbrg = jnsbrg;
    }

    public String getKdbrg() {
        return kdbrg;
    }
    public void setKdbrg(String kdbrg) {
        this.kdbrg = kdbrg;
    }

    public String getNmbrg() {
        return nmbrg;
    }
    public void setNmbrg(String nmbrg) {
        this.nmbrg = nmbrg;
    }
}

</code>```

row_arbrg.XML ( Layout file )

```<code><?xml version="1.0" encoding="utf-8"?>
<androidx.cardview.widget.CardView xmlns:android="http://schemas.android.com/apk/res/android"
    android:layout_width="match_parent"
    android:layout_height="200dp"
    app:cardBackgroundColor="#fff"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    app:cardCornerRadius="3dp"
    app:cardElevation="3dp"
    app:cardUseCompatPadding="true"
    app:contentPadding="5dp"
    android:layout_margin="10dp">

    <RelativeLayout
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:padding="5dp" >

        <TextView
            android:layout_width="wrap_content"
            android:layout_height="wrap_content"
            android:id="@+id/ra_tvNmbrg"
            android:text="Nama Barang"
            android:textSize="18sp"
            android:textColor="#000"
            android:textStyle="bold"
            android:layout_marginLeft="10dp"
            android:layout_marginStart="10dp"/>

        <ImageView
            android:layout_width="110dp"
            android:layout_height="130dp"
            android:id="@+id/imageIv"
            android:src="@mipmap/ic_launcher"
            android:layout_below="@id/ra_tvNmbrg"/>

        <TextView
            android:layout_width="wrap_content"
            android:layout_height="wrap_content"
            android:id="@+id/ra_tvKdbrg"
            android:text="Kode Barang"
            android:textSize="18sp"
            android:textColor="#000"
            android:layout_toRightOf="@id/imageIv"
            android:layout_toEndOf="@id/imageIv"
            android:layout_marginStart="10dp"
            android:layout_marginTop="20dp"/>

        <TextView
            android:id="@+id/ra_tvBarcode"
            android:layout_width="wrap_content"
            android:layout_height="wrap_content"
            android:layout_below="@id/ra_tvKdbrg"
            android:layout_marginStart="10dp"
            android:layout_marginTop="0dp"
            android:layout_toEndOf="@id/imageIv"
            android:layout_toRightOf="@id/imageIv"
            android:text="Bar code"
            android:textColor="#000"
            android:textSize="18sp" />
        <TextView
            android:id="@+id/ra_tvHargaJual"
            android:layout_width="wrap_content"
            android:layout_height="wrap_content"
            android:layout_below="@id/ra_tvBarcode"
            android:layout_marginStart="10dp"
            android:layout_marginTop="0dp"
            android:layout_toEndOf="@id/imageIv"
            android:layout_toRightOf="@id/imageIv"
            android:text="Harga Jual"
            android:textColor="#000"
            android:textSize="18sp" />
        <TextView
            android:id="@+id/ra_tvDivBrg"
            android:layout_width="wrap_content"
            android:layout_height="wrap_content"
            android:layout_below="@id/ra_tvHargaJual"
            android:layout_marginStart="10dp"
            android:layout_marginTop="0dp"
            android:layout_toEndOf="@id/imageIv"
            android:layout_toRightOf="@id/imageIv"
            android:text="Divisi barang"
            android:textColor="#000"
            android:textSize="18sp" />
        <TextView
            android:id="@+id/ra_tvJnsBrg"
            android:layout_width="wrap_content"
            android:layout_height="wrap_content"
            android:layout_below="@id/ra_tvDivBrg"
            android:layout_marginStart="10dp"
            android:layout_marginTop="0dp"
            android:layout_toEndOf="@id/imageIv"
            android:layout_toRightOf="@id/imageIv"
            android:text="Jenis barang"
            android:textColor="#000"
            android:textSize="18sp" />
    </RelativeLayout>

</androidx.cardview.widget.CardView>

</code>```

ViewHolderARBRG.java

```<code>package com.example.fb_app2;

import android.content.Context;
import android.view.View;
import android.widget.TextView;

import androidx.annotation.NonNull;
import androidx.recyclerview.widget.RecyclerView;

public class ViewHolderARBRG extends RecyclerView.ViewHolder {
    View mView;

    public ViewHolderARBRG(@NonNull View itemView) {
        super(itemView);
        mView = itemView;

        itemView.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View view) {
                mClickListener.onItemClick(view, getAdapterPosition()); }
        });

        itemView.setOnLongClickListener(new View.OnLongClickListener() {
            @Override
            public boolean onLongClick(View view) {
                mClickListener.onItemLongClick(view, getAdapterPosition());
                return true;  }
        });
    }

    public void setDetails(Context ctx, String barcode, String divbrg, String hrgjual, String jnsbrg, String kdbrg, String nmbrg) {
        TextView mbarcode = mView.findViewById(R.id.ra_tvBarcode);
        TextView mdivbrg = mView.findViewById(R.id.ra_tvDivBrg);
        TextView mhrgjual = mView.findViewById(R.id.ra_tvHargaJual);
        TextView mjnsbrg = mView.findViewById(R.id.ra_tvJnsBrg);
        TextView mkdbrg = mView.findViewById(R.id.ra_tvKdbrg);
        TextView mnmbrg = mView.findViewById(R.id.ra_tvNmbrg);

        mbarcode.setText(barcode);
        mdivbrg.setText(divbrg);
        mhrgjual.setText(hrgjual);
        mjnsbrg.setText(jnsbrg);
        mkdbrg.setText(kdbrg);
        mnmbrg.setText(nmbrg);
    }

    private ViewHolderARBRG.ClickListener mClickListener;
    public interface ClickListener {
        void onItemClick(View view, int position);
        void onItemLongClick(View view, int position);
    }

    public void setOnClickListener(ViewHolderARBRG.ClickListener clickListener){
        mClickListener = clickListener;
    }
}

</code>```

Database Structure is as follow :

```<code>{
  "ArBrg" : {
    "-M0CKcYgvjxeYF6sGCma" : {
      "barcode" : "001",
      "divbrg" : "Alba satu",
      "hrgjual" : "899787",
      "jnsbrg" : "ALBA",
      "kdbrg" : "AAM",
      "nmbrg" : "500000"
    }
  },
  "Arsale" : {
    "-M02z2kgntKpso966nVB" : {
      "kdsale" : "SSR",
      "nmsale" : "Showroom salesperson"
    },
    "-M02z5me5mwau1jSMtlV" : {
      "kdsale" : "ANDI",
      "nmsale" : "ANDI"
    },
    "-M02z899rEkNHbjfANi7" : {
      "kdsale" : "BUDI",
      "nmsale" : "BUDI"
    }
  },
  "DivBrg" : {
    "-M02sY8LQIykdvaQph3a" : {
      "divbrg" : "LORUS",
      "nmdivbrg" : "LORUS"
    },
    "-M02scwzlD5-iFz_7r0j" : {
      "divbrg" : "ALBA",
      "nmdivbrg" : "ALBA"
    },
    "-M02sgYbypIab-m4gTAx" : {
      "divbrg" : "SEIKO",
      "nmdivbrg" : "SEIKO"
    },
    "-M02skAS79c-xKK_xdag" : {
      "divbrg" : "ASTRON",
      "nmdivbrg" : "ASTRON"
    },
    "vSrawft4GiUgMO1Z7ZZTwaU50H42" : {
      "divbrg" : "JEMIS",
      "nmdivbrg" : "JEMIS"
    }
  },
  "JnsBrg" : {
    "-M02w-iyneYG_rjxFDj4" : {
      "divbrg" : "ALBA",
      "jnsbrg" : "AAL",
      "nmjnsbrg" : "Analog Ladies"
    },
    "-M02w3YsFzX6SWjz0z0m" : {
      "divbrg" : "ALBA",
      "jnsbrg" : "AAM",
      "nmjnsbrg" : "Analog Men"
    },
    "-M02wC9M0cjJKEKbQHJl" : {
      "divbrg" : "SEIKO",
      "jnsbrg" : "KTL",
      "nmjnsbrg" : "kinetic ladies Seiko"
    }
  },
  "UserProfil" : {
    "2GGGxrGCQkQqaK9LFIBSdJJQ22i2" : {
      "email" : "maestro.info@gmail.com",
      "name" : "budi handoyo",
      "phone" : "+6281888112576"
    },
    "9RHvuwMVbJRkxYZsx2j5f9Gqlwx1" : {
      "email" : "maestro280620@gmail.com",
      "memail" : "maestro280620@gmail.com",
      "mname" : "Api Email 2",
      "mphone" : "+6282288887529",
      "name" : "Api Email 2",
      "phone" : "+6282225397529"
    },
    "GTxmtkcY5LPBWuUK9su0JQ8Dtcv2" : {
      "email" : "celia.candr@yahoo.com",
      "name" : "celia ",
      "phone" : "+6281255082289"
    },
    "JBmBE4jd1cXARJJtzP7ZpMWi3TU2" : {
      "email" : "maestro.infosis@gmail.com",
      "memail" : "maestro.infosis@gmail.com",
      "mname" : "Andi Hidajat 2",
      "mphone" : "+6281888992276",
      "name" : "Andi Hidajat 2",
      "phone" : "+6281288972576"
    },
    "vSrawft4GiUgMO1Z7ZZTwaU50H42" : {
      "email" : "maestro_info@yahoo.com",
      "memail" : "maestro_info@yahoo.com",
      "mname" : "Andi Hidajat",
      "mphone" : "+62818902712",
      "name" : "Andi Hidajat",
      "phone" : "+628189888512"
    }
  }
}

</code>```

When building the APK, it shows no error but the product data from Arbrg firebase table is not showing when I execute the viewARBRG.java. Any help

Regards,
Andi

答:
Your database reference is not correct. It's case sensitive. So, Please correct it.

Use

```<code>mDatabaseReference = mFirebaseDatabase.getReference("ArBrg");
</code>```

Instead of

```<code>mDatabaseReference = mFirebaseDatabase.getReference("Arbrg");
</code>```

Body:
I'm a novice in Android programming. I've been able to create a userprofile table in firebase and can show user data from userprofile table in recyclerview but I cannot get my products table named "Arbrg" to show in firebaserecyclerview. Please tell me where I've done wrong in the codes. Your help is highly appreciated.

My codes as follow:

viewARBRG.java ( called on Button pressed )

package com.example.fb_app2;

import androidx.annotation.NonNull;
import androidx.appcompat.app.AppCompatActivity;
import androidx.recyclerview.widget.LinearLayoutManager;
import androidx.recyclerview.widget.RecyclerView;

import android.content.Intent;
import android.os.Bundle;
import android.view.LayoutInflater;
import android.view.View;
import android.view.ViewGroup;
import android.widget.Toast;

import com.firebase.ui.database.FirebaseRecyclerAdapter;
import com.firebase.ui.database.FirebaseRecyclerOptions;
import com.google.firebase.database.DatabaseReference;
import com.google.firebase.database.FirebaseDatabase;

public class viewARBRG extends AppCompatActivity {
    LinearLayoutManager mLinearLayoutManager;
    RecyclerView mRecyclerView;
    FirebaseDatabase mFirebaseDatabase;
    DatabaseReference mDatabaseReference;
    FirebaseRecyclerAdapter<MasterBarangClass, ViewHolderARBRG> firebaseRecyclerAdapter;
    FirebaseRecyclerOptions<MasterBarangClass> options;

    @Override
    protected void onStart() {
        super.onStart();
        firebaseRecyclerAdapter.startListening();

    }

    @Override
    protected void onStop() {
        super.onStop();
        firebaseRecyclerAdapter.stopListening();
    }

    @Override
    public void onBackPressed() {
    }

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_view_arbrg);

        mLinearLayoutManager = new LinearLayoutManager(this);
        mLinearLayoutManager.setReverseLayout(true);
        mLinearLayoutManager.setStackFromEnd(true);

        mRecyclerView = findViewById(R.id.recyclerView_arbrg);
        mFirebaseDatabase = FirebaseDatabase.getInstance();

        showData();
    }

    private void showData() {
        mDatabaseReference = mFirebaseDatabase.getReference("Arbrg");

        options = new FirebaseRecyclerOptions.Builder<MasterBarangClass>()
                .setQuery(mDatabaseReference, MasterBarangClass.class).build();

        Toast.makeText(getApplicationContext(), "Table : " + mDatabaseReference.toString(), Toast.LENGTH_SHORT).show();

        firebaseRecyclerAdapter = new FirebaseRecyclerAdapter<MasterBarangClass, ViewHolderARBRG>(options){
            @Override
            protected void onBindViewHolder(@NonNull ViewHolderARBRG viewHolderARBRG, int i, @NonNull MasterBarangClass masterBarangClass) {
                final String barcode = masterBarangClass.getBarcode();
                final String divbrg  = masterBarangClass.getDivbrg();
                final String hrgjual = masterBarangClass.getHrgjual();
                final String jnsbrg  = masterBarangClass.getJnsbrg();
                final String kdbrg = masterBarangClass.getKdbrg();
                final String nmbrg = masterBarangClass.getNmbrg();

                viewHolderARBRG.setDetails(viewARBRG.this,
                        masterBarangClass.getBarcode(),
                        masterBarangClass.getDivbrg(),
                        masterBarangClass.getHrgjual(),
                        masterBarangClass.getJnsbrg(),
                        masterBarangClass.getKdbrg(),
                        masterBarangClass.getNmbrg());

                viewHolderARBRG.mView.setOnClickListener(new View.OnClickListener() {
                    @Override
                    public void onClick(View v) {
                        Intent intent = new Intent(getApplicationContext(), showuser2klik.class);
                        intent.putExtra("barcode", barcode);
                        intent.putExtra("divbrg", divbrg);
                        intent.putExtra("hrgjual", hrgjual);
                        intent.putExtra("jnsbrg", jnsbrg);
                        intent.putExtra("kdbrg", kdbrg);
                        intent.putExtra("nmbrg", nmbrg);
                        startActivity(intent);
                    }
                });
            }
            @NonNull
            @Override
            public ViewHolderARBRG onCreateViewHolder(@NonNull ViewGroup parent, int viewType) {
                final View itemView = LayoutInflater.from(parent.getContext()).inflate(R.layout.row_arbrg, parent, false);

                final ViewHolderARBRG viewHolder = new ViewHolderARBRG(itemView);

                viewHolder.setOnClickListener(new ViewHolderARBRG.ClickListener() {
                    @Override
                    public void onItemClick(View view, int position) {
                        Toast.makeText(getApplicationContext(), "clicked on " +position, Toast.LENGTH_SHORT).show();
                    }

                    @Override
                    public void onItemLongClick(View view, int position) {
                        Toast.makeText(getApplicationContext(), "onItem Long click...", Toast.LENGTH_SHORT).show();
                    }
                });
                return viewHolder;
            }
        };
        mRecyclerView.setLayoutManager(mLinearLayoutManager);
        firebaseRecyclerAdapter.startListening();
        mRecyclerView.setAdapter(firebaseRecyclerAdapter);
    }

    public void back2main(View view) {
        finish();
        Intent int1 = new Intent(viewARBRG.this, MenuMasterActivity.class);
        startActivity(int1);
    }
}

activity_view_arbrg.xml

<?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    xmlns:tools="http://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:orientation="vertical"
    tools:context=".viewARBRG">

    <TextView
        android:id="@+id/textView3"
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:text="RecyclerView - MASTER BARANG VIEW" />
    <TextView
        android:id="@+id/textViewUserList"
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:textSize="20dp"
        android:text="LIST BARANG (ALL)" />
    <Button
        android:id="@+id/buttonBack"
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:onClick="back2main"
        android:text="BACK" />

    <androidx.recyclerview.widget.RecyclerView
        android:id="@+id/recyclerView_arbrg"
        android:layout_width="match_parent"
        android:layout_height="wrap_content" />
</LinearLayout>

MasterBarangClass.java ( Model class )

package com.example.fb_app2;

public class MasterBarangClass {
    public String barcode, divbrg, hrgjual, jnsbrg,  kdbrg, nmbrg;

    public MasterBarangClass(){}

    public MasterBarangClass(String barcode, String divbrg, String hrgjual, String jnsbrg, String kdbrg, String nmbrg) {
        this.barcode = barcode;
        this.divbrg = divbrg;
        this.hrgjual = hrgjual;
        this.jnsbrg = jnsbrg;
        this.kdbrg = kdbrg;
        this.nmbrg = nmbrg;
    }

    public String getBarcode() {
        return barcode;
    }
    public void setBarcode(String barcode) {
        this.barcode = barcode;
    }

    public String getDivbrg() {
        return divbrg;
    }
    public void setDivbrg(String divbrg) {
        this.divbrg = divbrg;
    }

    public String getHrgjual() {
        return hrgjual;
    }
    public void setHrgjual(String hrgjual) {
        this.hrgjual = hrgjual;
    }

    public String getJnsbrg() {
        return jnsbrg;
    }
    public void setJnsbrg(String jnsbrg) {
        this.jnsbrg = jnsbrg;
    }

    public String getKdbrg() {
        return kdbrg;
    }
    public void setKdbrg(String kdbrg) {
        this.kdbrg = kdbrg;
    }

    public String getNmbrg() {
        return nmbrg;
    }
    public void setNmbrg(String nmbrg) {
        this.nmbrg = nmbrg;
    }
}

row_arbrg.XML ( Layout file )

<?xml version="1.0" encoding="utf-8"?>
<androidx.cardview.widget.CardView xmlns:android="http://schemas.android.com/apk/res/android"
    android:layout_width="match_parent"
    android:layout_height="200dp"
    app:cardBackgroundColor="#fff"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    app:cardCornerRadius="3dp"
    app:cardElevation="3dp"
    app:cardUseCompatPadding="true"
    app:contentPadding="5dp"
    android:layout_margin="10dp">

    <RelativeLayout
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:padding="5dp" >

        <TextView
            android:layout_width="wrap_content"
            android:layout_height="wrap_content"
            android:id="@+id/ra_tvNmbrg"
            android:text="Nama Barang"
            android:textSize="18sp"
            android:textColor="#000"
            android:textStyle="bold"
            android:layout_marginLeft="10dp"
            android:layout_marginStart="10dp"/>

        <ImageView
            android:layout_width="110dp"
            android:layout_height="130dp"
            android:id="@+id/imageIv"
            android:src="@mipmap/ic_launcher"
            android:layout_below="@id/ra_tvNmbrg"/>

        <TextView
            android:layout_width="wrap_content"
            android:layout_height="wrap_content"
            android:id="@+id/ra_tvKdbrg"
            android:text="Kode Barang"
            android:textSize="18sp"
            android:textColor="#000"
            android:layout_toRightOf="@id/imageIv"
            android:layout_toEndOf="@id/imageIv"
            android:layout_marginStart="10dp"
            android:layout_marginTop="20dp"/>

        <TextView
            android:id="@+id/ra_tvBarcode"
            android:layout_width="wrap_content"
            android:layout_height="wrap_content"
            android:layout_below="@id/ra_tvKdbrg"
            android:layout_marginStart="10dp"
            android:layout_marginTop="0dp"
            android:layout_toEndOf="@id/imageIv"
            android:layout_toRightOf="@id/imageIv"
            android:text="Bar code"
            android:textColor="#000"
            android:textSize="18sp" />
        <TextView
            android:id="@+id/ra_tvHargaJual"
            android:layout_width="wrap_content"
            android:layout_height="wrap_content"
            android:layout_below="@id/ra_tvBarcode"
            android:layout_marginStart="10dp"
            android:layout_marginTop="0dp"
            android:layout_toEndOf="@id/imageIv"
            android:layout_toRightOf="@id/imageIv"
            android:text="Harga Jual"
            android:textColor="#000"
            android:textSize="18sp" />
        <TextView
            android:id="@+id/ra_tvDivBrg"
            android:layout_width="wrap_content"
            android:layout_height="wrap_content"
            android:layout_below="@id/ra_tvHargaJual"
            android:layout_marginStart="10dp"
            android:layout_marginTop="0dp"
            android:layout_toEndOf="@id/imageIv"
            android:layout_toRightOf="@id/imageIv"
            android:text="Divisi barang"
            android:textColor="#000"
            android:textSize="18sp" />
        <TextView
            android:id="@+id/ra_tvJnsBrg"
            android:layout_width="wrap_content"
            android:layout_height="wrap_content"
            android:layout_below="@id/ra_tvDivBrg"
            android:layout_marginStart="10dp"
            android:layout_marginTop="0dp"
            android:layout_toEndOf="@id/imageIv"
            android:layout_toRightOf="@id/imageIv"
            android:text="Jenis barang"
            android:textColor="#000"
            android:textSize="18sp" />
    </RelativeLayout>

</androidx.cardview.widget.CardView>

ViewHolderARBRG.java

package com.example.fb_app2;

import android.content.Context;
import android.view.View;
import android.widget.TextView;

import androidx.annotation.NonNull;
import androidx.recyclerview.widget.RecyclerView;

public class ViewHolderARBRG extends RecyclerView.ViewHolder {
    View mView;

    public ViewHolderARBRG(@NonNull View itemView) {
        super(itemView);
        mView = itemView;

        itemView.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View view) {
                mClickListener.onItemClick(view, getAdapterPosition()); }
        });

        itemView.setOnLongClickListener(new View.OnLongClickListener() {
            @Override
            public boolean onLongClick(View view) {
                mClickListener.onItemLongClick(view, getAdapterPosition());
                return true;  }
        });
    }

    public void setDetails(Context ctx, String barcode, String divbrg, String hrgjual, String jnsbrg, String kdbrg, String nmbrg) {
        TextView mbarcode = mView.findViewById(R.id.ra_tvBarcode);
        TextView mdivbrg = mView.findViewById(R.id.ra_tvDivBrg);
        TextView mhrgjual = mView.findViewById(R.id.ra_tvHargaJual);
        TextView mjnsbrg = mView.findViewById(R.id.ra_tvJnsBrg);
        TextView mkdbrg = mView.findViewById(R.id.ra_tvKdbrg);
        TextView mnmbrg = mView.findViewById(R.id.ra_tvNmbrg);

        mbarcode.setText(barcode);
        mdivbrg.setText(divbrg);
        mhrgjual.setText(hrgjual);
        mjnsbrg.setText(jnsbrg);
        mkdbrg.setText(kdbrg);
        mnmbrg.setText(nmbrg);
    }

    private ViewHolderARBRG.ClickListener mClickListener;
    public interface ClickListener {
        void onItemClick(View view, int position);
        void onItemLongClick(View view, int position);
    }

    public void setOnClickListener(ViewHolderARBRG.ClickListener clickListener){
        mClickListener = clickListener;
    }
}

Database Structure is as follow :

{
  "ArBrg" : {
    "-M0CKcYgvjxeYF6sGCma" : {
      "barcode" : "001",
      "divbrg" : "Alba satu",
      "hrgjual" : "899787",
      "jnsbrg" : "ALBA",
      "kdbrg" : "AAM",
      "nmbrg" : "500000"
    }
  },
  "Arsale" : {
    "-M02z2kgntKpso966nVB" : {
      "kdsale" : "SSR",
      "nmsale" : "Showroom salesperson"
    },
    "-M02z5me5mwau1jSMtlV" : {
      "kdsale" : "ANDI",
      "nmsale" : "ANDI"
    },
    "-M02z899rEkNHbjfANi7" : {
      "kdsale" : "BUDI",
      "nmsale" : "BUDI"
    }
  },
  "DivBrg" : {
    "-M02sY8LQIykdvaQph3a" : {
      "divbrg" : "LORUS",
      "nmdivbrg" : "LORUS"
    },
    "-M02scwzlD5-iFz_7r0j" : {
      "divbrg" : "ALBA",
      "nmdivbrg" : "ALBA"
    },
    "-M02sgYbypIab-m4gTAx" : {
      "divbrg" : "SEIKO",
      "nmdivbrg" : "SEIKO"
    },
    "-M02skAS79c-xKK_xdag" : {
      "divbrg" : "ASTRON",
      "nmdivbrg" : "ASTRON"
    },
    "vSrawft4GiUgMO1Z7ZZTwaU50H42" : {
      "divbrg" : "JEMIS",
      "nmdivbrg" : "JEMIS"
    }
  },
  "JnsBrg" : {
    "-M02w-iyneYG_rjxFDj4" : {
      "divbrg" : "ALBA",
      "jnsbrg" : "AAL",
      "nmjnsbrg" : "Analog Ladies"
    },
    "-M02w3YsFzX6SWjz0z0m" : {
      "divbrg" : "ALBA",
      "jnsbrg" : "AAM",
      "nmjnsbrg" : "Analog Men"
    },
    "-M02wC9M0cjJKEKbQHJl" : {
      "divbrg" : "SEIKO",
      "jnsbrg" : "KTL",
      "nmjnsbrg" : "kinetic ladies Seiko"
    }
  },
  "UserProfil" : {
    "2GGGxrGCQkQqaK9LFIBSdJJQ22i2" : {
      "email" : "maestro.info@gmail.com",
      "name" : "budi handoyo",
      "phone" : "+6281888112576"
    },
    "9RHvuwMVbJRkxYZsx2j5f9Gqlwx1" : {
      "email" : "maestro280620@gmail.com",
      "memail" : "maestro280620@gmail.com",
      "mname" : "Api Email 2",
      "mphone" : "+6282288887529",
      "name" : "Api Email 2",
      "phone" : "+6282225397529"
    },
    "GTxmtkcY5LPBWuUK9su0JQ8Dtcv2" : {
      "email" : "celia.candr@yahoo.com",
      "name" : "celia ",
      "phone" : "+6281255082289"
    },
    "JBmBE4jd1cXARJJtzP7ZpMWi3TU2" : {
      "email" : "maestro.infosis@gmail.com",
      "memail" : "maestro.infosis@gmail.com",
      "mname" : "Andi Hidajat 2",
      "mphone" : "+6281888992276",
      "name" : "Andi Hidajat 2",
      "phone" : "+6281288972576"
    },
    "vSrawft4GiUgMO1Z7ZZTwaU50H42" : {
      "email" : "maestro_info@yahoo.com",
      "memail" : "maestro_info@yahoo.com",
      "mname" : "Andi Hidajat",
      "mphone" : "+62818902712",
      "name" : "Andi Hidajat",
      "phone" : "+628189888512"
    }
  }
}

When building the APK, it shows no error but the product data from Arbrg firebase table is not showing when I execute the viewARBRG.java. Any help

Regards,
Andi

Body:
Your database reference is not correct. It's case sensitive. So, Please correct it.

Use

mDatabaseReference = mFirebaseDatabase.getReference("ArBrg");

Instead of

mDatabaseReference = mFirebaseDatabase.getReference("Arbrg");


问:
Unity samsung galaxy s2 lag spike on first touch
I'm experiencing a huge lag spike on the Samsung galaxy s2 when I click on the button. This happens only on the first touch after that it's buttery smooth.

I have also tested on my Huawei p9 lite and it runs just fine without any lag spike.

Then I thought that something is wrong with my game so I have tested this on an empty scene with only a single canvas and one button and this issue was still present.

The profiler shows that Event.SystemUpdate is the culprit:

Any ideas how to solve this issue?

答:
I had a similar issue. But in my case, it was a call to DateTime.Now somewhere deep in the methods that was slow!
take a look at this:
https://issuetracker.unity3d.com/issues/ios-il2cpp-system-dot-datetime-dot-now-is-slow-on-the-first-run
I replaced the code with Time.time and it's fixed now. </p>