问:
Keras Tensorflow fails to learn simple linear relationship
I am fairly new to Tensorflow/Keras and am trying to set up an LSTM model. I have successfully run my code already, but my results have failed to give me meaningful results. I, therefore - as a test - let my LSTM network learn one of the features I am inputting. I am aware that the LSTM and relu use nonlinear relationships, however, I was still expecting the output to be somewhat similar to the input feature I was trying to learn which it is not at all.

I am using a modified version from what I learned on https://keras.io/getting-started/sequential-model-guide/

feature_set = features.iloc[:-3,:].transpose() #23 features
target_set = features.iloc[-4:,:].transpose().iloc[:,0] #picking the 23rd feature
X_train,X_test,y_train,y_test = train_test_split(feature_set, target_set, test_size=0.2, shuffle=False, random_state=42)

rnn_units = 256
batch_size = 1
features_dim = 23
output = 1
def build_model(rnn_units):
    model = tf.keras.Sequential([
            tf.keras.layers.Dense(rnn_units, batch_input_shape=[batch_size, None, features_dim], activation='relu'),
            tf.keras.layers.Dropout(0.1),
            tf.keras.layers.CuDNNLSTM(rnn_units,
                return_sequences=True,
                stateful=True),
            tf.keras.layers.Dropout(0.1),
            tf.keras.layers.CuDNNLSTM(rnn_units,
                return_sequences=True,
                stateful=True),
            tf.keras.layers.Dense(output)
          ])
    return model

model = build_model(rnn_units=rnn_units)

model.compile(optimizer = tf.train.AdamOptimizer(), loss = tf.keras.losses.mean_squared_error, metrics=['mse', 'mae', 'mape', 'cosine'])

reshape_train = int(X_train.values.shape[0]/batch_size)
reshape_test = int(X_test.values.shape[0]/batch_size)

history = model.fit(X_train.values[:reshape_train*batch_size].reshape(reshape_train*batch_size, -1, features_dim), y_train.values[:reshape_train*batch_size].reshape(reshape_train*batch_size, -1, output), epochs=EPOCHS, batch_size=batch_size, validation_data=(X_test.values[:reshape_test*batch_size].reshape(reshape_test*batch_size, 1, features_dim), y_test.values[:reshape_test*batch_size].reshape(reshape_test*batch_size, 1, output)), callbacks=[checkpoint_callback,tensorboard])
```

<p>As you can see I am inputting a feature set of 23 values and am trying to learn the 23rd feature. I am using 256 nodes in every layer, with one Dense layout at the beginning and at the end and 2 LSTM layers followed by Dropout layers.</p>

<p>I am using mean-square as it is supposed to be a regression on time series data.</p>

<p>This is, for instance, one run of my training:</p>

<pre class="lang-py prettyprint-override"><code>Epoch 5/5
10329/10329 [==============================] - 93s 9ms/sample - loss: 0.0182 - mean_squared_error: 0.0182 - mean_absolute_error: 0.0424 - mean_absolute_percentage_error: 94.4916 - cosine_proximity: -0.9032 - val_loss: 0.0193 - val_mean_squared_error: 0.0193 - val_mean_absolute_error: 0.0438 - val_mean_absolute_percentage_error: 58.2152 - val_cosine_proximity: -0.9443
</code>```

And when I run

result = model.predict(feature_set.values.reshape(-1, 1, features_dim))

feature_set.transpose().append(pd.DataFrame(result.reshape(-1), columns = ['Prediction 5min']).set_index(features.columns).transpose()).transpose()
```

<p>I get for instance</p>

<pre class="lang-py prettyprint-override"><code>2019-03-04 01:00:00 82.0105414589   0.0704929618    -0.1165011768   -0.3369084807   -1.8137642288   -0.2780955060   -4.3090711538   6.2721520391    9.5553857757    -1.2900340169   ... -29.8867675862  1.9178869544    -1.4765772054   1.0000000000    0.0000000000    0.0000000000    0.0000000000    0.0080950060    -0.3594492457   0.0056902645
</code>```

where the last 2 values should be equal but they are

-0.3594492457   0.0056902645
```

Any idea what I am doing wrong in my model? Can I use LSTM to learn such relationships?

Thanks!

答:
A few issues:

Typically, LSTM layers go at the start, followed by a few dense layers.

Also, the LSTM layer before the dense layer needs to have return_sequence set to False.

However, I'm not sure that they are the reason to cause this problem, I'm just pointing out the problems. I think it is more likely to be the dataset that is the issue, you should check if the dataset has the pattern.

Body:
I am fairly new to Tensorflow/Keras and am trying to set up an LSTM model. I have successfully run my code already, but my results have failed to give me meaningful results. I, therefore - as a test - let my LSTM network learn one of the features I am inputting. I am aware that the LSTM and relu use nonlinear relationships, however, I was still expecting the output to be somewhat similar to the input feature I was trying to learn which it is not at all.

I am using a modified version from what I learned on https://keras.io/getting-started/sequential-model-guide/

feature_set = features.iloc[:-3,:].transpose() #23 features
target_set = features.iloc[-4:,:].transpose().iloc[:,0] #picking the 23rd feature
X_train,X_test,y_train,y_test = train_test_split(feature_set, target_set, test_size=0.2, shuffle=False, random_state=42)

rnn_units = 256
batch_size = 1
features_dim = 23
output = 1
def build_model(rnn_units):
    model = tf.keras.Sequential([
            tf.keras.layers.Dense(rnn_units, batch_input_shape=[batch_size, None, features_dim], activation='relu'),
            tf.keras.layers.Dropout(0.1),
            tf.keras.layers.CuDNNLSTM(rnn_units,
                return_sequences=True,
                stateful=True),
            tf.keras.layers.Dropout(0.1),
            tf.keras.layers.CuDNNLSTM(rnn_units,
                return_sequences=True,
                stateful=True),
            tf.keras.layers.Dense(output)
          ])
    return model

model = build_model(rnn_units=rnn_units)

model.compile(optimizer = tf.train.AdamOptimizer(), loss = tf.keras.losses.mean_squared_error, metrics=['mse', 'mae', 'mape', 'cosine'])

reshape_train = int(X_train.values.shape[0]/batch_size)
reshape_test = int(X_test.values.shape[0]/batch_size)

history = model.fit(X_train.values[:reshape_train*batch_size].reshape(reshape_train*batch_size, -1, features_dim), y_train.values[:reshape_train*batch_size].reshape(reshape_train*batch_size, -1, output), epochs=EPOCHS, batch_size=batch_size, validation_data=(X_test.values[:reshape_test*batch_size].reshape(reshape_test*batch_size, 1, features_dim), y_test.values[:reshape_test*batch_size].reshape(reshape_test*batch_size, 1, output)), callbacks=[checkpoint_callback,tensorboard])

As you can see I am inputting a feature set of 23 values and am trying to learn the 23rd feature. I am using 256 nodes in every layer, with one Dense layout at the beginning and at the end and 2 LSTM layers followed by Dropout layers.

I am using mean-square as it is supposed to be a regression on time series data.

This is, for instance, one run of my training:

Epoch 5/5
10329/10329 [==============================] - 93s 9ms/sample - loss: 0.0182 - mean_squared_error: 0.0182 - mean_absolute_error: 0.0424 - mean_absolute_percentage_error: 94.4916 - cosine_proximity: -0.9032 - val_loss: 0.0193 - val_mean_squared_error: 0.0193 - val_mean_absolute_error: 0.0438 - val_mean_absolute_percentage_error: 58.2152 - val_cosine_proximity: -0.9443

And when I run

result = model.predict(feature_set.values.reshape(-1, 1, features_dim))

feature_set.transpose().append(pd.DataFrame(result.reshape(-1), columns = ['Prediction 5min']).set_index(features.columns).transpose()).transpose()

I get for instance

2019-03-04 01:00:00 82.0105414589   0.0704929618    -0.1165011768   -0.3369084807   -1.8137642288   -0.2780955060   -4.3090711538   6.2721520391    9.5553857757    -1.2900340169   ... -29.8867675862  1.9178869544    -1.4765772054   1.0000000000    0.0000000000    0.0000000000    0.0000000000    0.0080950060    -0.3594492457   0.0056902645

where the last 2 values should be equal but they are

-0.3594492457   0.0056902645

Any idea what I am doing wrong in my model? Can I use LSTM to learn such relationships?

Thanks!


来源: stackexchange

问:
Using Regex to split hashtag and search items
I have a regex expression to list the hashtags in a search entered into my app that looks like this:

```<code>text.match(/#[a-z]+/gi);
</code>```

and a regex expression to list all the words in a search

```<code>text.match(/\S+\s*/g);
</code>```

What I want to be able to do is to split hashtags and words into two separate lists so I want to be able to update the second regex expression so that it ignores hashtags. How would I achieve this?

For example, if the input is:

"Hello this is #my #question for today"

The expected output should be 2 arrays:

```<code>[Hello, this, is, for, today],
[my, question]
</code>```

I would also like to make the hashtag regex return the words without the hashtags if possible otherwise I could just string split the words in the array later if not possible

答:
You can simply split on space characters and than you can check if the word starts with # or not using this regex

```<code>`/^#.+/`
</code>```

The above regex with match # followed by atleast one character, if you want to match single # also as than you can simply use startsWith

```<code>/^#.+/.test(inp)   ---> inp.startsWith('#')
</code>```

let str = `Hello this is #my #question for today`

let {withHash,withOutHash} = str.split(/\s+/).reduce((op,inp)=>{
  let hash = /^#.+/.test(inp)
  let key = hash ? 'withHash' : 'withOutHash'
  op[key] = op[key] || []
  op[key].push(inp)
  return op
},{})

console.log(withHash)
console.log(withOutHash)```

ContentLicense: CC BY-SA 4.0

Body:
I have a regex expression to list the hashtags in a search entered into my app that looks like this:

text.match(/#[a-z]+/gi);

and a regex expression to list all the words in a search

text.match(/\S+\s*/g);

What I want to be able to do is to split hashtags and words into two separate lists so I want to be able to update the second regex expression so that it ignores hashtags. How would I achieve this?

For example, if the input is:

"Hello this is #my #question for today"

The expected output should be 2 arrays:

[Hello, this, is, for, today],
[my, question]

I would also like to make the hashtag regex return the words without the hashtags if possible otherwise I could just string split the words in the array later if not possible

Body:
You can simply split on space characters and than you can check if the word starts with # or not using this regex

`/^#.+/`

The above regex with match # followed by atleast one character, if you want to match single # also as than you can simply use startsWith

/^#.+/.test(inp)   ---> inp.startsWith('#')

let str = `Hello this is #my #question for today`

let {withHash,withOutHash} = str.split(/\s+/).reduce((op,inp)=>{
  let hash = /^#.+/.test(inp)
  let key = hash ? 'withHash' : 'withOutHash'
  op[key] = op[key] || []
  op[key].push(inp)
  return op
},{})

console.log(withHash)
console.log(withOutHash)


问:
UnsupportedAudioFileException from stereo but not two-channel audio
I'm working on doing some basic audio processing in Java using AudioInputStream. My program can correctly load in 16 bit integer PCM audio that's encoded with two tracks, but not with stereo. When presented with a file that (according to ffprobe) is

```<code>Input #0, wav, from XXX:
  Duration: 00:00:02.80, bitrate: 1411 kb/s
    Stream #0:0: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 44100 Hz, 2 channels, s16, 1411 kb/s
</code>```

Everything's fine. However, when presented with

```<code>  Duration: 01:19:15.58, bitrate: 1536 kb/s
    Stream #0:0: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 48000 Hz, stereo, s16, 1536 kb/s
</code>```

(which is the file I actually want to edit), it fails with

```<code>Exception in thread "main" javax.sound.sampled.UnsupportedAudioFileException: could not get audio input stream from input file
    at javax.sound.sampled.AudioSystem.getAudioInputStream(Unknown Source)
</code>```

The encoding is the same, and it seems unlikely the sample rate differences would cause the issue. It seems strongly like it's the stereo vs. 2 channels that's causing this exception, but stereo should be two channels, shouldn't it?

I can replicate this issue with

```<code>AudioSystem.getAudioInputStream(new File("file under test"));
</code>```

答:
You can build the corresponding AudioFormat manually and check it with isFormatSupported(AudioFormat)

Support for 44100 is standard in Java, but 48000 is more hit-or-miss and depends on the OS of the computer running the code. See 6.3 Audio Formats

Audacity works well as a tool for converting audio resources if you don't want to hassle the coding complications.

Body:
I'm working on doing some basic audio processing in Java using AudioInputStream. My program can correctly load in 16 bit integer PCM audio that's encoded with two tracks, but not with stereo. When presented with a file that (according to ffprobe) is

Input #0, wav, from XXX:
  Duration: 00:00:02.80, bitrate: 1411 kb/s
    Stream #0:0: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 44100 Hz, 2 channels, s16, 1411 kb/s

Everything's fine. However, when presented with

  Duration: 01:19:15.58, bitrate: 1536 kb/s
    Stream #0:0: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 48000 Hz, stereo, s16, 1536 kb/s

(which is the file I actually want to edit), it fails with

Exception in thread "main" javax.sound.sampled.UnsupportedAudioFileException: could not get audio input stream from input file
    at javax.sound.sampled.AudioSystem.getAudioInputStream(Unknown Source)

The encoding is the same, and it seems unlikely the sample rate differences would cause the issue. It seems strongly like it's the stereo vs. 2 channels that's causing this exception, but stereo should be two channels, shouldn't it?

I can replicate this issue with

AudioSystem.getAudioInputStream(new File("file under test"));


问:
Reading between the lines (perl or awk)
Hey all, sorry for posting this here I could not find an answer anywhere and my solution is not working. I have a log file being written in the following fashion (don't ask):

```<code>=============================

04-12-2011 11:37:10 SOMETHING_GOES_HERE

Variable1

Something Goes Here

=============================

04-12-2011 11:37:20 SOMETHING_GOES_HERE

Variable2

Anything different may be here

=============================

04-12-2011 11:37:30 SOMETHING_GOES_HERE

Variable3

is altogether different here

=============================
</code>```

What I'd like to do (either in perl or awk as this is an RTOS) is:

Take a look at the file, if Variable1 exists, then start at Variable2 and print everything between the equal signs:

E.g.

```<code>=============================
04-12-2011 11:37:10 SOMETHING_GOES_HERE

Variable1

Mary had a little lamb

=============================
04-12-2011 11:37:20 SOMETHING_GOES_HERE

Variable2

The cow jumped over the moon
=============================
</code>```

awk '/Mary had/{getline;getline;getline;print}'

will only print

```<code>04-12-2011 11:37:20 SOMETHING_GOES_HERE
</code>```

but what I need is everything between the equal signs. I tried butchering up a perl script which isn't working either. Any thoughts?

Alright, this worked (sort of)

```<code>#!perl -w
use strict;

my $LOGFILE = "/home/mydir/MyTestFile";
open my $fh, "<$LOGFILE" or die("could not open log file: $!");
my $in = 0;

while(<$fh>)
{
    $in = 1 if /Variable1/i;
    print if($in);
    $in = 0 if /Variable2/i;
}
</code>```

In the sense that a lot was stripped out. Now another q I have is selective printing a-la awk. Typically, I can get the line before something using:

```<code>echo "
test
hello
foo
bar" | awk '/foo/{print x};{x=$0}'
</code>```

Will print test, however haven't found a way to get the word test (will always be a different word, but the word foo will always remain). Any takers (by the way many thanks in advance)

答:
I'm not totally clear on if "Variable1/2" etc are constant strings or actually varying quanitites, but does this work:</p>

```<code>if [ $(grep Variable1 $file | wc -l) -gt 0 ]; then
  sed -n '/Variable2/,/=======/' $file
fi
</code>```

ContentLicense: CC BY-SA 3.0

Title: Reading between the lines (perl or awk)

Body:
Hey all, sorry for posting this here I could not find an answer anywhere and my solution is not working. I have a log file being written in the following fashion (don't ask):

=============================

04-12-2011 11:37:10 SOMETHING_GOES_HERE

Variable1

Something Goes Here

=============================

04-12-2011 11:37:20 SOMETHING_GOES_HERE

Variable2

Anything different may be here

=============================

04-12-2011 11:37:30 SOMETHING_GOES_HERE

Variable3

is altogether different here

=============================

What I'd like to do (either in perl or awk as this is an RTOS) is:

Take a look at the file, if Variable1 exists, then start at Variable2 and print everything between the equal signs:

E.g.

=============================
04-12-2011 11:37:10 SOMETHING_GOES_HERE

Variable1

Mary had a little lamb

=============================
04-12-2011 11:37:20 SOMETHING_GOES_HERE

Variable2

The cow jumped over the moon
=============================

awk '/Mary had/{getline;getline;getline;print}'

will only print

04-12-2011 11:37:20 SOMETHING_GOES_HERE

but what I need is everything between the equal signs. I tried butchering up a perl script which isn't working either. Any thoughts?

Alright, this worked (sort of)

#!perl -w
use strict;

my $LOGFILE = "/home/mydir/MyTestFile";
open my $fh, "<$LOGFILE" or die("could not open log file: $!");
my $in = 0;

while(<$fh>)
{
    $in = 1 if /Variable1/i;
    print if($in);
    $in = 0 if /Variable2/i;
}

In the sense that a lot was stripped out. Now another q I have is selective printing a-la awk. Typically, I can get the line before something using:

echo "
test
hello
foo
bar" | awk '/foo/{print x};{x=$0}'

Will print test, however haven't found a way to get the word test (will always be a different word, but the word foo will always remain). Any takers (by the way many thanks in advance)

Body:
I'm not totally clear on if "Variable1/2" etc are constant strings or actually varying quanitites, but does this work:

if [ $(grep Variable1 $file | wc -l) -gt 0 ]; then
  sed -n '/Variable2/,/=======/' $file
fi


问:
Need to use shared_ptr?
I have two classes: Lattice and ModelGUI. I want to pass function from Lattice to GUI as a callback. I implemented Lattice as unique_ptr. Some code:

ModelGUI.h:

```<code>using CheckTypeClbk = std::function<Enums::AgentType(int)>;
ModelGUI(const Matrix* matrix_, CheckTypeClbk callback, float windowHeight_, float windowWidth_, float latticeWidth_);
</code>```

main.cpp:

```<code>std::unique_ptr<ILattice> lattice(new Lattice(5, qMap));
ModelGUI gui(lattice->getLattice(), std::bind(&ILattice::checkAgentType, lattice, std::placeholders::_1),
800, 1200, 800);
</code>```

With this implementation I got strange compilation errors about templates:

```<code>1>main.cpp
1>d:\program files (x86)\microsoft visual studio\2017\community\vc\tools\msvc\14.14.26428\include\xutility(390): error C2664: 'std::tuple<std::unique_ptr<ILattice,std::default_delete<_Ty>>,std::_Ph<1>>::tuple(std::tuple<std::unique_ptr<_Ty,std::default_delete<_Ty>>,std::_Ph<1>> &&)': cannot convert argument 1 from 'std::unique_ptr<ILattice,std::default_delete<_Ty>>' to 'std::allocator_arg_t'
1>        with
1>        [
1>            _Ty=ILattice
1>        ]
1>d:\program files (x86)\microsoft visual studio\2017\community\vc\tools\msvc\14.14.26428\include\xutility(389): note: No user-defined-conversion operator available that can perform this conversion, or the operator cannot be called
1>d:\program files (x86)\microsoft visual studio\2017\community\vc\tools\msvc\14.14.26428\include\functional(1902): note: see reference to function template instantiation 'std::_Compressed_pair<Enums::AgentType (__cdecl ILattice::* )(int),std::tuple<std::unique_ptr<ILattice,std::default_delete<_Ty>>,std::_Ph<1>>,false>::_Compressed_pair<Enums::AgentType(__cdecl ILattice::* )(int),_Cv_TiD&,const std::_Ph<1>&>(std::_One_then_variadic_args_t,_Other1 &&,_Cv_TiD &,const std::_Ph<1> &)' being compiled
1>        with
1>        [
1>            _Ty=ILattice,
1>            _Cv_TiD=std::unique_ptr<ILattice,std::default_delete<ILattice>>,
1>            _Other1=Enums::AgentType (__cdecl ILattice::* )(int)
1>        ]
1>d:\program files (x86)\microsoft visual studio\2017\community\vc\tools\msvc\14.14.26428\include\functional(1903): note: see reference to function template instantiation 'std::_Compressed_pair<Enums::AgentType (__cdecl ILattice::* )(int),std::tuple<std::unique_ptr<ILattice,std::default_delete<_Ty>>,std::_Ph<1>>,false>::_Compressed_pair<Enums::AgentType(__cdecl ILattice::* )(int),_Cv_TiD&,const std::_Ph<1>&>(std::_One_then_variadic_args_t,_Other1 &&,_Cv_TiD &,const std::_Ph<1> &)' being compiled
1>        with
1>        [
1>            _Ty=ILattice,
1>            _Cv_TiD=std::unique_ptr<ILattice,std::default_delete<ILattice>>,
1>            _Other1=Enums::AgentType (__cdecl ILattice::* )(int)
1>        ]
1>d:\program files (x86)\microsoft visual studio\2017\community\vc\tools\msvc\14.14.26428\include\functional(1902): note: while compiling class template member function 'std::_Binder<std::_Unforced,Enums::AgentType (__cdecl ILattice::* )(int),std::unique_ptr<ILattice,std::default_delete<_Ty>> &,const std::_Ph<1> &>::_Binder(_Fx &&,std::unique_ptr<_Ty,std::default_delete<_Ty>> &,const std::_Ph<1> &)'
1>        with
1>        [
1>            _Ty=ILattice,
1>            _Fx=Enums::AgentType (__cdecl ILattice::* )(int)
1>        ]
1>d:\program files (x86)\microsoft visual studio\2017\community\vc\tools\msvc\14.14.26428\include\functional(1929): note: see reference to function template instantiation 'std::_Binder<std::_Unforced,Enums::AgentType (__cdecl ILattice::* )(int),std::unique_ptr<ILattice,std::default_delete<_Ty>> &,const std::_Ph<1> &>::_Binder(_Fx &&,std::unique_ptr<_Ty,std::default_delete<_Ty>> &,const std::_Ph<1> &)' being compiled
1>        with
1>        [
1>            _Ty=ILattice,
1>            _Fx=Enums::AgentType (__cdecl ILattice::* )(int)
1>        ]
1>d:\predator-prey\predator-prey\main.cpp(16): note: see reference to class template instantiation 'std::_Binder<std::_Unforced,Enums::AgentType (__cdecl ILattice::* )(int),std::unique_ptr<ILattice,std::default_delete<_Ty>> &,const std::_Ph<1> &>' being compiled
1>        with
1>        [
1>            _Ty=ILattice
1>        ]
</code>```

But when I am using shared_ptr instead of unique_ptr, everything works fine. Is a good practice? I've heard to avoid shared_ptr as much as I can, unless they are totally necessary.

答:
Do you need shared_ptr?

No. At least not for the given example.

If lattice and gui are defined in different scopes with different lifespans and being used all over the place, wowie-zowie, we can talk about shared_ptr.

Why?

Let's start with a really simple example that shows why unique_ptr is causing grief.

```<code>#include <functional>
#include <iostream>

struct test
{
    test() = default;
    test(const test &)
    {
        std::cout << "copied" << std::endl;
    }
    void func(int i)
    {
        std::cout << i << std::endl;
    }
};

int main()
{
    test t;
    std::function<void(int)> f1 = std::bind(&test::func, t, std::placeholders::_1);
    f1(1);
}
</code>```

test doesn't do much other than tell us when the object is copied and prove that the function ran. Executing it we'll see that t is copied and produced the expected output from the function.

std::unique_ptr cannot be copied because that would pretty much ruin the whole unique part of the job description. We see that if we change main a little to use a unique_ptr and get a little closer to the question as posed.

```<code>int main()
{
    std::unique_ptr<test> tp = std::make_unique<test>();
    std::function<void(int)> f1 = std::bind(&test::func, tp, std::placeholders::_1);
}
</code>```

As expected, this doesn't compile. We can make this compile by using a std::reference_wrapper

```<code>std::function<void(int)> f1 = std::bind(&test::func, std::reference_wrapper<std::unique_ptr<test>>(tp), std::placeholders::_1);
</code>```

or provide a raw pointer to bind

```<code>std::function<void(int)> f1 = std::bind(&test::func, tp.get(), std::placeholders::_1);    f1(1);
</code>```

but this requires tp to have wider scope and be guaranteed to outlive f1. What this really comes down to is why use more than test t; in the first place? Do we really need a pointer here at all?

But let's go with it for now because we can at least make this look a lot prettier before we head off to greener pastures. Here is the same thing with a lambda expression

```<code>std::function<void(int)> f1 = [&tp](int i) { tp->func(i); };
</code>```

Normally I'm not a champion of "Lambda is easier to read than bind," but this case is a pretty convincing argument.

Getting back to basics,  it's not really all that different from

```<code>int main()
{
    test t;
    std::function<void(int)> f1 = [&t](int i) { t.func(i); };
    f1(1);
}
</code>```

and completely eliminates the pointer. No pointer, no shared_ptr.

If t can be fire and forget, the only user is the callback, let the lambda carry around a copy of t and let the original die.

```<code>std::function<void(int)> scopedemo()
{
    test t;
    return [t](int i) mutable { t.func(i); }; //
}

int main()
{
    auto f1 = scopedemo();
    f1(1);

}
</code>```

Note the mutable. Lambda defaults to carrying around constants and can't be used to invoke non-const methods or be used as a non-const parameter.

Title: Need to use shared_ptr?

Body:
I have two classes: Lattice and ModelGUI. I want to pass function from Lattice to GUI as a callback. I implemented Lattice as unique_ptr. Some code:

ModelGUI.h:

using CheckTypeClbk = std::function<Enums::AgentType(int)>;
ModelGUI(const Matrix* matrix_, CheckTypeClbk callback, float windowHeight_, float windowWidth_, float latticeWidth_);

main.cpp:

std::unique_ptr<ILattice> lattice(new Lattice(5, qMap));
ModelGUI gui(lattice->getLattice(), std::bind(&ILattice::checkAgentType, lattice, std::placeholders::_1),
800, 1200, 800);

With this implementation I got strange compilation errors about templates:

1>main.cpp
1>d:\program files (x86)\microsoft visual studio\2017\community\vc\tools\msvc\14.14.26428\include\xutility(390): error C2664: 'std::tuple<std::unique_ptr<ILattice,std::default_delete<_Ty>>,std::_Ph<1>>::tuple(std::tuple<std::unique_ptr<_Ty,std::default_delete<_Ty>>,std::_Ph<1>> &&)': cannot convert argument 1 from 'std::unique_ptr<ILattice,std::default_delete<_Ty>>' to 'std::allocator_arg_t'
1>        with
1>        [
1>            _Ty=ILattice
1>        ]
1>d:\program files (x86)\microsoft visual studio\2017\community\vc\tools\msvc\14.14.26428\include\xutility(389): note: No user-defined-conversion operator available that can perform this conversion, or the operator cannot be called
1>d:\program files (x86)\microsoft visual studio\2017\community\vc\tools\msvc\14.14.26428\include\functional(1902): note: see reference to function template instantiation 'std::_Compressed_pair<Enums::AgentType (__cdecl ILattice::* )(int),std::tuple<std::unique_ptr<ILattice,std::default_delete<_Ty>>,std::_Ph<1>>,false>::_Compressed_pair<Enums::AgentType(__cdecl ILattice::* )(int),_Cv_TiD&,const std::_Ph<1>&>(std::_One_then_variadic_args_t,_Other1 &&,_Cv_TiD &,const std::_Ph<1> &)' being compiled
1>        with
1>        [
1>            _Ty=ILattice,
1>            _Cv_TiD=std::unique_ptr<ILattice,std::default_delete<ILattice>>,
1>            _Other1=Enums::AgentType (__cdecl ILattice::* )(int)
1>        ]
1>d:\program files (x86)\microsoft visual studio\2017\community\vc\tools\msvc\14.14.26428\include\functional(1903): note: see reference to function template instantiation 'std::_Compressed_pair<Enums::AgentType (__cdecl ILattice::* )(int),std::tuple<std::unique_ptr<ILattice,std::default_delete<_Ty>>,std::_Ph<1>>,false>::_Compressed_pair<Enums::AgentType(__cdecl ILattice::* )(int),_Cv_TiD&,const std::_Ph<1>&>(std::_One_then_variadic_args_t,_Other1 &&,_Cv_TiD &,const std::_Ph<1> &)' being compiled
1>        with
1>        [
1>            _Ty=ILattice,
1>            _Cv_TiD=std::unique_ptr<ILattice,std::default_delete<ILattice>>,
1>            _Other1=Enums::AgentType (__cdecl ILattice::* )(int)
1>        ]
1>d:\program files (x86)\microsoft visual studio\2017\community\vc\tools\msvc\14.14.26428\include\functional(1902): note: while compiling class template member function 'std::_Binder<std::_Unforced,Enums::AgentType (__cdecl ILattice::* )(int),std::unique_ptr<ILattice,std::default_delete<_Ty>> &,const std::_Ph<1> &>::_Binder(_Fx &&,std::unique_ptr<_Ty,std::default_delete<_Ty>> &,const std::_Ph<1> &)'
1>        with
1>        [
1>            _Ty=ILattice,
1>            _Fx=Enums::AgentType (__cdecl ILattice::* )(int)
1>        ]
1>d:\program files (x86)\microsoft visual studio\2017\community\vc\tools\msvc\14.14.26428\include\functional(1929): note: see reference to function template instantiation 'std::_Binder<std::_Unforced,Enums::AgentType (__cdecl ILattice::* )(int),std::unique_ptr<ILattice,std::default_delete<_Ty>> &,const std::_Ph<1> &>::_Binder(_Fx &&,std::unique_ptr<_Ty,std::default_delete<_Ty>> &,const std::_Ph<1> &)' being compiled
1>        with
1>        [
1>            _Ty=ILattice,
1>            _Fx=Enums::AgentType (__cdecl ILattice::* )(int)
1>        ]
1>d:\predator-prey\predator-prey\main.cpp(16): note: see reference to class template instantiation 'std::_Binder<std::_Unforced,Enums::AgentType (__cdecl ILattice::* )(int),std::unique_ptr<ILattice,std::default_delete<_Ty>> &,const std::_Ph<1> &>' being compiled
1>        with
1>        [
1>            _Ty=ILattice
1>        ]

But when I am using shared_ptr instead of unique_ptr, everything works fine. Is a good practice? I've heard to avoid shared_ptr as much as I can, unless they are totally necessary.

Body:
Do you need shared_ptr?

No. At least not for the given example.

If lattice and gui are defined in different scopes with different lifespans and being used all over the place, wowie-zowie, we can talk about shared_ptr.

Why?

Let's start with a really simple example that shows why unique_ptr is causing grief.

#include <functional>
#include <iostream>

struct test
{
    test() = default;
    test(const test &)
    {
        std::cout << "copied" << std::endl;
    }
    void func(int i)
    {
        std::cout << i << std::endl;
    }
};

int main()
{
    test t;
    std::function<void(int)> f1 = std::bind(&test::func, t, std::placeholders::_1);
    f1(1);
}

test doesn't do much other than tell us when the object is copied and prove that the function ran. Executing it we'll see that t is copied and produced the expected output from the function.

std::unique_ptr cannot be copied because that would pretty much ruin the whole unique part of the job description. We see that if we change main a little to use a unique_ptr and get a little closer to the question as posed.

int main()
{
    std::unique_ptr<test> tp = std::make_unique<test>();
    std::function<void(int)> f1 = std::bind(&test::func, tp, std::placeholders::_1);
}

As expected, this doesn't compile. We can make this compile by using a std::reference_wrapper

std::function<void(int)> f1 = std::bind(&test::func, std::reference_wrapper<std::unique_ptr<test>>(tp), std::placeholders::_1);

or provide a raw pointer to bind

std::function<void(int)> f1 = std::bind(&test::func, tp.get(), std::placeholders::_1);    f1(1);

but this requires tp to have wider scope and be guaranteed to outlive f1. What this really comes down to is why use more than test t; in the first place? Do we really need a pointer here at all?

But let's go with it for now because we can at least make this look a lot prettier before we head off to greener pastures. Here is the same thing with a lambda expression

std::function<void(int)> f1 = [&tp](int i) { tp->func(i); };

Normally I'm not a champion of "Lambda is easier to read than bind," but this case is a pretty convincing argument.

Getting back to basics,  it's not really all that different from

int main()
{
    test t;
    std::function<void(int)> f1 = [&t](int i) { t.func(i); };
    f1(1);
}

and completely eliminates the pointer. No pointer, no shared_ptr.

If t can be fire and forget, the only user is the callback, let the lambda carry around a copy of t and let the original die.

std::function<void(int)> scopedemo()
{
    test t;
    return [t](int i) mutable { t.func(i); }; //
}

int main()
{
    auto f1 = scopedemo();
    f1(1);

}

Note the mutable. Lambda defaults to carrying around constants and can't be used to invoke non-const methods or be used as a non-const parameter.


问:
How to setup ramp down time in gatling?
I have a jmeter scripts to ramp up 20 users over 20 minutes(1 user every minute) and hold for 30 minutes and ramp down 1 user per minute.

How do i achieve this in Gatling?

I tried below setting in gatling,

```<code>SAAPIBase.scn_SA_Auth.inject(rampUsers(20) during (20 minutes)).protocols(httpConf)
).maxDuration(50 minutes)
</code>```

This ramps up 20 users for 20 minutes and holds for 30 minutes with 20 users. But after that suddenly it drops to zero.

How do I slowly ramp down 1 by 1 user every minute?

答:
Note how long your simulation runs is heavily impacted by the scenarios you're running - all the gatling DSL injection steps control when users get started, and that is all. So if you have scenarios that are very long running or unpredictable, then it's difficult to guarantee something exactly like what you've specified.

In your example

```<code>.inject(rampUsers(20) during (20 minutes)).protocols(httpConf)
).maxDuration(50 minutes)
</code>```

you inject 20 users evenly over 20 minutes (so one every minute). If that is indeed being held for 30 minutes after the ramp then your scenarios are probably looping and the users get stopped by the maxDuration statement.

for a ramp-down, you could change your injection to be

```<code>.inject(
  rampConcurrentUsers(1) to(20) during(20 minutes),
  constantConcurrentUsers(20) during (30 minutes),
  rampConcurrentUsers(20) to(1) during(20 minutes)
)
</code>```

this will give you the same ramping up over 20 minutes, then will keep injecting users as others finish to maintain 20 concurrent users over the next 30 minutes, then gradually slow the rate of injection over the final 20 minutes. However, if your scenario is using loops like .forever to keep running, this ramp-down won't work as the users injected in the 1st command will never stop.

Title: How to setup ramp down time in gatling?

Body:
I have a jmeter scripts to ramp up 20 users over 20 minutes(1 user every minute) and hold for 30 minutes and ramp down 1 user per minute.

How do i achieve this in Gatling?

I tried below setting in gatling,

SAAPIBase.scn_SA_Auth.inject(rampUsers(20) during (20 minutes)).protocols(httpConf)
).maxDuration(50 minutes)

This ramps up 20 users for 20 minutes and holds for 30 minutes with 20 users. But after that suddenly it drops to zero.

How do I slowly ramp down 1 by 1 user every minute?

Body:
Note how long your simulation runs is heavily impacted by the scenarios you're running - all the gatling DSL injection steps control when users get started, and that is all. So if you have scenarios that are very long running or unpredictable, then it's difficult to guarantee something exactly like what you've specified.

In your example

.inject(rampUsers(20) during (20 minutes)).protocols(httpConf)
).maxDuration(50 minutes)

you inject 20 users evenly over 20 minutes (so one every minute). If that is indeed being held for 30 minutes after the ramp then your scenarios are probably looping and the users get stopped by the maxDuration statement.

for a ramp-down, you could change your injection to be

.inject(
  rampConcurrentUsers(1) to(20) during(20 minutes),
  constantConcurrentUsers(20) during (30 minutes),
  rampConcurrentUsers(20) to(1) during(20 minutes)
)

this will give you the same ramping up over 20 minutes, then will keep injecting users as others finish to maintain 20 concurrent users over the next 30 minutes, then gradually slow the rate of injection over the final 20 minutes. However, if your scenario is using loops like .forever to keep running, this ramp-down won't work as the users injected in the 1st command will never stop.


问:
Vector multiplication algorithm
Let A,B be matrixes of R^n space and b belong to R^n.Describe a fast algorithm to compute A^-2*B*A^-3*b.How many computations will the algorithm make?

This is an exam question I have for numerical analysis.I tried brute forcing an algorithm but i believe that the answer is more mathematical.

We haven't yet talked about Big O notations so the question asks for strictly the actions of the algorithm.How would you go about answering this question?

答:
I would just work the problem from right to left, using a linear solver when dealing with the inverse of A, and matrix multiply when dealing with B:

```<code>x1 = linsolve(A, b)
x2 = linsolve(A, x1)
x3 = linsolve(A, x2)
y = B*x3
z1 = linsolve(A,y)
result = linsolve(A,z1)
</code>```

You can reduce by a constant multiplier the number of operations by keeping the LU decomposition of A in memory, but unless you are given more structure on A and B, quadratic complexity seems to be the best you can aim for.

Title: Vector multiplication algorithm

Body:
I would just work the problem from right to left, using a linear solver when dealing with the inverse of A, and matrix multiply when dealing with B:

x1 = linsolve(A, b)
x2 = linsolve(A, x1)
x3 = linsolve(A, x2)
y = B*x3
z1 = linsolve(A,y)
result = linsolve(A,z1)

You can reduce by a constant multiplier the number of operations by keeping the LU decomposition of A in memory, but unless you are given more structure on A and B, quadratic complexity seems to be the best you can aim for.


问:
pd.DataFrame.select_dtypes() inculdes timedelta dtype
Why is it expected behavior that this test code:

test = pd.DataFrame({'bool' :[False, True], 'int':[-1,2], 'float': [-2.5, 3.4],
                     'compl':np.array([1-1j, 5]),
                     'dt'   :[pd.Timestamp('2013-01-02'), pd.Timestamp('2016-10-20')],
                     'td'   :[pd.Timestamp('2012-03-02')- pd.Timestamp('2016-10-20'),
                              pd.Timestamp('2010-07-12')- pd.Timestamp('2000-11-10')]})
test.dtypes
test.select_dtypes(np.number)
```

<p>Produces DataFrame with <code>TimeDelta</code> column included?</p>

<pre class="lang-py prettyprint-override"><code>>>> bool                bool
>>> int                int64
>>> float            float64
>>> compl         complex128
>>> dt        datetime64[ns]
>>> td       timedelta64[ns]
>>> dtype: object

>>>     int     float   compl   td
>>> 0    -1     -2.5    (1-1j)  -1693 days
>>> 1     2      3.4    (5+0j)   3531 days
</code>```

EDIT:

For someone (including me) the following may be helpful:

I've also found the reason why this behavior was unexpected for me at first. The reason was another way to check if dtype of pd.DataFrame is numeric. Namely via pd.api.types.is_numeric_dtype:

for col in test.columns:
    if pd.api.types.is_numeric_dtype(test[col]):
        print (test[col].dtype)

>>> bool
>>> int64
>>> float64
>>> complex128
```

Which produces more 'human-desired' output.

答:
Because that's how it has been implemented:

```<code>np.issubdtype(np.timedelta64, np.number)
# True
</code>```

More specifically,

```<code>np.issubdtype(np.timedelta64, np.integer)
# True
</code>```

timedelta and datetime dtypes in numpy are internally represented by integer. This makes it easy to represent in memory, and makes arithmetic on datetimes fast.

If you want to exclude these types from your checks, you can specify an exclude argument:

```<code>test.select_dtypes(include=['number'], exclude=['datetime', 'timedelta'])

   int  float   compl
0   -1   -2.5  (1-1j)
1    2    3.4  (5+0j)
</code>```

Body:
Why is it expected behavior that this test code:

test = pd.DataFrame({'bool' :[False, True], 'int':[-1,2], 'float': [-2.5, 3.4],
                     'compl':np.array([1-1j, 5]),
                     'dt'   :[pd.Timestamp('2013-01-02'), pd.Timestamp('2016-10-20')],
                     'td'   :[pd.Timestamp('2012-03-02')- pd.Timestamp('2016-10-20'),
                              pd.Timestamp('2010-07-12')- pd.Timestamp('2000-11-10')]})
test.dtypes
test.select_dtypes(np.number)

Produces DataFrame with TimeDelta column included?

>>> bool                bool
>>> int                int64
>>> float            float64
>>> compl         complex128
>>> dt        datetime64[ns]
>>> td       timedelta64[ns]
>>> dtype: object

>>>     int     float   compl   td
>>> 0    -1     -2.5    (1-1j)  -1693 days
>>> 1     2      3.4    (5+0j)   3531 days

EDIT:

For someone (including me) the following may be helpful:

I've also found the reason why this behavior was unexpected for me at first. The reason was another way to check if dtype of pd.DataFrame is numeric. Namely via pd.api.types.is_numeric_dtype:

for col in test.columns:
    if pd.api.types.is_numeric_dtype(test[col]):
        print (test[col].dtype)

>>> bool
>>> int64
>>> float64
>>> complex128

Which produces more 'human-desired' output.

Body:
Because that's how it has been implemented:

np.issubdtype(np.timedelta64, np.number)
# True

More specifically,

np.issubdtype(np.timedelta64, np.integer)
# True

timedelta and datetime dtypes in numpy are internally represented by integer. This makes it easy to represent in memory, and makes arithmetic on datetimes fast.

If you want to exclude these types from your checks, you can specify an exclude argument:

test.select_dtypes(include=['number'], exclude=['datetime', 'timedelta'])

   int  float   compl
0   -1   -2.5  (1-1j)
1    2    3.4  (5+0j)


问:
Angular 8 - Lazy loading modules : Error TS1323: Dynamic import is only supported when '--module' flag is 'commonjs' or 'esNext'
When I updated Angular from 7 to Angular 8, getting error for lazy loading modules

I have tried the options, which are there in the angular upgradation guide

Made the below changes:

Before

```<code>    loadChildren: '../feature/path/sample-
                         tage.module#SameTagModule'
</code>```

After

```<code>   loadChildren: () => import('../feature/path/sample-
                      tags.module').then(m => m.CreateLinksModule)
</code>```

error TS1323: Dynamic import is only supported when '--module' flag is
  'commonjs' or 'esNext'.

答: Just adding to @Tony's anwser, you might also need to do the same (change to "module": "esnext" ) in the tsconfig.app.json. In my case the tsconfig.json was already using esnext as the module but the tsconfig.app.json was still using es2015 and that caused this error.</p>

Body:
When I updated Angular from 7 to Angular 8, getting error for lazy loading modules

I have tried the options, which are there in the angular upgradation guide

Made the below changes:

Before

    loadChildren: '../feature/path/sample-
                         tage.module#SameTagModule'

After

   loadChildren: () => import('../feature/path/sample-
                      tags.module').then(m => m.CreateLinksModule)

error TS1323: Dynamic import is only supported when '--module' flag is
  'commonjs' or 'esNext'.


问:
Is there a way to add characters based on a variable?
I'm making a text-based game about Hangman and so far I have 10 words with different amounts of characters in each word. The lowest character count is three, so I'm planning to put in three underscores at the start for me to be able to add underscores after the string.

I have a variable set up, called wordPlayingLength which counts up an integer based on the amount of characters in the word that has been selected through a random statement. My question is if there's a way to add underscores that matches up with the word's length in characters?

```<code>if (randomNumber == 10):
    wordPlaying = str("Didgeridoo") ## states the word being played
    wordPlayingLength = int(len(wordPlaying)) ## calculates the character length of the word being played
    print(str(wordPlayingLength) + " letters!")

    underscoreCount = (wordPlayingLength)
    print("_ _ _ " + ) ## this is where I got stuck, no idea here
</code>```

答:
Here is your program containing the logic I think you're describing:

```<code>if randomNumber == 10:
    wordPlaying = "Didgeridoo"
    print("{}  letters!".format(len(wordPlaying)))
    print(' '.join('_' * len(wordPlaying)))
</code>```

I've removed some unnecessary sets of parentheses and uses of str() and int().  Since calling len() of a string or list is a constant-time operation, there's no need to save use a variable to store the length here.

The key behaviors you need to know about are:

str.join(), which takes the elements of an iterable and makes a string out of them by interleaving the elements of the iterable with the contents of the string it's called on.

The integer n * sequence s operator returns the sequence you get if you add n copies of s together.

Body:
I'm making a text-based game about Hangman and so far I have 10 words with different amounts of characters in each word. The lowest character count is three, so I'm planning to put in three underscores at the start for me to be able to add underscores after the string.

I have a variable set up, called wordPlayingLength which counts up an integer based on the amount of characters in the word that has been selected through a random statement. My question is if there's a way to add underscores that matches up with the word's length in characters?

if (randomNumber == 10):
    wordPlaying = str("Didgeridoo") ## states the word being played
    wordPlayingLength = int(len(wordPlaying)) ## calculates the character length of the word being played
    print(str(wordPlayingLength) + " letters!")

    underscoreCount = (wordPlayingLength)
    print("_ _ _ " + ) ## this is where I got stuck, no idea here

Body:
Here is your program containing the logic I think you're describing:

if randomNumber == 10:
    wordPlaying = "Didgeridoo"
    print("{}  letters!".format(len(wordPlaying)))
    print(' '.join('_' * len(wordPlaying)))

I've removed some unnecessary sets of parentheses and uses of str() and int().  Since calling len() of a string or list is a constant-time operation, there's no need to save use a variable to store the length here.

The key behaviors you need to know about are:

str.join(), which takes the elements of an iterable and makes a string out of them by interleaving the elements of the iterable with the contents of the string it's called on.

The integer n * sequence s operator returns the sequence you get if you add n copies of s together.


问:
Perl and Complex SOAP Request
I need to make a somewhat complex soap query using Perl, preferably using SOAP::Lite.  I know the service is active and have been successful in getting errors back from the other end.  Here is the soap query I need to make:

```<code><?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
<soap:Body>
    <GetCategories xmlns="http://webservices.uship.com">
        <token>string</token>
    </GetCategories>
</soap:Body>
</soap:Envelope>
</code>```

I've researched this via Google to no avail.

Update: The code used so far is

```<code>use SOAP::Lite;
print SOAP::Lite
 -> uri('webservices.uship.com/uShipsvc.asmx?WSDL';)
 -> proxy('http:/webservices.uship.com')
 -> GetCategories('myToken')
 -> result;
</code>```

This returns

500 bad hostname, 500 Can't connect to :80 (Bad hostname '') at soap2.pl line 2

答:
From SOAP::Lite's Getting Started Guide your code should look something like:

```<code>#!perl -w
use SOAP::Lite;
print SOAP::Lite
  uri('http://www.soaplite.com/Temperatures')
  proxy('http://webservices.uship.com')
  GetCategories('string')
  result;
</code>```

Plug in the URI for the returned object in uri()

Title: Perl and Complex SOAP Request

Body:
I need to make a somewhat complex soap query using Perl, preferably using SOAP::Lite.  I know the service is active and have been successful in getting errors back from the other end.  Here is the soap query I need to make:

<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
<soap:Body>
    <GetCategories xmlns="http://webservices.uship.com">
        <token>string</token>
    </GetCategories>
</soap:Body>
</soap:Envelope>

I've researched this via Google to no avail.

Update: The code used so far is

use SOAP::Lite;
print SOAP::Lite
 -> uri('webservices.uship.com/uShipsvc.asmx?WSDL';)
 -> proxy('http:/webservices.uship.com')
 -> GetCategories('myToken')
 -> result;

This returns

500 bad hostname, 500 Can't connect to :80 (Bad hostname '') at soap2.pl line 2

Body:
From SOAP::Lite's Getting Started Guide your code should look something like:

#!perl -w
use SOAP::Lite;
print SOAP::Lite
  uri('http://www.soaplite.com/Temperatures')
  proxy('http://webservices.uship.com')
  GetCategories('string')
  result;

Plug in the URI for the returned object in uri()


问:
Correct way to codesign Qt-application .app and dmg on MacOS?
I have a cross-platform application written in Qt (5.9.6), running on Win, Mac and Ubuntu.

On mac, an application is built using Qt tools only (without Xcode).

The application has a 3rd-party library ffpmeg. We do not compile it, we use a prebuilt from https://ffmpeg.org/download.html#build-mac.

At the initial stage, the application is supposed to be distributed through the Developer-ID-signed dmg-image (otherwise Gatekepeer is giving a message of untrusted developer).

The dmg-image is built via hdutil (not via the macdeployqt util).

I tried to sign the .app file and dmg-image with certificates from the Apple Developer Program, in particular:

macOS -> Production -> Developer ID Sign versions of your Mac
application, Mac kernel extension and Mac Installer Package for
distribution outside of the Mac App Store -> Developer ID Installer
This certificate is used to sign your app's Installer Package for
distribution outside of the Mac App Store

macOS -> Production -> Developer ID Sign versions of your Mac application, Mac kernel extension and Mac Installer Package for
distribution outside of the Mac App Store -> Developer ID
Application This certificate is used to code sign your app for
distribution outside of the Mac App Store.

(As I understand it - these are the certificates for our case - distribution outside the AppStore)

All certificates are correctly installed in the system and visible in the "Keychain".

Application signing through codesign manually.

If i try to sign the Application.app file directly, then codesign responds "code object is not signed at all". The application is not considered signed.

Then, after few hours googling, i found out that i need to sign all .framework, .dylib-files and plugins inside the dmg.

Then i became sign all files, until codesign on the app stops giving an error. Like this:

```<code>...
$ codesign -s 9509FE0B2EBCC89D9047541AC762418395FCB40E Application.app
Application.app/: code object is not signed at all
# error, go next

$ codesign -s 9509FE0B2EBCC89D9047541AC762418395FCB40E Application.app/Contents/Frameworks/libavcodec.58.dylib
# ffmpeg is also shuld be signed - otherwise macOS give a not-at-all-error
...

$ codesign -s 9509FE0B2EBCC89D9047541AC762418395FCB40E Application.app/Contents/Frameworks/QtCore.framework
# Qt also sign
...

$ codesign -s 9509FE0B2EBCC89D9047541AC762418395FCB40E Application.app
# Ok, dmg is successfully signed.
</code>```

Codesign stops swearing on a not fully signed package.

Then run Application.app on our machine - everything is ok.
Then run on another Mac - the result when opening the Application.app file:

"The Application.app program is corrupted and cannot be opened. Move the program to the basket."

It refuses to launch even through the "hack" for untrusted applications (via Command-button).

But - an incompletely signed application on another Mac runs fine (but swears at an untrusted developer).

So, my question is:

How to sign Qt-application  .app and dmg with 3rd-party libraries (ffmpeg)?

Thanks!

答:
The application cannot run because it violated some restriction of codesign. You  need to sign with entitlements option to disable those restrictions.

Here's a sample entitlements.xml

```<code><?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>com.apple.security.cs.allow-jit</key>
    <true/>
    <key>com.apple.security.cs.allow-unsigned-executable-memory</key>
    <true/>
    <key>com.apple.security.cs.disable-executable-page-protection</key>
    <true/>
    <key>com.apple.security.cs.disable-library-validation</key>
        <true/>
        <key>com.apple.security.cs.allow-dyld-environment-variables</key>
        <true/>
</dict>
</plist>
</code>```

macOS notarization error: "The signature algorithm used is too weak"

Body:
I have a cross-platform application written in Qt (5.9.6), running on Win, Mac and Ubuntu.

On mac, an application is built using Qt tools only (without Xcode).

The application has a 3rd-party library ffpmeg. We do not compile it, we use a prebuilt from https://ffmpeg.org/download.html#build-mac.

At the initial stage, the application is supposed to be distributed through the Developer-ID-signed dmg-image (otherwise Gatekepeer is giving a message of untrusted developer).

The dmg-image is built via hdutil (not via the macdeployqt util).

I tried to sign the .app file and dmg-image with certificates from the Apple Developer Program, in particular:

macOS -> Production -> Developer ID Sign versions of your Mac
application, Mac kernel extension and Mac Installer Package for
distribution outside of the Mac App Store -> Developer ID Installer
This certificate is used to sign your app's Installer Package for
distribution outside of the Mac App Store

macOS -> Production -> Developer ID Sign versions of your Mac application, Mac kernel extension and Mac Installer Package for
distribution outside of the Mac App Store -> Developer ID
Application This certificate is used to code sign your app for
distribution outside of the Mac App Store.

(As I understand it - these are the certificates for our case - distribution outside the AppStore)

All certificates are correctly installed in the system and visible in the "Keychain".

Application signing through codesign manually.

If i try to sign the Application.app file directly, then codesign responds "code object is not signed at all". The application is not considered signed.

Then, after few hours googling, i found out that i need to sign all .framework, .dylib-files and plugins inside the dmg.

Then i became sign all files, until codesign on the app stops giving an error. Like this:

...
$ codesign -s 9509FE0B2EBCC89D9047541AC762418395FCB40E Application.app
Application.app/: code object is not signed at all
# error, go next

$ codesign -s 9509FE0B2EBCC89D9047541AC762418395FCB40E Application.app/Contents/Frameworks/libavcodec.58.dylib
# ffmpeg is also shuld be signed - otherwise macOS give a not-at-all-error
...

$ codesign -s 9509FE0B2EBCC89D9047541AC762418395FCB40E Application.app/Contents/Frameworks/QtCore.framework
# Qt also sign
...

$ codesign -s 9509FE0B2EBCC89D9047541AC762418395FCB40E Application.app
# Ok, dmg is successfully signed.

Codesign stops swearing on a not fully signed package.

Then run Application.app on our machine - everything is ok.
Then run on another Mac - the result when opening the Application.app file:

"The Application.app program is corrupted and cannot be opened. Move the program to the basket."

It refuses to launch even through the "hack" for untrusted applications (via Command-button).

But - an incompletely signed application on another Mac runs fine (but swears at an untrusted developer).

So, my question is:

How to sign Qt-application  .app and dmg with 3rd-party libraries (ffmpeg)?

Thanks!

Body:
The application cannot run because it violated some restriction of codesign. You  need to sign with entitlements option to disable those restrictions.

Here's a sample entitlements.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>com.apple.security.cs.allow-jit</key>
    <true/>
    <key>com.apple.security.cs.allow-unsigned-executable-memory</key>
    <true/>
    <key>com.apple.security.cs.disable-executable-page-protection</key>
    <true/>
    <key>com.apple.security.cs.disable-library-validation</key>
        <true/>
        <key>com.apple.security.cs.allow-dyld-environment-variables</key>
        <true/>
</dict>
</plist>

macOS notarization error: "The signature algorithm used is too weak"


问:
CSS Selector-How to locate Parent Element
Is there any way to locate parent element in CSS Selector?
i am using below code but i am not getting the parent element.

```<code>WebElement we=dr.findElement(By.cssSelector("div[id='gf-BIG']:parent"));
</code>```

I know there is a way in XPath but please let me know that how we can locate parent in CSS Selector.

答:
If the parent is say div, you could try something like below</p>

```<code>div div[id='gf-BIG']
</code>```

Body:
Is there any way to locate parent element in CSS Selector?
i am using below code but i am not getting the parent element.

WebElement we=dr.findElement(By.cssSelector("div[id='gf-BIG']:parent"));

I know there is a way in XPath but please let me know that how we can locate parent in CSS Selector.

Body:
If the parent is say div, you could try something like below

div div[id='gf-BIG']


问:
In Android EditText, how to force writing uppercase?
In my Android application I have different EditText where the user can enter information. But I need to force user to write in uppercase letters.
Do you know a function to do that?

答:
A Java 1-liner of the proposed solution could be:

```<code>editText.setFilters(Lists.asList(new InputFilter.AllCaps(), editText.getFilters())
    .toArray(new InputFilter[editText.getFilters().length + 1]));
</code>```

Note it needs com.google.common.collect.Lists.

Body:
A Java 1-liner of the proposed solution could be:

editText.setFilters(Lists.asList(new InputFilter.AllCaps(), editText.getFilters())
    .toArray(new InputFilter[editText.getFilters().length + 1]));

Note it needs com.google.common.collect.Lists.


问:
Visual Studio project folder conventions
A few versions back, Visual Studio started using C:\Users\Jonathan\source\repos as the default project folder (instead of C:\Users\Jonathan\Documents\Visual Studio 2019\Projects). I prefer the old location under Documents, but I decided to just accept the newer location because I figure that is more likely to avoid conflicts, avoid having to change it with each new release and just generally the path of least resistance moving forward.

Now I want to create a GitHub repository. But the default Local path is the same path where the project is stored! So without changing this path, I get the following error:

Repository with same name already exists at this location.

Obviously, I can change this path. But I'm trying to go with Microsoft's recommended folder locations to avoid conflicts. And so it seems ironic that I've run into a conflict using these locations!

Questions:

Does anyone know why Visual Studio needs a local repository path in addition to the current project folder?

Does anyone know why Visual Studio is defaulting to the same path for both? What is the intended convention for these locations?

答:
VS has default project location: Tools -> Options -> Project and Solutions -> Locations:

and default location for git repos: Team Explorer -> Settings -> Git -> Global Settings:

and they should be the same because project and local repo should be in the same folder.

But you can change them what I did. You don't need to follow default MS recommendations, just select folder convenient for you.

And this answer is correct. Just select your Solution, right click and press Add Solution to source control...


问:
Difference Between Deactivate and Dispose?
In Flutter, StatefulWidget has dispose() and deactivate(). How are they different?

答:
dispose is definitive. deactivate is not.

deactivate is called when a widget may be disposed. But that is not guaranteed.

A typical situation where deactivate is called but not dispose,is when moving widgets in the widget tree using a GlobalKey.


问:
Angular Material mat-ripple isn't working
I'm trying to add ripple effect to one button, but it crashes whole webpage.
When I just add matRipple to my button nothing happens. When I try to add color [matRippleColor]="white" I get "Can't bind to 'matRippleColor' since it isn't a known property of 'button'".

That strange because I enabled ripples on other page of website and it works just fine. (So yes, I've imported MatRippleModule):

HTML:

```<code><button  matRipple class="submit-button" color="accent" type="submit" >
  <mat-icon>search</mat-icon><span>Szukaj</span>
</button>
</code>```

答:
That is because the input property matRippleColor expect a string and you are passing a variable that Angular supposed you have initialized before and in this case you have a variable that I supposed doesn't exist in your component.

If you want the white color be recognized you have to pass it as string since you are using the binding feature:

```<code>[matRippleColor]="'white'"
</code>```

or you can remove the curly brackets and using it like this:

```<code>matRippleColor="white"
</code>```

Body:
I'm trying to add ripple effect to one button, but it crashes whole webpage.
When I just add matRipple to my button nothing happens. When I try to add color [matRippleColor]="white" I get "Can't bind to 'matRippleColor' since it isn't a known property of 'button'".

That strange because I enabled ripples on other page of website and it works just fine. (So yes, I've imported MatRippleModule):

HTML:

<button  matRipple class="submit-button" color="accent" type="submit" >
  <mat-icon>search</mat-icon>Szukaj
</button>

Body:
That is because the input property matRippleColor expect a string and you are passing a variable that Angular supposed you have initialized before and in this case you have a variable that I supposed doesn't exist in your component.

If you want the white color be recognized you have to pass it as string since you are using the binding feature:

[matRippleColor]="'white'"

or you can remove the curly brackets and using it like this:

matRippleColor="white"


问:
How do I use python sockets to send an api response over a connection?
I'm trying to have the server wait for a client to establish a connection, get data from the client, use that data to make an api request, and send the response to the client.  Everything works fine except for sending the response back to the client (client function just times out).  If put conn.sendall("anythinghere") inside of the while True loop, this works but obviously doesn't return what I need it to return.  Putting conn.sendall outside of the loop causes the timeout on the client side.

Server Code

```<code>conn, addr = s.accept()

print('Connected by', addr)
fragments = []
while True:
    chunk = conn.recv(4096)
    if not chunk:
        break
    fragments.append(chunk)

action = "".join(fragments)

URL = "http://172.31.55.211/" + action

print("making request")
r = requests.get(url = URL)
data = r.json()

conn.sendall(data['message'])
</code>```

Client Code

```<code>    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    print(s)
    s.connect((HOST, PORT))
    s.sendall(b'hellocvs')
    data = s.recv(1024)

    print('Received', repr(data))

    return {
        'statusCode': 200,
        'body': json.dumps(str(data))
    }
</code>```

答: At line action = "".join(fragments) server code will not run because fragments is bytes not string so you need b''.join(fragments) to join bytes

Body:
I'm trying to have the server wait for a client to establish a connection, get data from the client, use that data to make an api request, and send the response to the client.  Everything works fine except for sending the response back to the client (client function just times out).  If put conn.sendall("anythinghere") inside of the while True loop, this works but obviously doesn't return what I need it to return.  Putting conn.sendall outside of the loop causes the timeout on the client side.

Server Code

conn, addr = s.accept()

print('Connected by', addr)
fragments = []
while True:
    chunk = conn.recv(4096)
    if not chunk:
        break
    fragments.append(chunk)

action = "".join(fragments)

URL = "http://172.31.55.211/" + action

print("making request")
r = requests.get(url = URL)
data = r.json()

conn.sendall(data['message'])

Client Code

    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    print(s)
    s.connect((HOST, PORT))
    s.sendall(b'hellocvs')
    data = s.recv(1024)

    print('Received', repr(data))

    return {
        'statusCode': 200,
        'body': json.dumps(str(data))
    }


问:
Excel formula for calculating difference between two date columns
I have to 10k rows with two date columns. Can excel experts please provide an excel formula to calculate time difference(looking for diff in minutes) between below two date columns.

I have tried the formula =A1-B1 but its throwing #VALUE!

column A1 = 05/30/2019 21:16:48 +00:00

column B1 = 05/30/2019 21:10:28 +00:00

答:
As mentioned in a comment by @Ron Rosenfeld, your times are just Strings and not true date/time values.

If you can't alter the original data (with a find/replace to eliminate all the +00:00), then a simple SUBSTITUTE and your original subtraction should work. Also, multiply by 1440 to convert the result to minutes: 60 minutes/hour * 24 hours/day = 1440 minutes/day.

```<code>=(SUBSTITUTE(A1," +00:00","")-SUBSTITUTE(B1," +00:00",""))*1440
</code>```

Body:
As mentioned in a comment by @Ron Rosenfeld, your times are just Strings and not true date/time values.

If you can't alter the original data (with a find/replace to eliminate all the +00:00), then a simple SUBSTITUTE and your original subtraction should work. Also, multiply by 1440 to convert the result to minutes: 60 minutes/hour * 24 hours/day = 1440 minutes/day.

=(SUBSTITUTE(A1," +00:00","")-SUBSTITUTE(B1," +00:00",""))*1440


问:
How to extract multiple columns according to condition
I am new to this data analysis, I am currently working with a company's data. Where I have more than 100 columns worth of data. These columns are interrelated to each other, where the first four columns provide the description of the data so (id, domain, description, actual target). Rest of the other columns are just other numerical value that basically states the target value they reached for that day.

So, basically the question the company asked me to find is, which id have fluctuated the most and which id have failing to reach target.

I wanted to know, how i can properly execute the question, as its my very first time doing data analysis for a company.

What i did was firstly, i grouped the domain to see how many id's are associated with different domains.

Then I went about finding mean of every id in each domain (grouped). In finding the mean i can basically see if the id is failing to reach the target for that domain. Self explanatory, if most id's are failing to reach the target in specific domain, that means the domain/id is failing to reach the target.

```<code>df.groupby('Domain)['Id'].count() /*This tells me different domain with how many id's are there */
df[(df.Domain == 'Finance')].mean(axis = 1) /*Finance is just one of the domain that has different id's associated to it */
</code>```

But what the problem i am facing is that i can extract the mean of each id in grouped domain, however I can't seem to open the id and target columns that is associated to those mean values. So i can compare if the mean value is reaching the target value.

答:
Hello I managed to find answer to this issue
So basically this is what I did

df["avg"] = df.iloc[:,3:].mean(axis = 1)

df[(df.Domain == 'Finance')].loc[:,['ID','Domain','Target','avg']]

Body:
I am new to this data analysis, I am currently working with a company's data. Where I have more than 100 columns worth of data. These columns are interrelated to each other, where the first four columns provide the description of the data so (id, domain, description, actual target). Rest of the other columns are just other numerical value that basically states the target value they reached for that day.

So, basically the question the company asked me to find is, which id have fluctuated the most and which id have failing to reach target.

I wanted to know, how i can properly execute the question, as its my very first time doing data analysis for a company.

What i did was firstly, i grouped the domain to see how many id's are associated with different domains.

Then I went about finding mean of every id in each domain (grouped). In finding the mean i can basically see if the id is failing to reach the target for that domain. Self explanatory, if most id's are failing to reach the target in specific domain, that means the domain/id is failing to reach the target.

df.groupby('Domain)['Id'].count() /*This tells me different domain with how many id's are there */
df[(df.Domain == 'Finance')].mean(axis = 1) /*Finance is just one of the domain that has different id's associated to it */

But what the problem i am facing is that i can extract the mean of each id in grouped domain, however I can't seem to open the id and target columns that is associated to those mean values. So i can compare if the mean value is reaching the target value.


问:
Case-Insensitive array search
I am trying to search an NSMutableArray for a string.  I am using containsObject: currently but it looks like this is case sensitive.  I need to search for all combinations of the given string (trip).  Any ideas would be greatly appreciated.

```<code>if ([self.theArray containsObject:trip]) {
</code>```

}

答:
Not that hard:   </p>

```<code>BOOL found = NO;
for (NSString* str in self.theArray) {
   if ([str caseInsensitiveCompare:trip] == NSOrderedSame) {
       found = YES;
       break;
   }
}
</code>```

Title: Case-Insensitive array search

Body:
I am trying to search an NSMutableArray for a string.  I am using containsObject: currently but it looks like this is case sensitive.  I need to search for all combinations of the given string (trip).  Any ideas would be greatly appreciated.

if ([self.theArray containsObject:trip]) {

}

Body:
Not that hard:

BOOL found = NO;
for (NSString* str in self.theArray) {
   if ([str caseInsensitiveCompare:trip] == NSOrderedSame) {
       found = YES;
       break;
   }
}


答:
Since numpy.timedelta is belong to numpy.number, if you only want the number numeric columns return

```<code>num= ['int16', 'int32', 'int64', 'float16', 'float32', 'float64','complex128']
test.select_dtypes(include=num)
Out[715]:
    compl  float  int
0  (1-1j)   -2.5   -1
1  (5+0j)    3.4    2
</code>```

Title: pd.DataFrame.select_dtypes() inculdes timedelta dtype

Body:
Why is it expected behavior that this test code:

test = pd.DataFrame({'bool' :[False, True], 'int':[-1,2], 'float': [-2.5, 3.4],
                     'compl':np.array([1-1j, 5]),
                     'dt'   :[pd.Timestamp('2013-01-02'), pd.Timestamp('2016-10-20')],
                     'td'   :[pd.Timestamp('2012-03-02')- pd.Timestamp('2016-10-20'),
                              pd.Timestamp('2010-07-12')- pd.Timestamp('2000-11-10')]})
test.dtypes
test.select_dtypes(np.number)

Produces DataFrame with TimeDelta column included?

>>> bool                bool
>>> int                int64
>>> float            float64
>>> compl         complex128
>>> dt        datetime64[ns]
>>> td       timedelta64[ns]
>>> dtype: object

>>>     int     float   compl   td
>>> 0    -1     -2.5    (1-1j)  -1693 days
>>> 1     2      3.4    (5+0j)   3531 days

EDIT:

For someone (including me) the following may be helpful:

I've also found the reason why this behavior was unexpected for me at first. The reason was another way to check if dtype of pd.DataFrame is numeric. Namely via pd.api.types.is_numeric_dtype:

for col in test.columns:
    if pd.api.types.is_numeric_dtype(test[col]):
        print (test[col].dtype)

>>> bool
>>> int64
>>> float64
>>> complex128

Which produces more 'human-desired' output.

Body:
Since numpy.timedelta is belong to numpy.number, if you only want the number numeric columns return

num= ['int16', 'int32', 'int64', 'float16', 'float32', 'float64','complex128']
test.select_dtypes(include=num)
Out[715]:
    compl  float  int
0  (1-1j)   -2.5   -1
1  (5+0j)    3.4    2


问:
SpringBoot not able to read from vault generic location
In my Vault and Consul integration i have following secrets engine enble

```<code>[root@user ConsulSetup]# vault secrets list -detailed
Path          Plugin       Accessor              Default TTL    Max TTL    Force No Cache    Replication    Seal Wrap    Options           Description
----          ------       --------              -----------    -------    --------------    -----------    ---------    -------           -----------
cubbyhole/    cubbyhole    cubbyhole_f6352b88    n/a            n/a        false             local          false        map[]             per-token private secret storage
identity/     identity     identity_9ba1824c     system         system     false             replicated     false        map[]             identity store
kv/           kv           kv_864b2492           system         system     false             replicated     false        map[version:2]    n/a
sys/          system       system_0064db30       n/a            n/a        false             replicated     false        map[]             system endpoints used for control, policy and debugging
</code>```

After this i added key/value from this command

```<code>vault kv put kv/demo/dev spring.datasource.username=demo-user-dev spring.datasource.password=demo-pass-dev
</code>```

Now i want to read these value via SpringBoot Application so here is my bootstrap.yml file

```<code>spring:
    cloud:
        # Configuration for a vault server running in dev mode
        vault:
            scheme: http
            host: <HOSTIP>
            port: 8200
            connection-timeout: 5000
            read-timeout: 15000
            authentication: TOKEN
            token: <TOKEN>
            kv.enabled: true
            generic:
                enabled: true
                backend: kv
                profile-separator: '/'

logging:
    level:
        ROOT: WARN
</code>```

Now i am trying to run the code

```<code>package com.example.demo;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

import javax.annotation.PostConstruct;
import javax.sql.DataSource;
import java.sql.SQLException;

import static org.springframework.util.Assert.isTrue;

@SpringBootApplication
public class DemoApplication {

    @Value("${spring.profiles.active:#{null}}")
    private String profiles;

    @Value("${spring.datasource.username}")
    private String username;

    @Value("${spring.datasource.password}")
    private String password;

    @Value("${other.username}")
    private String other;

    @Autowired
    private DataSource dataSource;

    public static void main(String[] args) {
        SpringApplication.run(DemoApplication.class, args);
    }

    @PostConstruct
    private void postConstruct() throws SQLException {
        System.out.println("##########################");
        System.out.println("profile(s): " + profiles);
        System.out.println("username: " + username);
        System.out.println("password: " + password);
        System.out.println("other: " + other);

        dataSource.getConnection();
        System.out.println("Successfully connected to database");
        System.out.println("##########################");

        isTrue (!username.equals("to-be-overwritten-by-vault-value"), "Username " + username);
        isTrue (!password.equals("to-be-overwritten-by-vault-value"), "Password " + password);
    }

}
</code>```

Which is end with following error

  #################### profile(s): null username: to-be-overwritten-by-vault-value password:

to-be-overwritten-by-vault-value other:
  to-be-overwritten-by-vault-value Successfully connected to database

  #################### 2019-05-30 15:31:39.147  WARN 18680 --- [           main] s.c.a.AnnotationConfigApplicationContext : Exception encountered

during context initialization - cancelling refresh attempt:
  org.springframework.beans.factory.BeanCreationException: Error
  creating bean with name 'demoApplication': Invocation of init method
  failed; nested exception is java.lang.IllegalArgumentException:
  Username to-be-overwritten-by-vault-value 2019-05-30 15:31:39.178
  ERROR 18680 --- [           main] o.s.boot.SpringApplication

  : Application startup failed

org.springframework.beans.factory.BeanCreationException: Error
  creating bean with name 'demoApplication': Invocation of init method
  failed; nested exception is java.lang.IllegalArgumentException:
  Username to-be-overwritten-by-vault-value     at
  org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:137)
  ~[spring-beans-4.3.9.RELEASE.jar:4.3.9.RELEASE]   at
  org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:409)
  ~[spring-beans-4.3.9.RELEASE.jar:4.3.9.RELEASE]   at
  org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1620)
  ~[spring-beans-4.3.9.RELEASE.jar:4.3.9.RELEASE]   at
  org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:555)
  ~[spring-beans-4.3.9.RELEASE.jar:4.3.9.RELEASE]   at
  org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:483)
  ~[spring-beans-4.3.9.RELEASE.jar:4.3.9.RELEASE]   at
  org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306)
  ~[spring-beans-4.3.9.RELEASE.jar:4.3.9.RELEASE]   at
  org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
  ~[spring-beans-4.3.9.RELEASE.jar:4.3.9.RELEASE]   at
  org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302)
  ~[spring-beans-4.3.9.RELEASE.jar:4.3.9.RELEASE]   at
  org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)
  ~[spring-beans-4.3.9.RELEASE.jar:4.3.9.RELEASE]   at
  org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:761)
  ~[spring-beans-4.3.9.RELEASE.jar:4.3.9.RELEASE]   at
  org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:867)
  ~[spring-context-4.3.9.RELEASE.jar:4.3.9.RELEASE]     at
  org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:543)
  ~[spring-context-4.3.9.RELEASE.jar:4.3.9.RELEASE]     at
  org.springframework.boot.SpringApplication.refresh(SpringApplication.java:693)
  [spring-boot-1.5.4.RELEASE.jar:1.5.4.RELEASE]     at
  org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:360)
  [spring-boot-1.5.4.RELEASE.jar:1.5.4.RELEASE]     at
  org.springframework.boot.SpringApplication.run(SpringApplication.java:303)
  [spring-boot-1.5.4.RELEASE.jar:1.5.4.RELEASE]     at
  org.springframework.boot.SpringApplication.run(SpringApplication.java:1118)
  [spring-boot-1.5.4.RELEASE.jar:1.5.4.RELEASE]     at
  org.springframework.boot.SpringApplication.run(SpringApplication.java:1107)
  [spring-boot-1.5.4.RELEASE.jar:1.5.4.RELEASE]     at
  com.example.demo.DemoApplication.main(DemoApplication.java:33)
  [classes/:na] Caused by: java.lang.IllegalArgumentException: Username
  to-be-overwritten-by-vault-value  at
  org.springframework.util.Assert.isTrue(Assert.java:92)
  ~[spring-core-4.3.9.RELEASE.jar:4.3.9.RELEASE]    at
  com.example.demo.DemoApplication.postConstruct(DemoApplication.java:48)
  [classes/:na]     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
  Method) ~[na:1.8.0_171]   at
  sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
  ~[na:1.8.0_171]   at
  sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
  ~[na:1.8.0_171]   at java.lang.reflect.Method.invoke(Unknown Source)
  ~[na:1.8.0_171]   at
  org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:366)
  ~[spring-beans-4.3.9.RELEASE.jar:4.3.9.RELEASE]   at
  org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:311)
  ~[spring-beans-4.3.9.RELEASE.jar:4.3.9.RELEASE]   at
  org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:134)
  ~[spring-beans-4.3.9.RELEASE.jar:4.3.9.RELEASE]   ... 17 common frames
  omitted

When i have vault integrated with file system that time path was something like secret\key it was working fine with this new changes its stop working.

答:
The exception actually does not seem to be related to vault though. It says

```<code>java.lang.IllegalArgumentException: Username to-be-overwritten-by-vault-value
</code>```

Is it that you have a space in the default values that you have defined in the properties file?

Body:
In my Vault and Consul integration i have following secrets engine enble

[root@user ConsulSetup]# vault secrets list -detailed
Path          Plugin       Accessor              Default TTL    Max TTL    Force No Cache    Replication    Seal Wrap    Options           Description
----          ------       --------              -----------    -------    --------------    -----------    ---------    -------           -----------
cubbyhole/    cubbyhole    cubbyhole_f6352b88    n/a            n/a        false             local          false        map[]             per-token private secret storage
identity/     identity     identity_9ba1824c     system         system     false             replicated     false        map[]             identity store
kv/           kv           kv_864b2492           system         system     false             replicated     false        map[version:2]    n/a
sys/          system       system_0064db30       n/a            n/a        false             replicated     false        map[]             system endpoints used for control, policy and debugging

After this i added key/value from this command

vault kv put kv/demo/dev spring.datasource.username=demo-user-dev spring.datasource.password=demo-pass-dev

Now i want to read these value via SpringBoot Application so here is my bootstrap.yml file

spring:
    cloud:
        # Configuration for a vault server running in dev mode
        vault:
            scheme: http
            host: <HOSTIP>
            port: 8200
            connection-timeout: 5000
            read-timeout: 15000
            authentication: TOKEN
            token: <TOKEN>
            kv.enabled: true
            generic:
                enabled: true
                backend: kv
                profile-separator: '/'

logging:
    level:
        ROOT: WARN

Now i am trying to run the code

package com.example.demo;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

import javax.annotation.PostConstruct;
import javax.sql.DataSource;
import java.sql.SQLException;

import static org.springframework.util.Assert.isTrue;

@SpringBootApplication
public class DemoApplication {

    @Value("${spring.profiles.active:#{null}}")
    private String profiles;

    @Value("${spring.datasource.username}")
    private String username;

    @Value("${spring.datasource.password}")
    private String password;

    @Value("${other.username}")
    private String other;

    @Autowired
    private DataSource dataSource;

    public static void main(String[] args) {
        SpringApplication.run(DemoApplication.class, args);
    }

    @PostConstruct
    private void postConstruct() throws SQLException {
        System.out.println("##########################");
        System.out.println("profile(s): " + profiles);
        System.out.println("username: " + username);
        System.out.println("password: " + password);
        System.out.println("other: " + other);

        dataSource.getConnection();
        System.out.println("Successfully connected to database");
        System.out.println("##########################");

        isTrue (!username.equals("to-be-overwritten-by-vault-value"), "Username " + username);
        isTrue (!password.equals("to-be-overwritten-by-vault-value"), "Password " + password);
    }

}

Which is end with following error

  #################### profile(s): null username: to-be-overwritten-by-vault-value password:

to-be-overwritten-by-vault-value other:
  to-be-overwritten-by-vault-value Successfully connected to database

  #################### 2019-05-30 15:31:39.147  WARN 18680 --- [           main] s.c.a.AnnotationConfigApplicationContext : Exception encountered

during context initialization - cancelling refresh attempt:
  org.springframework.beans.factory.BeanCreationException: Error
  creating bean with name 'demoApplication': Invocation of init method
  failed; nested exception is java.lang.IllegalArgumentException:
  Username to-be-overwritten-by-vault-value 2019-05-30 15:31:39.178
  ERROR 18680 --- [           main] o.s.boot.SpringApplication

  : Application startup failed

org.springframework.beans.factory.BeanCreationException: Error
  creating bean with name 'demoApplication': Invocation of init method
  failed; nested exception is java.lang.IllegalArgumentException:
  Username to-be-overwritten-by-vault-value     at
  org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:137)
  ~[spring-beans-4.3.9.RELEASE.jar:4.3.9.RELEASE]   at
  org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:409)
  ~[spring-beans-4.3.9.RELEASE.jar:4.3.9.RELEASE]   at
  org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1620)
  ~[spring-beans-4.3.9.RELEASE.jar:4.3.9.RELEASE]   at
  org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:555)
  ~[spring-beans-4.3.9.RELEASE.jar:4.3.9.RELEASE]   at
  org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:483)
  ~[spring-beans-4.3.9.RELEASE.jar:4.3.9.RELEASE]   at
  org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306)
  ~[spring-beans-4.3.9.RELEASE.jar:4.3.9.RELEASE]   at
  org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
  ~[spring-beans-4.3.9.RELEASE.jar:4.3.9.RELEASE]   at
  org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302)
  ~[spring-beans-4.3.9.RELEASE.jar:4.3.9.RELEASE]   at
  org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)
  ~[spring-beans-4.3.9.RELEASE.jar:4.3.9.RELEASE]   at
  org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:761)
  ~[spring-beans-4.3.9.RELEASE.jar:4.3.9.RELEASE]   at
  org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:867)
  ~[spring-context-4.3.9.RELEASE.jar:4.3.9.RELEASE]     at
  org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:543)
  ~[spring-context-4.3.9.RELEASE.jar:4.3.9.RELEASE]     at
  org.springframework.boot.SpringApplication.refresh(SpringApplication.java:693)
  [spring-boot-1.5.4.RELEASE.jar:1.5.4.RELEASE]     at
  org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:360)
  [spring-boot-1.5.4.RELEASE.jar:1.5.4.RELEASE]     at
  org.springframework.boot.SpringApplication.run(SpringApplication.java:303)
  [spring-boot-1.5.4.RELEASE.jar:1.5.4.RELEASE]     at
  org.springframework.boot.SpringApplication.run(SpringApplication.java:1118)
  [spring-boot-1.5.4.RELEASE.jar:1.5.4.RELEASE]     at
  org.springframework.boot.SpringApplication.run(SpringApplication.java:1107)
  [spring-boot-1.5.4.RELEASE.jar:1.5.4.RELEASE]     at
  com.example.demo.DemoApplication.main(DemoApplication.java:33)
  [classes/:na] Caused by: java.lang.IllegalArgumentException: Username
  to-be-overwritten-by-vault-value  at
  org.springframework.util.Assert.isTrue(Assert.java:92)
  ~[spring-core-4.3.9.RELEASE.jar:4.3.9.RELEASE]    at
  com.example.demo.DemoApplication.postConstruct(DemoApplication.java:48)
  [classes/:na]     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
  Method) ~[na:1.8.0_171]   at
  sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
  ~[na:1.8.0_171]   at
  sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
  ~[na:1.8.0_171]   at java.lang.reflect.Method.invoke(Unknown Source)
  ~[na:1.8.0_171]   at
  org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:366)
  ~[spring-beans-4.3.9.RELEASE.jar:4.3.9.RELEASE]   at
  org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:311)
  ~[spring-beans-4.3.9.RELEASE.jar:4.3.9.RELEASE]   at
  org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:134)
  ~[spring-beans-4.3.9.RELEASE.jar:4.3.9.RELEASE]   ... 17 common frames
  omitted

When i have vault integrated with file system that time path was something like secret\key it was working fine with this new changes its stop working.

Body:
The exception actually does not seem to be related to vault though. It says

java.lang.IllegalArgumentException: Username to-be-overwritten-by-vault-value

Is it that you have a space in the default values that you have defined in the properties file?


问:
Ways to determine returning "anonymous" guests in PHP
Two types of users visit my website: registered users and guests. Registered users are tracked and retained by PHP session, cookies and individual logins. Guests I find trickier to manage so I give little to no authority to contribute content.

I want to open up editing for users (registered or not) to save favourites, shopping carts, vote in polls, moderate, tag, upload and comment. At the same time, protect the data from other non-registered users.

What are some options or best practices to determine a unique visitor from another and some security measures to consider when allowing them to contribute more? Should I work around their security/restriction settings to provide contribution service or should I expect them to meet in the middle and relax some of their settings to allow cookies etc?

IP Address - The IP is an option but only temporary. It can change for a user reconnecting to their Internet with a different IP, and for that matter another user may eventually share the same IP. IP can also be anonymous, shared or misleading.

PHP Sessions - OK for a session (and if enabled) but lost on closing the browser.

Cookies - Can store session data but is easily (and often) disabled from the client-side.

Header data - combining known details of he user might at least group users - ISP, browser, operating system, referring website etc.

Edit: I'm still having trouble getting my head around all the the key factors involved... we set up a cookie for a guest. Attach a bunch of edits and user history to that session. If the cookie is removed, the data floats around attached to nothing and the user loses their data. Or if the user logs in, the guest and user data should be merged...

答:
You could use a combination of all of the identifying points that you can find, that are not likely to change, and are likely to be unique - looking at panopticlick you can gather a bunch of data such as installed fonts and browser plugins. You could take those kinds of more unique data points and hash them to give you an id, and then compare it against the less unique data like useragents and ip addresses.

But honestly, that's super complicated and sneaky. Use cookies/session. If the user doesn't want to enable cookies then they don't want all your anonymous tracking for a reason, and you should honor their decision.


问:
How do I set state from a child component that requires a function to set state with react-dates?
I'm using react-date to try to set state of dates that are selected:

```<code><DateRangePicker
  ...
  onDatesChange={({ startDate, endDate }) => this.setState({ startDate,
  endDate })}
  ...
/> // PropTypes.func.isRequired
</code>```

This works fine if it is in the main component where the state lives, but I now have the DateRangePicker in a child component of that component, so I can't just set the state from the child.

I've tried writing a function that lives in the parent component to set the state, but this isn't working:

In the child component:

```<code><DateRangePicker
  ...
  onDatesChange={({ startDate, endDate }) => props.onDatesChange({ startDate,
  endDate })}
  ...
/>
</code>```

In the parent component:

```<code>onDatesChange = (startDate, endDate) => {
    this.setState({
      startDate,
      endDate
    })
  };
</code>```

Although this works for another parameter in DateRangePicker that sets the state in the parent:

In the child:

```<code><DateRangePicker
  ...
  onFocusChange={props.onFocusChange}
  ...
/>
</code>```

In the parent:

```<code>  onFocusChange = focusedInput => this.setState({ focusedInput })
</code>```

答:
It looks like you're calling props.onDatesChange with an object, but the parent component's onDatesChange function takes two parameters.

```<code>// Parent:
onDatesChange = (startDate, endDate) => {

// Child:
props.onDatesChange({ startDate, endDate })
</code>```

I would probably change the parent to take an object, to keep it consistent with the other similar functions.

Body:
I'm using react-date to try to set state of dates that are selected:

<DateRangePicker
  ...
  onDatesChange={({ startDate, endDate }) => this.setState({ startDate,
  endDate })}
  ...
/> // PropTypes.func.isRequired

This works fine if it is in the main component where the state lives, but I now have the DateRangePicker in a child component of that component, so I can't just set the state from the child.

I've tried writing a function that lives in the parent component to set the state, but this isn't working:

In the child component:

<DateRangePicker
  ...
  onDatesChange={({ startDate, endDate }) => props.onDatesChange({ startDate,
  endDate })}
  ...
/>

In the parent component:

onDatesChange = (startDate, endDate) => {
    this.setState({
      startDate,
      endDate
    })
  };

Although this works for another parameter in DateRangePicker that sets the state in the parent:

In the child:

<DateRangePicker
  ...
  onFocusChange={props.onFocusChange}
  ...
/>

In the parent:

  onFocusChange = focusedInput => this.setState({ focusedInput })

Body:
It looks like you're calling props.onDatesChange with an object, but the parent component's onDatesChange function takes two parameters.

// Parent:
onDatesChange = (startDate, endDate) => {

// Child:
props.onDatesChange({ startDate, endDate })

I would probably change the parent to take an object, to keep it consistent with the other similar functions.


问:
Google Cloud Vision API returning "429 - Quota Exceeded"
I've had a Cloud Function that makes Vision API requests, specifically Document Text Detection requests. My peak requests rate is usually around ~120-150 requests per minute on an average day.

I've suddenly been getting resource quota exceeded errors for Vision API requests with a request rate at 2500 requests per minute. Some things to note:

I've had no code changes in 3 months

I deleted and redeployed the Cloud Function making these requests to stop any problematic image that was causing a runaway loop

My code calling the API nor the cloud functions themselves were getting retried so there really wasn't a way that I could exponentially increase my request rate overnight with no changes introduced.

The service account making the Vision calls is making the normal amount of requests and is only in use by the cloud function i.e. not being used by someone's local script

I've since turned on retries to mitigate this issue since it'll "work" with exponential back off but this is expensive to do, especially with the vision API. Is there anything I can do to find out the root cause of this issue?

答:
To identify the specific quota being exceeded, Stackdriver API helps by using Monitoring quota metrics as explained here .

GCP lets you specify quota being exceeded in greater depth using the Stackdriver API and UI, with quota metrics appearing in the Metrics Explorer.