问:
Python Conditional Regex
My program is given an object with parameters, and I need to get the parameters' values.

The object my program is given will look like this:

Object = """{{objectName|
parameter1=random text|
parameter2=that may or may not|
parameter3=contain any letter (well, almost)|
parameter4=this is some [[problem|problematic text]], Houston, we have a problem!|
otherParameters=(order of parameters is random, but their name is fixed)}}"""
```

<p>(all parameters might or might not exist)</p>

<p>I am trying to get the properties values.</p>

<p>In the first 3 lines, its pretty easy. a simple regex will find it:</p>

<pre class="lang-py prettyprint-override"><code>if "parameter1" in Object:
    parameter1 = re.split(r"parameter1=(.*?)[\|\}]", Object)[1]

if "parameter2" in Object:
    parameter2 = re.split(r"parameter2=(.*?)[\|\}]", Object)[1]
</code>```

and so on.

The problem is with parameter4, the above regex (property4=(.*?)[\|\}]) will only return this is some [[problem, since the regex stops at the vertical bar.

Now here is the thing: vertical bar will only appear as part of the text inside "[[]]".

For example, parameter1=a[[b|c]]d might appear, but parameter1=a|bc| will never appear.

I need a regex which will stop at vertical bar, unless it is inside double square brackets. So for example, for parameter4, I will get this is some [[problem|problematic text]], Houston, we have a problem!

答:
Worked here when I removed the "?":

```<code>parameter4 = re.split(r"parameter4=(.*)[\|\}]", object_)[1]
</code>```

I also changed the name of the variable to "object_" because "object" is a built-in object in Python

Best.

Title: Python Conditional Regex

Body:
My program is given an object with parameters, and I need to get the parameters' values.

The object my program is given will look like this:

Object = """{{objectName|
parameter1=random text|
parameter2=that may or may not|
parameter3=contain any letter (well, almost)|
parameter4=this is some [[problem|problematic text]], Houston, we have a problem!|
otherParameters=(order of parameters is random, but their name is fixed)}}"""

(all parameters might or might not exist)

I am trying to get the properties values.

In the first 3 lines, its pretty easy. a simple regex will find it:

if "parameter1" in Object:
    parameter1 = re.split(r"parameter1=(.*?)[\|\}]", Object)[1]

if "parameter2" in Object:
    parameter2 = re.split(r"parameter2=(.*?)[\|\}]", Object)[1]

and so on.

The problem is with parameter4, the above regex (property4=(.*?)[\|\}]) will only return this is some [[problem, since the regex stops at the vertical bar.

Now here is the thing: vertical bar will only appear as part of the text inside "[[]]".

For example, parameter1=a[[b|c]]d might appear, but parameter1=a|bc| will never appear.

I need a regex which will stop at vertical bar, unless it is inside double square brackets. So for example, for parameter4, I will get this is some [[problem|problematic text]], Houston, we have a problem!

Body:
Worked here when I removed the "?":

parameter4 = re.split(r"parameter4=(.*)[\|\}]", object_)[1]

I also changed the name of the variable to "object_" because "object" is a built-in object in Python

Best.


来源: stackexchange

问:
Mongoengine / MongoDB model filtering on basis of key presence in Dictionary field
I'm building a data-model using Mongoengine to store some metada for, in this case, email files. The very stripped down model, with only the field relevant for my issue, is:

```<code>class Email(Document):
    headers = DictField()
</code>```

The headers dictionary will contain extracted header-data from emails which works in a key-value relationship.

In this header-data I know, at times, there will be a header present with the name (and dictionary key): x-mailer (all header keys are auto lower-cased). I've build a simple query to see if a header contains this key like this:

```<code>xmailer_emails = Email.objects.filter(headers__exists='x-mailer')
</code>```

However, the results of this don't contain all the entries in the email collection but it contains quite a few that don't have x-mailer as a key in the headers dictionary. Here is some code I used to check the resulting data:

```<code>xmailer_emails = Email.objects.filter(headers__exists='x-mailer'))
log(xmailer_emails._query)

log('Total email count: ' + str(Email.objects.count()))
log('X-Mailer email acount: ' + str(xmailer_emails.count()))

no_xmailer = len([e for e in xmailer_emails if 'x-mailer' not in e.headers])
log('Filtered no x-mailer count: ' + str(no_xmailer))

has_xmailer = len([e for e in xmailer_emails if 'x-mailer' in e.headers])
log('Filtered has x-mailer count: ' + str(has_xmailer))
</code>```

Here is the output I get:

```<code>[21:32:54 05/27/19] {'headers': {'$exists': 'x-mailer'}}
[21:32:54 05/27/19] Total email count: 86
[21:32:54 05/27/19] X-Mailer email count: 79
[21:32:54 05/27/19] Filtered no x-mailer count: 55
[21:32:54 05/27/19] Filtered has x-mailer count: 24
</code>```

So while there are 86 entries in the collection it pulls back 79 of them on a filter for x-mailer in headers. Filtering out of those only 24 actually have that key-value in the dictionary. I seem to be doing something wrong but I don't know what.

Here's a short dump of a few of the items that it also pulls back that do not contain the x-mailer key, these are the keys:

```<code>dict_keys(['x-receiver', 'to', 'mime-version', 'received', 'x-priority', 'x-sender', 'date', 'content-type', 'message-id', 'subject', 'x-riferimento-message-id', 'from'])
dict_keys(['content-type', 'mime-version'])
dict_keys(['x-wum-to', 'date', 'x-uidl', 'message-id', 'in-reply-to', 'x-wum-replyto', 'x-message-delivery', 'mime-version', 'received', 'x-savecopy', 'authentication-results', 'x-wum-nature', 'x-account-key', 'x-wum-from', 'message-context', 'x-originalarrivaltime', 'x-me-spamrating', 'x-me-spamlevel', 'to', 'x-dkim-result', 'x-mozilla-status', 'x-mozilla-status2', 'references', 'x-auth-result', 'x-store-info', 'return-path', 'x-message-info', 'x-sid-pra', 'x-message-status', 'sender', 'x-wum-cci', 'content-type', 'reply-to', 'subject', 'from'])
dict_keys(['x-ms-has-attach', 'to', 'mime-version', 'received', 'thread-topic', 'x-auto-response-suppress', 'x-ms-exchange-organization-authsource', 'content-language', 'content-type', 'date', 'subject', 'message-id', 'thread-index', 'reply-to', 'from'])
</code>```

I can't figure out what is going on, while the exists operator works it also includes non-matching documents but not every one in the collection.

I've also raised this question as an issue on the mongoengine repository with no reply (besides a linked SO issue for the exists operator): https://github.com/MongoEngine/mongoengine/issues/2059

答:
The pymongo query you actually need to execute is the following:

```<code>c.find({'headers.x-mailer': {'$exists': True}})
</code>```

If you have simple keys (e.g "xmailer" instead of "x-mailer"), you could achieve this in mongoengine with:
Email.objects.filter(headers__xmailer__exists=True))

But you can achieve it using the raw operator:

Email.objects.filter(__raw__={'headers.x-mailer': {'$exists': True}})

ContentLicense: CC BY-SA 4.0

Body:
I'm building a data-model using Mongoengine to store some metada for, in this case, email files. The very stripped down model, with only the field relevant for my issue, is:

class Email(Document):
    headers = DictField()

The headers dictionary will contain extracted header-data from emails which works in a key-value relationship.

In this header-data I know, at times, there will be a header present with the name (and dictionary key): x-mailer (all header keys are auto lower-cased). I've build a simple query to see if a header contains this key like this:

xmailer_emails = Email.objects.filter(headers__exists='x-mailer')

However, the results of this don't contain all the entries in the email collection but it contains quite a few that don't have x-mailer as a key in the headers dictionary. Here is some code I used to check the resulting data:

xmailer_emails = Email.objects.filter(headers__exists='x-mailer'))
log(xmailer_emails._query)

log('Total email count: ' + str(Email.objects.count()))
log('X-Mailer email acount: ' + str(xmailer_emails.count()))

no_xmailer = len([e for e in xmailer_emails if 'x-mailer' not in e.headers])
log('Filtered no x-mailer count: ' + str(no_xmailer))

has_xmailer = len([e for e in xmailer_emails if 'x-mailer' in e.headers])
log('Filtered has x-mailer count: ' + str(has_xmailer))

Here is the output I get:

[21:32:54 05/27/19] {'headers': {'$exists': 'x-mailer'}}
[21:32:54 05/27/19] Total email count: 86
[21:32:54 05/27/19] X-Mailer email count: 79
[21:32:54 05/27/19] Filtered no x-mailer count: 55
[21:32:54 05/27/19] Filtered has x-mailer count: 24

So while there are 86 entries in the collection it pulls back 79 of them on a filter for x-mailer in headers. Filtering out of those only 24 actually have that key-value in the dictionary. I seem to be doing something wrong but I don't know what.

Here's a short dump of a few of the items that it also pulls back that do not contain the x-mailer key, these are the keys:

dict_keys(['x-receiver', 'to', 'mime-version', 'received', 'x-priority', 'x-sender', 'date', 'content-type', 'message-id', 'subject', 'x-riferimento-message-id', 'from'])
dict_keys(['content-type', 'mime-version'])
dict_keys(['x-wum-to', 'date', 'x-uidl', 'message-id', 'in-reply-to', 'x-wum-replyto', 'x-message-delivery', 'mime-version', 'received', 'x-savecopy', 'authentication-results', 'x-wum-nature', 'x-account-key', 'x-wum-from', 'message-context', 'x-originalarrivaltime', 'x-me-spamrating', 'x-me-spamlevel', 'to', 'x-dkim-result', 'x-mozilla-status', 'x-mozilla-status2', 'references', 'x-auth-result', 'x-store-info', 'return-path', 'x-message-info', 'x-sid-pra', 'x-message-status', 'sender', 'x-wum-cci', 'content-type', 'reply-to', 'subject', 'from'])
dict_keys(['x-ms-has-attach', 'to', 'mime-version', 'received', 'thread-topic', 'x-auto-response-suppress', 'x-ms-exchange-organization-authsource', 'content-language', 'content-type', 'date', 'subject', 'message-id', 'thread-index', 'reply-to', 'from'])

I can't figure out what is going on, while the exists operator works it also includes non-matching documents but not every one in the collection.

I've also raised this question as an issue on the mongoengine repository with no reply (besides a linked SO issue for the exists operator): https://github.com/MongoEngine/mongoengine/issues/2059

Body:
The pymongo query you actually need to execute is the following:

c.find({'headers.x-mailer': {'$exists': True}})

If you have simple keys (e.g "xmailer" instead of "x-mailer"), you could achieve this in mongoengine with:
Email.objects.filter(headers__xmailer__exists=True))

But you can achieve it using the raw operator:

Email.objects.filter(__raw__={'headers.x-mailer': {'$exists': True}})


问:
APACHE 2.4 Remote Proxy mod_substitute and ProxyHTMLExtended don't work
I built a reverse proxy with Apache 2.4 on a cento's 7 server. It works with standard html pages but i need to substitute some url stored in .js files too. The directive:

```<code>ProxyHTMLExtended On
</code>```

should enable the parsing inside external .css and .js files but it doesn't work. In the log file I can see:

```<code>[proxy_html:trace1] [pid 3263] mod_proxy_html.c(823): [client xxx] Non-HTML content; not inserting proxy-html filter
</code>```

I tried to use mod_substitute, this is the interesting part in my httpd.conf:

```<code>ProxyPass /mylocation/ http://remoteserver/
<Location /mylocation/>
  ProxyHTMLEnable On
  ProxyHTMLExtended On

  LogLevel debug proxy_html:trace3  substitute_module:debug
  RequestHeader    unset  Accept-Encoding

  AddOutputFilterByType SUBSTITUTE text/javascript text/html
  Substitute "s|/css/|/mylocation/css/|ni"
  Substitute "s|/js/|/mylocation/js/|ni"
  Substitute "s|/custom_logo/|/mylocation/custom_logo/|ni"
  Substitute "s|/html/|/mylocation/html/|ni"
  Substitute "s|/current_config/|/mylocation/current_config/|ni"
  Substitute "s|/web_lang/|/mylocation/web_lang/|ni"
  Substitute "s|/custom_lang/|/mylocation/custom_lang/|ni"

  ProxyPassReverse /

  ProxyHTMLURLMap //remoteserver /mylocation/
  ProxyHTMLURLMap http://remoteserver /mylocation/
  ProxyHTMLURLMap /mylocation /mylocation
  ProxyHTMLURLMap ^\/(.*) /mylocation/$1 R
</Location>
</code>```

But in the log file there aren't any mod_substitute trace. It seems mod_substitute is never called.

The proxyHTMLURLMap rules works fine, but only on regular html files.

Depending on the  .js file I'm asking to the server, I can see in the log file:

```<code>[xml2enc:debug] [pid 3259] mod_xml2enc.c(254): [client xxx] AH01434: Charset ISO-8859-1 not supported by libxml2; trying apr_xlate
</code>```

or

```<code>[proxy_html:trace1] [pid 3263] mod_proxy_html.c(823): [client xxx] Non-HTML content; not inserting proxy-html filter
</code>```

then the process stop, I receive the file but nothing has been replaced on it.

1) Wy the "ProxyHTMLExtended On" rule don't parse external .js files as described in Apache documentation?

2) Wy the mod_substitute don't work?

答:
I have a similar problem where mod substitute was not working for me.

Then I read somewhere that normally, mod_subsittue is actually by default only working if the HTTP response you get from the server has mime type txt/html.

This was not my scenario.
My scenario was that I want to re-write the content of an XML, namely a JEE web service that was being reverse proxied by apache httpd.

In order to manage to make the mod substitute alter the reply content it was necessary to do:

```<code><Location /mockOutgoingWebServicePortBinding>

              # core authentication and mod_auth_basic configuration
              # for mod_authn_dbd
              AuthType Basic
              Authname "Password Required"

             # core authorization configuration
             AuthUserFile C:\Apache24\conf\htpasswd
             Require valid-user

             # mod_filter to be able to subsitute text xml
             AddOutputFilterByType SUBSTITUTE text/xml text/html
             Substitute "s|http://someHostName:8088/|http://localhost:80/|i"
</Location>
</code>```

The magic step was enabling the mod_filter and adding the directive: AddOutputFilterByType .

When this was added in, the substitue altered the body of the xml. replacing the endpoint address.

ContentLicense: CC BY-SA 3.0

Body:
I built a reverse proxy with Apache 2.4 on a cento's 7 server. It works with standard html pages but i need to substitute some url stored in .js files too. The directive:

ProxyHTMLExtended On

should enable the parsing inside external .css and .js files but it doesn't work. In the log file I can see:

[proxy_html:trace1] [pid 3263] mod_proxy_html.c(823): [client xxx] Non-HTML content; not inserting proxy-html filter

I tried to use mod_substitute, this is the interesting part in my httpd.conf:

ProxyPass /mylocation/ http://remoteserver/
<Location /mylocation/>
  ProxyHTMLEnable On
  ProxyHTMLExtended On

  LogLevel debug proxy_html:trace3  substitute_module:debug
  RequestHeader    unset  Accept-Encoding

  AddOutputFilterByType SUBSTITUTE text/javascript text/html
  Substitute "s|/css/|/mylocation/css/|ni"
  Substitute "s|/js/|/mylocation/js/|ni"
  Substitute "s|/custom_logo/|/mylocation/custom_logo/|ni"
  Substitute "s|/html/|/mylocation/html/|ni"
  Substitute "s|/current_config/|/mylocation/current_config/|ni"
  Substitute "s|/web_lang/|/mylocation/web_lang/|ni"
  Substitute "s|/custom_lang/|/mylocation/custom_lang/|ni"

  ProxyPassReverse /

  ProxyHTMLURLMap //remoteserver /mylocation/
  ProxyHTMLURLMap http://remoteserver /mylocation/
  ProxyHTMLURLMap /mylocation /mylocation
  ProxyHTMLURLMap ^\/(.*) /mylocation/$1 R
</Location>

But in the log file there aren't any mod_substitute trace. It seems mod_substitute is never called.

The proxyHTMLURLMap rules works fine, but only on regular html files.

Depending on the  .js file I'm asking to the server, I can see in the log file:

[xml2enc:debug] [pid 3259] mod_xml2enc.c(254): [client xxx] AH01434: Charset ISO-8859-1 not supported by libxml2; trying apr_xlate

or

[proxy_html:trace1] [pid 3263] mod_proxy_html.c(823): [client xxx] Non-HTML content; not inserting proxy-html filter

then the process stop, I receive the file but nothing has been replaced on it.

1) Wy the "ProxyHTMLExtended On" rule don't parse external .js files as described in Apache documentation?

2) Wy the mod_substitute don't work?

Body:
I have a similar problem where mod substitute was not working for me.

Then I read somewhere that normally, mod_subsittue is actually by default only working if the HTTP response you get from the server has mime type txt/html.

This was not my scenario.
My scenario was that I want to re-write the content of an XML, namely a JEE web service that was being reverse proxied by apache httpd.

In order to manage to make the mod substitute alter the reply content it was necessary to do:

<Location /mockOutgoingWebServicePortBinding>

              # core authentication and mod_auth_basic configuration
              # for mod_authn_dbd
              AuthType Basic
              Authname "Password Required"

             # core authorization configuration
             AuthUserFile C:\Apache24\conf\htpasswd
             Require valid-user

             # mod_filter to be able to subsitute text xml
             AddOutputFilterByType SUBSTITUTE text/xml text/html
             Substitute "s|http://someHostName:8088/|http://localhost:80/|i"
</Location>

The magic step was enabling the mod_filter and adding the directive: AddOutputFilterByType .

When this was added in, the substitue altered the body of the xml. replacing the endpoint address.


问:
How to get the html document contained inside an HTML tag using selenium in python
I want to get the source code of HTML document which is inside an HTML tag which is generated after some JavaScript and store it in a variable. Here, the HTML tag is  and it contains a variable kind of something that looks like #document and when I expand this, I get an HTML document which looks something like <!DOCTYPE html> ...

To summarize:

```<code><iframe src="https://www.XXXXXX.com/" allow="autoplay; fullscreen" frameborder="no" scrolling="no" allowfullscreen="yes" style="width: 100%; height: 100%;">
    #document
        <!DOCTYPE html>
        <html>...</html>  // a whole new HTML document
</iframe>
</code>```

I want to store all the content of this HTML document as a string in python

What I have done:

```<code>driver.find_element_by_xpath('/path/to/iframe/tag').get_attribute('innerHTML')
</code>```

but, this just returns an empty string. Also, I have checked if it is working with BeautifulSoup

```<code>html = driver.execute_script("return document.body.innerHTML")
soup = BeautifulSoup(html, 'html5lib')
print(soup.prettify())
</code>```

but, this also isn't working

NOTE: I run these test only after the script is executed, also, I guess the problem seems to be with the #document thing

答:
The answer is simple,
I just switched from current frame to the frame of  element

Code:

```<code>driver.switch_to.default_content()
frame = driver.find_element_by_xpath('//iframe')
driver.switch_to.frame(frame)
</code>```

Body:
I want to get the source code of HTML document which is inside an HTML tag which is generated after some JavaScript and store it in a variable. Here, the HTML tag is  and it contains a variable kind of something that looks like #document and when I expand this, I get an HTML document which looks something like <!DOCTYPE html> ...

To summarize:

    #document
        <!DOCTYPE html>
        ...  // a whole new HTML document

I want to store all the content of this HTML document as a string in python

What I have done:

driver.find_element_by_xpath('/path/to/iframe/tag').get_attribute('innerHTML')

but, this just returns an empty string. Also, I have checked if it is working with BeautifulSoup

html = driver.execute_script("return document.body.innerHTML")
soup = BeautifulSoup(html, 'html5lib')
print(soup.prettify())

but, this also isn't working

NOTE: I run these test only after the script is executed, also, I guess the problem seems to be with the #document thing

Body:
The answer is simple,
I just switched from current frame to the frame of <iframe> element

Code:

driver.switch_to.default_content()
frame = driver.find_element_by_xpath('//iframe')
driver.switch_to.frame(frame)


问:
Regex giving incorrect results
Working in Javascript attempting to use a regular expression to capture data in a string.

My string appears as this starting with the left bracket

```<code>['ABC']['ABC.5']['ABC.5.1']
</code>```

My goal is to get each piece of the regular expression as a chunk or in array.
I have reviewed and see that the match function might be a good choice.

```<code>var myString = "['ABC']['ABC.5']['ABC.5.1']";
myString.match(/\[/g]);
</code>```

The output I see is only the [ for each element.

I would like the array to be like this for example

```<code>myString[0] = ['ABC']
myString[1] = ['ABC.5']
myString[2] = ['ABC.5.1']
</code>```

What is the correct regular expression and or function to get the above-desired output?

答:
If you just want to separate them, you can use a simple expression or better than that you can split them:

```<code>\[\'(.+?)'\]
</code>```

const regex = /\[\'(.+?)'\]/gm;
const str = `['ABC']['ABC.5']['ABC.5.1']`;
const subst = `['$1']\n`;

// The substituted value will be contained in the result variable
const result = str.replace(regex, subst);

console.log('Substitution result: ', result);```

DEMO

Title: Regex giving incorrect results

Body:
Working in Javascript attempting to use a regular expression to capture data in a string.

My string appears as this starting with the left bracket

['ABC']['ABC.5']['ABC.5.1']

My goal is to get each piece of the regular expression as a chunk or in array.
I have reviewed and see that the match function might be a good choice.

var myString = "['ABC']['ABC.5']['ABC.5.1']";
myString.match(/\[/g]);

The output I see is only the [ for each element.

I would like the array to be like this for example

myString[0] = ['ABC']
myString[1] = ['ABC.5']
myString[2] = ['ABC.5.1']

What is the correct regular expression and or function to get the above-desired output?

Body:
If you just want to separate them, you can use a simple expression or better than that you can split them:

\[\'(.+?)'\]

const regex = /\[\'(.+?)'\]/gm;
const str = `['ABC']['ABC.5']['ABC.5.1']`;
const subst = `['$1']\n`;

// The substituted value will be contained in the result variable
const result = str.replace(regex, subst);

console.log('Substitution result: ', result);

DEMO


问:
Func<T extends IComponent>( ): T[] => IComponent[] - doesn't assign to T
I trying to realize ECS pattern in typescript, and i created class ComponentStore which contain components of entities for next handling.
Components like this:

class Health implements IComponent {
  name: EComponents = EComponents.health;
}
```

<p>But methods <code>setComponent</code> and <code>getComponents</code> throw errors:</p>

<blockquote>
  <p>Error:(12, 11) TS2322: Type 'IComponent[]' is not assignable to type 'T[]'.
   Type 'IComponent' is not assignable to type 'T'.</p>

  <p>Error:(17, 5) TS2322: Type 'IComponent[]' is not assignable to type 'T[]'.
   Type 'IComponent' is not assignable to type 'T'.</p>
</blockquote>

<p>I do this like in example in Generics in Typescript, but it doesn't work.</p>

<pre class="lang-js prettyprint-override"><code>class ComponentStore implements IComponentStore {
  private components: Map<EComponents, IComponent[]> = new Map();

  setComponent<T extends IComponent>( componentName: EComponents, component: IComponent): void {
    const components: T[] = this.components.get(componentName) || [];
    this.components.set(componentName, [...components, component ]);
  }

  getComponents<T extends IComponent>( componentName: EComponents): T[] {
    return this.components.get(componentName) || [];
  }
}

enum EComponents {
  health = 'health',
}

interface IComponent {
  name: Ecomponents;
}
</code>```

Of course i can use casts, but i don't think that is good idea.
I wanted that type detection work correctly for these methods
My goal in set Map with some name of EComponent which keep array of one type IComponent like IHealth and then transfer it to the system.

答:
I am not exactly sure what you want, but perhaps you can look at something like this:

```<code>interface EComponents {
  //...
}

interface IComponent {
  //...
}

interface IComponentStore<T extends IComponent> {
  setComponent(componentName: EComponents, component: T): void;
  getComponents(componentName: EComponents): T[];
}

class ComponentStore<T extends IComponent> implements IComponentStore<T> {
  private components = new Map<EComponents, T[]>();

  setComponent(componentName: EComponents, component: T): void {
    const components = this.components.get(componentName) || [];
    this.components.set(componentName, [...components, component ]);
  }

  getComponents(componentName: EComponents): T[] {
    return this.components.get(componentName) || [];
  }
}
</code>```

The idea here is that ComponentStore and IComponentStore already get the generic type T for the IComponent instances it should handle...

I assumed that IComponent and EComponents are interfaces. I provided a dummy implementation for them. This code should compile correctly. It generates the following JavaScript (targeting ESNext):

```<code>"use strict";
class ComponentStore {
    constructor() {
        this.components = new Map();
    }
    setComponent(componentName, component) {
        const components = this.components.get(componentName) || [];
        this.components.set(componentName, [...components, component]);
    }
    getComponents(componentName) {
        return this.components.get(componentName) || [];
    }
}
</code>```

Hope it helps you.

Edit:

By the way, IComponentStore does not need to be generic for this. The following code works too:

```<code>interface EComponents {
  //...
}

interface IComponent {
  //...
}

interface IComponentStore {
  setComponent(componentName: EComponents, component: IComponent): void;
  getComponents(componentName: EComponents): IComponent[];
}

class ComponentStore<T extends IComponent> implements IComponentStore {
  private components = new Map<EComponents, T[]>();

  setComponent(componentName: EComponents, component: T): void {
    const components = this.components.get(componentName) || [];
    this.components.set(componentName, [...components, component ]);
  }

  getComponents(componentName: EComponents): T[] {
    return this.components.get(componentName) || [];
  }
}
</code>```

It generates the same JavaScript.

Edit 2

Five days have passed since my latest edit. But I promised I would update my answer again based on the new information provided in your comments. Sorry for the delay.

I assume you have already found a fitting answer yourself, but if not, you may perhaps find some inspiration from the following code. Hope it helps.

```<code>enum EComponents {
  health = 'health',
  wealth = 'wealth'
}

interface IComponent {
  name: EComponents;
  value: string;
}

interface IComponentStore {
  setComponent<T extends IComponent>(component: T): void;
  getComponents<T extends IComponent>(componentName: EComponents): T[];
}

class ComponentStore implements IComponentStore {
  private components: Map<EComponents, IComponent[]> = new Map();

  setComponent<T extends IComponent>(component: T): void {
    const components: IComponent[] = this.components.get(component.name) || [];
    this.components.set(component.name, [...components, component]);
  }

  getComponents<T extends IComponent>(componentName: EComponents): T[] {
    return (this.components.get(componentName) || []) as T[];
  }
}

class Health implements IComponent {
  name: EComponents = EComponents.health;

  constructor(public value: string) {
  }
}

class Wealth implements IComponent {
  name: EComponents = EComponents.wealth;

  constructor(public value: string) {
  }
}

const store = new ComponentStore();

store.setComponent(new Health('ill'));
store.setComponent(new Health('fairly healthy'));
store.setComponent(new Health('top condition'));

store.setComponent(new Wealth('poor'));
store.setComponent(new Wealth('prosperous'));
store.setComponent(new Wealth('filthy rich'));

console.log(store.getComponents(EComponents.health));
console.log(store.getComponents(EComponents.wealth));
</code>```

Body:
I trying to realize ECS pattern in typescript, and i created class ComponentStore which contain components of entities for next handling.
Components like this:

class Health implements IComponent {
  name: EComponents = EComponents.health;
}

But methods setComponent and getComponents throw errors:

Error:(12, 11) TS2322: Type 'IComponent[]' is not assignable to type 'T[]'.
   Type 'IComponent' is not assignable to type 'T'.

Error:(17, 5) TS2322: Type 'IComponent[]' is not assignable to type 'T[]'.
   Type 'IComponent' is not assignable to type 'T'.

I do this like in example in Generics in Typescript, but it doesn't work.

class ComponentStore implements IComponentStore {
  private components: Map<EComponents, IComponent[]> = new Map();

  setComponent<T extends IComponent>( componentName: EComponents, component: IComponent): void {
    const components: T[] = this.components.get(componentName) || [];
    this.components.set(componentName, [...components, component ]);
  }

  getComponents<T extends IComponent>( componentName: EComponents): T[] {
    return this.components.get(componentName) || [];
  }
}

enum EComponents {
  health = 'health',
}

interface IComponent {
  name: Ecomponents;
}

Of course i can use casts, but i don't think that is good idea.
I wanted that type detection work correctly for these methods
My goal in set Map with some name of EComponent which keep array of one type IComponent like IHealth and then transfer it to the system.

Body:
I am not exactly sure what you want, but perhaps you can look at something like this:

interface EComponents {
  //...
}

interface IComponent {
  //...
}

interface IComponentStore<T extends IComponent> {
  setComponent(componentName: EComponents, component: T): void;
  getComponents(componentName: EComponents): T[];
}

class ComponentStore<T extends IComponent> implements IComponentStore<T> {
  private components = new Map<EComponents, T[]>();

  setComponent(componentName: EComponents, component: T): void {
    const components = this.components.get(componentName) || [];
    this.components.set(componentName, [...components, component ]);
  }

  getComponents(componentName: EComponents): T[] {
    return this.components.get(componentName) || [];
  }
}

The idea here is that ComponentStore and IComponentStore already get the generic type T for the IComponent instances it should handle...

I assumed that IComponent and EComponents are interfaces. I provided a dummy implementation for them. This code should compile correctly. It generates the following JavaScript (targeting ESNext):

"use strict";
class ComponentStore {
    constructor() {
        this.components = new Map();
    }
    setComponent(componentName, component) {
        const components = this.components.get(componentName) || [];
        this.components.set(componentName, [...components, component]);
    }
    getComponents(componentName) {
        return this.components.get(componentName) || [];
    }
}

Hope it helps you.

Edit:

By the way, IComponentStore does not need to be generic for this. The following code works too:

interface EComponents {
  //...
}

interface IComponent {
  //...
}

interface IComponentStore {
  setComponent(componentName: EComponents, component: IComponent): void;
  getComponents(componentName: EComponents): IComponent[];
}

class ComponentStore<T extends IComponent> implements IComponentStore {
  private components = new Map<EComponents, T[]>();

  setComponent(componentName: EComponents, component: T): void {
    const components = this.components.get(componentName) || [];
    this.components.set(componentName, [...components, component ]);
  }

  getComponents(componentName: EComponents): T[] {
    return this.components.get(componentName) || [];
  }
}

It generates the same JavaScript.

Edit 2

Five days have passed since my latest edit. But I promised I would update my answer again based on the new information provided in your comments. Sorry for the delay.

I assume you have already found a fitting answer yourself, but if not, you may perhaps find some inspiration from the following code. Hope it helps.

enum EComponents {
  health = 'health',
  wealth = 'wealth'
}

interface IComponent {
  name: EComponents;
  value: string;
}

interface IComponentStore {
  setComponent<T extends IComponent>(component: T): void;
  getComponents<T extends IComponent>(componentName: EComponents): T[];
}

class ComponentStore implements IComponentStore {
  private components: Map<EComponents, IComponent[]> = new Map();

  setComponent<T extends IComponent>(component: T): void {
    const components: IComponent[] = this.components.get(component.name) || [];
    this.components.set(component.name, [...components, component]);
  }

  getComponents<T extends IComponent>(componentName: EComponents): T[] {
    return (this.components.get(componentName) || []) as T[];
  }
}

class Health implements IComponent {
  name: EComponents = EComponents.health;

  constructor(public value: string) {
  }
}

class Wealth implements IComponent {
  name: EComponents = EComponents.wealth;

  constructor(public value: string) {
  }
}

const store = new ComponentStore();

store.setComponent(new Health('ill'));
store.setComponent(new Health('fairly healthy'));
store.setComponent(new Health('top condition'));

store.setComponent(new Wealth('poor'));
store.setComponent(new Wealth('prosperous'));
store.setComponent(new Wealth('filthy rich'));

console.log(store.getComponents(EComponents.health));
console.log(store.getComponents(EComponents.wealth));


问:
Disallowing HTTP verbs: System.Web.HttpMethodNotAllowedHandler is never invoked
On my site, I want to disallow HTTP HEAD requests and have them answered with the 405 status code (Method not allowed). To achieve this I have the following in my web.config file:

```<code><system.webServer>
  <modules runAllManagedModulesForAllRequests="true">
    <remove name="TelemetryCorrelationHttpModule" />
    <add name="TelemetryCorrelationHttpModule" type="Microsoft.AspNet.TelemetryCorrelation.TelemetryCorrelationHttpModule, Microsoft.AspNet.TelemetryCorrelation" preCondition="integratedMode,managedHandler" />
    <remove name="ApplicationInsightsWebTracking" />
    <add name="ApplicationInsightsWebTracking" type="Microsoft.ApplicationInsights.Web.ApplicationInsightsHttpModule, Microsoft.AI.Web" preCondition="managedHandler" />
  </modules>
  <handlers>
    <clear />
    <add name="DenyHead" path="*" verb="HEAD" type="System.Web.HttpMethodNotAllowedHandler" />
    <add name="DebugAttachHandler" path="DebugAttach.aspx" verb="DEBUG" type="System.Web.HttpDebugHandler" preCondition="integratedMode,runtimeVersionv4.0" />
    <add name="StaticFile" path="*" verb="*" modules="StaticFileModule" resourceType="Either" requireAccess="Read" />
    <add name="ExtensionlessUrlHandler-Integrated-4.0" path="*." verb="GET,POST,DEBUG" type="System.Web.Handlers.TransferRequestHandler" preCondition="integratedMode,runtimeVersionv4.0" />
  </handlers>
  <security>
    <requestFiltering allowDoubleEscaping="true">
      <verbs allowUnlisted="false">
        <add verb="GET" allowed="true" />
        <add verb="POST" allowed="true" />
        <add verb="HEAD" allowed="true" />
        <add verb="DEBUG" allowed="true" />
      </verbs>
    </requestFiltering>
  </security>
</system.webServer>
</code>```

Unfortunately, this doesn't work - I'm receiving bog-standard 404s instead.

Enabling failed request tracing yields the following:

```<code>20  HANDLER_CHANGED OldHandlerName
                    NewHandlerName DenyHead
                    NewHandlerType System.Web.HttpMethodNotAllowedHandler

...

61  AspNetPipelineEnter Data1 <Application_BeginRequest in my ASP.NET application>

...

135 HANDLER_CHANGED OldHandlerName System.Web.HttpMethodNotAllowedHandler
                    NewHandlerName System.Web.Mvc.MvcHandler

...

169 MODULE_SET_RESPONSE_ERROR_STATUS Notification EXECUTE_REQUEST_HANDLER
                                     HttpStatus   404
</code>```

This seems to show that the DenyHead handler is somehow being replaced/overridden by my MVC application, but there's no code in my app that does anything of the sort.

I've tried alternative recommendations such as the answers here, but they give the same result.

Request filtering isn't an option because the status code it returns is not configurable (it always returns a 404).

Action filters aren't an option because they won't be hit for static content, and I don't want to send everything through the MVC pipeline.

答:
I wouldn't use IIS configuration as it gets you dependant on IIS, even though you might already be. Using a filter removes that dependency, just like that:</p>

```<code>public class VerbFilter : IAsyncActionFilter
{
    public async Task OnActionExecutionAsync(ActionExecutingContext context, ActionExecutionDelegate next)
    {
        if (context.HttpContext.Request.Method == "HEAD")
        {
            context.Result = new StatusCodeResult(405);
        }
        else
        {
            await next();
        }
    }
}
</code>```

Body:
On my site, I want to disallow HTTP HEAD requests and have them answered with the 405 status code (Method not allowed). To achieve this I have the following in my web.config file:

<system.webServer>
  <modules runAllManagedModulesForAllRequests="true">
    <remove name="TelemetryCorrelationHttpModule" />
    <add name="TelemetryCorrelationHttpModule" type="Microsoft.AspNet.TelemetryCorrelation.TelemetryCorrelationHttpModule, Microsoft.AspNet.TelemetryCorrelation" preCondition="integratedMode,managedHandler" />
    <remove name="ApplicationInsightsWebTracking" />
    <add name="ApplicationInsightsWebTracking" type="Microsoft.ApplicationInsights.Web.ApplicationInsightsHttpModule, Microsoft.AI.Web" preCondition="managedHandler" />
  </modules>
  <handlers>
    <clear />
    <add name="DenyHead" path="*" verb="HEAD" type="System.Web.HttpMethodNotAllowedHandler" />
    <add name="DebugAttachHandler" path="DebugAttach.aspx" verb="DEBUG" type="System.Web.HttpDebugHandler" preCondition="integratedMode,runtimeVersionv4.0" />
    <add name="StaticFile" path="*" verb="*" modules="StaticFileModule" resourceType="Either" requireAccess="Read" />
    <add name="ExtensionlessUrlHandler-Integrated-4.0" path="*." verb="GET,POST,DEBUG" type="System.Web.Handlers.TransferRequestHandler" preCondition="integratedMode,runtimeVersionv4.0" />
  </handlers>
  <security>
    <requestFiltering allowDoubleEscaping="true">
      <verbs allowUnlisted="false">
        <add verb="GET" allowed="true" />
        <add verb="POST" allowed="true" />
        <add verb="HEAD" allowed="true" />
        <add verb="DEBUG" allowed="true" />
      </verbs>
    </requestFiltering>
  </security>
</system.webServer>

Unfortunately, this doesn't work - I'm receiving bog-standard 404s instead.

Enabling failed request tracing yields the following:

20  HANDLER_CHANGED OldHandlerName
                    NewHandlerName DenyHead
                    NewHandlerType System.Web.HttpMethodNotAllowedHandler

...

61  AspNetPipelineEnter Data1 <Application_BeginRequest in my ASP.NET application>

...

135 HANDLER_CHANGED OldHandlerName System.Web.HttpMethodNotAllowedHandler
                    NewHandlerName System.Web.Mvc.MvcHandler

...

169 MODULE_SET_RESPONSE_ERROR_STATUS Notification EXECUTE_REQUEST_HANDLER
                                     HttpStatus   404

This seems to show that the DenyHead handler is somehow being replaced/overridden by my MVC application, but there's no code in my app that does anything of the sort.

I've tried alternative recommendations such as the answers here, but they give the same result.

Request filtering isn't an option because the status code it returns is not configurable (it always returns a 404).

Action filters aren't an option because they won't be hit for static content, and I don't want to send everything through the MVC pipeline.

Body:
I wouldn't use IIS configuration as it gets you dependant on IIS, even though you might already be. Using a filter removes that dependency, just like that:

public class VerbFilter : IAsyncActionFilter
{
    public async Task OnActionExecutionAsync(ActionExecutingContext context, ActionExecutionDelegate next)
    {
        if (context.HttpContext.Request.Method == "HEAD")
        {
            context.Result = new StatusCodeResult(405);
        }
        else
        {
            await next();
        }
    }
}


问:
Error defining an input shape in keras for (60000, 28, 28) array
I am setting up my first neural network with keras and tensorflow. I got my input into an array of shape (60000, 28, 28), but when I try and feed it to the model I get an error that the input shape is wrong.

I have tried multiple different input shapes including (60000, 28, 28) (1, 28, 28) (28, 28) (28, 28, 1) but none of them seem to work.

model = kr.Sequential()

model.add(InputLayer(input_shape=(60000, 28, 28)))
model.add(Dense(units=784, activation='relu'))
model.add(Dense(units=392, activation='relu'))
model.add(Dense(units=196, activation='relu'))
model.add(Dense(units=10, activation='softmax'))

model.compile(loss='categorical_crossentropy', optimizer='Adam', metrics=['accuracy'])

training = model.fit(x=images_array, y=labels_array, epochs=10, batch_size=256)
```

<p>I would expect it to work with input shape (60000, 28, 28) but I always get this error:</p>

<blockquote>
  <p>ValueError: Error when checking input: expected input_1 to have 4
  dimensions, but got array with shape (60000, 28, 28)</p>
</blockquote>

<p><strong>Edit:</strong></p>

<p>Thanks to everyone who answerd. cho_uc answer indeed worked, which is why I accepted it.
What I shold have mentioned in the post was, that I was trying to build a model consisting only of Dense layers, so I can use it as a benchmark for future models.</p>

<p>I solved the input layer problem with:</p>

<pre class="lang-py prettyprint-override"><code>images_array = images_array.reshape(-1, 28 * 28)

model.add(InputLayer(input_shape=(784, )))
</code>```

答:
Two corrections are required.

TF and Keras expects image dimension as (Width, Height, Channels), channels being 3 for RGB images and 1 for greyscale images.

```<code>model.add(InputLayer(input_shape=(28, 28, 1)))
</code>```

The training input to fit() method must be of dimension (Number of samples, Width, Height, Channels).

```<code>assert images_array.shape == (60000, 28, 28, 1)
</code>```

Body:
I am setting up my first neural network with keras and tensorflow. I got my input into an array of shape (60000, 28, 28), but when I try and feed it to the model I get an error that the input shape is wrong.

I have tried multiple different input shapes including (60000, 28, 28) (1, 28, 28) (28, 28) (28, 28, 1) but none of them seem to work.

model = kr.Sequential()

model.add(InputLayer(input_shape=(60000, 28, 28)))
model.add(Dense(units=784, activation='relu'))
model.add(Dense(units=392, activation='relu'))
model.add(Dense(units=196, activation='relu'))
model.add(Dense(units=10, activation='softmax'))

model.compile(loss='categorical_crossentropy', optimizer='Adam', metrics=['accuracy'])

training = model.fit(x=images_array, y=labels_array, epochs=10, batch_size=256)

I would expect it to work with input shape (60000, 28, 28) but I always get this error:

ValueError: Error when checking input: expected input_1 to have 4
  dimensions, but got array with shape (60000, 28, 28)

Edit:

Thanks to everyone who answerd. cho_uc answer indeed worked, which is why I accepted it.
What I shold have mentioned in the post was, that I was trying to build a model consisting only of Dense layers, so I can use it as a benchmark for future models.

I solved the input layer problem with:

images_array = images_array.reshape(-1, 28 * 28)

model.add(InputLayer(input_shape=(784, )))

Body:
Two corrections are required.

TF and Keras expects image dimension as (Width, Height, Channels), channels being 3 for RGB images and 1 for greyscale images.

model.add(InputLayer(input_shape=(28, 28, 1)))

The training input to fit() method must be of dimension (Number of samples, Width, Height, Channels).

assert images_array.shape == (60000, 28, 28, 1)


问:
Looking for a quicker way of fuzzy string matching
I am using fuzzywuzzy in python for fuzzy string matching. I have a set of names in a list named HKCP_list which I am matching against a pandas column iteratively to get the best possible match. Given below is the code for it

```<code>import fuzzywuzzy
from fuzzywuzzy import fuzz,process

def search_func(row):
    chk = process.extract(row,HKCP_list,scorer=fuzz_token_sort_ratio)[0]
    return chk

wc_df['match']=wc_df['concat_name'].map(search_func)
</code>```

The wc_df dataframe contains the column 'concat_name' which needs to be matched with every name in the list HKCP_list. The above code took around 2 hours to run with 6K names in the list and 11K names in the column 'concat_name'.

I have to rerun this on another data set where are 89K names in the list and 120K names in the column. In order to speed up the process, I got an idea in the following question on Stackoverflow

Vectorizing or Speeding up Fuzzywuzzy String Matching on PANDAS Column

In one of the comments in the answer in the above, it has been advised to compare names that have the same 1st letter. The 'concat_name' column that I am comparing with is a derived column obtained by concatenating 'first_name' and 'last_name' columns in the dataframe. Hence I am using the following function to match the 1st letter (since this is a token sort score that I am considering, I am comparing the 1st letter of both the first_name and last_name with the elements in the list). Given below is the code:

```<code>wc_df['first_name_1stletter'] = wc_df['first_name'].str[0]
wc_df['last_name_1stletter'] = wc_df['last_name'].str[0]

import time
start_time=time.time()
def match_func(row):
    CP_subset=[x for x in HKCP_list if x[0]==row['first_name_1stletter'] or x[0]==row['last_name_1stletter']]
    return CP_subset
wc_df['list_to_match']=wc_df.apply(match_func,axis=1)
end_time=time.time()
print(end_time-start_time)
</code>```

The above step took 1600 second with 6K X 11K data. The 'list_to_match' column contains the list of names to be compared for each concat_name. Now here I have to again take the list_to_match element and pass individual elements in a list and do the fuzzy string matching using the process.extract method. Is there a more elegant and faster way of doing this in the same step as above?

PS: Editing this to add an example as to how the list and the dataframe column looks like.

```<code>HKCp_list=['jeff bezs','michael blomberg','bill gtes','tim coook','elon musk']
concat_name=['jeff bezos','michael bloomberg','bill gates','tim cook','elon musk','donald trump','kim jong un', 'narendra modi','michael phelps']
first_name=['jeff','michael','bill','tim','elon','donald','kim','narendra','michael']
last_name=['bezos','bloomberg','gates','cook','musk','trump','jong un', 'modi','phelps']
import pandas as pd
df=pd.DataFrame({'first_name':first_name,'last_name':last_name,'concat_name':concat_name})
</code>```

Each row of the 'concat_name' in df has to be compared against the elements of HKcp_list.

PS: editing today to reflect the ":" and the row in the 2nd snippet of code I missed yesterday

Regards,
Nirvik

答:
Given below is the code that I have used to make the comparison of list dynamic for each instance:

```<code>import fuzzywuzzy
from fuzzywuzzy import fuzz,process

wc_df['first_name_1stletter'] = wc_df['first_name'].str[0]
wc_df['last_name_1stletter'] = wc_df['last_name'].str[0]

import time
start_time=time.time()
def match_func(row):

    CP_subset=[x for x in HKCP_list if x[0]==row['first_name_1stletter'] or x[0]==row['last_name_1stletter']]
    if len(CP_subset)>0:
        chk=process.extract(row['concat_name'],CP_subset,scorer=fuzz.token_sort_ratio)[0]
    else:
        chk = "No item to match"

    return chk

wc_df['match']=wc_df.apply(match_func,axis=1)

end_time=time.time()
print(end_time-start_time)
</code>```

The above code for 6K X 11K comparisons took around 2600 seconds instead of the 7000 seconds as per the 1st snippet of the code posted in the question.

Body:
I am using fuzzywuzzy in python for fuzzy string matching. I have a set of names in a list named HKCP_list which I am matching against a pandas column iteratively to get the best possible match. Given below is the code for it

import fuzzywuzzy
from fuzzywuzzy import fuzz,process

def search_func(row):
    chk = process.extract(row,HKCP_list,scorer=fuzz_token_sort_ratio)[0]
    return chk

wc_df['match']=wc_df['concat_name'].map(search_func)

The wc_df dataframe contains the column 'concat_name' which needs to be matched with every name in the list HKCP_list. The above code took around 2 hours to run with 6K names in the list and 11K names in the column 'concat_name'.

I have to rerun this on another data set where are 89K names in the list and 120K names in the column. In order to speed up the process, I got an idea in the following question on Stackoverflow

Vectorizing or Speeding up Fuzzywuzzy String Matching on PANDAS Column

In one of the comments in the answer in the above, it has been advised to compare names that have the same 1st letter. The 'concat_name' column that I am comparing with is a derived column obtained by concatenating 'first_name' and 'last_name' columns in the dataframe. Hence I am using the following function to match the 1st letter (since this is a token sort score that I am considering, I am comparing the 1st letter of both the first_name and last_name with the elements in the list). Given below is the code:

wc_df['first_name_1stletter'] = wc_df['first_name'].str[0]
wc_df['last_name_1stletter'] = wc_df['last_name'].str[0]

import time
start_time=time.time()
def match_func(row):
    CP_subset=[x for x in HKCP_list if x[0]==row['first_name_1stletter'] or x[0]==row['last_name_1stletter']]
    return CP_subset
wc_df['list_to_match']=wc_df.apply(match_func,axis=1)
end_time=time.time()
print(end_time-start_time)

The above step took 1600 second with 6K X 11K data. The 'list_to_match' column contains the list of names to be compared for each concat_name. Now here I have to again take the list_to_match element and pass individual elements in a list and do the fuzzy string matching using the process.extract method. Is there a more elegant and faster way of doing this in the same step as above?

PS: Editing this to add an example as to how the list and the dataframe column looks like.

HKCp_list=['jeff bezs','michael blomberg','bill gtes','tim coook','elon musk']
concat_name=['jeff bezos','michael bloomberg','bill gates','tim cook','elon musk','donald trump','kim jong un', 'narendra modi','michael phelps']
first_name=['jeff','michael','bill','tim','elon','donald','kim','narendra','michael']
last_name=['bezos','bloomberg','gates','cook','musk','trump','jong un', 'modi','phelps']
import pandas as pd
df=pd.DataFrame({'first_name':first_name,'last_name':last_name,'concat_name':concat_name})

Each row of the 'concat_name' in df has to be compared against the elements of HKcp_list.

PS: editing today to reflect the ":" and the row in the 2nd snippet of code I missed yesterday

Regards,
Nirvik

Body:
Given below is the code that I have used to make the comparison of list dynamic for each instance:

import fuzzywuzzy
from fuzzywuzzy import fuzz,process

wc_df['first_name_1stletter'] = wc_df['first_name'].str[0]
wc_df['last_name_1stletter'] = wc_df['last_name'].str[0]

import time
start_time=time.time()
def match_func(row):

    CP_subset=[x for x in HKCP_list if x[0]==row['first_name_1stletter'] or x[0]==row['last_name_1stletter']]
    if len(CP_subset)>0:
        chk=process.extract(row['concat_name'],CP_subset,scorer=fuzz.token_sort_ratio)[0]
    else:
        chk = "No item to match"

    return chk

wc_df['match']=wc_df.apply(match_func,axis=1)

end_time=time.time()
print(end_time-start_time)

The above code for 6K X 11K comparisons took around 2600 seconds instead of the 7000 seconds as per the 1st snippet of the code posted in the question.


问:
When do I need to use/have a makefile?
I am trying to learn how to code on a Raspberry PI.
I am coming from coding using Windows and VS Code.
Now I am using Linux Mint 19.1 and ssh to access Raspbian 4.14.

The problem is that, after a long battle with downloading the library that I am trying to use, after installing a compiler, creating a file, and finding the right command to run and include paths I get undefined reference to errors.

I am trying to compile the simplest example code from https://github.com/hzeller/rpi-rgb-led-matrix
because I started with this project. I don't have any other code or example.

Here are the commands that I've wrote in command line:

```<code>sudo apt-get install g++
mkdir 4panel
cd 4panel
sudo nano main.cpp
git clone https://github.com/hzeller/rpi-rgb-led-matrix.git
sudo g++ -Wall -Irpi-rgb-led-matrix/include  main.cpp -o main
</code>```

The following commands should be:

```<code>sudo chmod +x main
./main
</code>```

main.cpp contains the same code as https://github.com/hzeller/rpi-rgb-led-matrix/blob/master/examples-api-use/minimal-example.cc

```<code>#include "led-matrix.h"

#include <unistd.h>
#include <math.h>
#include <stdio.h>
#include <signal.h>

using rgb_matrix::GPIO;
using rgb_matrix::RGBMatrix;
using rgb_matrix::Canvas;

volatile bool interrupt_received = false;
static void InterruptHandler(int signo) {
  interrupt_received = true;
}

static void DrawOnCanvas(Canvas *canvas) {
  /*
   * Let's create a simple animation. We use the canvas to draw
   * pixels. We wait between each step to have a slower animation.
   */
  canvas->Fill(0, 0, 255);

  int center_x = canvas->width() / 2;
  int center_y = canvas->height() / 2;
  float radius_max = canvas->width() / 2;
  float angle_step = 1.0 / 360;
  for (float a = 0, r = 0; r < radius_max; a += angle_step, r += angle_step) {
    if (interrupt_received)
      return;
    float dot_x = cos(a * 2 * M_PI) * r;
    float dot_y = sin(a * 2 * M_PI) * r;
    canvas->SetPixel(center_x + dot_x, center_y + dot_y,
                     255, 0, 0);
    usleep(1 * 1000);  // wait a little to slow down things.
  }
}

int main(int argc, char *argv[]) {
  RGBMatrix::Options defaults;
  defaults.hardware_mapping = "regular";  // or e.g. "adafruit-hat"
  defaults.rows = 32;
  defaults.chain_length = 1;
  defaults.parallel = 1;
  defaults.show_refresh_rate = true;
  Canvas *canvas = rgb_matrix::CreateMatrixFromFlags(&argc, &argv, &defaults);
  if (canvas == NULL)
    return 1;

  // It is always good to set up a signal handler to cleanly exit when we
  // receive a CTRL-C for instance. The DrawOnCanvas() routine is looking
  // for that.
  signal(SIGTERM, InterruptHandler);
  signal(SIGINT, InterruptHandler);

  DrawOnCanvas(canvas);    // Using the canvas.

  // Animation finished. Shut down the RGB matrix.
  canvas->Clear();
  delete canvas;

  return 0;
}
</code>```

Errors:

```<code>/tmp/cci8MGL5.o: In function `main':
main.cpp:(.text+0x264): undefined reference to `rgb_matrix::RGBMatrix::Options::Options()'
main.cpp:(.text+0x2a8): undefined reference to `rgb_matrix::CreateMatrixFromFlags(int*, char***, rgb_matrix::RGBMatrix::Options*, rgb_matrix::RuntimeOptions*, bool)'
collect2: error: ld returned 1 exit status

</code>```

Possible fixes that I found:

Link .cpp files in a strict order when compiling: sudo g++ main1.cpp main2.cpp

This

The need of a makefile

While searching for similar issues and hopefully finding some magic fix, I ended up more puzzled than before. I have no idea if what I did is what I was supposed to do.

What cpp files to link if I have just one ?

Do I need to link all the .cpp and .h files that the library has ?

I don't understand a thing about makefile or why it even exists.

My issue may be specific to my situation but I believe that the answers will help me with future projects.

You are free to answer all of my questions and misunderstandings and help me understand what is going on or answer just the question related to the title: Why makefile ?

答:
The problem is that, after a long battle with downloading the library
  that I am trying to use, after installing a compiler, creating a file,
  and finding the right command to run and include paths I get undefined
  reference to errors.

There are already numerous questions and answers on SO pertaining to such problems, especially What is an undefined reference/unresolved external symbol error and how do I fix it?

Possible fixes that I found:

Link .cpp files in a strict order when compiling: sudo g++ main1.cpp main2.cpp

[SO question with answers saying roughly the same as the previous]

Not typically relevant.  Although link order of different objects / libraries is significant, that rarely affects the case when all the needed C and / or C++ source files are specified in the same compilation command.

The need of a makefile

Using make does not inherently solve the problem, and it is certainly not necessary for solving the problem.  Writing a makefile that provides for a successful build, whether directly or via a build tool such as CMake or the Autotools, requires you to understand at least at a high level how compilers and linkers work, what their inputs and outputs are, and what their arguments and options mean.

What cpp files to link if I have just one ?

You do not link a .cpp (directly) at all.  You compile it and link the result to any needed libraries and any other objects. g++ will try to do that all in one step by default, in which case the process is conventionally called "compiling", not "linking", even though both functions are performed. g++ can be made to compile only, without linking, by specifying the -c option to it (which is common in makefiles, as it turns out), and it will happily link objects and libraries without compiling anything if you don't name any source files to it.

Anyway, your case is not at all that of just one .cpp file.  It is your one .cpp file plus all those of the library you are trying to use.

Do I need to link all the .cpp and .h files that the library has ?

You do not need to compile (directly) or link any .h files.  But if you want to use the library whose sources you downloaded, you do need to compile all the sources, and you probably should link them into an actual library.  Furthermore, unless you produce and use a static library, you should also install the library you build.  It looks like the library sources include a makefile, so you would be well advised to use make to perform those steps.

I don't understand a thing about makefile or why it even exists.

A makefile is the input to make.  A well-written one defines rules for how to build one or more "targets", and expresses the prerequisites (generally original sources or other targets) for doing so.  make uses these, together with a set of built-in rules, to figure out what steps to take to build the target(s) you ask it to build, and also what steps it can skip.  It is typical, for example, to write makefiles so that if you build a multisource project, modify only one source file, and then build again, only the modified source gets recompiled.

This may seem trivial when you're used to projects with only a handful of sources, but it becomes important in larger projects, where full rebuilds are expensive -- many hours, in some cases.  But even if a full build takes only a few minutes, if you really only needed to spend a few seconds then that adds up to a lot of wasted time over a full day of work.

Overall, then:

When do I need to use/have a makefile?

Never.  But providing a good one and using make to build your project has these among its advantages:

memorializing all the needed build commands, including options and arguments, in a persistent, actionable form, and therefore

providing for consistent, automated builds.

Those can also be achieved by a simple build script, but make also can give you

faster builds by building only those pieces that are out of date, and

built-in knowledge of how to build certain kinds of targets from corresponding sources.

Title: When do I need to use/have a makefile?

Body:
I am trying to learn how to code on a Raspberry PI.
I am coming from coding using Windows and VS Code.
Now I am using Linux Mint 19.1 and ssh to access Raspbian 4.14.

The problem is that, after a long battle with downloading the library that I am trying to use, after installing a compiler, creating a file, and finding the right command to run and include paths I get undefined reference to errors.

I am trying to compile the simplest example code from https://github.com/hzeller/rpi-rgb-led-matrix
because I started with this project. I don't have any other code or example.

Here are the commands that I've wrote in command line:

sudo apt-get install g++
mkdir 4panel
cd 4panel
sudo nano main.cpp
git clone https://github.com/hzeller/rpi-rgb-led-matrix.git
sudo g++ -Wall -Irpi-rgb-led-matrix/include  main.cpp -o main

The following commands should be:

sudo chmod +x main
./main

main.cpp contains the same code as https://github.com/hzeller/rpi-rgb-led-matrix/blob/master/examples-api-use/minimal-example.cc

#include "led-matrix.h"

#include <unistd.h>
#include <math.h>
#include <stdio.h>
#include <signal.h>

using rgb_matrix::GPIO;
using rgb_matrix::RGBMatrix;
using rgb_matrix::Canvas;

volatile bool interrupt_received = false;
static void InterruptHandler(int signo) {
  interrupt_received = true;
}

static void DrawOnCanvas(Canvas *canvas) {
  /*
   * Let's create a simple animation. We use the canvas to draw
   * pixels. We wait between each step to have a slower animation.
   */
  canvas->Fill(0, 0, 255);

  int center_x = canvas->width() / 2;
  int center_y = canvas->height() / 2;
  float radius_max = canvas->width() / 2;
  float angle_step = 1.0 / 360;
  for (float a = 0, r = 0; r < radius_max; a += angle_step, r += angle_step) {
    if (interrupt_received)
      return;
    float dot_x = cos(a * 2 * M_PI) * r;
    float dot_y = sin(a * 2 * M_PI) * r;
    canvas->SetPixel(center_x + dot_x, center_y + dot_y,
                     255, 0, 0);
    usleep(1 * 1000);  // wait a little to slow down things.
  }
}

int main(int argc, char *argv[]) {
  RGBMatrix::Options defaults;
  defaults.hardware_mapping = "regular";  // or e.g. "adafruit-hat"
  defaults.rows = 32;
  defaults.chain_length = 1;
  defaults.parallel = 1;
  defaults.show_refresh_rate = true;
  Canvas *canvas = rgb_matrix::CreateMatrixFromFlags(&argc, &argv, &defaults);
  if (canvas == NULL)
    return 1;

  // It is always good to set up a signal handler to cleanly exit when we
  // receive a CTRL-C for instance. The DrawOnCanvas() routine is looking
  // for that.
  signal(SIGTERM, InterruptHandler);
  signal(SIGINT, InterruptHandler);

  DrawOnCanvas(canvas);    // Using the canvas.

  // Animation finished. Shut down the RGB matrix.
  canvas->Clear();
  delete canvas;

  return 0;
}

Errors:

/tmp/cci8MGL5.o: In function `main':
main.cpp:(.text+0x264): undefined reference to `rgb_matrix::RGBMatrix::Options::Options()'
main.cpp:(.text+0x2a8): undefined reference to `rgb_matrix::CreateMatrixFromFlags(int*, char***, rgb_matrix::RGBMatrix::Options*, rgb_matrix::RuntimeOptions*, bool)'
collect2: error: ld returned 1 exit status

Possible fixes that I found:

Link .cpp files in a strict order when compiling: sudo g++ main1.cpp main2.cpp

This

The need of a makefile

While searching for similar issues and hopefully finding some magic fix, I ended up more puzzled than before. I have no idea if what I did is what I was supposed to do.

What cpp files to link if I have just one ?

Do I need to link all the .cpp and .h files that the library has ?

I don't understand a thing about makefile or why it even exists.

My issue may be specific to my situation but I believe that the answers will help me with future projects.

You are free to answer all of my questions and misunderstandings and help me understand what is going on or answer just the question related to the title: Why makefile ?


问:
How to replace zeros in Pandas Data Frame by negative 1
Replacing negative numbers to zero is straightforward, see
How to replace negative numbers in Pandas Data Frame by zero but I want to know how to do the opposite.

I want to replace the 0 here to -1:

But df.replace(0,-1) or df[df == 0] = -1 gives me

It seems like I am able to replace the zeros into positive integers but I cannot replace them with negative integers (i.e. df.replace(0,-3) replaces all my zeros with -253).

Am I missing something here?

答:
I believe you have a data type of 8 bit unsigned integer.  For that data type, there are no negatives and therefore a -1 overflows(underflows?) to the largest such number.

```<code>df = pd.DataFrame([[0, 1], [1, 0]], dtype=np.uint8)

df.replace(0, -1)

     0    1
0  255    1
1    1  255
</code>```

Where 255 is the largest such number.

```<code>np.iinfo(np.uint8).max

255
</code>```

Instead, set the data type first

```<code>df.astype(int).replace(0, -1)

   0  1
0 -1  1
1  1 -1
</code>```

Body:
I believe you have a data type of 8 bit unsigned integer.  For that data type, there are no negatives and therefore a -1 overflows(underflows?) to the largest such number.

df = pd.DataFrame([[0, 1], [1, 0]], dtype=np.uint8)

df.replace(0, -1)

     0    1
0  255    1
1    1  255

Where 255 is the largest such number.

np.iinfo(np.uint8).max

255

Instead, set the data type first

df.astype(int).replace(0, -1)

   0  1
0 -1  1
1  1 -1


问:
How do I create a dev and prod instance of the same cloud function using serverless?
I have a Cloud Function named getVendors that is deployed using serverless. I need to create both a dev and production instance of the same function. When I deploy using the dev variable in my yml file, it overwrites the function currently in GCP.

How do I deploy a dev instance of the same cloud function without overwriting the production instance?

For reference, I have two stacks (dev and prod) visible in the Deployment Manager. It's when I look at Cloud Functions in the project, there is only one function listed in the project. This is the function that gets overwritten.

```<code>service: get-vendor-info

provider:
  name: google
  stage: production
  runtime: nodejs8
  region: us-central1
  project: {PROJECT NAME IS HIDDEN}
  # the path to the credentials file needs to be absolute
  credentials:{MY_CREDENTIALS}.json

plugins:
  - serverless-google-cloudfunctions

package:
  exclude:
    - node_modules/**
    - .gitignore
    - .git/**

functions:
  getVendors:
    handler: getVendors
    events:
      - event:
          eventType: providers/cloud.pubsub/eventTypes/topic.publish
          resource: projects/{MY_PROJECT_NAME}/topics/getVendors

</code>```

答:
I use two different GCP projects; one for dev/test and another for prod. If there are multiple developers on the team you should consider giving each of them their own dev project as well.

By doing that you reduce the risk of development or testing work messing up production data. You could copy the production database into the dev/test project nightly, and potentially add special test data to it as well. This special test data could be corner cases, well-known data to run automated tests against, etc.

For a more in-depth discussion, check out this talk from Google Cloud Next: https://youtu.be/41QvqGfbz9o?t=1142

Body:
I have a Cloud Function named getVendors that is deployed using serverless. I need to create both a dev and production instance of the same function. When I deploy using the dev variable in my yml file, it overwrites the function currently in GCP.

How do I deploy a dev instance of the same cloud function without overwriting the production instance?

For reference, I have two stacks (dev and prod) visible in the Deployment Manager. It's when I look at Cloud Functions in the project, there is only one function listed in the project. This is the function that gets overwritten.

service: get-vendor-info

provider:
  name: google
  stage: production
  runtime: nodejs8
  region: us-central1
  project: {PROJECT NAME IS HIDDEN}
  # the path to the credentials file needs to be absolute
  credentials:{MY_CREDENTIALS}.json

plugins:
  - serverless-google-cloudfunctions

package:
  exclude:
    - node_modules/**
    - .gitignore
    - .git/**

functions:
  getVendors:
    handler: getVendors
    events:
      - event:
          eventType: providers/cloud.pubsub/eventTypes/topic.publish
          resource: projects/{MY_PROJECT_NAME}/topics/getVendors


问:
Integration between Webservices and Website
I have some web services running on a central server(Debian) on Apache.
I have multiple remote machines which call these webservices.
One of the webservice is a status/healthcheck webservice - each remote machine has a cronjob which calls the status webservice every minute. Basically the point of this status webservice is to send requests back to the machines as a response to the status webservice.

I also have a website running on the same central server. I want to have communication between the status webservice & the website.

i.e. someone can click on an option on the website & ask to send request 'X' to Remote Machine 'Y'. If the webservice were a running process, the website would just communicate this to the webservice & the webservice would send this next time it gets a status call from that Machine.

However, since the webservice is not a process - I am confused as to how I can achieve this - does anyone have any suggestions.

Everything is running on Apache/PHP.

答:
You can have a DB table with these fields: id, request, remote_machine, processed and save the request from the website here immediately.

Later when each time the remote machines call the health check web service, it can query that table for request matching the remote machine and processed = 0. It can then send the request as a response to the health check web service.

Hope this helps.


问:
Closures in Groovy not capturing outside variables
In the context of Jenkins pipelines, I have some Groovy code that's enumerating a list, creating closures, and then using that value in the closure as a key to lookup another value in a map. This appears to be rife with some sort of anomaly or race condition almost every time.

This is a simplification of the code:

```<code>def tasks = [:]
for (platformName in platforms) {
  // ...

  tasks[platformName] = {
    def componentUploadPath = componentUploadPaths[platformName]

    echo "Uploading for platform [${platformName}] to [${componentUploadPath}]."

    // ...
}

tasks.failFast = true
parallel(tasks)
</code>```

platforms has two values. I will usually see two iterations and two tasks registered and the keys in tasks will be correct, but the echo statement inside the closure indicates that we're just running one of the platforms twice:

```<code>14:20:02 [platform2] Uploading for platform [platform1] to [some_path/platform1].
14:20:02 [platform1] Uploading for platform [platform1] to [some_path/platform1].
</code>```

It's ridiculous.

What do I need to add or do differently?

答:
It's the same issue as you'd see in Javascript.

When you generate the closures in a for loop, they are bound to a variable, not the value of the variable.

When the loop exits, and the closures are run, they will all be using the same value...that is -- the last value in the for loop before it exited

For example, you'd expect the following to print 1 2 3 4, but it doesn't

```<code>def closures = []

for (i in 1..4) {
    closures << { -> println i }
}

closures.each { it() }
</code>```

It prints 4 4 4 4

To fix this, you need to do one of two things...  First, you could capture the value in a locally scoped variable, then close over this variable:

```<code>for (i in 1..4) {
    def n = i
    closures << { -> println n }
}
</code>```

The second thing you could do is use groovy's each or collect as each time they are called, the variable is a different instance, so it works again:

```<code>(1..4).each { i ->
    closures << { -> println i }
}
</code>```

For your case, you can loop over platforms and collect into a map at the same time by using collectEntries:

```<code>def tasks = platforms.collectEntries { platformName ->
  [
     platformName,
     { ->
        def componentUploadPath = componentUploadPaths[platformName]
        echo "Uploading for platform [${platformName}] to [${componentUploadPath}]."
     }
  ]
}
</code>```

Hope this helps!

Body:
In the context of Jenkins pipelines, I have some Groovy code that's enumerating a list, creating closures, and then using that value in the closure as a key to lookup another value in a map. This appears to be rife with some sort of anomaly or race condition almost every time.

This is a simplification of the code:

def tasks = [:]
for (platformName in platforms) {
  // ...

  tasks[platformName] = {
    def componentUploadPath = componentUploadPaths[platformName]

    echo "Uploading for platform [${platformName}] to [${componentUploadPath}]."

    // ...
}

tasks.failFast = true
parallel(tasks)

platforms has two values. I will usually see two iterations and two tasks registered and the keys in tasks will be correct, but the echo statement inside the closure indicates that we're just running one of the platforms twice:

14:20:02 [platform2] Uploading for platform [platform1] to [some_path/platform1].
14:20:02 [platform1] Uploading for platform [platform1] to [some_path/platform1].

It's ridiculous.

What do I need to add or do differently?

Body:
It's the same issue as you'd see in Javascript.

When you generate the closures in a for loop, they are bound to a variable, not the value of the variable.

When the loop exits, and the closures are run, they will all be using the same value...that is -- the last value in the for loop before it exited

For example, you'd expect the following to print 1 2 3 4, but it doesn't

def closures = []

for (i in 1..4) {
    closures << { -> println i }
}

closures.each { it() }

It prints 4 4 4 4

To fix this, you need to do one of two things...  First, you could capture the value in a locally scoped variable, then close over this variable:

for (i in 1..4) {
    def n = i
    closures << { -> println n }
}

The second thing you could do is use groovy's each or collect as each time they are called, the variable is a different instance, so it works again:

(1..4).each { i ->
    closures << { -> println i }
}

For your case, you can loop over platforms and collect into a map at the same time by using collectEntries:

def tasks = platforms.collectEntries { platformName ->
  [
     platformName,
     { ->
        def componentUploadPath = componentUploadPaths[platformName]
        echo "Uploading for platform [${platformName}] to [${componentUploadPath}]."
     }
  ]
}

Hope this helps!


问:
Pushing Fortio in cloud foundry using docker image
I am trying to push Fortio(a load testing tool for microservices) as an application in cloud foundry. I am trying it with a docker image from hub.

```<code>   cf push <app-name> --docker-image fortio:fortio random-route
</code>```

Application crashes when it tries to start the application. cf logs says : executable file not found in $PATH

It is working fine in my local docker setup but not in cloud foundry.

Any help??

答:
https://docs.cloudfoundry.org/devguide/deploy-apps/push-docker.html

A Docker image that meets the following requirements:
  The Docker image must contain an /etc/passwd file with an entry for the root user. In addition, the home directory and the shell for that root user must be present in the image file system.

Unfortunately, public Docker image fortio/fortio doesn't meet this particular requirement.

Solutions:

build customized Docker app image with shell and /etc/passwd

use source code and push it as a native golang app and not a Dockerized app

Body:
I am trying to push Fortio(a load testing tool for microservices) as an application in cloud foundry. I am trying it with a docker image from hub.

   cf push <app-name> --docker-image fortio:fortio random-route

Application crashes when it tries to start the application. cf logs says : executable file not found in $PATH

It is working fine in my local docker setup but not in cloud foundry.

Any help??


问:
Firebase Security Rules Access newData Node Name
I couldn't find any way to access the name of the data that is being written in Firebase Rules, which is very critical for me.
Does Firebase Rules supports such thing?

```<code>"tests":{
    "$test":{

      ".read": "auth.uid != null",
     ".write": "auth.uid  != null
         "$user_id":{".validate": "newData.isBoolean() && auth.uid == NewData.NAME(?)"}

    }
  }
</code>```

Sample data that is gonna be written:

```<code>{
  "YHer1txgZyReM8dEkSCTDg4l2ro1": true
}
</code>```

Sample database structure:

```<code>"tests":{
    "test1":{

       "YHer1txgZyReM8dEkSCTDg4l2ro1": true,
       "A4sfgYhred1festfsefsSRGFsgr3": true,
       "7JIsfgYhred1festfsefsSRGFsg8": true,

    }
 "test2":{

      "YHer1txgZyReM8dEkSCTDg4l2ro1": true,
      "A4sfgYhred1festfsefsSRGFsgr3": true,
      "7JIsfgYhred1festfsefsSRGFsg8": true,

    }
  }
</code>```

答:
Instead of newData.NAME(?) use $user_id.

So:

```<code> "tests":{
    "$test":{

      ".read": "auth.uid != null",
     ".write": "auth.uid  != null
         "$user_id":{".validate": "newData.isBoolean() && auth.uid == $user_id"}"

    }
  }
</code>```

Also see the Firebase documentation on securing user data, which contains an example that is very close to what you need.

Body:
I couldn't find any way to access the name of the data that is being written in Firebase Rules, which is very critical for me.
Does Firebase Rules supports such thing?

"tests":{
    "$test":{

      ".read": "auth.uid != null",
     ".write": "auth.uid  != null
         "$user_id":{".validate": "newData.isBoolean() && auth.uid == NewData.NAME(?)"}

    }
  }

Sample data that is gonna be written:

{
  "YHer1txgZyReM8dEkSCTDg4l2ro1": true
}

Sample database structure:

"tests":{
    "test1":{

       "YHer1txgZyReM8dEkSCTDg4l2ro1": true,
       "A4sfgYhred1festfsefsSRGFsgr3": true,
       "7JIsfgYhred1festfsefsSRGFsg8": true,

    }
 "test2":{

      "YHer1txgZyReM8dEkSCTDg4l2ro1": true,
      "A4sfgYhred1festfsefsSRGFsgr3": true,
      "7JIsfgYhred1festfsefsSRGFsg8": true,

    }
  }

Body:
Instead of newData.NAME(?) use $user_id.

So:

 "tests":{
    "$test":{

      ".read": "auth.uid != null",
     ".write": "auth.uid  != null
         "$user_id":{".validate": "newData.isBoolean() && auth.uid == $user_id"}"

    }
  }

Also see the Firebase documentation on securing user data, which contains an example that is very close to what you need.


问:
xml.Marshal ignore struct if it's empty
I need to output an XML file, and I've constructed a few structs that represent it As a basic example, say this:

```<code>type Parent struct {
    XMLName xml.Name `xml:"parent"`
    Name    string   `xml:"name,omitempty"`
    Age     int64    `xml:"age,omitempty"`
    Child   Child    `xml:"child,omitempty`
}

type Child struct {
    XMLName  xml.Name `xml:"child,omitempty"`
    Name     string   `xml:"name,omitempty"`
    Gender   string   `xml:"gender,omitempty"`
    Thoughts string   `xml:",innerxml,omitempty"`
}
</code>```

I expect that when I create a Parent without defining the child, and then marshal it into an XML file...

```<code>parent := Parent{
    Name: "Beatrice",
    Age: "23",
}
_ = xml.MarshalIndent(parent, "", "    ")
</code>```

...that I should get an XML file not containing a child tag:

```<code><parent>
    <name>Beatrice</name>
    <age>23</age>
</parent>
</code>```

Instead, I get this:

```<code><parent>
    <name>Beatrice</name>
    <age>23</age>
    <child></child>
</parent>
</code>```

Why is the empty <child></child> tag there, and how can I get rid of it?

答:
You have a few syntax errors but you can set child as a pointer:

```<code>type Parent struct {
    XMLName xml.Name `xml:"parent"`
    Name    string   `xml:"name,omitempty"`
    Age     int64    `xml:"age,omitempty"`
    Child   *Child    `xml:"child,omitempty"`
}
</code>```

When it's nil, it will be empty.

Working Demo

Title: xml.Marshal ignore struct if it's empty

Body:
I need to output an XML file, and I've constructed a few structs that represent it As a basic example, say this:

type Parent struct {
    XMLName xml.Name `xml:"parent"`
    Name    string   `xml:"name,omitempty"`
    Age     int64    `xml:"age,omitempty"`
    Child   Child    `xml:"child,omitempty`
}

type Child struct {
    XMLName  xml.Name `xml:"child,omitempty"`
    Name     string   `xml:"name,omitempty"`
    Gender   string   `xml:"gender,omitempty"`
    Thoughts string   `xml:",innerxml,omitempty"`
}

I expect that when I create a Parent without defining the child, and then marshal it into an XML file...

parent := Parent{
    Name: "Beatrice",
    Age: "23",
}
_ = xml.MarshalIndent(parent, "", "    ")

...that I should get an XML file not containing a child tag:

<parent>
    <name>Beatrice</name>
    <age>23</age>
</parent>

Instead, I get this:

<parent>
    <name>Beatrice</name>
    <age>23</age>
    <child></child>
</parent>

Why is the empty <child></child> tag there, and how can I get rid of it?

Body:
You have a few syntax errors but you can set child as a pointer:

type Parent struct {
    XMLName xml.Name `xml:"parent"`
    Name    string   `xml:"name,omitempty"`
    Age     int64    `xml:"age,omitempty"`
    Child   *Child    `xml:"child,omitempty"`
}

When it's nil, it will be empty.

Working Demo


问:
Android location manager error 'Exiting with error onLocationChanged line 152 "1"'
I have a location manager in my Android app that sometimes works. Sometimes I'll run the app and it will get location updates (with some errors). Sometimes I'll run it and it will just throw this error every couple seconds without receiving any location updates:

```<code>E/IzatSvc_PassiveLocListener: Exiting with error onLocationChanged line 152 "1"
</code>```

Here is my class for managing location events:

```<code>package com.company.AppName;

import android.app.job.JobParameters;
import android.app.job.JobService;
import android.content.Context;
import android.content.Intent;
import android.location.Criteria;
import android.location.Location;
import android.location.LocationManager;
import android.os.Bundle;
import android.util.Log;

public class LocationListenerService extends JobService {
  private static final String TAG = "LocationListenerService";
  private LocationManager locationManager = null;
  private LocationListener locationListener = null;
  private String locationProvider = null;

  public LocationListenerService() {}

  @Override
  public int onStartCommand(Intent intent, int flags, int startId) {
    return START_STICKY;
  }

  @Override
  public boolean onStartJob(JobParameters params) {
    Log.i(TAG, "onStartJob");
    startLocationManager(params);
    return true;
  }

  @Override
  public boolean onStopJob(JobParameters params) {
    Log.i(TAG, "onStopJob");
    return false;
  }

  public void startLocationManager(JobParameters params) {
    if(locationManager != null) return;

    Criteria criteria = new Criteria();
    criteria.setAccuracy(Criteria.ACCURACY_FINE);
  //    criteria.setPowerRequirement(Criteria.POWER_LOW);
    criteria.setAltitudeRequired(false);
    criteria.setBearingRequired(false);

    locationManager = (LocationManager) getApplicationContext().getSystemService(Context.LOCATION_SERVICE);
    locationProvider = locationManager.getBestProvider(criteria, true);
    locationListener = new LocationListener();

    if (locationProvider != null) {
      Log.v(TAG, "Location provider: " + locationProvider);
    } else {
      Log.e(TAG, "Location provider is null. Location events will not work.");
      return;
    }

    if (locationListener == null) {
      Log.e(TAG, "Location listener is null. Location events will not work.");
      return;
    }

    // Finish job after first time updating location with the server
    NativeApp.shared().getLocationData((NativeApp app, String response) -> {
      Log.i(TAG, "Received location data response. Finishing job.");
      jobFinished(params, true);
    });

    try {
      locationManager.requestLocationUpdates(locationProvider, 0, 0, locationListener);
    } catch (java.lang.SecurityException ex) {
      Log.e(TAG, "fail to request location update, ignore", ex);
    } catch (IllegalArgumentException ex) {
      Log.e(TAG, "network provider does not exist, " + ex.getMessage());
    }
  }

  private class LocationListener implements android.location.LocationListener {
    @Override
    public void onLocationChanged(Location location) {
      if(location == null) {
        Log.w(TAG, "onLocationChanged skipped: null location");
        return;
      }
      Log.i(TAG, "onLocationChanged: " + location.toString());
      NativeApp.shared().updateLocation(location);
    }

    @Override
    public void onProviderDisabled(String provider) {
      Log.i(TAG, "onProviderDisabled: " + provider);
    }

    @Override
    public void onProviderEnabled(String provider) {
      Log.i(TAG, "onProviderEnabled: " + provider);
    }

    @Override
    public void onStatusChanged(String provider, int status, Bundle extras) {
      Log.i(TAG, "onStatusChanged: " + provider);
    }
  }
}
</code>```

Why is this happening? Why do location updates work sometimes but not other times?

EDIT: After giving up for a few hours and rerunning, the app is still throwing the error repeatedly, but after about 10 seconds logs this and starts receiving location updates:

```<code>E/XTCC-6.1.2.10: [FDAL_OSListener] handleLocationUpdate:  failed: 2
D/LocationManagerService: incoming location: gps
I/LgeGnssLocationProvider: Intent - android.location.GPS_FIX_CHANGE
D/LgeGnssLocationProvider: GPS_FIX_CHANGE_ACTION! , mGpsNavigating =true
D/LocationManagerService: incoming location: gps
</code>```

Device/version info:

The device is an LG G6 (LG-H872) running Android 8.0.0

And here is the contents of my AndroidManifest.xml file:

```<code><?xml version='1.0' encoding='utf-8'?>
<manifest android:hardwareAccelerated="true" android:versionCode="10000" android:versionName="1.0.0" package="com.company.AppName" xmlns:android="http://schemas.android.com/apk/res/android">
    <supports-screens android:anyDensity="true" android:largeScreens="true" android:normalScreens="true" android:resizeable="true" android:smallScreens="true" android:xlargeScreens="true" />
    <uses-permission android:name="android.permission.ACCESS_COARSE_LOCATION" />
    <uses-permission android:name="android.permission.ACCESS_FINE_LOCATION" />
    <uses-permission android:name="android.permission.INTERNET" />
    <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
    <application android:hardwareAccelerated="true" android:icon="@mipmap/icon" android:label="@string/app_name" android:supportsRtl="true">
        <activity android:configChanges="orientation|keyboardHidden|keyboard|screenSize|locale" android:label="@string/activity_name" android:launchMode="singleTop" android:name="MainActivity" android:screenOrientation="portrait" android:theme="@android:style/Theme.DeviceDefault.NoActionBar" android:windowSoftInputMode="adjustResize">
            <intent-filter android:label="@string/launcher_name">
                <action android:name="android.intent.action.MAIN" />
                <category android:name="android.intent.category.LAUNCHER" />
            </intent-filter>
        </activity>
        <provider android:authorities="${applicationId}.provider" android:exported="false" android:grantUriPermissions="true" android:name="org.apache.cordova.camera.FileProvider">
            <meta-data android:name="android.support.FILE_PROVIDER_PATHS" android:resource="@xml/camera_provider_paths" />
        </provider>
        <service android:enabled="true" android:name="com.company.AppName.LocationListenerService" android:permission="android.permission.BIND_JOB_SERVICE" />
        <service android:exported="false" android:name="com.company.AppName.AppFirebaseMessagingService">
            <intent-filter>
                <action android:name="com.google.firebase.MESSAGING_EVENT" />
            </intent-filter>
        </service>
    </application>
    <uses-sdk android:minSdkVersion="21" android:targetSdkVersion="26" />
</manifest>
</code>```

答:
I had the same problem because my device (General Mobile GM8d) 8.1 OS updated to Android 9 (Pie). After, I tried to take location in below way with LocationManager:

```<code>LocationManager locationManager = (LocationManager) context.getSystemService(LOCATION_SERVICE);
locationManager.requestLocationUpdates(
      LocationManager.GPS_PROVIDER,
      0,
      0,
      new LocationListener() {
          @Override
          public void onLocationChanged(Location location) {
              double latitude = location.getLatitude();
              double longitude = location.getLongitude();
          }
      });
</code>```

However, after Android 9 update, I had the same error. For solution, follow these steps:

Restarted device.

After, open map application (Maps). In this way, GPS will be active by native system.

After that, clear cache and storage of your app and uninstall it from device.

On top bar of Android Studio, File > Invalidate Caches/Restart...

Finally, run your application on your device

Location will taken in 13-24 seconds, depends on your area.

Done!

Body:
I have a location manager in my Android app that sometimes works. Sometimes I'll run the app and it will get location updates (with some errors). Sometimes I'll run it and it will just throw this error every couple seconds without receiving any location updates:

E/IzatSvc_PassiveLocListener: Exiting with error onLocationChanged line 152 "1"

Here is my class for managing location events:

package com.company.AppName;

import android.app.job.JobParameters;
import android.app.job.JobService;
import android.content.Context;
import android.content.Intent;
import android.location.Criteria;
import android.location.Location;
import android.location.LocationManager;
import android.os.Bundle;
import android.util.Log;

public class LocationListenerService extends JobService {
  private static final String TAG = "LocationListenerService";
  private LocationManager locationManager = null;
  private LocationListener locationListener = null;
  private String locationProvider = null;

  public LocationListenerService() {}

  @Override
  public int onStartCommand(Intent intent, int flags, int startId) {
    return START_STICKY;
  }

  @Override
  public boolean onStartJob(JobParameters params) {
    Log.i(TAG, "onStartJob");
    startLocationManager(params);
    return true;
  }

  @Override
  public boolean onStopJob(JobParameters params) {
    Log.i(TAG, "onStopJob");
    return false;
  }

  public void startLocationManager(JobParameters params) {
    if(locationManager != null) return;

    Criteria criteria = new Criteria();
    criteria.setAccuracy(Criteria.ACCURACY_FINE);
  //    criteria.setPowerRequirement(Criteria.POWER_LOW);
    criteria.setAltitudeRequired(false);
    criteria.setBearingRequired(false);

    locationManager = (LocationManager) getApplicationContext().getSystemService(Context.LOCATION_SERVICE);
    locationProvider = locationManager.getBestProvider(criteria, true);
    locationListener = new LocationListener();

    if (locationProvider != null) {
      Log.v(TAG, "Location provider: " + locationProvider);
    } else {
      Log.e(TAG, "Location provider is null. Location events will not work.");
      return;
    }

    if (locationListener == null) {
      Log.e(TAG, "Location listener is null. Location events will not work.");
      return;
    }

    // Finish job after first time updating location with the server
    NativeApp.shared().getLocationData((NativeApp app, String response) -> {
      Log.i(TAG, "Received location data response. Finishing job.");
      jobFinished(params, true);
    });

    try {
      locationManager.requestLocationUpdates(locationProvider, 0, 0, locationListener);
    } catch (java.lang.SecurityException ex) {
      Log.e(TAG, "fail to request location update, ignore", ex);
    } catch (IllegalArgumentException ex) {
      Log.e(TAG, "network provider does not exist, " + ex.getMessage());
    }
  }

  private class LocationListener implements android.location.LocationListener {
    @Override
    public void onLocationChanged(Location location) {
      if(location == null) {
        Log.w(TAG, "onLocationChanged skipped: null location");
        return;
      }
      Log.i(TAG, "onLocationChanged: " + location.toString());
      NativeApp.shared().updateLocation(location);
    }

    @Override
    public void onProviderDisabled(String provider) {
      Log.i(TAG, "onProviderDisabled: " + provider);
    }

    @Override
    public void onProviderEnabled(String provider) {
      Log.i(TAG, "onProviderEnabled: " + provider);
    }

    @Override
    public void onStatusChanged(String provider, int status, Bundle extras) {
      Log.i(TAG, "onStatusChanged: " + provider);
    }
  }
}

Why is this happening? Why do location updates work sometimes but not other times?

EDIT: After giving up for a few hours and rerunning, the app is still throwing the error repeatedly, but after about 10 seconds logs this and starts receiving location updates:

E/XTCC-6.1.2.10: [FDAL_OSListener] handleLocationUpdate:  failed: 2
D/LocationManagerService: incoming location: gps
I/LgeGnssLocationProvider: Intent - android.location.GPS_FIX_CHANGE
D/LgeGnssLocationProvider: GPS_FIX_CHANGE_ACTION! , mGpsNavigating =true
D/LocationManagerService: incoming location: gps

Device/version info:

The device is an LG G6 (LG-H872) running Android 8.0.0

And here is the contents of my AndroidManifest.xml file:

<?xml version='1.0' encoding='utf-8'?>
<manifest android:hardwareAccelerated="true" android:versionCode="10000" android:versionName="1.0.0" package="com.company.AppName" xmlns:android="http://schemas.android.com/apk/res/android">
    <supports-screens android:anyDensity="true" android:largeScreens="true" android:normalScreens="true" android:resizeable="true" android:smallScreens="true" android:xlargeScreens="true" />
    <uses-permission android:name="android.permission.ACCESS_COARSE_LOCATION" />
    <uses-permission android:name="android.permission.ACCESS_FINE_LOCATION" />
    <uses-permission android:name="android.permission.INTERNET" />
    <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
    <application android:hardwareAccelerated="true" android:icon="@mipmap/icon" android:label="@string/app_name" android:supportsRtl="true">
        <activity android:configChanges="orientation|keyboardHidden|keyboard|screenSize|locale" android:label="@string/activity_name" android:launchMode="singleTop" android:name="MainActivity" android:screenOrientation="portrait" android:theme="@android:style/Theme.DeviceDefault.NoActionBar" android:windowSoftInputMode="adjustResize">
            <intent-filter android:label="@string/launcher_name">
                <action android:name="android.intent.action.MAIN" />
                <category android:name="android.intent.category.LAUNCHER" />
            </intent-filter>
        </activity>
        <provider android:authorities="${applicationId}.provider" android:exported="false" android:grantUriPermissions="true" android:name="org.apache.cordova.camera.FileProvider">
            <meta-data android:name="android.support.FILE_PROVIDER_PATHS" android:resource="@xml/camera_provider_paths" />
        </provider>
        <service android:enabled="true" android:name="com.company.AppName.LocationListenerService" android:permission="android.permission.BIND_JOB_SERVICE" />
        <service android:exported="false" android:name="com.company.AppName.AppFirebaseMessagingService">
            <intent-filter>
                <action android:name="com.google.firebase.MESSAGING_EVENT" />
            </intent-filter>
        </service>
    </application>
    <uses-sdk android:minSdkVersion="21" android:targetSdkVersion="26" />
</manifest>

Body:
I had the same problem because my device (General Mobile GM8d) 8.1 OS updated to Android 9 (Pie). After, I tried to take location in below way with LocationManager:

LocationManager locationManager = (LocationManager) context.getSystemService(LOCATION_SERVICE);
locationManager.requestLocationUpdates(
      LocationManager.GPS_PROVIDER,
      0,
      0,
      new LocationListener() {
          @Override
          public void onLocationChanged(Location location) {
              double latitude = location.getLatitude();
              double longitude = location.getLongitude();
          }
      });

However, after Android 9 update, I had the same error. For solution, follow these steps:

Restarted device.

After, open map application (Maps). In this way, GPS will be active by native system.

After that, clear cache and storage of your app and uninstall it from device.

On top bar of Android Studio, File > Invalidate Caches/Restart...

Finally, run your application on your device

Location will taken in 13-24 seconds, depends on your area.

Done!