问:
How to select a window of values from the list?
I have a stream of data which is of the type

```<code>list  = [22 , 15 , 6 ,12 ,30 , 45, 200 , 238 , 220 , 6000, 6250 , 6900, 6700,6500, 0 , 250 , 6000 ,6800,220, 250,200]
</code>```

As you can see the stream of data has some very high values in the middle which is what we desire to extract. This block can also contain low values (0 and 250 in the above list) but definitely higher values at the start and end of block ( 6000,6500). How can we extract this particular window of data.

The output should be

```<code>new_list = [6000,6250,6900,600,6500,0, 250 , 6000 , 6800]
</code>```

I usually work on MATLAB. So the way i would do this is by finding the first and the last peak in the data

答:
Below is a plot of your data:

The traditional way to do what you're talking about would be to interpolate the data. This means adding a curve of best fit, such as a cubic spline.

After finding a differentiable curve passing through the points, you find the first derivative (rate of change). Wherever the first derivative has a local maximum is where your data suddenly went uphill or downhill.

However, all of that maybe overkill for your purposes. I think the following code will do what you want:

```<code>class OnOffRecorder:
    """
    """
    def __init__(self, is_active = False):
        """
        By default, is not recording data
        """
        self.is_active = is_active
        self.data = dict()

    def turn_recording_on(self):
        self.is_active = True

    def turn_recording_off(self):
        self.is_active = False

    def toggle_recording(self):
        self.is_active = not(self.is_active)

    def push(self, key, value):
        if self.is_active:
            self.data[key] = value
        return

    def pop(self):
        old_data = self.data
        self.data = dict()
        return old_data

def get_peak_data(data, delta):
    """
    `delta` represents percentage distance between
            minimum and maximum of the data.

            if data suddenly increases by delta, then begin recording
            if data suddenly decreases by delta, then stop recording
    """
    mind = min(data)
    maxd = max(data)
    raynge = maxd - mind
    # `li` == `left index`
    # `ri` == `right index`
    record = OnOffRecorder()
    record.turn_recording_off()
    for li in range(0, -1 + len(data)):
        ri = li + 1
        ld = data[li] # left data
        rd = data[ri] # right data
        if abs(rd-ld)/raynge > delta:
            if rd > ld :
                record.turn_recording_on()
            elif rd < ld:
                record.turn_recording_off()
        record.push(ri, rd)
    return record.pop()

data = [22 , 15 , 6 ,12 ,30 , 45, 200 , 238 ,
        220 , 6000, 6250 , 6900, 6700, 6500,
        0 , 250 , 6000 ,6800,220, 250, 200]

# if data suddenly increases, then begin recording that data
# if data suddenly decreases, then stop recording that data.
delta = .25
peak_data = get_peak_data(data, delta)
print(peak_data.values())
</code>```

EDIT:

I developed a second/different approach which does not require you to manually specify a delta value. This is described below:

Sort the data.

traverse the sorted data from minimum to maximum

record differences/jumps.

find maximum of the jumps.

That maximimum jump is where low signals transition to high signals

Below is code which implements the step-by-step process outlined above:

```<code>def get_dividing_line(data):
    sdata = sorted(data)
    jumps = [sdata[i+1] - sdata[i] for i in range(0, -1 + len(sdata))]
    jumps_max = max(jumps)
    jumps_max_left = jumps.index(jumps_max)
    jumps_max_right = [x for x in reversed(jumps)].index(jumps_max)
    jumps_max_right = len(sdata) - jumps_max_right
    jump_start = sdata[jumps_max_left]
    jump_end = sdata[jumps_max_right]
    return (jump_start + jump_end)/2

def get_high_signals(data):
    threshold = get_dividing_line(data)
    return [x for x in data if x >= threshold]

data = [22 , 15 , 6 ,12 ,30 , 45, 200 , 238 ,
        220 , 6000, 6250 , 6900, 6700, 6500,
        0 , 250 , 6000 ,6800,220, 250, 200]

high_signals = get_high_signals(data)
print(high_signals)
# prints [6000, 6250, 6900, 6700, 6500, 6000, 6800]
</code>```

Body:
I have a stream of data which is of the type

list  = [22 , 15 , 6 ,12 ,30 , 45, 200 , 238 , 220 , 6000, 6250 , 6900, 6700,6500, 0 , 250 , 6000 ,6800,220, 250,200]

As you can see the stream of data has some very high values in the middle which is what we desire to extract. This block can also contain low values (0 and 250 in the above list) but definitely higher values at the start and end of block ( 6000,6500). How can we extract this particular window of data.

The output should be

new_list = [6000,6250,6900,600,6500,0, 250 , 6000 , 6800]

I usually work on MATLAB. So the way i would do this is by finding the first and the last peak in the data

Body:
Below is a plot of your data:

The traditional way to do what you're talking about would be to interpolate the data. This means adding a curve of best fit, such as a cubic spline.

After finding a differentiable curve passing through the points, you find the first derivative (rate of change). Wherever the first derivative has a local maximum is where your data suddenly went uphill or downhill.

However, all of that maybe overkill for your purposes. I think the following code will do what you want:

class OnOffRecorder:
    """
    """
    def __init__(self, is_active = False):
        """
        By default, is not recording data
        """
        self.is_active = is_active
        self.data = dict()

    def turn_recording_on(self):
        self.is_active = True

    def turn_recording_off(self):
        self.is_active = False

    def toggle_recording(self):
        self.is_active = not(self.is_active)

    def push(self, key, value):
        if self.is_active:
            self.data[key] = value
        return

    def pop(self):
        old_data = self.data
        self.data = dict()
        return old_data

def get_peak_data(data, delta):
    """
    `delta` represents percentage distance between
            minimum and maximum of the data.

            if data suddenly increases by delta, then begin recording
            if data suddenly decreases by delta, then stop recording
    """
    mind = min(data)
    maxd = max(data)
    raynge = maxd - mind
    # `li` == `left index`
    # `ri` == `right index`
    record = OnOffRecorder()
    record.turn_recording_off()
    for li in range(0, -1 + len(data)):
        ri = li + 1
        ld = data[li] # left data
        rd = data[ri] # right data
        if abs(rd-ld)/raynge > delta:
            if rd > ld :
                record.turn_recording_on()
            elif rd < ld:
                record.turn_recording_off()
        record.push(ri, rd)
    return record.pop()

data = [22 , 15 , 6 ,12 ,30 , 45, 200 , 238 ,
        220 , 6000, 6250 , 6900, 6700, 6500,
        0 , 250 , 6000 ,6800,220, 250, 200]

# if data suddenly increases, then begin recording that data
# if data suddenly decreases, then stop recording that data.
delta = .25
peak_data = get_peak_data(data, delta)
print(peak_data.values())

EDIT:

I developed a second/different approach which does not require you to manually specify a delta value. This is described below:

Sort the data.

traverse the sorted data from minimum to maximum

record differences/jumps.

find maximum of the jumps.

That maximimum jump is where low signals transition to high signals

Below is code which implements the step-by-step process outlined above:

def get_dividing_line(data):
    sdata = sorted(data)
    jumps = [sdata[i+1] - sdata[i] for i in range(0, -1 + len(sdata))]
    jumps_max = max(jumps)
    jumps_max_left = jumps.index(jumps_max)
    jumps_max_right = [x for x in reversed(jumps)].index(jumps_max)
    jumps_max_right = len(sdata) - jumps_max_right
    jump_start = sdata[jumps_max_left]
    jump_end = sdata[jumps_max_right]
    return (jump_start + jump_end)/2

def get_high_signals(data):
    threshold = get_dividing_line(data)
    return [x for x in data if x >= threshold]

data = [22 , 15 , 6 ,12 ,30 , 45, 200 , 238 ,
        220 , 6000, 6250 , 6900, 6700, 6500,
        0 , 250 , 6000 ,6800,220, 250, 200]

high_signals = get_high_signals(data)
print(high_signals)
# prints [6000, 6250, 6900, 6700, 6500, 6000, 6800]


来源: stackexchange

问:
Difference between two lists
I Have two generic list filled with CustomsObjects.

I need to retrieve the difference between those two lists(Items who are in the first without the items in the second one)  in a third one.

I was thinking using .Except() was a good idea but I don't see how to use this..
Help!

答: Since the Except extension method operates on two IEumerables, it seems to me that it will be a O(n^2) operation.   If performance is an issue (if say your lists are large), I'd suggest creating a HashSet from list1 and use HashSet's ExceptWith method.</p>

ContentLicense: CC BY-SA 3.0

Title: Difference between two lists


问:
I need something in c# which work like the setw() in c++
I am using a richTextBox in c#. I need to display strings of different length inside one richTextBox but these strings should be perfectly aligned.. this is an example..

```<code>abcd   abcde  abcde
ab     abc    abcdef
</code>```

I know how to do this in c++ using the setw function.. but I could not find an equivalent in c#.

答:
I created a function for that purpose:

```<code>public string tab(string s, int w)
{
    //w is the desired width
    int stringwidth = s.Length;
    int i;
    string resultstring = s;

    for(i=0;i<=(w-stringwidth)/8;i++)
    {
        resultstring = resultstring + "\t";
    }
    return resultstring;
 }
</code>```

Then, adding it to a ListBox, for example:

```<code>listBox.Items.Add(tab("MyFullNameHere",30)+ tab("MyContact - xxxxx",12));
listBox.Items.Add(tab("MyWifeFullNameHereVeryLong", 30) + tab("HerContact - xxxxx", 12));
</code>```

Body:
I am using a richTextBox in c#. I need to display strings of different length inside one richTextBox but these strings should be perfectly aligned.. this is an example..

abcd   abcde  abcde
ab     abc    abcdef

I know how to do this in c++ using the setw function.. but I could not find an equivalent in c#.

Body:
I created a function for that purpose:

public string tab(string s, int w)
{
    //w is the desired width
    int stringwidth = s.Length;
    int i;
    string resultstring = s;

    for(i=0;i<=(w-stringwidth)/8;i++)
    {
        resultstring = resultstring + "\t";
    }
    return resultstring;
 }

Then, adding it to a ListBox, for example:

listBox.Items.Add(tab("MyFullNameHere",30)+ tab("MyContact - xxxxx",12));
listBox.Items.Add(tab("MyWifeFullNameHereVeryLong", 30) + tab("HerContact - xxxxx", 12));

ContentLicense: CC BY-SA 4.0


问:
In Power Automate, how do I filter a list and save the result?
I am trying to filter a list in Power Automate. I followed the instructions, and it's correct, but it doesn't save the result to my variable and there is no "output" variable for me to save it manually. How do I save the result?

答: Oh nevermind; I found the results, it was in the "Body" variable.</p>


问:
Using RDF, request for suggestions on how to model one to ORed (all) members of a set and one to ANDed (all) members of a set
I am looking for a way to model using RDF a one to ORed (all) members of a set relationship as well as a one to ANDed (all) members of a set.

The task at hand how to represent (using RDF statement triples) a treatment regimen such as the one below.

Regimen "Regimen_A" consists of drug (DB14502 OR DB09449) AND DB11094

Using RDF/XML, I am now stuck at presenting the drug combination (regimen) consisting of one-of (alternative) drug from list A plus all the (non-optional) drugs from list B.

a) It is possible for a drug regimen to have zero, one or more than one alternative list as well as zero, one or more one non-optional drug list.

b) I would also like to present each drug as a resource having URIref.
c) The drugs in these lists (the ORed list and the ANDed list) can be from different databases such as drugbank, ChEBI or WHOCC.
For example the URIref for drugbank drug DB14502 would be "https://www.drugbank.ca/drugs/DB14502". The URI for ChEBI drug would be "http://purl.obolibrary.org/obo/CHEBI_17012"
In addition to the URIref, there is a requirement to somehow explicitly provide an indication of the drug's source database.

I have been reading the RDF primer https://www.w3.org/TR/rdf-primer/ with the aim of encoding into RDF/XML the solution below as well as extend it to address the above requirements but I have not made much progress in this area.

Below is the RDF data (provided by Jeen Broekstra as an answer to my original question) I would like to transform to RDF/XML

```<code>:regimenA :consistsOf [ a :DrugAlternative;
                        :option :DB14502;
                        :option :DB09449 ],
                      :DB11094.
</code>```

答:
You can express it by introducing your own vocabulary concept for an OR.

```<code>:regimenA :consistsOf [ a :Or;
                        :option :DB14502;
                        :option :DB09449 ],
                      :DB11094.
</code>```

For AND you don't need any special vocabulary. Perhaps better in your case, rather than a generic "logical OR", is to capture your domain semantics in the vocabulary that you introduce:

```<code>:regimenA :consistsOf [ a :DrugAlternative;
                        :option :DB14502;
                        :option :DB09449 ],
                      :DB11094.
</code>```

You can also use standard RDF vocabulary for this kind of thing, using an rdf:Alt container. It is semantically not quite the same as a logical OR, but for most intents and purposes it comes close:

```<code>:regimenA :consistsOf [ a rdf:Alt;
                        rdfs:member :DB14502;
                        rdfs:member :DB09449 ],
                      :DB11094.
</code>```

RDF also has standard vocabulary for expressing a list, the RDF Collection vocabulary. You could use this as follows:

```<code>:regimenA :consistsOf (:DB14502 :DB09449), :DB11094;
</code>```

Potential downside here is that this way of modelling leaves the fact that the items in the list are alternatives implicit.

Of course, if you want entailment to apply to any of this, you will have to go beyond RDF, and use either a rule language like SPIN/SHACL, or a more expressive ontology language, like OWL. However, if your goal is to have a way to express this such that you can query it later through e.g. SPARQL, then the above solutions are fine.

Body:
I am looking for a way to model using RDF a one to ORed (all) members of a set relationship as well as a one to ANDed (all) members of a set.

The task at hand how to represent (using RDF statement triples) a treatment regimen such as the one below.

Regimen "Regimen_A" consists of drug (DB14502 OR DB09449) AND DB11094

Using RDF/XML, I am now stuck at presenting the drug combination (regimen) consisting of one-of (alternative) drug from list A plus all the (non-optional) drugs from list B.

a) It is possible for a drug regimen to have zero, one or more than one alternative list as well as zero, one or more one non-optional drug list.

b) I would also like to present each drug as a resource having URIref.
c) The drugs in these lists (the ORed list and the ANDed list) can be from different databases such as drugbank, ChEBI or WHOCC.
For example the URIref for drugbank drug DB14502 would be "https://www.drugbank.ca/drugs/DB14502". The URI for ChEBI drug would be "http://purl.obolibrary.org/obo/CHEBI_17012"
In addition to the URIref, there is a requirement to somehow explicitly provide an indication of the drug's source database.

I have been reading the RDF primer https://www.w3.org/TR/rdf-primer/ with the aim of encoding into RDF/XML the solution below as well as extend it to address the above requirements but I have not made much progress in this area.

Below is the RDF data (provided by Jeen Broekstra as an answer to my original question) I would like to transform to RDF/XML

:regimenA :consistsOf [ a :DrugAlternative;
                        :option :DB14502;
                        :option :DB09449 ],
                      :DB11094.

Body:
You can express it by introducing your own vocabulary concept for an OR.

:regimenA :consistsOf [ a :Or;
                        :option :DB14502;
                        :option :DB09449 ],
                      :DB11094.

For AND you don't need any special vocabulary. Perhaps better in your case, rather than a generic "logical OR", is to capture your domain semantics in the vocabulary that you introduce:

:regimenA :consistsOf [ a :DrugAlternative;
                        :option :DB14502;
                        :option :DB09449 ],
                      :DB11094.

You can also use standard RDF vocabulary for this kind of thing, using an rdf:Alt container. It is semantically not quite the same as a logical OR, but for most intents and purposes it comes close:

:regimenA :consistsOf [ a rdf:Alt;
                        rdfs:member :DB14502;
                        rdfs:member :DB09449 ],
                      :DB11094.

RDF also has standard vocabulary for expressing a list, the RDF Collection vocabulary. You could use this as follows:

:regimenA :consistsOf (:DB14502 :DB09449), :DB11094;

Potential downside here is that this way of modelling leaves the fact that the items in the list are alternatives implicit.

Of course, if you want entailment to apply to any of this, you will have to go beyond RDF, and use either a rule language like SPIN/SHACL, or a more expressive ontology language, like OWL. However, if your goal is to have a way to express this such that you can query it later through e.g. SPARQL, then the above solutions are fine.


问:
Calculating average number of customers that were contacted before successful conversion
I three tables:

Table Name: Customer

```<code>Column Name:        Data Type:              Description:
id                   integer        Unique id for customer
company_id         integer(FK to company.id) Unique id for a company that the customer works for
contact_name           varchar           Full name of customer
contact_email          varchar           Email of customer
create_dt              date          Date when customer added to database
</code>```

Table Name: Company

```<code>Column Name:          Data Type:              Description:
id                    integer        Unique id for company
company_name          varchar             Name of company
create_dt             date     Date when company added to database
converted             boolean        if customer was converted
converted_dt          date       Date company purchased the product Value is null if the company did not
</code>```

Table Name: Contacted (actions/ how the customer was contacted)

```<code>Column Name:          Data Type:              Description:
customer_id       integer(FK to Customer.id)    Unique id for contact
rep_name          varchar      Name of the rep who made the contact with the customer
rep_email         varchar      Email of rep who made the contact
contact_channel   varchar      Channel used to contact the customer(email,phone, in-person...)
contact_dt         date        Date of contact
</code>```

I'm trying to write a query that to show the average number of customers that were contacted before they were successfully converted as a customer, on a company level.
I'm able to get the number of counts before customers were successfully converted but not sure how to get the avg count and factor in instances where they were eventually converted.

```<code>COUNT(CASE WHEN converted = FALSE AND converted_dt is NULL THEN 1 ELSE 0 END) as contacts_made_before_conversion,

CASE WHEN Company.converted_dt is NOT null THEN count(contact_dt) over(partition by company_name order by contact_dt DESC END) as contacted_counts

FROM Contacted

LEFT JOIN
  (SELECT Customer.id as customer_id , company_name,
   Company.converted,
   FROM Customer LEFT JOIN Company
   ON Company.id = Customer.company_id) cc

ON cc.customer_id = Contacted.customer_id

GROUP BY company_name, customer_id
</code>```

答:
This is basically a left join with aggregation:

```<code>select c.company_id,
       count(distinct c.id) as num_customers,
       count(co.id) as num_contacts,
       count(co.id) * 1.0 / count(distinct c.id) as num_per_customer
from customer c left join
     contacted co
     on co.customer_id = c.id and
        co.converted_dt < c.created_dt
group by c.company_id
</code>```

Body:
I three tables:

Table Name: Customer

Column Name:        Data Type:              Description:
id                   integer        Unique id for customer
company_id         integer(FK to company.id) Unique id for a company that the customer works for
contact_name           varchar           Full name of customer
contact_email          varchar           Email of customer
create_dt              date          Date when customer added to database

Table Name: Company

Column Name:          Data Type:              Description:
id                    integer        Unique id for company
company_name          varchar             Name of company
create_dt             date     Date when company added to database
converted             boolean        if customer was converted
converted_dt          date       Date company purchased the product Value is null if the company did not

Table Name: Contacted (actions/ how the customer was contacted)

Column Name:          Data Type:              Description:
customer_id       integer(FK to Customer.id)    Unique id for contact
rep_name          varchar      Name of the rep who made the contact with the customer
rep_email         varchar      Email of rep who made the contact
contact_channel   varchar      Channel used to contact the customer(email,phone, in-person...)
contact_dt         date        Date of contact

I'm trying to write a query that to show the average number of customers that were contacted before they were successfully converted as a customer, on a company level.
I'm able to get the number of counts before customers were successfully converted but not sure how to get the avg count and factor in instances where they were eventually converted.

COUNT(CASE WHEN converted = FALSE AND converted_dt is NULL THEN 1 ELSE 0 END) as contacts_made_before_conversion,

CASE WHEN Company.converted_dt is NOT null THEN count(contact_dt) over(partition by company_name order by contact_dt DESC END) as contacted_counts

FROM Contacted

LEFT JOIN
  (SELECT Customer.id as customer_id , company_name,
   Company.converted,
   FROM Customer LEFT JOIN Company
   ON Company.id = Customer.company_id) cc

ON cc.customer_id = Contacted.customer_id

GROUP BY company_name, customer_id

Body:
This is basically a left join with aggregation:

select c.company_id,
       count(distinct c.id) as num_customers,
       count(co.id) as num_contacts,
       count(co.id) * 1.0 / count(distinct c.id) as num_per_customer
from customer c left join
     contacted co
     on co.customer_id = c.id and
        co.converted_dt < c.created_dt
group by c.company_id


问:
Web crawler in PHP not able to create records in mysql database. How to fix?
I am creating a web crawler for a search engine like Google. The web crawler works well, that's what I see when I run it through terminal, but it is not writing any records in the mysql database.

I have already tried granting all permissions to the database user the web crawler uses, but it was of no use. My server is flawless, that I can be sure of.

```<code><?php
$start = "http://localhost/mariophp/test.html";
$already_crawled=array();
$crawling=array();
function get_details($url)
{
    $options=array('http'=>array('method'=>"GET", 'headers'=>"User-Agent: ZeroBot/0.2\n"));
    $context=stream_context_create($options);
    $doc = new DOMDocument();
    @$doc->loadHTML(@file_get_contents($url,false,$context));
    $title=$doc->getElementsByTagName("title");
    $title=$title->item(0)->nodeValue;
    $simg=$doc->getElementsByTagName("img");
    //$simg=$simg->getAttribute("src");
    //$simg=$simg->item(0)->nodeValue;
    $description="";
    $keywords="";
    $metas=$doc->getElementsByTagName("meta");
    for($i=0; $i<$metas->length; $i++)
    {
        $meta=$metas->item($i);
        if($meta->getAttribute("name")==strtolower("description"))
            $description=$meta->getAttribute("content");
        if($meta->getAttribute("name")==strtolower("keywords"))
            $keywords=$meta->getAttribute("content");
    }
    $_con=mysqli_connect("localhost","augustus","password");
    mysqli_select_db($_con,"websited");

    $title=$_POST["title"];
    $url=$_POST["url"];
    $keywords=$_POST["keywords"];
    $description=$_POST["description"];
    $simg=$_POST["simg"];

    $sql="insert into websited(stitle,slink,skey,sdesc,simg) values('$title','$url',$keywords',$description','$simg')";
    if(!mysqli_query($_con,$sql))
       {
        echo "Error: mysqli_error($_con))";
       }

}
function follow_links($url)
{
    global $already_crawled;
    global $crawling;
    $options=array('http'=>array('method'=>"GET", 'headers'=>"User-Agent: MarioBot/0.1\n"));
    $context=stream_context_create($options);
    $doc = new DOMDocument();
    @$doc->loadHTML(@file_get_contents($url,false,$context));
    $linklist = $doc->getElementsByTagName("a");
    foreach ($linklist as $link)
    {
        $l = $link->getAttribute("href");
        if(substr($l,0,1)=="/" && substr($l,0,2)!="//")
        {
            $l=parse_url($url)["scheme"]."://".parse_url($url)["host"].$l;
        }
        else if (substr($l,0,2)=="//")
        {
            $l=parse_url($url)["scheme"].":".$l;
        }
        else if(substr($l,0,2)=="./")
        {
            $l=parse_url($url)["scheme"]."://".parse_url($url)["host"].dirname(parse_url($url)["path"]).substr($l,1);
        }
        else if(substr($l,0,1)=="#")
        {
            $l=parse_url($url)["scheme"]."://".parse_url($url)["host"].parse_url($url)["path"].$l;
        }
        else if(substr($l,0,3)=="../")
        {
            $l=parse_url($url)["scheme"]."://".parse_url($url)["host"]."/".$l;
        }
        else if(substr($l,0,11)=="javascript:")
        {
            continue;
        }
        else if(substr($l,0,5)!="https" && substr($l,0,4)!="http")
        {
            $l=parse_url($url)["scheme"]."://".parse_url($url)["host"]."/".$l;
        }
        if(!in_array($l,$already_crawled))
        {
            $already_crawled[]=$l;
            $crawling[]=$l;
            echo get_details($l)."\n";
            //echo $l."\n";
        }
    }
    array_shift($crawling);
    foreach ($crawling as $site) {
        follow_links($site);
    }
}
follow_links($start);
print_r($already_crawled);
?>
</code>```

Note:The test.html file mentioned in the code is a simple file containing links to different websites.
You may need to setup a server first to successfully run this code.
I am getting the following output right now.
     [augustoandro@Augustus zerophp]$ php crawle2.php
     PHP Notice:  Undefined index: title in /srv/http/zerophp/crawle2.php on line 30
     PHP Notice:  Undefined index: url in /srv/http/zerophp/crawle2.php on line 31
     PHP Notice:  Undefined index: keywords in /srv/http/zerophp/crawle2.php on line 32
     PHP Notice:  Undefined index: description in /srv/http/zerophp/crawle2.php on line 33
     PHP Notice:  Undefined index: simg in /srv/http/zerophp/crawle2.php on line 34
     PHP Recoverable fatal error:  Object of class mysqli could not be converted to string in /srv/http/zerophp/crawle2.php on line 39
     [augustoandro@Augustoandro zerophp]$

Please help.

答:
Get rid of these lines:

```<code>    $title=$_POST["title"];
    $url=$_POST["url"];
    $keywords=$_POST["keywords"];
    $description=$_POST["description"];
    $simg=$_POST["simg"];
</code>```

These are overwriting the variables that you got from scraping the website. $_POST is for getting parameters that are submitted from a form or AJAX, they're not needed here.

The call to mysqli_error() should not be inside a string. Change

```<code>    if(!mysqli_query($_con,$sql))
       {
        echo "Error: mysqli_error($_con))";
       }
</code>```

to

```<code>    if(!mysqli_query($_con,$sql))
       {
        echo "Error: " . mysqli_error($_con));
       }
</code>```

Body:
I am creating a web crawler for a search engine like Google. The web crawler works well, that's what I see when I run it through terminal, but it is not writing any records in the mysql database.

I have already tried granting all permissions to the database user the web crawler uses, but it was of no use. My server is flawless, that I can be sure of.

<?php
$start = "http://localhost/mariophp/test.html";
$already_crawled=array();
$crawling=array();
function get_details($url)
{
    $options=array('http'=>array('method'=>"GET", 'headers'=>"User-Agent: ZeroBot/0.2\n"));
    $context=stream_context_create($options);
    $doc = new DOMDocument();
    @$doc->loadHTML(@file_get_contents($url,false,$context));
    $title=$doc->getElementsByTagName("title");
    $title=$title->item(0)->nodeValue;
    $simg=$doc->getElementsByTagName("img");
    //$simg=$simg->getAttribute("src");
    //$simg=$simg->item(0)->nodeValue;
    $description="";
    $keywords="";
    $metas=$doc->getElementsByTagName("meta");
    for($i=0; $i<$metas->length; $i++)
    {
        $meta=$metas->item($i);
        if($meta->getAttribute("name")==strtolower("description"))
            $description=$meta->getAttribute("content");
        if($meta->getAttribute("name")==strtolower("keywords"))
            $keywords=$meta->getAttribute("content");
    }
    $_con=mysqli_connect("localhost","augustus","password");
    mysqli_select_db($_con,"websited");

    $title=$_POST["title"];
    $url=$_POST["url"];
    $keywords=$_POST["keywords"];
    $description=$_POST["description"];
    $simg=$_POST["simg"];

    $sql="insert into websited(stitle,slink,skey,sdesc,simg) values('$title','$url',$keywords',$description','$simg')";
    if(!mysqli_query($_con,$sql))
       {
        echo "Error: mysqli_error($_con))";
       }

}
function follow_links($url)
{
    global $already_crawled;
    global $crawling;
    $options=array('http'=>array('method'=>"GET", 'headers'=>"User-Agent: MarioBot/0.1\n"));
    $context=stream_context_create($options);
    $doc = new DOMDocument();
    @$doc->loadHTML(@file_get_contents($url,false,$context));
    $linklist = $doc->getElementsByTagName("a");
    foreach ($linklist as $link)
    {
        $l = $link->getAttribute("href");
        if(substr($l,0,1)=="/" && substr($l,0,2)!="//")
        {
            $l=parse_url($url)["scheme"]."://".parse_url($url)["host"].$l;
        }
        else if (substr($l,0,2)=="//")
        {
            $l=parse_url($url)["scheme"].":".$l;
        }
        else if(substr($l,0,2)=="./")
        {
            $l=parse_url($url)["scheme"]."://".parse_url($url)["host"].dirname(parse_url($url)["path"]).substr($l,1);
        }
        else if(substr($l,0,1)=="#")
        {
            $l=parse_url($url)["scheme"]."://".parse_url($url)["host"].parse_url($url)["path"].$l;
        }
        else if(substr($l,0,3)=="../")
        {
            $l=parse_url($url)["scheme"]."://".parse_url($url)["host"]."/".$l;
        }
        else if(substr($l,0,11)=="javascript:")
        {
            continue;
        }
        else if(substr($l,0,5)!="https" && substr($l,0,4)!="http")
        {
            $l=parse_url($url)["scheme"]."://".parse_url($url)["host"]."/".$l;
        }
        if(!in_array($l,$already_crawled))
        {
            $already_crawled[]=$l;
            $crawling[]=$l;
            echo get_details($l)."\n";
            //echo $l."\n";
        }
    }
    array_shift($crawling);
    foreach ($crawling as $site) {
        follow_links($site);
    }
}
follow_links($start);
print_r($already_crawled);
?>

Note:The test.html file mentioned in the code is a simple file containing links to different websites.
You may need to setup a server first to successfully run this code.
I am getting the following output right now.
     [augustoandro@Augustus zerophp]$ php crawle2.php
     PHP Notice:  Undefined index: title in /srv/http/zerophp/crawle2.php on line 30
     PHP Notice:  Undefined index: url in /srv/http/zerophp/crawle2.php on line 31
     PHP Notice:  Undefined index: keywords in /srv/http/zerophp/crawle2.php on line 32
     PHP Notice:  Undefined index: description in /srv/http/zerophp/crawle2.php on line 33
     PHP Notice:  Undefined index: simg in /srv/http/zerophp/crawle2.php on line 34
     PHP Recoverable fatal error:  Object of class mysqli could not be converted to string in /srv/http/zerophp/crawle2.php on line 39
     [augustoandro@Augustoandro zerophp]$

Please help.

Body:
Get rid of these lines:

    $title=$_POST["title"];
    $url=$_POST["url"];
    $keywords=$_POST["keywords"];
    $description=$_POST["description"];
    $simg=$_POST["simg"];

These are overwriting the variables that you got from scraping the website. $_POST is for getting parameters that are submitted from a form or AJAX, they're not needed here.

The call to mysqli_error() should not be inside a string. Change

    if(!mysqli_query($_con,$sql))
       {
        echo "Error: mysqli_error($_con))";
       }

to

    if(!mysqli_query($_con,$sql))
       {
        echo "Error: " . mysqli_error($_con));
       }


问:
Equivalent of template context in Pyramid (pylons user)
What is the equivalent of template context in Pyramid?

Does the IBeforeRender event in pyramid have anything to with this? I've gone through the official documentation but diffcult to understand what the IBeforeRender event is exactly.

答:
If instead you are hoping for some "global bag" where you can stuff variables that will be available to every template, then your question about IBeforeRender is appropriate.

```<code>from pyramid.events import subscriber
from pyramid.events import BeforeRender

@subscriber(BeforeRender)
def add_global(event):
    event['name'] = 'Pyramid Developer'
</code>```

There is an alternative way of adding globals when setting up the Configurator as well.  You can see the full info at: http://docs.pylonsproject.org/projects/pyramid/en/latest/narr/hooks.html#using-the-before-render-event

ContentLicense: CC BY-SA 2.5

Body:
If instead you are hoping for some "global bag" where you can stuff variables that will be available to every template, then your question about IBeforeRender is appropriate.

from pyramid.events import subscriber
from pyramid.events import BeforeRender

@subscriber(BeforeRender)
def add_global(event):
    event['name'] = 'Pyramid Developer'

There is an alternative way of adding globals when setting up the Configurator as well.  You can see the full info at: http://docs.pylonsproject.org/projects/pyramid/en/latest/narr/hooks.html#using-the-before-render-event


问:
RegEx for an autocomplete feature
I am writing a search bar with an autocomplete feature that is hooked up to an endpoint. I am using regex to determine the "context" that I am in inside of the query I type in the search bar. The three contexts are "attribute," "value," and "operator." The two operators that are allowed are "AND" and "OR." Below is an example of an example query.

Color: Blue AND Size: "Women's Large" (<-- multi-word values or attribute names are surrounded by quotation marks)

I need my regex to match after you put a space after Blue, and if the user begins type "A/AN/AND/O/OR", I need it to match. Once they have put a space after the operator, I need it to stop matching.

This is the expression I have come up with.

```<code>const contextIsOperator = /[\w\d\s"]+: *[\w\s\d"]+ [\w]*$/
</code>```

It matches once I put a space after "Blue," but matches for everything I put after that. If I replace the last * in the expression with a +, it works when I put a space after "Blue" and start manually typing one of the operators, but not if I just have a space after "Blue."

The pattern I have in my head written in words is:

group of one or more characters/digits/spaces/quotation marks

followed by a colon

followed by an optional space

followed by another group of one or more characters/digits/space/quotation marks

followed by a space (after the value)

followed by one or more characters (this is the operator)

How do I solve this problem?

答:
Is as, your language is not deterministic enough to be properly modeled with a regex.  That being said, there are 2 approaches you can take:

Require all values (the stuff after a : and before an operator) to be enclosed in quotes

Build a simple state machine that can parse the data more intelligently.  (Google Finite State Machine Parser)

If you choose to use the first method, you can use the following regex:

```<code>^(("?[\w\s]+"?): ?("[\w\s']+")( (AND|OR) )?)+$
</code>```

I would explain the different components, but regex101 already does for me with really good visuals and detail.

Title: RegEx for an autocomplete feature

Body:
I am writing a search bar with an autocomplete feature that is hooked up to an endpoint. I am using regex to determine the "context" that I am in inside of the query I type in the search bar. The three contexts are "attribute," "value," and "operator." The two operators that are allowed are "AND" and "OR." Below is an example of an example query.

Color: Blue AND Size: "Women's Large" (<-- multi-word values or attribute names are surrounded by quotation marks)

I need my regex to match after you put a space after Blue, and if the user begins type "A/AN/AND/O/OR", I need it to match. Once they have put a space after the operator, I need it to stop matching.

This is the expression I have come up with.

const contextIsOperator = /[\w\d\s"]+: *[\w\s\d"]+ [\w]*$/

It matches once I put a space after "Blue," but matches for everything I put after that. If I replace the last * in the expression with a +, it works when I put a space after "Blue" and start manually typing one of the operators, but not if I just have a space after "Blue."

The pattern I have in my head written in words is:

group of one or more characters/digits/spaces/quotation marks

followed by a colon

followed by an optional space

followed by another group of one or more characters/digits/space/quotation marks

followed by a space (after the value)

followed by one or more characters (this is the operator)

How do I solve this problem?

Body:
Is as, your language is not deterministic enough to be properly modeled with a regex.  That being said, there are 2 approaches you can take:

Require all values (the stuff after a : and before an operator) to be enclosed in quotes

Build a simple state machine that can parse the data more intelligently.  (Google Finite State Machine Parser)

If you choose to use the first method, you can use the following regex:

^(("?[\w\s]+"?): ?("[\w\s']+")( (AND|OR) )?)+$

I would explain the different components, but regex101 already does for me with really good visuals and detail.


问:
Failing to Split String
I am having a stupid problem where there are blank items in the list. There are if statements in it to attempt to offset it, but even with them, the output contains empty parts.

I have been playing around with string.split() a lot and found that what it takes is really specific, and each time i attempt to make it work, it fails. What's going wrong?

```<code>a = 'Beautiful, is==; better*than\nugly'
import re

a = re.split(',|\s|=|;|\*|\n| ',a)

for x in a:
    if x == '\n':
        a.remove(x)
    elif x == ' ':
        a.remove(x)
    elif x == '':
        a.remove(x)

print(a)
print("REE")
</code>```

I want the outcome to only be:
['Beautiful', 'is', 'better', 'than', 'ugly']

But the actual outcome is:
['Beautiful', '', 'is', '', '', '', 'better', 'than', 'ugly']

答:
You want to split on groups of any number of your delimiter characters.  The way you're doing it now, you're splitting on only one at a time.  This gives you what you want:

```<code>import re

a = 'Beautiful, is==; better*than\nugly'

a = re.split(r'[,\s=;*\n]+',a)

print(a)
</code>```

Result:

```<code>['Beautiful', 'is', 'better', 'than', 'ugly']
</code>```

If you want to split on all non-alphanumeric characters, you can use this expression instead:

```<code>a = re.split(r'[^\w]+',a)
</code>```

Title: Failing to Split String

Body:
I am having a stupid problem where there are blank items in the list. There are if statements in it to attempt to offset it, but even with them, the output contains empty parts.

I have been playing around with string.split() a lot and found that what it takes is really specific, and each time i attempt to make it work, it fails. What's going wrong?

a = 'Beautiful, is==; better*than\nugly'
import re

a = re.split(',|\s|=|;|\*|\n| ',a)

for x in a:
    if x == '\n':
        a.remove(x)
    elif x == ' ':
        a.remove(x)
    elif x == '':
        a.remove(x)

print(a)
print("REE")

I want the outcome to only be:
['Beautiful', 'is', 'better', 'than', 'ugly']

But the actual outcome is:
['Beautiful', '', 'is', '', '', '', 'better', 'than', 'ugly']

Body:
You want to split on groups of any number of your delimiter characters.  The way you're doing it now, you're splitting on only one at a time.  This gives you what you want:

import re

a = 'Beautiful, is==; better*than\nugly'

a = re.split(r'[,\s=;*\n]+',a)

print(a)

Result:

['Beautiful', 'is', 'better', 'than', 'ugly']

If you want to split on all non-alphanumeric characters, you can use this expression instead:

a = re.split(r'[^\w]+',a)


问:
Empty variable due to asynchronous callback
There is some very simplified pseudo-ish code to understand the structure of my code.

What would be a simple approach to solve my asynchronous callback dilemma.

The token will be empty when it's used in requestOption. So the following usage of requestOption will not give any results due to the missing token.

```<code>var token = ''
var auth1 = someData
var auth2 = someFunc(auth1)
var auth3 = someOtherData
var auth4 = someOtherFunc(auth3)
var finalAuth = someFinalData
var finalAuth2 = finalAuth.someFinalFunc(auth4, cb)
  cb.onSuccess(result)
    token = result.getToken()

var requestOption = {
  someData: ...,
  'authToken': token,
  someMoreData: ...
}
req = request.post(requestOption)
</code>```

There are many solutions and most of them includes using functions to some extend.

Maybe it is useful to modularize the authentication and processing the the requests in another file.

But I would like to keep it simple and not be dependant on something like this

```<code>var empty = ''
var data = 'xyz'
auth(data, {
    onSuccess: function(res) {
        doSomething(res)
    }
})

function doSomething(var) {
 // do something
}
</code>```

What I found so far couldn't help me without overcomplicating my code.

答:
Basic JavaScript will provide you with three primary options for handling asynchronous behavior.

Callback functions (what you're doing now)

function callback() {
    console.log('do this');
    setTimeout(() => console.log('later do this'), 2000);
}
callback();```
</div>
</div>
</p>

<hr>

<ol start="2">
<li><strong>Promises</strong> - Used to provide a more structured way of handling asynchronous behavior, and cleanup the <em>callback hell</em> that starts to form when there is a lot of asynchronous behavior being handled by callback functions.</li>
</ol>

<p>A promise is declared with a function with two function parameters - resolve and reject. Resolve is used to resolve the promise normally, with a payload as an argument, which then becomes available in a chained <code>.then( ... )</code> function. Reject can be used to resolve the promise exceptionally. See Mozilla docs on Promises for further explanation and examples.</p>

<p><div class="snippet" data-lang="js" data-hide="false" data-console="true" data-babel="false">
<div class="snippet-code">
<pre class="snippet-code-js lang-js prettyprint-override"><code>function asynchronousFunction() {
  return new Promise( (resolve, reject) => {
    setTimeout( () => {
      return resolve('then do this');
    }, 2000);
  });
}

function promise() {
  console.log('do this');
  asynchronousFunction()
    .then((data) => console.log(data));
};

promise();</code>```

Async/Await - Essentially syntactic sugar for promises that hides most of the promises code, which can complicate and crowd out internal logic.

function asynchronousFunction() {
  return new Promise( (resolve, reject) => {
    setTimeout( () => {
      return resolve('then do this');
    }, 2000);
  });
}

/**
* async functions implicitly return promises with the return value being the resolved value
await keyword will implicitly unwrap the promise when it resolves and (to some degre) pause execution to until resolution
*/
async function asyncFunction() {
  console.log('do this');
  console.log(await asynchronousFunction());
};

asyncFunction();```

The skinny on it:

Callbacks are okay for general purpose stuff, and under the hood this is how things really work. Understanding callbacks is important for understanding JavaScript. However, with a heavily asynchronous application, it can be very difficult to understand the order code executes in, because of the nested behavior.

Promises solves the former problem, but adds its own layer of complexity to the code. You should be careful to use promises effectively to make the code more readable. If they are not used well, sometimes they can impede code readability. This is generally better structure than endless callbacks, but still is not nearly as readable and organized as synchronous code.

Async/await is fundamentally promises, but hidden from the user. These tools can make your code much easier to read, but you need to fundamentally understand promises to really understand how it works. The big benefit here is that the code begins to look more like synchronous code and it makes it easier to focus on the internal logic rather than the asynchronous behavior.

Body:
There is some very simplified pseudo-ish code to understand the structure of my code.

What would be a simple approach to solve my asynchronous callback dilemma.

The token will be empty when it's used in requestOption. So the following usage of requestOption will not give any results due to the missing token.

var token = ''
var auth1 = someData
var auth2 = someFunc(auth1)
var auth3 = someOtherData
var auth4 = someOtherFunc(auth3)
var finalAuth = someFinalData
var finalAuth2 = finalAuth.someFinalFunc(auth4, cb)
  cb.onSuccess(result)
    token = result.getToken()

var requestOption = {
  someData: ...,
  'authToken': token,
  someMoreData: ...
}
req = request.post(requestOption)

There are many solutions and most of them includes using functions to some extend.

Maybe it is useful to modularize the authentication and processing the the requests in another file.

But I would like to keep it simple and not be dependant on something like this

var empty = ''
var data = 'xyz'
auth(data, {
    onSuccess: function(res) {
        doSomething(res)
    }
})

function doSomething(var) {
 // do something
}

What I found so far couldn't help me without overcomplicating my code.

Body:
Basic JavaScript will provide you with three primary options for handling asynchronous behavior.

Callback functions (what you're doing now)

function callback() {
    console.log('do this');
    setTimeout(() => console.log('later do this'), 2000);
}
callback();

Promises - Used to provide a more structured way of handling asynchronous behavior, and cleanup the callback hell that starts to form when there is a lot of asynchronous behavior being handled by callback functions.

A promise is declared with a function with two function parameters - resolve and reject. Resolve is used to resolve the promise normally, with a payload as an argument, which then becomes available in a chained .then( ... ) function. Reject can be used to resolve the promise exceptionally. See Mozilla docs on Promises for further explanation and examples.

function asynchronousFunction() {
  return new Promise( (resolve, reject) => {
    setTimeout( () => {
      return resolve('then do this');
    }, 2000);
  });
}

function promise() {
  console.log('do this');
  asynchronousFunction()
    .then((data) => console.log(data));
};

promise();

Async/Await - Essentially syntactic sugar for promises that hides most of the promises code, which can complicate and crowd out internal logic.

function asynchronousFunction() {
  return new Promise( (resolve, reject) => {
    setTimeout( () => {
      return resolve('then do this');
    }, 2000);
  });
}

/**
* async functions implicitly return promises with the return value being the resolved value
await keyword will implicitly unwrap the promise when it resolves and (to some degre) pause execution to until resolution
*/
async function asyncFunction() {
  console.log('do this');
  console.log(await asynchronousFunction());
};

asyncFunction();

The skinny on it:

Callbacks are okay for general purpose stuff, and under the hood this is how things really work. Understanding callbacks is important for understanding JavaScript. However, with a heavily asynchronous application, it can be very difficult to understand the order code executes in, because of the nested behavior.

Promises solves the former problem, but adds its own layer of complexity to the code. You should be careful to use promises effectively to make the code more readable. If they are not used well, sometimes they can impede code readability. This is generally better structure than endless callbacks, but still is not nearly as readable and organized as synchronous code.

Async/await is fundamentally promises, but hidden from the user. These tools can make your code much easier to read, but you need to fundamentally understand promises to really understand how it works. The big benefit here is that the code begins to look more like synchronous code and it makes it easier to focus on the internal logic rather than the asynchronous behavior.


问:
How do I create a javascript function that add html elements dynamically?
I’m trying to creat a javascript function that adds new html elements, with new ID's for each element. for example this form; To let the user submit languages he can speak and his level on each one. But he can add a new input to type other languages if he speaks many.

```<code><form id="form">
Language spoken1: <input id="language1">
Level: <input id="level1">
<form>
<input type="button" value="add new language" id="add" onclick="myFunction">
</code>```

To get a result like this

```<code><form id="form">
Language spoken1: <input id="language1">
Level: <input id="level1">
Language spoken2: <input id="language2">
Level: <input id="level2">
<form>
<input type="button" value="add new language" id="add" onclick="myFunction">
</code>```

And when he submit result like this will be on the page

Language spoken1: English, level: native
Language spoken2: Spanish, level: good

答:
You can use document.getElementById("ID").innerHTML

document.getElementById("form").innerHTML='Language spoken1:<input id="language1"> Level: <input id="level1">
 Language spoken2: <input id="language2"> Level: <input id="level2">'```
<pre class="snippet-code-html lang-html prettyprint-override"><code><form id="form">

</form>
<input type="button" value="add new language" id="add" onclick="myFunction"></code>```

Body:
I’m trying to creat a javascript function that adds new html elements, with new ID's for each element. for example this form; To let the user submit languages he can speak and his level on each one. But he can add a new input to type other languages if he speaks many.

<form id="form">
Language spoken1: <input id="language1">
Level: <input id="level1">
<form>
<input type="button" value="add new language" id="add" onclick="myFunction">

To get a result like this

<form id="form">
Language spoken1: <input id="language1">
Level: <input id="level1">
Language spoken2: <input id="language2">
Level: <input id="level2">
<form>
<input type="button" value="add new language" id="add" onclick="myFunction">

And when he submit result like this will be on the page

Language spoken1: English, level: native
Language spoken2: Spanish, level: good

Body:
You can use document.getElementById("ID").innerHTML

document.getElementById("form").innerHTML='Language spoken1:<input id="language1"> Level: <input id="level1"><br> Language spoken2: <input id="language2"> Level: <input id="level2">'

<form id="form">

</form>
<input type="button" value="add new language" id="add" onclick="myFunction">


问:
Use variable storing key from dict1 as new key for dict2
I want to pull keys and their values from a dictionary that meet conditions, then store them in another dictionary. What I thought of so far was:

```<code>dict = {'a': 1, 'b': 2}
new_dict = {}
for x in dict:
    new_dict.update(x=dict[x])
print (dict)
print (new_dict)
</code>```

Here, x stores a key 'a' with value 1, but when I update new_dict, it stores x: 2, meaning during the first loop it created a new key x and reinitialized the key during the second loop. It means that Python is interpreting x as a literal.

Is there anyway to have new_dict store 'a': 1 and 'b': 2?

答:
This worked for me:

```<code>$ python
Python 3.7.2 (default, Dec 27 2018, 07:35:06)
>>> d1 = {'a':1,'b':2}
>>> d2 = {}
>>> for k,v in d1.items():
...     if True:
...             d2[k] = v
...
>>> d2
{'a': 1, 'b': 2}
</code>```

You would just replace if True with your condition.

See here for some more info on looping over dictionaries: https://docs.python.org/3/tutorial/datastructures.html#looping-techniques

Body:
I want to pull keys and their values from a dictionary that meet conditions, then store them in another dictionary. What I thought of so far was:

dict = {'a': 1, 'b': 2}
new_dict = {}
for x in dict:
    new_dict.update(x=dict[x])
print (dict)
print (new_dict)

Here, x stores a key 'a' with value 1, but when I update new_dict, it stores x: 2, meaning during the first loop it created a new key x and reinitialized the key during the second loop. It means that Python is interpreting x as a literal.

Is there anyway to have new_dict store 'a': 1 and 'b': 2?

Body:
This worked for me:

$ python
Python 3.7.2 (default, Dec 27 2018, 07:35:06)
>>> d1 = {'a':1,'b':2}
>>> d2 = {}
>>> for k,v in d1.items():
...     if True:
...             d2[k] = v
...
>>> d2
{'a': 1, 'b': 2}

You would just replace if True with your condition.

See here for some more info on looping over dictionaries: https://docs.python.org/3/tutorial/datastructures.html#looping-techniques


问:
Git checkout only giving me pointer files on some machines
I have some Jenkins pipeline jobs that run on multiple windows machines.  The machines have, as far as I can tell, identical configurations.  But on some of the machines, the checkout of git repositories leaves me with the git-lfs pointer files instead of the binary files that they point to.

If I then call git lfs pull, it will resolve the binary files.  But I'm trying to figure out why it isn't resolving them on checkout.

I have confirmed that on two machines, one in which it's working correctly and one where it's not, have identical versions of git (2.21.0) and git-lfs (2.7.2).

What might be causing git-lfs to not resolve the binary files on checkout?

答:
In order for Git LFS to check out files automatically, you need to have the filter options set up properly. You can run git lfs env, which should print three git config commands at the bottom. When properly configured, they should look something like this:

```<code>git config filter.lfs.process = "git-lfs filter-process"
git config filter.lfs.smudge = "git-lfs smudge -- %f"
git config filter.lfs.clean = "git-lfs clean -- %f
</code>```

If the value is empty or it looks wrong for your system, you should configure it appropriately, or use git lfs install --skip-repo to install it on a per-user basis (or use one of the other options to that command to install it elsewhere).

Note that if you're not sure where it's set on the systems where it works, you can check by using git config -l --show-origin, and then copy the configuration to an identical place on the other system.

Body:
In order for Git LFS to check out files automatically, you need to have the filter options set up properly. You can run git lfs env, which should print three git config commands at the bottom. When properly configured, they should look something like this:

git config filter.lfs.process = "git-lfs filter-process"
git config filter.lfs.smudge = "git-lfs smudge -- %f"
git config filter.lfs.clean = "git-lfs clean -- %f

If the value is empty or it looks wrong for your system, you should configure it appropriately, or use git lfs install --skip-repo to install it on a per-user basis (or use one of the other options to that command to install it elsewhere).

Note that if you're not sure where it's set on the systems where it works, you can check by using git config -l --show-origin, and then copy the configuration to an identical place on the other system.


问:
Using a converted Unix Timestamp to query the database
I am currently using this:

```<code>$rawsql = "SELECT
*
FROM
    _erc_foffices n
INNER JOIN
    _erc_openings o ON n.id = o.branch_id AND o.dotw = DAYOFWEEK(CURRENT_DATE())
INNER JOIN
    _erc_openings_times t ON o.id = t.opening_id
WHERE
(
    UNIX_TIMESTAMP(CURRENT_TIMESTAMP()) BETWEEN
        UNIX_TIMESTAMP(CONCAT(CURRENT_DATE(), ' ', t.open))
    AND UNIX_TIMESTAMP(CONCAT(CURRENT_DATE(), ' ', t.close))
)
AND
(
    n.id = %d
)
    ;";
</code>```

to query the database, and return all of the details from the embassy if it is currently open.

Now, rather than the CURRENT_TIMESTAMP(), I need to find out whether the time stored in variable $current_local_time is between the 2 database values. Timetable times need to be stored in local times instead of London's times, so that's why I'm trying to convert them.

I have tried replacing the

```<code>WHERE
    (
        UNIX_TIMESTAMP(CURRENT_TIMESTAMP()) BETWEEN
            UNIX_TIMESTAMP(CONCAT(CURRENT_DATE(), ' ', t.open))
        AND UNIX_TIMESTAMP(CONCAT(CURRENT_DATE(), ' ', t.close))
    )
</code>```

with

```<code>WHERE
    (
        UNIX_TIMESTAMP($current_local_time) BETWEEN
            UNIX_TIMESTAMP(CONCAT(CURRENT_DATE(), ' ', t.open))
        AND UNIX_TIMESTAMP(CONCAT(CURRENT_DATE(), ' ', t.close))
    )
</code>```

but I just get error messages. Could someone please tell me how I would do this properly?

Thanks for any help

Edit: The $current_local_time variable is taken like so:

```<code>date_default_timezone_set('Europe/London');
$time = date("H:i", time());
$myDateTime = new DateTime($time, new DateTimeZone('GMT'));
$myDateTime->setTimezone(new DateTimeZone($timezone));
$current_local_time = $myDateTime->format('H:i:s');
</code>```

and outputs as whatever the current time in the other country is when echoed.

Edit 2:

I have changed the $current_local_time to this:

```<code>$current_local_time = $myDateTime->format('H:i:s');
$timestamp2 = strtotime($current_local_time);
</code>```

and the where clause to this:

```<code>WHERE
(
    '$timestamp2' BETWEEN
        UNIX_TIMESTAMP(CONCAT(CURRENT_DATE(), ' ', t.open))
    AND UNIX_TIMESTAMP(CONCAT(CURRENT_DATE(), ' ', t.close))
)
</code>```

This works OK, but if t.close is set at 23:30, it won't actually shut until 23:31. Is there a way to take a minute off of the captured time to sort this out?

答:
The timestamp that you're inserting needs to be quoted as a string, so you need

```<code>WHERE
    (
        UNIX_TIMESTAMP('$current_local_time') BETWEEN
            UNIX_TIMESTAMP(CONCAT(CURRENT_DATE(), ' ', t.open))
        AND UNIX_TIMESTAMP(CONCAT(CURRENT_DATE(), ' ', t.close))
    )
</code>```

It's even better to use a SQL framework that lets you do parameter binding, specify the query as follows, and bind the ? to $current_local_time in PHP.

```<code>WHERE
    (
        UNIX_TIMESTAMP(?) BETWEEN
            UNIX_TIMESTAMP(CONCAT(CURRENT_DATE(), ' ', t.open))
        AND UNIX_TIMESTAMP(CONCAT(CURRENT_DATE(), ' ', t.close))
    )
</code>```

EDIT: I think you want

```<code>(UNIX_TIMESTAMP(CONCAT(CURRENT_DATE(), ' ', t.open))     <=   '$timestamp2'
AND
'$timestamp2'    <    UNIX_TIMESTAMP(CONCAT(CURRENT_DATE(), ' ', t.close)))
--              ^^^ Note the less-than sign
</code>```

You probably don't need quotes around $timestamp2 anymore.

Body:
I am currently using this:

$rawsql = "SELECT
*
FROM
    _erc_foffices n
INNER JOIN
    _erc_openings o ON n.id = o.branch_id AND o.dotw = DAYOFWEEK(CURRENT_DATE())
INNER JOIN
    _erc_openings_times t ON o.id = t.opening_id
WHERE
(
    UNIX_TIMESTAMP(CURRENT_TIMESTAMP()) BETWEEN
        UNIX_TIMESTAMP(CONCAT(CURRENT_DATE(), ' ', t.open))
    AND UNIX_TIMESTAMP(CONCAT(CURRENT_DATE(), ' ', t.close))
)
AND
(
    n.id = %d
)
    ;";

to query the database, and return all of the details from the embassy if it is currently open.

Now, rather than the CURRENT_TIMESTAMP(), I need to find out whether the time stored in variable $current_local_time is between the 2 database values. Timetable times need to be stored in local times instead of London's times, so that's why I'm trying to convert them.

I have tried replacing the

WHERE
    (
        UNIX_TIMESTAMP(CURRENT_TIMESTAMP()) BETWEEN
            UNIX_TIMESTAMP(CONCAT(CURRENT_DATE(), ' ', t.open))
        AND UNIX_TIMESTAMP(CONCAT(CURRENT_DATE(), ' ', t.close))
    )

with

WHERE
    (
        UNIX_TIMESTAMP($current_local_time) BETWEEN
            UNIX_TIMESTAMP(CONCAT(CURRENT_DATE(), ' ', t.open))
        AND UNIX_TIMESTAMP(CONCAT(CURRENT_DATE(), ' ', t.close))
    )

but I just get error messages. Could someone please tell me how I would do this properly?

Thanks for any help

Edit: The $current_local_time variable is taken like so:

date_default_timezone_set('Europe/London');
$time = date("H:i", time());
$myDateTime = new DateTime($time, new DateTimeZone('GMT'));
$myDateTime->setTimezone(new DateTimeZone($timezone));
$current_local_time = $myDateTime->format('H:i:s');

and outputs as whatever the current time in the other country is when echoed.

Edit 2:

I have changed the $current_local_time to this:

$current_local_time = $myDateTime->format('H:i:s');
$timestamp2 = strtotime($current_local_time);

and the where clause to this:

WHERE
(
    '$timestamp2' BETWEEN
        UNIX_TIMESTAMP(CONCAT(CURRENT_DATE(), ' ', t.open))
    AND UNIX_TIMESTAMP(CONCAT(CURRENT_DATE(), ' ', t.close))
)

This works OK, but if t.close is set at 23:30, it won't actually shut until 23:31. Is there a way to take a minute off of the captured time to sort this out?

Body:
The timestamp that you're inserting needs to be quoted as a string, so you need

WHERE
    (
        UNIX_TIMESTAMP('$current_local_time') BETWEEN
            UNIX_TIMESTAMP(CONCAT(CURRENT_DATE(), ' ', t.open))
        AND UNIX_TIMESTAMP(CONCAT(CURRENT_DATE(), ' ', t.close))
    )

It's even better to use a SQL framework that lets you do parameter binding, specify the query as follows, and bind the ? to $current_local_time in PHP.

WHERE
    (
        UNIX_TIMESTAMP(?) BETWEEN
            UNIX_TIMESTAMP(CONCAT(CURRENT_DATE(), ' ', t.open))
        AND UNIX_TIMESTAMP(CONCAT(CURRENT_DATE(), ' ', t.close))
    )

EDIT: I think you want

(UNIX_TIMESTAMP(CONCAT(CURRENT_DATE(), ' ', t.open))     <=   '$timestamp2'
AND
'$timestamp2'    <    UNIX_TIMESTAMP(CONCAT(CURRENT_DATE(), ' ', t.close)))
--              ^^^ Note the less-than sign

You probably don't need quotes around $timestamp2 anymore.


问:
Threads not terminating on Linux but on Mac
I'm currently working on a genetic algorithm where a new population is computed with different independent threads. My program works fine on OSX but some threads do not terminate on a Linux machine.

I have the following method which each thread performs until I have enough individuals generated. I'm using a single mutex which is part of the population class (as this method is). Therefore the mutex object is not static. The method gets passed a set of parents of which it may select one on the first parents_max's parents to generate a new child. A child is generated by mutation or recombination which another parent object. Inside select_randomly and bool_with_prob I use several distributions from <random> (all local variables) to pick a random parent or provide me with a random bool.

I searched the internet for reasons why the threads won't terminate and added serveral (possibly unecessary) locks around method which use std::random_device or std::mt19937 objects.

```<code>void generate_childs(std::set<individual> &parents, double mutation_rate, size_t parents_max)
{

size_t individuals_size;

{
    boost::lock_guard<boost::mutex> lock(mutex);
    individuals_size = individuals.size();
}

auto selectable_parents_end = parents.begin();
std::advance(selectable_parents_end, parents_max);

while(individuals_size < size)
{
    mutex.lock();
    individual male = *utilities::container::select_randomly(parents.begin(), selectable_parents_end);
    bool generate_child = utilities::container::bool_with_prob(0.3);
    mutex.unlock();

    boost::optional<individual> ind;

    if(generate_child)
    {
        mutex.lock();
        individual female = *utilities::container::select_randomly(parents.begin(), parents.end());
        mutex.unlock();

        ind = mutation::combined_mutated_child(male, female, mutation_rate);
    } else
    {
        ind = mutation::mutated_child(male, 0.9);
    }

    if(ind && ind->is_valid())
    {
        boost::lock_guard<boost::mutex> lock(mutex);

        if (individuals.size() < size) {
            individuals.insert(*ind);
        }

    }

    {
        boost::lock_guard<boost::mutex> lock(mutex);
        individuals_size = individuals.size();
    }
}
}
</code>```

I'm starting the threads like this:

```<code>unsigned int number_of_threads = std::thread::hardware_concurrency();

auto parents = individuals;

std::vector<boost::thread> threads;

for(size_t i = 0; i<number_of_threads; i++)
{
    threads.emplace_back(&population::generate_childs,
                         this,
                         std::ref(parents),
                         mutation_rate,
                         parents_max);
}

for(auto &t: threads)
{
    t.join();
    std::cout << "Thread finished" << individuals.size()  << std::endl;
}
</code>```

When executing my program on the local (OSX) machine compiled with Clang it performs as expected. On my Linux machine, it does not finish. I even tried to set number_of_threads=1, which did not help. When the program is not terminating on my Linux machine I can not quit it with Ctrl+C. Any ideas where I might have a race condition or a deadlock?

EDIT

As suggested I printed out an comment with the thread id for each thread. Apparently, my usage of the lock when updating the size was inappropriate. Therefore I modified the last lock as follows:

```<code>std::cout << i << " updating size" << std::endl;

{
    std::cout << i << " updating size about to lock" << std::endl;
    boost::lock_guard<boost::mutex> lock(configuration::mutex);

    std::cout << i << " updating size about to locked" << std::endl;

    individuals_size = individuals.size();

    if(individuals_size >= size)
    {
        std::cout << i << " returning" << std::endl;
        return;
    }
}
</code>```

The output of my program is this (skipped the part where threads worked fine):

```<code>0 started
2 started
3 started
3 entered while
0 entered while
1 started
2 entered while
1 entered while
3 got male 1
0 got male 0
3 got female
2 got male 1
1 got male 1
2 got female
1 got female
0 got mutated
0 before is valid
0 inserting
0 inserted
0 updating size
0 updating size about to lock
0 updating size about to locked
0 returning
Thread finished10
2 got combined
2 before is valid
2 inserting
2 inserted
2 updating size
2 updating size about to lock
2 updating size about to locked
2 returning
</code>```

After this, I don't get any additional outputs. For me, it seems like the lock guard does not free the mutex. Is it the order in which I join the threads? Because I try to join thread 1 before 2 even though it is not finished yet?

答:
One thing that I don't like in your code is that you misuse locks. For example when you retrieve the size of the container you cannot be sure that the size you've just retrieved is correct right after you unlock the mutex. So the correct pattern could be to lock the block of code where you retrieve the size and work with the container in the assumption that this size is correct -- and unlock it when you don't need this container anymore.

So you should rework the code because there could be race conditions in many places. One possible answer to your question, look at the code below:

```<code>    mutex.lock();
    individual male = *utilities::container::select_randomly(parents.begin(), selectable_parents_end);
    bool generate_child = utilities::container::bool_with_prob(0.3);
    mutex.unlock();
</code>```

What is an exception is thrown in the select_randomly? You would never unlock the mutex and this is a deadlock condition. Why could it throw an exception? For example because the value selectable_parents_end have become obsolete due to the race condition.

Body:
I'm currently working on a genetic algorithm where a new population is computed with different independent threads. My program works fine on OSX but some threads do not terminate on a Linux machine.

I have the following method which each thread performs until I have enough individuals generated. I'm using a single mutex which is part of the population class (as this method is). Therefore the mutex object is not static. The method gets passed a set of parents of which it may select one on the first parents_max's parents to generate a new child. A child is generated by mutation or recombination which another parent object. Inside select_randomly and bool_with_prob I use several distributions from <random> (all local variables) to pick a random parent or provide me with a random bool.

I searched the internet for reasons why the threads won't terminate and added serveral (possibly unecessary) locks around method which use std::random_device or std::mt19937 objects.

void generate_childs(std::set<individual> &parents, double mutation_rate, size_t parents_max)
{

size_t individuals_size;

{
    boost::lock_guard<boost::mutex> lock(mutex);
    individuals_size = individuals.size();
}

auto selectable_parents_end = parents.begin();
std::advance(selectable_parents_end, parents_max);

while(individuals_size < size)
{
    mutex.lock();
    individual male = *utilities::container::select_randomly(parents.begin(), selectable_parents_end);
    bool generate_child = utilities::container::bool_with_prob(0.3);
    mutex.unlock();

    boost::optional<individual> ind;

    if(generate_child)
    {
        mutex.lock();
        individual female = *utilities::container::select_randomly(parents.begin(), parents.end());
        mutex.unlock();

        ind = mutation::combined_mutated_child(male, female, mutation_rate);
    } else
    {
        ind = mutation::mutated_child(male, 0.9);
    }

    if(ind && ind->is_valid())
    {
        boost::lock_guard<boost::mutex> lock(mutex);

        if (individuals.size() < size) {
            individuals.insert(*ind);
        }

    }

    {
        boost::lock_guard<boost::mutex> lock(mutex);
        individuals_size = individuals.size();
    }
}
}

I'm starting the threads like this:

unsigned int number_of_threads = std::thread::hardware_concurrency();

auto parents = individuals;

std::vector<boost::thread> threads;

for(size_t i = 0; i<number_of_threads; i++)
{
    threads.emplace_back(&population::generate_childs,
                         this,
                         std::ref(parents),
                         mutation_rate,
                         parents_max);
}

for(auto &t: threads)
{
    t.join();
    std::cout << "Thread finished" << individuals.size()  << std::endl;
}

When executing my program on the local (OSX) machine compiled with Clang it performs as expected. On my Linux machine, it does not finish. I even tried to set number_of_threads=1, which did not help. When the program is not terminating on my Linux machine I can not quit it with Ctrl+C. Any ideas where I might have a race condition or a deadlock?

EDIT

As suggested I printed out an comment with the thread id for each thread. Apparently, my usage of the lock when updating the size was inappropriate. Therefore I modified the last lock as follows:

std::cout << i << " updating size" << std::endl;

{
    std::cout << i << " updating size about to lock" << std::endl;
    boost::lock_guard<boost::mutex> lock(configuration::mutex);

    std::cout << i << " updating size about to locked" << std::endl;

    individuals_size = individuals.size();

    if(individuals_size >= size)
    {
        std::cout << i << " returning" << std::endl;
        return;
    }
}

The output of my program is this (skipped the part where threads worked fine):

0 started
2 started
3 started
3 entered while
0 entered while
1 started
2 entered while
1 entered while
3 got male 1
0 got male 0
3 got female
2 got male 1
1 got male 1
2 got female
1 got female
0 got mutated
0 before is valid
0 inserting
0 inserted
0 updating size
0 updating size about to lock
0 updating size about to locked
0 returning
Thread finished10
2 got combined
2 before is valid
2 inserting
2 inserted
2 updating size
2 updating size about to lock
2 updating size about to locked
2 returning

After this, I don't get any additional outputs. For me, it seems like the lock guard does not free the mutex. Is it the order in which I join the threads? Because I try to join thread 1 before 2 even though it is not finished yet?

Body:
One thing that I don't like in your code is that you misuse locks. For example when you retrieve the size of the container you cannot be sure that the size you've just retrieved is correct right after you unlock the mutex. So the correct pattern could be to lock the block of code where you retrieve the size and work with the container in the assumption that this size is correct -- and unlock it when you don't need this container anymore.

So you should rework the code because there could be race conditions in many places. One possible answer to your question, look at the code below:

    mutex.lock();
    individual male = *utilities::container::select_randomly(parents.begin(), selectable_parents_end);
    bool generate_child = utilities::container::bool_with_prob(0.3);
    mutex.unlock();

What is an exception is thrown in the select_randomly? You would never unlock the mutex and this is a deadlock condition. Why could it throw an exception? For example because the value selectable_parents_end have become obsolete due to the race condition.


问:
How do I set return_uri for GoogleWebAuthorizationBroker.AuthorizeAsync?
I am trying to use the Google Calendar API in my non-MVC .NET Web Application. (This appears to be an important distinction.)

I’ve tried to use code from this example at Google and this example at Daimto along with some helpful hints from a number of related posts here.

I have written the following method:

```<code>public void GetUserCredential( String userName )
{
    String clientId = ConfigurationManager.AppSettings[ "Google.ClientId" ];            //From Google Developer console https://console.developers.google.com
    String clientSecret = ConfigurationManager.AppSettings[ "Google.ClientSecret" ];    //From Google Developer console https://console.developers.google.com
    String[] scopes = new string[] {
            Google.Apis.Calendar.v3.CalendarService.Scope.Calendar
    };

    // here is where we Request the user to give us access, or use the Refresh Token that was previously stored in %AppData%
    UserCredential credential = GoogleWebAuthorizationBroker.AuthorizeAsync( new ClientSecrets
    {
        ClientId = clientId,
        ClientSecret = clientSecret
    }, scopes, userName, CancellationToken.None, new FileDataStore( "c:\\temp" ) ).Result;

    // TODO: Replace FileDataStore with DatabaseDataStore
}
</code>```

Problem is, when Google’s OAuth2 page is called, redirect_uri keeps getting set to http://localhost:<some-random-port>/authorize. I have no idea how to set this to something else, as in the following example URL generated by AuthorizeAsync:

```<code>https://accounts.google.com/o/oauth2/auth?access_type=offline
    &response_type=code
    &client_id=********.apps.googleusercontent.com
    &redirect_uri=http:%2F%2Flocalhost:40839%2Fauthorize%2F
    &scope=https:%2F%2Fwww.googleapis.com%2Fauth%2Fcalendar
</code>```

Google responds with a redirect_uri_mismatch error page with the message:

“The redirect URI in the request: http://localhost:XXXXX/authorize/ did not match a registered redirect URI”

I can only register so many Redirect URI’s in my Google Developer’s Console Credentials page. I’m not inclined to register 65535 ports, and I want to use a page other than /authorize on my site. Specifically, I want to use, during development, http://localhost:888/Pages/GoogleApiRedirect but have no clue as to where I would set this, beyond what I've done in the Developer’s Console.

How do I explicitly set the value of redirect_uri? I am also open to a response in the form “This approach is completely wrong.”

EDIT:

After playing with this over the past day, I've discovered that by using the Client ID/Client Secret for the Native Application rather than the Web Application, I can at least get to Google's web authorization page without it complaining about a redirect_uri_mismatch. This is still unacceptable, because it still returns to http://localhost:<some-random-port>/authorize, which is outside the control of my web application.

答:
If you are trying to use GoogleWebAuthorizationBroker.AuthorizeAsync in a .NET application NON-web server application i.e. C# Console App command line program, it's crucial when creating the Google OAuth profile (https://console.developers.google.com/apis) in the credentials to select the following.

DESKTOP APP - This also known now as an installed application

It's hidden under "Help me choose" new credential. You must create a new credential which shows Desktop app and not WebApp.

CLICK "HELP ME CHOOSE"

CHOOSE YOUR INTENDED API LIBRARY

ie (Google Calendar API, YouTube) Select "User Data"

STEP 3 - OAUTH CLIENT ID - Desktop App!

Yeah - NO AUTHORIZATION REQUIRED FILEDS"

ie Javascript & Redirect Now you have a profile without the Web Application redirect authorization

Use the "Download JSON" and save it to your application

Reference in the code below. When you look inside this file, you will notice a different set of parameters as well to tell the broker this is an application. In this example, I am accessing the scope Calendar API. Just change the scope to whatever API you are trying to access.**

```<code>string[] Scopes = { CalendarService.Scope.Calendar }; //requires full scope to get ACL list..

string ApplicationName = "Name Of Your Application In Authorization Screen";
//just reference the namespaces in your using block

using (var stream = new FileStream("other_client_id.json", FileMode.Open, FileAccess.Read))
   {
   // The file token.json stores the user's access and refresh tokens, and is created
    // automatically when the authorization flow completes for the first time.
   string credPath = "other_token.json";
   credential = GoogleWebAuthorizationBroker.AuthorizeAsync(
      GoogleClientSecrets.Load(stream).Secrets,
      Scopes,
      "user",
      CancellationToken.None,
      new FileDataStore(credPath, true)).Result;
    }

// Create Google Calendar API service.
var service = new CalendarService(new BaseClientService.Initializer()
{
   HttpClientInitializer = credential,
   ApplicationName = ApplicationName,
});

//Then your ready to grab data from here using the methods mentioned in Google Calendar API docs
</code>```

To run this in Docker

When you deploy this to docker you will get an exception that the operating system does not support running the process. Basically, its trying to open a browser on the Metal Box. which is not possible with docker.

To solve this. Modify the code to use full absolute path like this

```<code>var inputFolderAbsolute = Path.Combine(AppContext.BaseDirectory, "Auth.Store");
...
new FileDataStore(inputFolderAbsolute, true)

</code>```

Run this application as a console app on your local machine so the browser opens.

Select the account you want to work with

In the bin folder, a new folder and file will be created.

Copy that folder to the root path

Set the file to copy if newer

Deploy to docker

Because the refresh token is saved for the account you selected it will get a new access token and work.

NB: It is possible the refresh token expires to whatever reason. You will have to repeat the steps above

Body:
I am trying to use the Google Calendar API in my non-MVC .NET Web Application. (This appears to be an important distinction.)

I’ve tried to use code from this example at Google and this example at Daimto along with some helpful hints from a number of related posts here.

I have written the following method:

public void GetUserCredential( String userName )
{
    String clientId = ConfigurationManager.AppSettings[ "Google.ClientId" ];            //From Google Developer console https://console.developers.google.com
    String clientSecret = ConfigurationManager.AppSettings[ "Google.ClientSecret" ];    //From Google Developer console https://console.developers.google.com
    String[] scopes = new string[] {
            Google.Apis.Calendar.v3.CalendarService.Scope.Calendar
    };

    // here is where we Request the user to give us access, or use the Refresh Token that was previously stored in %AppData%
    UserCredential credential = GoogleWebAuthorizationBroker.AuthorizeAsync( new ClientSecrets
    {
        ClientId = clientId,
        ClientSecret = clientSecret
    }, scopes, userName, CancellationToken.None, new FileDataStore( "c:\\temp" ) ).Result;

    // TODO: Replace FileDataStore with DatabaseDataStore
}

Problem is, when Google’s OAuth2 page is called, redirect_uri keeps getting set to http://localhost:<some-random-port>/authorize. I have no idea how to set this to something else, as in the following example URL generated by AuthorizeAsync:

https://accounts.google.com/o/oauth2/auth?access_type=offline
    &response_type=code
    &client_id=********.apps.googleusercontent.com
    &redirect_uri=http:%2F%2Flocalhost:40839%2Fauthorize%2F
    &scope=https:%2F%2Fwww.googleapis.com%2Fauth%2Fcalendar

Google responds with a redirect_uri_mismatch error page with the message:

“The redirect URI in the request: http://localhost:XXXXX/authorize/ did not match a registered redirect URI”

I can only register so many Redirect URI’s in my Google Developer’s Console Credentials page. I’m not inclined to register 65535 ports, and I want to use a page other than /authorize on my site. Specifically, I want to use, during development, http://localhost:888/Pages/GoogleApiRedirect but have no clue as to where I would set this, beyond what I've done in the Developer’s Console.

How do I explicitly set the value of redirect_uri? I am also open to a response in the form “This approach is completely wrong.”

EDIT:

After playing with this over the past day, I've discovered that by using the Client ID/Client Secret for the Native Application rather than the Web Application, I can at least get to Google's web authorization page without it complaining about a redirect_uri_mismatch. This is still unacceptable, because it still returns to http://localhost:<some-random-port>/authorize, which is outside the control of my web application.

Body:
If you are trying to use GoogleWebAuthorizationBroker.AuthorizeAsync in a .NET application NON-web server application i.e. C# Console App command line program, it's crucial when creating the Google OAuth profile (https://console.developers.google.com/apis) in the credentials to select the following.

DESKTOP APP - This also known now as an installed application

It's hidden under "Help me choose" new credential. You must create a new credential which shows Desktop app and not WebApp.

CLICK "HELP ME CHOOSE"

CHOOSE YOUR INTENDED API LIBRARY

ie (Google Calendar API, YouTube) Select "User Data"

STEP 3 - OAUTH CLIENT ID - Desktop App!

Yeah - NO AUTHORIZATION REQUIRED FILEDS"

ie Javascript & Redirect Now you have a profile without the Web Application redirect authorization

Use the "Download JSON" and save it to your application

Reference in the code below. When you look inside this file, you will notice a different set of parameters as well to tell the broker this is an application. In this example, I am accessing the scope Calendar API. Just change the scope to whatever API you are trying to access.**

string[] Scopes = { CalendarService.Scope.Calendar }; //requires full scope to get ACL list..

string ApplicationName = "Name Of Your Application In Authorization Screen";
//just reference the namespaces in your using block

using (var stream = new FileStream("other_client_id.json", FileMode.Open, FileAccess.Read))
   {
   // The file token.json stores the user's access and refresh tokens, and is created
    // automatically when the authorization flow completes for the first time.
   string credPath = "other_token.json";
   credential = GoogleWebAuthorizationBroker.AuthorizeAsync(
      GoogleClientSecrets.Load(stream).Secrets,
      Scopes,
      "user",
      CancellationToken.None,
      new FileDataStore(credPath, true)).Result;
    }

// Create Google Calendar API service.
var service = new CalendarService(new BaseClientService.Initializer()
{
   HttpClientInitializer = credential,
   ApplicationName = ApplicationName,
});

//Then your ready to grab data from here using the methods mentioned in Google Calendar API docs

To run this in Docker

When you deploy this to docker you will get an exception that the operating system does not support running the process. Basically, its trying to open a browser on the Metal Box. which is not possible with docker.

To solve this. Modify the code to use full absolute path like this

var inputFolderAbsolute = Path.Combine(AppContext.BaseDirectory, "Auth.Store");
...
new FileDataStore(inputFolderAbsolute, true)

Run this application as a console app on your local machine so the browser opens.

Select the account you want to work with

In the bin folder, a new folder and file will be created.

Copy that folder to the root path

Set the file to copy if newer

Deploy to docker

Because the refresh token is saved for the account you selected it will get a new access token and work.

NB: It is possible the refresh token expires to whatever reason. You will have to repeat the steps above


问:
Sorting multidim array: prioritize if column contains substring, then order by a second column
I am currently creating a sorting method that consists of values from an mysql query.

Here's a brief view of the array:

```<code>    Array
    (
        [0] => Array
            (
                ['id'] = 1;
                ['countries'] = 'EN,CH,SP';
            )
        [1] => Array
            (
                ['id'] = 2;
                ['countries'] = 'GE,SP,SV';
            )
    )
</code>```

I have succeeded in making a normal usort based on the numeric id values, but I rather want to sort the array by the content of the "countries" field (if it contains a set string, a country code in this case), and then by the id field.

The following snippet was my first idea of how to do it, but I have no idea of how to incorporate it into an working function:

```<code>in_array('EN', explode(",",$a['countries']) );
</code>```

How would you do it?

Thanks!

I am really getting nowhere with this unfortunately.

Here is what I have for the moment, and its giving me nothing but errors: uasort() [function.uasort]: Invalid comparison function

```<code>function compare($a, $b) {
    global $usercountry;

        if ( in_array($usercountry, $a['countries']) && in_array($usercountry, $a['countries']) ) {
            $return = 0;
        }

        else if (in_array($usercountry, $a['countries'])) {
            $return = 1;
        }

        else {
            $return = -1;
        }

        return $return;

        }

        $array= usort($array, "compare");
</code>```

Is there anyone who might give me a hint of how to go on with it?

答:
I am currently creating a sorting method that consists of values from an mysql query.

TRUTH:
 Using anything other than MySQL to sort your result set will be less efficient (with php, a usort() or array_multisort() call will be more convoluted and harder to maintain) and therefore inappropriate.

SQL: (Demo)

```<code>ORDER BY IF(LOCATE('EN', countries), 0, 1), id;
</code>```

This prioritizes countries column values that contain EN then sorts on id ASC.

For anyone who isn't handling a sql result set or cannot manipulate the query for some reason, I endorse usort().
PHP7 offers a beautiful new operator that performs a comparison and returns one of three values (-1, 0, 1). This operator is affectionately called the "spaceship operator" and looks like this <=>.

PHP: (Demo)

```<code>$test = [
    ['id' => 1, 'countries' => 'EN,CH,SP'],
    ['id' => 2, 'countries' => 'GE,SP,SV'],
    ['id' => 3, 'countries' => 'PR,SP,IT'],
    ['id' => 4, 'countries' => 'EN'],
    ['id' => 5, 'countries' => 'SP,EN'],
    ['id' => 6, 'countries' => 'SV,SP,EN'],
    ['id' => 7, 'countries' => 'GE,SP'],
    ['id' => 8, 'countries' => 'FR'],
    ['id' => 9, 'countries' => 'RU,EN'],
    ['id' => 10, 'countries' => 'EN,SP,IT'],
    ['id' => 11, 'countries' => 'SP,GR'],
    ['id' => 12, 'countries' => 'GR,EN']
];

usort($test, function($a, $b) {
    return [strpos($a['countries'], 'EN') === false, $a['id']]
           <=>
           [strpos($b['countries'], 'EN') === false, $b['id']];
});

var_export($test);
</code>```

PHP8 alternatives:

```<code>return [str_contains($b['countries'], 'EN'), $a['id']]
       <=>
       [str_contains($a['countries'], 'EN'), $b['id']];
</code>```

or

```<code>return str_contains($b['countries'], 'EN') <=> [str_contains($a['countries'], 'EN')
       ?: $a['id']] <=> $b['id']];
</code>```

Output:

```<code>array (
  0 =>
  array (
    'id' => 1,
    'countries' => 'EN,CH,SP',
  ),
  1 =>
  array (
    'id' => 4,
    'countries' => 'EN',
  ),
  2 =>
  array (
    'id' => 5,
    'countries' => 'SP,EN',
  ),
  3 =>
  array (
    'id' => 6,
    'countries' => 'SV,SP,EN',
  ),
  4 =>
  array (
    'id' => 9,
    'countries' => 'RU,EN',
  ),
  5 =>
  array (
    'id' => 10,
    'countries' => 'EN,SP,IT',
  ),
  6 =>
  array (
    'id' => 12,
    'countries' => 'GR,EN',
  ),
  7 =>
  array (
    'id' => 2,
    'countries' => 'GE,SP,SV',
  ),
  8 =>
  array (
    'id' => 3,
    'countries' => 'PR,SP,IT',
  ),
  9 =>
  array (
    'id' => 7,
    'countries' => 'GE,SP',
  ),
  10 =>
  array (
    'id' => 8,
    'countries' => 'FR',
  ),
  11 =>
  array (
    'id' => 11,
    'countries' => 'SP,GR',
  ),
)
</code>```

The array elements on either side of the spaceship operator are evaluated from left to right (leftside [0] vs rightside [0], then moving onto the pair of [1] values if there is a "tie" between the two [0] values).

If the === false looks backwards, let me explain...

If EN is found in the countries string, the condition will evaluate as false. When comparing true and false, remember that true equates to 1 and false equates to 0.  We want ASC sorting, so we want to put false outcomes before true outcomes, ergo strings containing EN need to return false.  Hopefully that clears up the logic.

Body:
I am currently creating a sorting method that consists of values from an mysql query.

Here's a brief view of the array:

    Array
    (
        [0] => Array
            (
                ['id'] = 1;
                ['countries'] = 'EN,CH,SP';
            )
        [1] => Array
            (
                ['id'] = 2;
                ['countries'] = 'GE,SP,SV';
            )
    )

I have succeeded in making a normal usort based on the numeric id values, but I rather want to sort the array by the content of the "countries" field (if it contains a set string, a country code in this case), and then by the id field.

The following snippet was my first idea of how to do it, but I have no idea of how to incorporate it into an working function:

in_array('EN', explode(",",$a['countries']) );

How would you do it?

Thanks!

I am really getting nowhere with this unfortunately.

Here is what I have for the moment, and its giving me nothing but errors: uasort() [function.uasort]: Invalid comparison function

function compare($a, $b) {
    global $usercountry;

        if ( in_array($usercountry, $a['countries']) && in_array($usercountry, $a['countries']) ) {
            $return = 0;
        }

        else if (in_array($usercountry, $a['countries'])) {
            $return = 1;
        }

        else {
            $return = -1;
        }

        return $return;

        }

        $array= usort($array, "compare");

Is there anyone who might give me a hint of how to go on with it?

Body:
I am currently creating a sorting method that consists of values from an mysql query.

TRUTH:
 Using anything other than MySQL to sort your result set will be less efficient (with php, a usort() or array_multisort() call will be more convoluted and harder to maintain) and therefore inappropriate.

SQL: (Demo)

ORDER BY IF(LOCATE('EN', countries), 0, 1), id;

This prioritizes countries column values that contain EN then sorts on id ASC.

For anyone who isn't handling a sql result set or cannot manipulate the query for some reason, I endorse usort().
PHP7 offers a beautiful new operator that performs a comparison and returns one of three values (-1, 0, 1). This operator is affectionately called the "spaceship operator" and looks like this <=>.

PHP: (Demo)

$test = [
    ['id' => 1, 'countries' => 'EN,CH,SP'],
    ['id' => 2, 'countries' => 'GE,SP,SV'],
    ['id' => 3, 'countries' => 'PR,SP,IT'],
    ['id' => 4, 'countries' => 'EN'],
    ['id' => 5, 'countries' => 'SP,EN'],
    ['id' => 6, 'countries' => 'SV,SP,EN'],
    ['id' => 7, 'countries' => 'GE,SP'],
    ['id' => 8, 'countries' => 'FR'],
    ['id' => 9, 'countries' => 'RU,EN'],
    ['id' => 10, 'countries' => 'EN,SP,IT'],
    ['id' => 11, 'countries' => 'SP,GR'],
    ['id' => 12, 'countries' => 'GR,EN']
];

usort($test, function($a, $b) {
    return [strpos($a['countries'], 'EN') === false, $a['id']]
           <=>
           [strpos($b['countries'], 'EN') === false, $b['id']];
});

var_export($test);

PHP8 alternatives:

return [str_contains($b['countries'], 'EN'), $a['id']]
       <=>
       [str_contains($a['countries'], 'EN'), $b['id']];

or

return str_contains($b['countries'], 'EN') <=> [str_contains($a['countries'], 'EN')
       ?: $a['id']] <=> $b['id']];

Output:

array (
  0 =>
  array (
    'id' => 1,
    'countries' => 'EN,CH,SP',
  ),
  1 =>
  array (
    'id' => 4,
    'countries' => 'EN',
  ),
  2 =>
  array (
    'id' => 5,
    'countries' => 'SP,EN',
  ),
  3 =>
  array (
    'id' => 6,
    'countries' => 'SV,SP,EN',
  ),
  4 =>
  array (
    'id' => 9,
    'countries' => 'RU,EN',
  ),
  5 =>
  array (
    'id' => 10,
    'countries' => 'EN,SP,IT',
  ),
  6 =>
  array (
    'id' => 12,
    'countries' => 'GR,EN',
  ),
  7 =>
  array (
    'id' => 2,
    'countries' => 'GE,SP,SV',
  ),
  8 =>
  array (
    'id' => 3,
    'countries' => 'PR,SP,IT',
  ),
  9 =>
  array (
    'id' => 7,
    'countries' => 'GE,SP',
  ),
  10 =>
  array (
    'id' => 8,
    'countries' => 'FR',
  ),
  11 =>
  array (
    'id' => 11,
    'countries' => 'SP,GR',
  ),
)

The array elements on either side of the spaceship operator are evaluated from left to right (leftside [0] vs rightside [0], then moving onto the pair of [1] values if there is a "tie" between the two [0] values).

If the === false looks backwards, let me explain...

If EN is found in the countries string, the condition will evaluate as false. When comparing true and false, remember that true equates to 1 and false equates to 0.  We want ASC sorting, so we want to put false outcomes before true outcomes, ergo strings containing EN need to return false.  Hopefully that clears up the logic.