问:
ssh works in terminal but not through Python
I am trying to write data to a disk on a remote machine via ssh using a Python script. However it is giving an error of dd: /dev/xbd2d: Device not configured.

```<code>import argparse
import os
import time

parser = argparse.ArgumentParser(description='basis')
parser.add_argument("-g", default=1, help="")
args = parser.parse_args()
volume = args.g
instance_ip=10.1.12.3
cmd_ssh='ssh -tt -i basis.pem root@'+instance_ip+ ''' "date | dd of=/dev/xbd2d"'''
os.system(cmd_ssh)
</code>```

What is quite unusual is that if use the command:

```<code>ssh -tt -i basis.pem root@10.1.12.3 "date | dd of=/dev/xbd2d"
</code>```

in a terminal, it executes correctly without any problem and writes the data to the disk. I wrote the same script in C++ and it worked fine but for some reason Python gives me dd: /dev/xbd2d: Device not configured.

答:
Checking the quotes around the variable and also using \" instead of the triple quotes solved my problem as recommended by Eran</p>

```<code>import argparse
import os
import time

parser = argparse.ArgumentParser(description='basis')
parser.add_argument("-g", default=1, help="")
args = parser.parse_args()
volume = args.g
instance_ip=10.1.12.3
cmd_ssh="ssh -tt -i basis.pem root@"+str(instance_ip)+" \"date | dd of=/dev/xbd2d\""
os.system(cmd_ssh)
</code>```

Body:
I am trying to write data to a disk on a remote machine via ssh using a Python script. However it is giving an error of dd: /dev/xbd2d: Device not configured.

import argparse
import os
import time

parser = argparse.ArgumentParser(description='basis')
parser.add_argument("-g", default=1, help="")
args = parser.parse_args()
volume = args.g
instance_ip=10.1.12.3
cmd_ssh='ssh -tt -i basis.pem root@'+instance_ip+ ''' "date | dd of=/dev/xbd2d"'''
os.system(cmd_ssh)

What is quite unusual is that if use the command:

ssh -tt -i basis.pem root@10.1.12.3 "date | dd of=/dev/xbd2d"

in a terminal, it executes correctly without any problem and writes the data to the disk. I wrote the same script in C++ and it worked fine but for some reason Python gives me dd: /dev/xbd2d: Device not configured.

Body:
Checking the quotes around the variable and also using \" instead of the triple quotes solved my problem as recommended by Eran

import argparse
import os
import time

parser = argparse.ArgumentParser(description='basis')
parser.add_argument("-g", default=1, help="")
args = parser.parse_args()
volume = args.g
instance_ip=10.1.12.3
cmd_ssh="ssh -tt -i basis.pem root@"+str(instance_ip)+" \"date | dd of=/dev/xbd2d\""
os.system(cmd_ssh)


来源: stackexchange

问:
php file_get_contents ('....datafile.xml') - get the datafile from my local desktop?
I have a database that exports xmls only locally (to my computer).  It will not export to a remote server.

I have a php file (located on remote server) that sends the xml to a webserivce.
Can I have the remote php file use "file_get_contents" function to extract xml data from my local computer?

If so, what is the proper syntax?
Thanks in advance.

答:
You have a couple of options...

You could run a local web server on your computer and publish the files via that. Your remote PHP script could then read the file.

A potentially better alternative would be to use something like Dropbox. Save the database files in your Dropbox folder. They will be automatically uploaded to the Dropbox cloud storage.

Your remote PHP script could then retrieve them via the Dropbox API or if they are in the "public" folder, directly via URL.

ContentLicense: CC BY-SA 3.0


问:
C# get unreferenced assemblies

The problem

I'm working on a project with a plugin like system. The project structure is defined as follows:

```<code>- Solution
    - Project A (Cli)
    - Project B (Core logic)
    - Project C (Plugin)
</code>```

Project A is the startup project, Project B contains the core logic and Project C is a plugin, which implements an interface in project B. Project A only has a reference to Project B and Project C only has a reference to Project B.

To visualize:

Since I do not reference Project C from Project B (and do not instantiate the class directly), code like AppDomain.CurrentDomain.GetAssemblies() (inside Project B) does not return Project C.

However, once I do (from Project B): var MyImplementation = new MyImplementation(), AppDomain.CurrentDomain.GetAssemblies() does return the assembly, since it's known and loaded at runtime as I instantiated the class MyImplementation explicitly from code.

The question

From Project B, I want to get all assemblies which implement a specific interface from Project B and instantiate them.

So, how do I get these (seemingly) unknown assemblies to get loaded at runtime?

答:
I don't think it's possible to know if an assembly contains a class that implement your interface without loading it.

What you can do is provide a directory where you put all your plugins assemblies.

So you can load your plugins in your AppDomain like this :

public static Assembly[] MyPlugins { get; private set; }

public static void LoadAssemblies()
{
   MyPlugins = Directory.GetFiles(lookingDirectory, "*.dll")
                .Select(assemblyPath =>
                {
                    AssemblyName an = AssemblyName.GetAssemblyName(assemblyPath);
                    return Assembly.Load(an);
                })
                .ToArray();
}

```

<p>And then you can find all types that implements your interface like this :</p>

<pre class="lang-cs prettyprint-override"><code>MyPlugins.SelectMany(assembly => assembly.GetTypes())
    .Where(type => typeof(IYourInterface).IsAssignableFrom(type))
    .Select(type => (IYourInterface)Activator.CreateInstance(type))
</code>```

And then you can iterate on each implementation of your interface to call a shared method for example.

ContentLicense: CC BY-SA 4.0

Title: C# get unreferenced assemblies

Body:
The problem

I'm working on a project with a plugin like system. The project structure is defined as follows:

- Solution
    - Project A (Cli)
    - Project B (Core logic)
    - Project C (Plugin)

Project A is the startup project, Project B contains the core logic and Project C is a plugin, which implements an interface in project B. Project A only has a reference to Project B and Project C only has a reference to Project B.

To visualize:

Since I do not reference Project C from Project B (and do not instantiate the class directly), code like AppDomain.CurrentDomain.GetAssemblies() (inside Project B) does not return Project C.

However, once I do (from Project B): var MyImplementation = new MyImplementation(), AppDomain.CurrentDomain.GetAssemblies() does return the assembly, since it's known and loaded at runtime as I instantiated the class MyImplementation explicitly from code.

The question

From Project B, I want to get all assemblies which implement a specific interface from Project B and instantiate them.

So, how do I get these (seemingly) unknown assemblies to get loaded at runtime?

Body:
I don't think it's possible to know if an assembly contains a class that implement your interface without loading it.

What you can do is provide a directory where you put all your plugins assemblies.

So you can load your plugins in your AppDomain like this :

public static Assembly[] MyPlugins { get; private set; }

public static void LoadAssemblies()
{
   MyPlugins = Directory.GetFiles(lookingDirectory, "*.dll")
                .Select(assemblyPath =>
                {
                    AssemblyName an = AssemblyName.GetAssemblyName(assemblyPath);
                    return Assembly.Load(an);
                })
                .ToArray();
}

And then you can find all types that implements your interface like this :

MyPlugins.SelectMany(assembly => assembly.GetTypes())
    .Where(type => typeof(IYourInterface).IsAssignableFrom(type))
    .Select(type => (IYourInterface)Activator.CreateInstance(type))

And then you can iterate on each implementation of your interface to call a shared method for example.


问:
How does character array with single quote affects an character array without the quotes?
```<code>char a[5] = { 1, 2, 3, 4, 5 };
char a[5] = { '1','2', '3', '4', '5' };
</code>```

what is the difference? It does not raise any errors. why?

答: Characters like '1' actually kind of like macros to their actual ASCII values. So saying '1' is same as saying 49 since that is its ASCII code value. A char type essentially is a signed 8 bit integer by the way so you can assign numbers to it if thats confuse you.

Body:
char a[5] = { 1, 2, 3, 4, 5 };
char a[5] = { '1','2', '3', '4', '5' };

what is the difference? It does not raise any errors. why?


问:
Combining acspline with filledcurve on data
I know how to use filledcurve with using 1:2:3 to fill the area between the curves described by columns 2 and 3.

I also know how to smooth a single curve using acsplines with using 1:2:(0.1) (with 0.1 being the smoothing weight for each point).

But I fail to combine the two, i.e. fill the area between two smoothed curves.  All variants I try give me error messages like duplicated or contradicting arguments in datafile options.

This is what I tried:

```<code>plot 'datafile' using 1:2:3:(0.1):(0.1) smooth acs with lines
plot 'datafile' using 1:2:(0.1):3:(0.1) smooth acs with lines
</code>```

Is this combination possible at all?  How is the syntax then?

答:
I guess you can't combine smooth acspline and with filledcurves, but I also don't know why this shouldn't be possible.
Well, as a workaround, plot your smooth acspline curve into a data table first and then plot it with filledcurves.

Code:

```<code>### filled acspline curve
reset session

$Data <<EOD
1 2
2 4
3 3
4 1
EOD

# plot your acspline data into a table
set table $ACSpline
    plot $Data u 1:2 smooth acspline
unset table

set style fill transparent solid 0.1

plot $Data u 1:2 w lp pt 7 lc rgb "black" ti "Original data", \
     $Data u 1:2 smooth acspline lw 2 lc rgb "red" ti "acspline", \
     $ACSpline u 1:2 w filledcurves x1 lc rgb "red" ti "acspline filled"
### end of code
</code>```

Result:

Addition:

There is another way to get data into a table. Check help set print.
With this, you can get the data of the two splines into one datablock and can plot the area between them. Maybe someone knows a simpler way to achieve this.

Code:

```<code>### filled curve between two acsplines
reset session

$Data <<EOD
1 2 4
2 4 1
3 3 0
4 1 3
EOD

# plot your acspline data into tables
set table $ACSpline1
    plot $Data u 1:2:(1) smooth acspline
unset table
set table $ACSpline2
    plot $Data u 1:3:(1) smooth acspline
unset table
set print $BetweenSplines
    do for [i=1:|$ACSpline1|] {
        print sprintf("%s %s %s",word($ACSpline1[i],1),word($ACSpline1[i],2),word($ACSpline2[i],2))
    }
set print

set style fill transparent solid 0.5

plot $Data u 1:2 w lp pt 7 lc rgb "black" ti "Original data col 2", \
     $Data u 1:3 w lp pt 7 lc rgb "blue" ti "Original data col 3", \
     $ACSpline1 u 1:2 w l lw 2 lc rgb "red" ti "acspline col 2", \
     $ACSpline2 u 1:2 w l lw 2 lc rgb "green" ti "acspline col 3", \
     $BetweenSplines u 1:2:3 w filledcurves lc rgb "yellow" ti "acspline filled"
### end of code
</code>```

Result:

Body:
I know how to use filledcurve with using 1:2:3 to fill the area between the curves described by columns 2 and 3.

I also know how to smooth a single curve using acsplines with using 1:2:(0.1) (with 0.1 being the smoothing weight for each point).

But I fail to combine the two, i.e. fill the area between two smoothed curves.  All variants I try give me error messages like duplicated or contradicting arguments in datafile options.

This is what I tried:

plot 'datafile' using 1:2:3:(0.1):(0.1) smooth acs with lines
plot 'datafile' using 1:2:(0.1):3:(0.1) smooth acs with lines

Is this combination possible at all?  How is the syntax then?

Body:
I guess you can't combine smooth acspline and with filledcurves, but I also don't know why this shouldn't be possible.
Well, as a workaround, plot your smooth acspline curve into a data table first and then plot it with filledcurves.

Code:

### filled acspline curve
reset session

$Data <<EOD
1 2
2 4
3 3
4 1
EOD

# plot your acspline data into a table
set table $ACSpline
    plot $Data u 1:2 smooth acspline
unset table

set style fill transparent solid 0.1

plot $Data u 1:2 w lp pt 7 lc rgb "black" ti "Original data", \
     $Data u 1:2 smooth acspline lw 2 lc rgb "red" ti "acspline", \
     $ACSpline u 1:2 w filledcurves x1 lc rgb "red" ti "acspline filled"
### end of code

Result:

Addition:

There is another way to get data into a table. Check help set print.
With this, you can get the data of the two splines into one datablock and can plot the area between them. Maybe someone knows a simpler way to achieve this.

Code:

### filled curve between two acsplines
reset session

$Data <<EOD
1 2 4
2 4 1
3 3 0
4 1 3
EOD

# plot your acspline data into tables
set table $ACSpline1
    plot $Data u 1:2:(1) smooth acspline
unset table
set table $ACSpline2
    plot $Data u 1:3:(1) smooth acspline
unset table
set print $BetweenSplines
    do for [i=1:|$ACSpline1|] {
        print sprintf("%s %s %s",word($ACSpline1[i],1),word($ACSpline1[i],2),word($ACSpline2[i],2))
    }
set print

set style fill transparent solid 0.5

plot $Data u 1:2 w lp pt 7 lc rgb "black" ti "Original data col 2", \
     $Data u 1:3 w lp pt 7 lc rgb "blue" ti "Original data col 3", \
     $ACSpline1 u 1:2 w l lw 2 lc rgb "red" ti "acspline col 2", \
     $ACSpline2 u 1:2 w l lw 2 lc rgb "green" ti "acspline col 3", \
     $BetweenSplines u 1:2:3 w filledcurves lc rgb "yellow" ti "acspline filled"
### end of code

Result:


问:
ssh: Could not resolve hostname sama5d27-som1-ek-sd:
I am trying to set up remote connection between my Desktop machine ( Windows machine ) and remote machine ( Linux machine ).

For that, I am following the steps described here :

https://learn.microsoft.com/en-us/cpp/linux/set-up-fips-compliant-secure-remote-linux-development?view=vs-2019#to-create-and-use-an-rsa-key-file

But, in step 2, when I try, From Windows, to copy the public key to the Linux machine using this command :

```<code>scp C:\Users\wiemz/.ssh/id_rsa.pub root@sama5d27-som1-ek-sd:
</code>```

This error occurs :

ssh: Could not resolve hostname sama5d27-som1-ek-sd: H\364te Unknown.
  lost connection

I verfied my Linux internet connection with ping command and it's going well. Besides, when I typed this command on the Linux machine :

```<code>ssh root@sama5d27-som1-ek-sd
</code>```

it says :

ssh: Could not resolve hostname sama5d27-som1-ek-sd: Temporary failure
  in name resolution

How can I fix this problem please ?

答:
The problem is your OS can't resolve the hostname you use. You should provide FQDN like web.example.com or IP address of the machine. For example

```<code>scp file root@IP:
</code>```

or

```<code>scp file root@web.example.com:
</code>```

Body:
I am trying to set up remote connection between my Desktop machine ( Windows machine ) and remote machine ( Linux machine ).

For that, I am following the steps described here :

https://learn.microsoft.com/en-us/cpp/linux/set-up-fips-compliant-secure-remote-linux-development?view=vs-2019#to-create-and-use-an-rsa-key-file

But, in step 2, when I try, From Windows, to copy the public key to the Linux machine using this command :

scp C:\Users\wiemz/.ssh/id_rsa.pub root@sama5d27-som1-ek-sd:

This error occurs :

ssh: Could not resolve hostname sama5d27-som1-ek-sd: H\364te Unknown.
  lost connection

I verfied my Linux internet connection with ping command and it's going well. Besides, when I typed this command on the Linux machine :

ssh root@sama5d27-som1-ek-sd

it says :

ssh: Could not resolve hostname sama5d27-som1-ek-sd: Temporary failure
  in name resolution

How can I fix this problem please ?

Body:
The problem is your OS can't resolve the hostname you use. You should provide FQDN like web.example.com or IP address of the machine. For example

scp file root@IP:

or

scp file root@web.example.com:


问:
What exactly is usage of cursor_advance in BPF?
I was looking through a slide by IOvisor project, https://events.static.linuxfound.org/sites/events/files/slides/iovisor-lc-bof-2016.pdf

#include <bcc/proto.h>

struct IPKey { u32 dip; u32 sip; };
BPF_TABLE("hash", struct IPKey, int, mytable, 1024);

int recv_packet(struct __sk_buff *skb) {
    struct IPKey key;
    u8 *cursor = 0;
    struct ethernet_t *ethernet = cursor_advance(cursor, sizeof(*ethernet));
    struct ip_t *ip = cursor_advance(cursor, sizeof(*ip));

    key.dip = ip->dst;
    key.sip = ip->src;
    int *leaf = mytable.lookup(&key);
    if (leaf)
        *(leaf)++;
    return 0;
}
```

This code is amongst the examples.
I've been using cursor_advance() quite often and now I'm trying to figure out what exactly it does.
I suspect that cursor is a pointer where we save the address of the packet we are parsing.
Then, with cursor_advance() we move the cursor by the size of the ethernet header, since ethernet_t contains all the ethernet header information.
Then, the cursor now at the address at the end of the ethernet header of the packet and if we use variables declared in the ethernet_t header, such as type, like : ethernet->type, we can access the information saved at type since the struct ethernet would read the values saved in that address?
I'm sorry my explanation is not really good.
I'm just looking for a general explanation or if my theory is correct.
Thanks!

答:
Your understanding sounds correct to me. Just think of it as a “cursor” used to successively parse the different headers of your packet. The cursor_advance() macro is defined as:

#define cursor_advance(_cursor, _len) \
        ({ void *_tmp = _cursor; _cursor += _len; _tmp; })
```

It adds _len to the _cursor, and returns the value _cursor had before we added _len.

So the first call to cursor_advance() returns the initial value: ethernet points to the beginning of the packet, and we can use its attributes to access the different fields of the Ethernet header. But this same call also moves the cursor forwards by the length of the Ethernet header, so now it points to the beginning of the next header (L3, e.g. IP). The second call to cursor_advance() returns the pointer to the L3 layer, which we store in ip. The cursor is also moved forward and, assuming the packet is IPv4, would now point at the L4 header.

Note: I do not believe this mechanism is widely used in BPF programs aside from the few networking examples available in BCC. Instead, programs often navigate through packet headers with skb->data and skb->data_end.

Body:
I was looking through a slide by IOvisor project, https://events.static.linuxfound.org/sites/events/files/slides/iovisor-lc-bof-2016.pdf

#include <bcc/proto.h>

struct IPKey { u32 dip; u32 sip; };
BPF_TABLE("hash", struct IPKey, int, mytable, 1024);

int recv_packet(struct __sk_buff *skb) {
    struct IPKey key;
    u8 *cursor = 0;
    struct ethernet_t *ethernet = cursor_advance(cursor, sizeof(*ethernet));
    struct ip_t *ip = cursor_advance(cursor, sizeof(*ip));

    key.dip = ip->dst;
    key.sip = ip->src;
    int *leaf = mytable.lookup(&key);
    if (leaf)
        *(leaf)++;
    return 0;
}

This code is amongst the examples.
I've been using cursor_advance() quite often and now I'm trying to figure out what exactly it does.
I suspect that cursor is a pointer where we save the address of the packet we are parsing.
Then, with cursor_advance() we move the cursor by the size of the ethernet header, since ethernet_t contains all the ethernet header information.
Then, the cursor now at the address at the end of the ethernet header of the packet and if we use variables declared in the ethernet_t header, such as type, like : ethernet->type, we can access the information saved at type since the struct ethernet would read the values saved in that address?
I'm sorry my explanation is not really good.
I'm just looking for a general explanation or if my theory is correct.
Thanks!

Body:
Your understanding sounds correct to me. Just think of it as a “cursor” used to successively parse the different headers of your packet. The cursor_advance() macro is defined as:

#define cursor_advance(_cursor, _len) \
        ({ void *_tmp = _cursor; _cursor += _len; _tmp; })

It adds _len to the _cursor, and returns the value _cursor had before we added _len.

So the first call to cursor_advance() returns the initial value: ethernet points to the beginning of the packet, and we can use its attributes to access the different fields of the Ethernet header. But this same call also moves the cursor forwards by the length of the Ethernet header, so now it points to the beginning of the next header (L3, e.g. IP). The second call to cursor_advance() returns the pointer to the L3 layer, which we store in ip. The cursor is also moved forward and, assuming the packet is IPv4, would now point at the L4 header.

Note: I do not believe this mechanism is widely used in BPF programs aside from the few networking examples available in BCC. Instead, programs often navigate through packet headers with skb->data and skb->data_end.


问:
Problems getting next page when scraping with scrapy
I have a scrapy code which doesn't crawl pagination links and i'm stuck.

The source of the page is:

https://www.levenhuk.bg/katalog/teleskopi/?page=1

My code is:

```<code>

import scrapy

class TelescopesSpider(scrapy.Spider):
    name = 'telescopes'
    allowed_domains = ['https://www.levenhuk.bg/']
    start_urls = ['https://www.levenhuk.bg/katalog/teleskopi/?page=1']
    download_delay = 3

    def parse(self, response):
        for product in response.xpath('//div[@class="catalog-item"]'):
            yield {

                # 'name': product.xpath('.//span[@itemprop="name" and contains(text(), "Levenhuk")]/text()').get(),
                'name': product.xpath('.//span[@itemprop="name"]/text()').get(),
                # 'price': product.xpath('.//div[@class="price"]/span/text()').get(),
                'price': product.xpath('.//span[@itemprop="price"]/text()').re_first(r'[0-9]+,[0-9]+'),
                'short_discr': product.xpath('.//div[@class="opis-item"]/p/strong/text()').get()

            }

            next_page_url = response.xpath('//*[@class="pagesCount"][1]//@href').get()
            if next_page_url is not None:
                yield scrapy.Request(response.urljoin(next_page_url))

</code>```

答:
This is a little tricky since usually standard practice is to just check if there is a next page button on a loop until there isn't.

Here's an example since there is no next page button we can figure out the total page count. There will be a duplicate request to page1 though with this method its not the most ideal situation.

```<code>import scrapy

class TelescopesSpider(scrapy.Spider):
    name = 'telescopes'
    allowed_domains = ['https://www.levenhuk.bg/']
    start_urls = ['https://www.levenhuk.bg/katalog/teleskopi/?page=1']
    download_delay = 3

    def parse(self, response):
        total_pages = response.css('.pagesCount a::text')[-1].get()
        total_pages = int(total_pages)
        pages_str = str(total_pages)
        for i in range(1, total_pages):
            url = 'https://www.levenhuk.bg/katalog/teleskopi/?page={}'.format(pages_str)
            yield scrapy.Request(url, callback=self.parse_item, dont_filter=True)

    def parse_item(self, response):
        for product in response.xpath('//div[@class="catalog-item"]'):
           yield {
            'name': product.xpath('.//span[@itemprop="name"]/text()').get(),
            'price': product.xpath('.//span[@itemprop="price"]/text()').re_first(r'[0-9]+,[0-9]+'),
            'short_discr': product.xpath('.//div[@class="opis-item"]/p/strong/text()').get()
        }

</code>```

Another method of doing this would be to just look at how many pages there are and over ride your start_requests method as follows:

```<code>class TelescopesSpider(scrapy.Spider):
    name = 'telescopes'
    allowed_domains = ['https://www.levenhuk.bg/']
    start_urls = ['https://www.levenhuk.bg/katalog/teleskopi/?page={}']
    download_delay = 3

    def start_requests(self):
        for i in range(1, 14):
            yield scrapy.Request(self.start_urls[0].format(str(i)), callback=self.parse)
</code>```

Body:
I have a scrapy code which doesn't crawl pagination links and i'm stuck.

The source of the page is:

https://www.levenhuk.bg/katalog/teleskopi/?page=1

My code is:

import scrapy

class TelescopesSpider(scrapy.Spider):
    name = 'telescopes'
    allowed_domains = ['https://www.levenhuk.bg/']
    start_urls = ['https://www.levenhuk.bg/katalog/teleskopi/?page=1']
    download_delay = 3

    def parse(self, response):
        for product in response.xpath('//div[@class="catalog-item"]'):
            yield {

                # 'name': product.xpath('.//span[@itemprop="name" and contains(text(), "Levenhuk")]/text()').get(),
                'name': product.xpath('.//span[@itemprop="name"]/text()').get(),
                # 'price': product.xpath('.//div[@class="price"]/span/text()').get(),
                'price': product.xpath('.//span[@itemprop="price"]/text()').re_first(r'[0-9]+,[0-9]+'),
                'short_discr': product.xpath('.//div[@class="opis-item"]/p/strong/text()').get()

            }

            next_page_url = response.xpath('//*[@class="pagesCount"][1]//@href').get()
            if next_page_url is not None:
                yield scrapy.Request(response.urljoin(next_page_url))

Body:
This is a little tricky since usually standard practice is to just check if there is a next page button on a loop until there isn't.

Here's an example since there is no next page button we can figure out the total page count. There will be a duplicate request to page1 though with this method its not the most ideal situation.

import scrapy

class TelescopesSpider(scrapy.Spider):
    name = 'telescopes'
    allowed_domains = ['https://www.levenhuk.bg/']
    start_urls = ['https://www.levenhuk.bg/katalog/teleskopi/?page=1']
    download_delay = 3

    def parse(self, response):
        total_pages = response.css('.pagesCount a::text')[-1].get()
        total_pages = int(total_pages)
        pages_str = str(total_pages)
        for i in range(1, total_pages):
            url = 'https://www.levenhuk.bg/katalog/teleskopi/?page={}'.format(pages_str)
            yield scrapy.Request(url, callback=self.parse_item, dont_filter=True)

    def parse_item(self, response):
        for product in response.xpath('//div[@class="catalog-item"]'):
           yield {
            'name': product.xpath('.//span[@itemprop="name"]/text()').get(),
            'price': product.xpath('.//span[@itemprop="price"]/text()').re_first(r'[0-9]+,[0-9]+'),
            'short_discr': product.xpath('.//div[@class="opis-item"]/p/strong/text()').get()
        }

Another method of doing this would be to just look at how many pages there are and over ride your start_requests method as follows:

class TelescopesSpider(scrapy.Spider):
    name = 'telescopes'
    allowed_domains = ['https://www.levenhuk.bg/']
    start_urls = ['https://www.levenhuk.bg/katalog/teleskopi/?page={}']
    download_delay = 3

    def start_requests(self):
        for i in range(1, 14):
            yield scrapy.Request(self.start_urls[0].format(str(i)), callback=self.parse)


问:
Can you do this using jquery?
Is it possible to do this from within a class?

```<code>   $("#" + field).click(this.validate);
</code>```

So basically I want to pass a function of the object that should be executed whenever something is clicked. Also, if there are more than 1 instances of this object, then the correct instance (i.e the one which runs this code for the given field) should be executed.

答:
Yes it is possible.

10 mins ago was writting a snipet because i had just that problem

```<code>$("div.myalarms_delete").click(function(){
    var mid = this.id;
    $("li#"+mid).fadeOut("fast");
});
</code>```

the div.myalarms_delete also as the id I needed

OwnerDisplayName: Click Upvote

ContentLicense: CC BY-SA 2.5

Title: Can you do this using jquery?

Body:
Is it possible to do this from within a class?

   $("#" + field).click(this.validate);

So basically I want to pass a function of the object that should be executed whenever something is clicked. Also, if there are more than 1 instances of this object, then the correct instance (i.e the one which runs this code for the given field) should be executed.

Body:
Yes it is possible.

10 mins ago was writting a snipet because i had just that problem

$("div.myalarms_delete").click(function(){
    var mid = this.id;
    $("li#"+mid).fadeOut("fast");
});

the div.myalarms_delete also as the id I needed

OwnerDisplayName: fmsf


问:
Threading to allow cancelling a web request
I'm writing a web app that allows users to download large files over http web request. I need to give them the option to cancel the request, so I create a thread for the request. But, while the download is happening, I still can't get the cancel event to fire. What am I doing wrong?
Thanks for any input!</p>

```<code>public class downloadThread  {
           public int isResume;
           public void downloadImage()
            { }
        }
            protected void btnDownload_Click(object sender, EventArgs e)
            {       var x = new downloadThread();
                    x.isResume = 0;

                    tRequest = new Thread(new ThreadStart(x.downloadImage));
                    tRequest.Start();
                    while (tRequest.IsAlive)
                    {
                      DownloadImage(); //this is where the rest request happens
                    } }
            protected void btnCancelRequest_Click(object sender, EventArgs e)
            {
                if (tRequest != null && tRequest.IsAlive)
                {

                    tRequest.Abort();

                }
            }
</code>```

答:
Aborting a thread with thread.Abort is maybe not the way you want to do this.

How about an asynchronous web request in your DownloadImage method instead? (See http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.begingetresponse.aspx ). That way you can call the web request's .Abort method rather than aborting the thread.

Body:
I'm writing a web app that allows users to download large files over http web request. I need to give them the option to cancel the request, so I create a thread for the request. But, while the download is happening, I still can't get the cancel event to fire. What am I doing wrong?
Thanks for any input!

public class downloadThread  {
           public int isResume;
           public void downloadImage()
            { }
        }
            protected void btnDownload_Click(object sender, EventArgs e)
            {       var x = new downloadThread();
                    x.isResume = 0;

                    tRequest = new Thread(new ThreadStart(x.downloadImage));
                    tRequest.Start();
                    while (tRequest.IsAlive)
                    {
                      DownloadImage(); //this is where the rest request happens
                    } }
            protected void btnCancelRequest_Click(object sender, EventArgs e)
            {
                if (tRequest != null && tRequest.IsAlive)
                {

                    tRequest.Abort();

                }
            }


问:
Wpf, C# - Contextmenu in TabItem Header
I wanted to add a contextmenu to a tabitem. But it should only be opened by clicking on the header of the tab

```<code>if (!(View.tabContent.Items[View.tabContent.SelectedIndex] is TabItem tabItem))
            return;

        ContextMenu ctm = new ContextMenu();

        MenuItem m1 = new MenuItem();
        m1.Header = "a";
        m1.Click += M1_Click;

        MenuItem m2 = new MenuItem();
        m2.Header = "b";
        m2.Click += M2_Click;

        ctm.Items.Add(m1);
        ctm.Items.Add(m2);

        TabControl tbc = View.tabContent;

        tbc.ContextMenu = ctm;
</code>```

答:
Replace each TabItem's Header with a ContentControl that wraps the actual content for that header while also replacing the ContextMenu:

```<code><TabItem>
    <TabItem.Header>
        <ContentControl Content="FirstTab">
            <ContentControl.ContextMenu>
                <ContextMenu>
                    <MenuItem Header="Foo" />
                    <MenuItem Header="Bar" />
                    <MenuItem Header="Baz" />
                </ContextMenu>
            </ContentControl.ContextMenu>
        </ContentControl>
    </TabItem.Header>
    <Grid Background="CornflowerBlue" />
</TabItem>
</code>```

Obviously this can also be done entirely in code-behind, and if you only need the menus to  be dynamic then you can just declare an empty ContextMenu like so:

```<code><ContentControl Content="FirstTab">
    <ContentControl.ContextMenu>
        <ContextMenu x:Name="FirstMenu" />
    </ContentControl.ContextMenu>
</ContentControl>
</code>```

...and then populate that in code-behind:

```<code>FirstMenu.Items.Add(new MenuItem { Header = "Foo" });
FirstMenu.Items.Add(new MenuItem { Header = "Bar" });
FirstMenu.Items.Add(new MenuItem { Header = "Baz" });
</code>```

Title: Wpf, C# - Contextmenu in TabItem Header

Body:
I wanted to add a contextmenu to a tabitem. But it should only be opened by clicking on the header of the tab

if (!(View.tabContent.Items[View.tabContent.SelectedIndex] is TabItem tabItem))
            return;

        ContextMenu ctm = new ContextMenu();

        MenuItem m1 = new MenuItem();
        m1.Header = "a";
        m1.Click += M1_Click;

        MenuItem m2 = new MenuItem();
        m2.Header = "b";
        m2.Click += M2_Click;

        ctm.Items.Add(m1);
        ctm.Items.Add(m2);

        TabControl tbc = View.tabContent;

        tbc.ContextMenu = ctm;

Body:
Replace each TabItem's Header with a ContentControl that wraps the actual content for that header while also replacing the ContextMenu:

<TabItem>
    <TabItem.Header>
        <ContentControl Content="FirstTab">
            <ContentControl.ContextMenu>
                <ContextMenu>
                    <MenuItem Header="Foo" />
                    <MenuItem Header="Bar" />
                    <MenuItem Header="Baz" />
                </ContextMenu>
            </ContentControl.ContextMenu>
        </ContentControl>
    </TabItem.Header>
    <Grid Background="CornflowerBlue" />
</TabItem>

Obviously this can also be done entirely in code-behind, and if you only need the menus to  be dynamic then you can just declare an empty ContextMenu like so:

<ContentControl Content="FirstTab">
    <ContentControl.ContextMenu>
        <ContextMenu x:Name="FirstMenu" />
    </ContentControl.ContextMenu>
</ContentControl>

...and then populate that in code-behind:

FirstMenu.Items.Add(new MenuItem { Header = "Foo" });
FirstMenu.Items.Add(new MenuItem { Header = "Bar" });
FirstMenu.Items.Add(new MenuItem { Header = "Baz" });


问:
CNN based regression with words input
I successfully trained a CNN with regression output layer to predict a numerical indicator of the image. Further, I would like to encode the description of each image into CNN to get better regression performance.

My data are a series of x-ray images in 512 x 512. Each image has a description recording how the image was taken. For example, the description of image #1 is

```<code>Voltage: 40V
Current: 2A
Shuttle speed: 1 sec
Filter material: rhodium
</code>```

the description of image #2 is

```<code>Voltage: 38V
Current: 1.7A
Shuttle speed: 0.7 sec
Filter material: tungsten
</code>```

the description of other images are very similar, just with different parameters.

My question is: how can I encode these description into CNN?

答:
If you have only dozens of possible values to the attributes, I would suggest learning embeddings for each of the attribute-value pairs and concatenate them with the CNN outputs and perhaps add one hidden layer on top of that so the textual and image features can more interact with each other.

If the texts are not so well structured, I would process the text with an RNN (i.e., tokenize the text, embed the tokens, feed them into an RNN and take the final state). This will give you a single vector for the text description that you can again concatenate to the CNN output.

Title: CNN based regression with words input

Body:
I successfully trained a CNN with regression output layer to predict a numerical indicator of the image. Further, I would like to encode the description of each image into CNN to get better regression performance.

My data are a series of x-ray images in 512 x 512. Each image has a description recording how the image was taken. For example, the description of image #1 is

Voltage: 40V
Current: 2A
Shuttle speed: 1 sec
Filter material: rhodium

the description of image #2 is

Voltage: 38V
Current: 1.7A
Shuttle speed: 0.7 sec
Filter material: tungsten

the description of other images are very similar, just with different parameters.

My question is: how can I encode these description into CNN?


问:
PhoneGap Direct user to safari
I am developing a iphone app using phoneGap. How can I direct user to safari when he clicks on a link or button inside of the application.

example:
 When user taps a button, safari browser will come to front and load corresponding site.

答:
This line just worked for me..</p>

```<code> location.href="http://www.google.com";
</code>```

Title: PhoneGap Direct user to safari

Body:
This line just worked for me..

 location.href="http://www.google.com";


问:
Copy parent query into another variable and add more conditions to it
I declared a variable which is basically a select query, and I need to copy this parent variable into another one, and then add some where conditions to this copied variable. The thing is that my where conditions are being applied to the first variable and not the one I'd like to.</p>

```<code>//main query
$query = News::select('news.id','news.created_at','news.idCategory')->with('newsTranslations');

if ($request->has('categories')) {
    $query = $query->whereIn('news.idCategory', $input['categories']);
}

if ($request->has('jobCategories')) {
    $query = $query->join('jobCategories_News','news.id','=','jobCategories_News.idNews')
    ->whereIn('jobCategories_News.idjobCategories', $input['jobCategories']);
}
$chartquery = $query; // copy of parent query variable
$chartquery= $chartquery->leftJoin('newsInteractions','news.id','=','newsInteractions.idNews');

//subqueries (applied to $query instead of $chartquery)
$vis_like = $chartquery->where('liked','=',1)->where('visualizationDate','!=', NULL)->count();
$no_vis_like= $chartquery->where('liked','=',1)->where('visualizationDate','=', NULL)->count();
$dislike = $chartquery->where('liked','=',0)->count();
$vis_no_like= $chartquery->where('liked','=',NULL)->where('visualizationDate','!=', NULL)->count();

</code>```

答:
The solution is:

```<code>  $query = News::select('news.id','news.created_at','news.idCategory')- >with('newsTranslations');

    if ($request->has('categories')) {
        $query = $query->whereIn('news.idCategory', $input['categories']);
    }

    if ($request->has('jobCategories')) {
        $query = $query->join('jobCategories_News','news.id','=','jobCategories_News.idNews')
        ->whereIn('jobCategories_News.idjobCategories', $input['jobCategories']);
    }

    $chartquery = clone $query;
    $chartquery= $chartquery->leftJoin('newsInteractions','news.id','=','newsInteractions.idNews');

    $vis_like = clone $query;
    $vis_like = $vis_like->where('liked','=',1)->where('visualizationDate','!=', NULL)->count();
</code>```

Thanks u guys!

Body:
I declared a variable which is basically a select query, and I need to copy this parent variable into another one, and then add some where conditions to this copied variable. The thing is that my where conditions are being applied to the first variable and not the one I'd like to.

//main query
$query = News::select('news.id','news.created_at','news.idCategory')->with('newsTranslations');

if ($request->has('categories')) {
    $query = $query->whereIn('news.idCategory', $input['categories']);
}

if ($request->has('jobCategories')) {
    $query = $query->join('jobCategories_News','news.id','=','jobCategories_News.idNews')
    ->whereIn('jobCategories_News.idjobCategories', $input['jobCategories']);
}
$chartquery = $query; // copy of parent query variable
$chartquery= $chartquery->leftJoin('newsInteractions','news.id','=','newsInteractions.idNews');

//subqueries (applied to $query instead of $chartquery)
$vis_like = $chartquery->where('liked','=',1)->where('visualizationDate','!=', NULL)->count();
$no_vis_like= $chartquery->where('liked','=',1)->where('visualizationDate','=', NULL)->count();
$dislike = $chartquery->where('liked','=',0)->count();
$vis_no_like= $chartquery->where('liked','=',NULL)->where('visualizationDate','!=', NULL)->count();

Body:
The solution is:

  $query = News::select('news.id','news.created_at','news.idCategory')- >with('newsTranslations');

    if ($request->has('categories')) {
        $query = $query->whereIn('news.idCategory', $input['categories']);
    }

    if ($request->has('jobCategories')) {
        $query = $query->join('jobCategories_News','news.id','=','jobCategories_News.idNews')
        ->whereIn('jobCategories_News.idjobCategories', $input['jobCategories']);
    }

    $chartquery = clone $query;
    $chartquery= $chartquery->leftJoin('newsInteractions','news.id','=','newsInteractions.idNews');

    $vis_like = clone $query;
    $vis_like = $vis_like->where('liked','=',1)->where('visualizationDate','!=', NULL)->count();

Thanks u guys!


问:
Markdown in iPython Notebook: how to highlight code?
In Markdown, how do you highlight code <--- just like that. I don't know how to search for it because I'm not sure what it's called. I also don't know what the little dashes are called either. I tried doing what I would do in SO, but it just reads it as normal text

Update:

This is what I have:

foo in SO it actually shows the hilighting

bar whereas in iPython Notebook, it doesn't, it only changes the fontstyle

答:
In short: Use <code> tag

In normal cases backticks work for highlighting text but are printed as literal characters when inside html lists.

A part of tensorflow tutorial written using markdown

```<code>Training the neural network model requires the following steps:

<ol>
    <li>Feed the training data into the model. In this example, the training data is in the <code>train_images</code> and <code>train_labels</code> arrays.</li>
    <li>The model learns to associate the images and labels.</li>
    <li>You ask the model to make predictions about a test set -- in this example the test_images array.</li>
    <li>Verify the predictions match the labels from the <code>test_labels</code> array.</li>
</ol>
</code>```

Output

(As seen in jupyter notebook)

Training the neural network model requires the following steps:

Feed the training data into the model. In this example, the training data is in the train_images and train_labels arrays.

The model learns to associate the images and labels.

You ask the model to make predictions about a test set -- in this example the test_images array.

Verify the predictions match the labels from the test_labels array.

Body:
In short: Use <code> tag

In normal cases backticks work for highlighting text but are printed as literal characters when inside html lists.

A part of tensorflow tutorial written using markdown

Training the neural network model requires the following steps:

<ol>
    <li>Feed the training data into the model. In this example, the training data is in the <code>train_images</code> and <code>train_labels</code> arrays.</li>
    <li>The model learns to associate the images and labels.</li>
    <li>You ask the model to make predictions about a test set -- in this example the test_images array.</li>
    <li>Verify the predictions match the labels from the <code>test_labels</code> array.</li>
</ol>

Output

(As seen in jupyter notebook)

Training the neural network model requires the following steps:

Feed the training data into the model. In this example, the training data is in the train_images and train_labels arrays.

The model learns to associate the images and labels.

You ask the model to make predictions about a test set -- in this example the test_images array.

Verify the predictions match the labels from the test_labels array.


问:
run php code with javascript code using crontab in linux
I am trying to a php code which contain JavaScript code using phantomjs and firefox browser into crontab. But it's not working.If i run using CURL then it's run but js code isn't work.My crontab command below.

```<code>* * * * * /usr/local/bin/phantomjs http://XX.XX.XXX.XXX/project/index.php > /dev/null 2>&1

* * * * * /usr/bin/firefox http://XX.XX.XXX.XXX/project/index.php > /dev/null 2>&1
</code>```

Someone tell me can i run php and js code into crontab?If yes then what is easier way?

答:
As PhantomJS, which is a JavaScript based automation framework available for Windows, Mac OS X, Linux, other *ix systems.

Using PhantomJS, you can do things like this:

```<code>    console.log('Loading a web page');

    var page = new WebPage();
    var url = "http://www.phantomjs.org/";

    page.open(url, function (status) {
        // perform your task once the page is ready ...
        phantom.exit();
    });
</code>```

Or evaluate a page's title:

```<code>    var page = require('webpage').create();
    page.open(url, function (status) {
        var title = page.evaluate(function () {
            return document.title;
        });
        console.log('Page title is ' + title);
    });

</code>```

Examples from PhantomJS' Quickstart page. You can even render a page to a PNG, JPEG or PDF using the render() method.

so in your crontab

```<code>* * * * * /usr/local/bin/phantomjs /home/user/custom.js > /dev/null 2>&1

</code>```

Body:
I am trying to a php code which contain JavaScript code using phantomjs and firefox browser into crontab. But it's not working.If i run using CURL then it's run but js code isn't work.My crontab command below.

* * * * * /usr/local/bin/phantomjs http://XX.XX.XXX.XXX/project/index.php > /dev/null 2>&1

* * * * * /usr/bin/firefox http://XX.XX.XXX.XXX/project/index.php > /dev/null 2>&1

Someone tell me can i run php and js code into crontab?If yes then what is easier way?

Body:
As PhantomJS, which is a JavaScript based automation framework available for Windows, Mac OS X, Linux, other *ix systems.

Using PhantomJS, you can do things like this:

    console.log('Loading a web page');

    var page = new WebPage();
    var url = "http://www.phantomjs.org/";

    page.open(url, function (status) {
        // perform your task once the page is ready ...
        phantom.exit();
    });

Or evaluate a page's title:

    var page = require('webpage').create();
    page.open(url, function (status) {
        var title = page.evaluate(function () {
            return document.title;
        });
        console.log('Page title is ' + title);
    });

Examples from PhantomJS' Quickstart page. You can even render a page to a PNG, JPEG or PDF using the render() method.

so in your crontab

* * * * * /usr/local/bin/phantomjs /home/user/custom.js > /dev/null 2>&1


问:
Split the string '1/10/2/20/3/30/4/40' into two columns check body for more details
```<code>string = '1/10/2/20/3/30/4/40'
output = col1  |   col2
           1   |    10
           2   |    20
           3   |    30
           4   |    40
</code>```

I tried below code:

```<code>DECLARE @TEXT VARCHAR(60) = '1/10/2/20/3/30/4/40', @POSITION INT=1
CREATE TABLE #TEMP(VAL1 INT,VAL2 INT)
WHILE (LEN(@TEXT) >= @POSITION)
BEGIN
  INSERT INTO #TEMP VALUES(SUBSTRING(@TEXT,@POSITION,1),SUBSTRING(@TEXT,@POSITION+2,2))
  SET @POSITION = @POSITION+5
END

SELECT * FROM #TEMP
</code>```

this is giving the output but this is more like hard coded.

答:
Next time please state your SQL-Server's version...

The following solution works with almost any version, if you are at v2016+ there might be better approaches. Try this out:

```<code>DECLARE @TEXT VARCHAR(60) = '1/10/2/20/3/30/4/40';

WITH Casted(ToXml) AS
(
    SELECT CAST('<x>' + REPLACE(@TEXT,'/','</x><x>') + '</x>' AS XML)
)
,TallyOddNumbers(OddNumber) AS
(
    SELECT TOP((SELECT ToXml.value('count(/x)','int')/2 FROM Casted)) (ROW_NUMBER() OVER(ORDER BY (SELECT NULL)))*2-1 FROM master..spt_values
)
SELECT ToXml.value('/x[sql:column("OddNumber")][1]','int') AS Col1
      ,ToXml.value('/x[sql:column("OddNumber")+1][1]','int') AS Col1
FROM Casted
CROSS JOIN TallyOddNumbers;
</code>```

The idea in short:

The first cte "Casted" will create an XML from your string.

The second cte "TallyOddNumbers" will use ROW_NUMBER() against master..spt_values (just any larger set is okay here) to create a set of running odd numbers

The computed TOP clause will use XML.value() with count() to find the needed number of element pairs.

The final query will use the tally to grab the values by their positions.

Hint: You can use the code above simply together with INSERT in order to get the set in a persistant table.

UPDATE  for version 2016+

In your own answer you are using string_split() (which is not a good idea due to the non-guaranteed sort order). But this points to a version of v2016+.

You might try this alternativ approach

```<code>DECLARE @TEXT VARCHAR(60) = '1/10/2/20/3/30/4/40';

WITH ToJson(j) AS(SELECT CONCAT('[',REPLACE(@TEXT,'/',','),']'))
SELECT p.Col1
      ,p.Col2
FROM
(
    SELECT A.[key] /2 AS GroupIndex
          ,CONCAT('Col',A.[key]%2 +1) AS ColumnName
          ,A.[value] AS TheValue
    FROM ToJson
    CROSS APPLY OPENJSON(j) A
) t
PIVOT
(
    MAX(TheValue) FOR ColumnName IN(Col1,Col2)
) p;
</code>```

The idea in short:

The cte will transform your CSV-list to a JSON array

We use OPENJSON to read this array

the key is the elements position. Together with %2 (modulo operator) we get a computed, alternating column name

Finally we can use PIVOT to spread your values into Col1 and Col2.

The A.[key] /2 will use a trick with integer divisions to create one number per group (otherwise you would see just to last row).

Body:
string = '1/10/2/20/3/30/4/40'
output = col1  |   col2
           1   |    10
           2   |    20
           3   |    30
           4   |    40

I tried below code:

DECLARE @TEXT VARCHAR(60) = '1/10/2/20/3/30/4/40', @POSITION INT=1
CREATE TABLE #TEMP(VAL1 INT,VAL2 INT)
WHILE (LEN(@TEXT) >= @POSITION)
BEGIN
  INSERT INTO #TEMP VALUES(SUBSTRING(@TEXT,@POSITION,1),SUBSTRING(@TEXT,@POSITION+2,2))
  SET @POSITION = @POSITION+5
END

SELECT * FROM #TEMP

this is giving the output but this is more like hard coded.

Body:
Next time please state your SQL-Server's version...

The following solution works with almost any version, if you are at v2016+ there might be better approaches. Try this out:

DECLARE @TEXT VARCHAR(60) = '1/10/2/20/3/30/4/40';

WITH Casted(ToXml) AS
(
    SELECT CAST('<x>' + REPLACE(@TEXT,'/','</x><x>') + '</x>' AS XML)
)
,TallyOddNumbers(OddNumber) AS
(
    SELECT TOP((SELECT ToXml.value('count(/x)','int')/2 FROM Casted)) (ROW_NUMBER() OVER(ORDER BY (SELECT NULL)))*2-1 FROM master..spt_values
)
SELECT ToXml.value('/x[sql:column("OddNumber")][1]','int') AS Col1
      ,ToXml.value('/x[sql:column("OddNumber")+1][1]','int') AS Col1
FROM Casted
CROSS JOIN TallyOddNumbers;

The idea in short:

The first cte "Casted" will create an XML from your string.

The second cte "TallyOddNumbers" will use ROW_NUMBER() against master..spt_values (just any larger set is okay here) to create a set of running odd numbers

The computed TOP clause will use XML.value() with count() to find the needed number of element pairs.

The final query will use the tally to grab the values by their positions.

Hint: You can use the code above simply together with INSERT in order to get the set in a persistant table.

UPDATE  for version 2016+

In your own answer you are using string_split() (which is not a good idea due to the non-guaranteed sort order). But this points to a version of v2016+.

You might try this alternativ approach

DECLARE @TEXT VARCHAR(60) = '1/10/2/20/3/30/4/40';

WITH ToJson(j) AS(SELECT CONCAT('[',REPLACE(@TEXT,'/',','),']'))
SELECT p.Col1
      ,p.Col2
FROM
(
    SELECT A.[key] /2 AS GroupIndex
          ,CONCAT('Col',A.[key]%2 +1) AS ColumnName
          ,A.[value] AS TheValue
    FROM ToJson
    CROSS APPLY OPENJSON(j) A
) t
PIVOT
(
    MAX(TheValue) FOR ColumnName IN(Col1,Col2)
) p;

The idea in short:

The cte will transform your CSV-list to a JSON array

We use OPENJSON to read this array

the key is the elements position. Together with %2 (modulo operator) we get a computed, alternating column name

Finally we can use PIVOT to spread your values into Col1 and Col2.

The A.[key] /2 will use a trick with integer divisions to create one number per group (otherwise you would see just to last row).


问:
How to replace images and files at build without referencing them in the app?
We are rewriting our existing Angular 8 PWA to Vuejs and we are facing a few issues concerning the build process.
(We are not here to discuss the reasons of that choice nor anyone's opinion on it)

Our app must be built for different customers with specific assets for each customers (manifest.json file, icons and images).

In Angular, we accomplish file replacements by defining rules in angular.json file using the fileReplacements key like this:

```<code>"configurations": {
        "production-customer-a": {
          "fileReplacements": [
            {
              "replace": "src/manifest.json",
              "with": "src/environments/customer-a/assets/manifest.json"
            },
          ],
          // Some more json stuff
</code>```

We tried to implement a similar process in our vue app in vue.config.js using webpack's NormalModuleReplacementPlugin like this:

```<code>configureWebpack: () => {
let plugins;
switch (customer) {
  case customer-a:
    plugins = [
      new NormalModuleReplacementPlugin(
        /\.\.\/public\/manifest\.json/,
        "../environments/customer-a/assets/manifest.json"
      ),
     // Some more config
</code>```

It works great if the concerned files are actually referenced in the app using its url at build.
Let me explain with an example:

```<code><img src="@/assets/myImageWhichShouldBeReplacedAtBuild.png" /> /* Will actually be replaced and the dist folder will contain the file properly */
<img src="@/assets/myImageWhichShouldBeReplacedAtBuildButIsBase64EncodedWithWebpackUrlLoader.png" /> /* Will be replaced as the base64 string but the dist folder wont contain the file and might be an issue if the image is to be used by files like manifest.json */
</code>```

The later is not a complete issue since we can override webpack's url-loader in vue.config.js like explained here. But still a good catch to know about.

Since we don't reference files like manifest.json or favicon.ico in the Vue templates, those files are not replaced.

Is there anyway to make sure every file we need will be replaced at build to obtain the same result as with Angular's angular.json config file ? Maybe using something different than NormalModuleReplacementPlugin ?

答:
So for any lost soul that comes around here, we managed to find a proper solution.

We got rid of webpack's NormalModuleReplacementPlugin and worked in combination with a task runner (gulp, in our case) and Vue static assets handling.

Since the concerned assets where always the same, we put the default ones in the public folder of the solution and recreated the same structure in our environments folders for each customer.

```<code>-public
--img
---icons

-environments
--customer
---img
----icons
</code>```

Making use of Vue's URL transform rule, we referenced all our assets in the app with an absolute path, so the files won't go through webpack's processing.
After the build, we run our gulpfile that takes all the customer's assets (passed as argument) and override them in the dist folder.

gulpfile.js:

```<code>const gulp = require('gulp');
const yargs = require('yargs').argv;

gulp.task('moveassets', () => {
  return gulp.src(`environments/${yargs.customer}/**/**/**`)
    .pipe(gulp.dest('dist'));
})
</code>```

package.json script:

```<code>"build:customer": "vue-cli-service build && gulp moveassets --customer customer"
</code>```

It might cause a bit of trouble with cache handling since webpack would have put hashes to files, but Angular is not doing it also so it won't change our habits.

Body:
We are rewriting our existing Angular 8 PWA to Vuejs and we are facing a few issues concerning the build process.
(We are not here to discuss the reasons of that choice nor anyone's opinion on it)

Our app must be built for different customers with specific assets for each customers (manifest.json file, icons and images).

In Angular, we accomplish file replacements by defining rules in angular.json file using the fileReplacements key like this:

"configurations": {
        "production-customer-a": {
          "fileReplacements": [
            {
              "replace": "src/manifest.json",
              "with": "src/environments/customer-a/assets/manifest.json"
            },
          ],
          // Some more json stuff

We tried to implement a similar process in our vue app in vue.config.js using webpack's NormalModuleReplacementPlugin like this:

configureWebpack: () => {
let plugins;
switch (customer) {
  case customer-a:
    plugins = [
      new NormalModuleReplacementPlugin(
        /\.\.\/public\/manifest\.json/,
        "../environments/customer-a/assets/manifest.json"
      ),
     // Some more config

It works great if the concerned files are actually referenced in the app using its url at build.
Let me explain with an example:

 /* Will actually be replaced and the dist folder will contain the file properly */
 /* Will be replaced as the base64 string but the dist folder wont contain the file and might be an issue if the image is to be used by files like manifest.json */

The later is not a complete issue since we can override webpack's url-loader in vue.config.js like explained here. But still a good catch to know about.

Since we don't reference files like manifest.json or favicon.ico in the Vue templates, those files are not replaced.

Is there anyway to make sure every file we need will be replaced at build to obtain the same result as with Angular's angular.json config file ? Maybe using something different than NormalModuleReplacementPlugin ?

Body:
So for any lost soul that comes around here, we managed to find a proper solution.

We got rid of webpack's NormalModuleReplacementPlugin and worked in combination with a task runner (gulp, in our case) and Vue static assets handling.

Since the concerned assets where always the same, we put the default ones in the public folder of the solution and recreated the same structure in our environments folders for each customer.

-public
--img
---icons

-environments
--customer
---img
----icons

Making use of Vue's URL transform rule, we referenced all our assets in the app with an absolute path, so the files won't go through webpack's processing.
After the build, we run our gulpfile that takes all the customer's assets (passed as argument) and override them in the dist folder.

gulpfile.js:

const gulp = require('gulp');
const yargs = require('yargs').argv;

gulp.task('moveassets', () => {
  return gulp.src(`environments/${yargs.customer}/**/**/**`)
    .pipe(gulp.dest('dist'));
})

package.json script:

"build:customer": "vue-cli-service build && gulp moveassets --customer customer"

It might cause a bit of trouble with cache handling since webpack would have put hashes to files, but Angular is not doing it also so it won't change our habits.


问:
How to search and modify floating values that follow a string in a file using Perl
Lets say we have "abc {{1.1 2.2}}" occurring multiple times in a file and I want to modify it to "xyz {{2.2 4.4}}" which is basically:

Change abc to xyz.

And multiply the floating values by 2 that appear in the curly braces (always preceded by abc keyword).

```<code>abc {{1.1 2.2}} ----------> xyz {{2.2 4.4}}
</code>```

This modification must be done to all instances of the file.
Here's the code that I tried but was able to replace only abc to xyz and couldn't figure out how to do the operations by extracting those floating values.

```<code>#!/usr/bin/perl -w

use strict;

open(FILE, "</tmp/yourfile.txt") || die "File not found";
my @lines = <FILE>;
close(FILE);

foreach(@lines) {
   $_ =~ s/abc/xyz/g;
}

open(FILE, ">/tmp/yourfile.txt") || die "File not found";
print FILE @lines;
close(FILE);
</code>```

Any sort of help is highly appreciated.

答:
I presume you'd want to keep the number of decimals (and/or control it), and that the floats may need be manipulated in ways more complex than mere *2

```<code>perl -wE' $_=q(abc {{0.2 14.55}});
    say;
    s{ abc \s+ \{\s*\{\s* \K
       ([0-9]+\.([0-9]+)) (\s+) ([0-9]+\.([0-9]+))  (?=\s*\}\s*\})
    }{
       ($l2,$l5) = map length, $2, $5;
       sprintf("%.${l2}f", 2*$1) .$3. sprintf("%.${l5}f", 2*$4)
    }ex;
    say'
</code>```

This makes some basic assumptions.  It prints

```
abc {{0.2 14.55}}
abc {{0.4 29.10}}
```

Comments

The \K makes it drop all matches up to that point; they need not be captured and re-entered as replacement as they haven't been "consumed."  So we match that first part and leave it alone

The }e modifier makes it evaluate the replacement side as Perl code. There we format numbers resulting from multiplication with the number of decimal places of the originals

The bit in the end of the matching part, (?=...), is a lookahead

If numbers may come with -/+ then these characters need be added to the pattern

```<code>([-+]?[0-9]+\.([0-9]+))
</code>```

If that + need be reproduced in output as well then you also need to capture the sign ([-+]?), so to be able to check in the replacement side and add it (if it was there).

Body:
Lets say we have "abc {{1.1 2.2}}" occurring multiple times in a file and I want to modify it to "xyz {{2.2 4.4}}" which is basically:

Change abc to xyz.

And multiply the floating values by 2 that appear in the curly braces (always preceded by abc keyword).

abc {{1.1 2.2}} ----------> xyz {{2.2 4.4}}

This modification must be done to all instances of the file.
Here's the code that I tried but was able to replace only abc to xyz and couldn't figure out how to do the operations by extracting those floating values.

#!/usr/bin/perl -w

use strict;

open(FILE, "</tmp/yourfile.txt") || die "File not found";
my @lines = <FILE>;
close(FILE);

foreach(@lines) {
   $_ =~ s/abc/xyz/g;
}

open(FILE, ">/tmp/yourfile.txt") || die "File not found";
print FILE @lines;
close(FILE);

Any sort of help is highly appreciated.

Body:
I presume you'd want to keep the number of decimals (and/or control it), and that the floats may need be manipulated in ways more complex than mere *2

perl -wE' $_=q(abc {{0.2 14.55}});
    say;
    s{ abc \s+ \{\s*\{\s* \K
       ([0-9]+\.([0-9]+)) (\s+) ([0-9]+\.([0-9]+))  (?=\s*\}\s*\})
    }{
       ($l2,$l5) = map length, $2, $5;
       sprintf("%.${l2}f", 2*$1) .$3. sprintf("%.${l5}f", 2*$4)
    }ex;
    say'

This makes some basic assumptions.  It prints

abc {{0.2 14.55}}
abc {{0.4 29.10}}

Comments

The \K makes it drop all matches up to that point; they need not be captured and re-entered as replacement as they haven't been "consumed."  So we match that first part and leave it alone

The }e modifier makes it evaluate the replacement side as Perl code. There we format numbers resulting from multiplication with the number of decimal places of the originals

The bit in the end of the matching part, (?=...), is a lookahead

If numbers may come with -/+ then these characters need be added to the pattern

([-+]?[0-9]+\.([0-9]+))

If that + need be reproduced in output as well then you also need to capture the sign ([-+]?), so to be able to check in the replacement side and add it (if it was there).


问:
How to include a .js file in PL/SQL on Oracle10gXE
again. I'm making a PL/SQL generated HTML5 web page. It's running a Oracle 10g XE server. Okay, now when the setup is clear, my problem - I need to include a Java Script file in the page. Simply

```<code>    HTP.P('<script type="text/javascript" src="js/ScriptFileName.js"></script>');
</code>```

Doesn't work of course. So i created a folder object and granted read,write to PUBLIC. Then changed the string to match the newly created object, instead of path. Still doesn't work. I know, i can write

```<code>    HTP.P(<script type="text/javascript"> MY JAVA SCRIPT HERE</script>);
</code>```

And i've done so with other scripts(Even had to write CSS this way). But this time this will not work. Reason being - the JavaScript i'm trying to run was normalized(or rather unnormalized), so it's written all in one line. And there is a lot of it too. I tried to reverse it to normal, but faild many a time.

So, I went online and searched for a solution. Found one. It seem's that this include should go not to the page, but to server config. Makes sense, since PL/SQL is server sided. But when i went looking for the usual httpd.conf, it's nowhere to be found in Database directory.So i went online again, result - NOT A WORD OF WHERE THE HELL ARE HTTP SERVER CONFIGS IN 10gXE IN ANY ORACLE MANUALS. Searched some forums - exactly 1 person asked where httpd.conf in XE is, and didn't get an answer. Please, help. I'm desperate.

P.S. I don't use APEX. I don't get that mumbo-jumbo. So i write in Notepad and run the scripts in SQL line.

答:
Firstly, XE has its own built in HTTP server called the 'Embedded PL/SQL Gateway' or EPG. But you don't HAVE to use that. You can use an Oracle HTTP Server with the mod_plsql plugin. Or you can use the Apex listener.

The question is on what server is "ScriptFileName.js" ?

Is it a flat file on the database server ? If so, you'll need to use the Oracle HTTP Server (or Apache or similar) to serve it. The database is pretty much unaware of files on its server and the EPG can't deliver them. [At least not in any practical sense, you could do weird things with chicken entrails and UTL_FILE, but you don't want to go there.]

Is it a file stored in the database ? That sounds exotic, but it is pretty much how all the CSS, images etc are served up through the EPG. The best explanation on how to get files in and out of there is by Dietmar

Is it a file stored on a separate machine ? Often the best answer. The "src=" directive will be read by the end users browser. That will do an HTTP get to the URL. It doesn't have to be a URL on the same domain/host as the rest of the page.

Body:
again. I'm making a PL/SQL generated HTML5 web page. It's running a Oracle 10g XE server. Okay, now when the setup is clear, my problem - I need to include a Java Script file in the page. Simply

    HTP.P('');

Doesn't work of course. So i created a folder object and granted read,write to PUBLIC. Then changed the string to match the newly created object, instead of path. Still doesn't work. I know, i can write

    HTP.P();

And i've done so with other scripts(Even had to write CSS this way). But this time this will not work. Reason being - the JavaScript i'm trying to run was normalized(or rather unnormalized), so it's written all in one line. And there is a lot of it too. I tried to reverse it to normal, but faild many a time.

So, I went online and searched for a solution. Found one. It seem's that this include should go not to the page, but to server config. Makes sense, since PL/SQL is server sided. But when i went looking for the usual httpd.conf, it's nowhere to be found in Database directory.So i went online again, result - NOT A WORD OF WHERE THE HELL ARE HTTP SERVER CONFIGS IN 10gXE IN ANY ORACLE MANUALS. Searched some forums - exactly 1 person asked where httpd.conf in XE is, and didn't get an answer. Please, help. I'm desperate.

P.S. I don't use APEX. I don't get that mumbo-jumbo. So i write in Notepad and run the scripts in SQL line.


问:
How to upload a database on Kiwi TCMS from json file?
I have kiwi TCMS on docker through docker-compose.yml. And i have file json with database (old version of kiwi)
To dump database from old serwer i use:

docker exec -it kiwi_web /Kiwi/manage.py dumpdata --all --indent 2 > database.json.

In new kiwi TCMS i want use my old database, but after enter this command

docker exec -it kiwi_web /bin/bash -c '/Kiwi/manage.py sqlflush | /Kiwi/manage.py dbshell'

i get:

CommandError: You appear not to have the 'mysql' program installed or on your path.
Exception ignored in: <_io.TextIOWrapper name='<stdout>' mode='w' encoding='UTF-8'>
BrokenPipeError: [Errno 32] Broken pipe

What should I do?
I use instructions from: kiwi blog

答:
mysql is indeed missing inside the web docker image (this is a side effect from the build process, before this binary was included in the image).

You can work around by docker exec -u 0 -it kiwi_web /bin/bash -> this will give you root privileges inside the container. Once you are inside "yum install mariadb" or "yum install mysql".

Without stopping the container try removing all data from all tables and restoring the json data.


问:
Is there a function in C to find a substring overlap?
in C, is there a function that when giver two strings, will return the substring overlap or size of the overlap? So like something that does:

```<code>char s1[5] = {cart};
char s2[4] = {car};
int overlap;
overlap = get_overlap(s1, s2); /*or have overlap be a string if it returns the overlap*.
</code>```

and then overlap would be 3.

If not, how do i make one that will return the int value of the overlap.

答:
There's no builtin function, but it's pretty simple to write:</p>

```<code>size_t overlap(const char *s1, const char *s2)
{
    size_t i = 0;

    while (s1[i] && s2[i] && s1[i] == s2[i])
        i++;

    return i;
}
</code>```

Body:
in C, is there a function that when giver two strings, will return the substring overlap or size of the overlap? So like something that does:

char s1[5] = {cart};
char s2[4] = {car};
int overlap;
overlap = get_overlap(s1, s2); /*or have overlap be a string if it returns the overlap*.

and then overlap would be 3.

If not, how do i make one that will return the int value of the overlap.

Body:
There's no builtin function, but it's pretty simple to write:

size_t overlap(const char *s1, const char *s2)
{
    size_t i = 0;

    while (s1[i] && s2[i] && s1[i] == s2[i])
        i++;

    return i;
}


问:
Implementing SCD type2 in Data Fusion
Am trying to implement type2 in Datafusion.Can someone help in performing insert and updating by using the pipeline transformations/action/conditions to achieve this. I was trying to generate hash using Wrangler for both source and Target and join them. I was struck here ,not sure how to determine which goes to update/insert and how to handle those.My Target is BigQuery tables.</p>

答:
Welcome on Stack overflow @Manasa!

Once you are considering Bigquery as a warehouse for the target sink, leveraging SCD type2 merging method, you might have first to get the practice way how to implement it composing particular SQL statement, reaching the desired logic. Saying this, I encourage you to study this great article written by @Hariharan Ramachandran, explaining SCD techniques with some hand on examples using Bigquery service as a destination data holder.

According to Data Fusion documentation, you can find a rich set of Plugins, extending standard data processing workflows.Therefore, in this particular case I would recommend implementing BigQuery Execute plugin which can be adjusted in your pipeline, executing SQL queries against Bigquery managed service and perform some SCD2 specific actions.

Title: Implementing SCD type2 in Data Fusion

Body:
Welcome on Stack overflow @Manasa!

Once you are considering <a href="https://cloud.google.com/bigquery/?utm_source=google&utm_medium=cpc&utm_campaign=emea-pl-all-en-dr-skws-all-all-trial-e-gcp-1008073&utm_content=text-ad-none-any-DEV_c-CRE_167401602652-ADGP_Hybrid%20%7C%20AW%20SEM%20%7C%20SKWS%20~%20EXA_1:1_PL_EN_Data_BigQuery_TOP_bigquery-KWID_43700016291537629-kwd-47616965283-userloc_9067437&utm_term=KW_bigquery-ST_bigquery&ds_rl=1242850&ds_rl=1245734&ds_rl=1245734&gclid=EAIaIQobChMIgMqHmaHY5wIVBp53Ch1Zjg7CEAAYASAAEgJfsPD_BwE" rel="nofollow noreferrer">Bigquery as a warehouse for the target sink, leveraging SCD type2 merging method, you might have first to get the practice way how to implement it composing particular SQL statement, reaching the desired logic. Saying this, I encourage you to study this great article written by @Hariharan Ramachandran, explaining SCD techniques with some hand on examples using Bigquery service as a destination data holder.

According to Data Fusion documentation, you can find a rich set of Plugins, extending standard data processing workflows.Therefore, in this particular case I would recommend implementing BigQuery Execute plugin which can be adjusted in your pipeline, executing SQL queries against Bigquery managed service and perform some SCD2 specific actions.


问:
Get value from property that's value is xml
I have a property that's contain :   <date xmlns="http://test.com/ds">20200222</date>.

And i want to get the value '20200222' how can i do it ?   knowing that if it's empty i get <date xmlns="http://test.com/ds"/>
please advise .

答:
You can use a ternary operator coupled with querySelector and get the text using innerHTML

document.querySelector('date').innerHTML==''?console.log(document.querySelector('date')):console.log(document.querySelector('date').innerHTML)```
<pre class="snippet-code-html lang-html prettyprint-override"><code><date xmlns="http://test.com/ds">20200222</date></code>```

Body:
You can use a ternary operator coupled with querySelector and get the text using innerHTML

document.querySelector('date').innerHTML==''?console.log(document.querySelector('date')):console.log(document.querySelector('date').innerHTML)

<date xmlns="http://test.com/ds">20200222</date>


问:
How to remove non UTF-8 characters from text
I need help removing non UTF-8 character from my word cloud. So far this is my code. I've tried gsub and removeWords and they are still there in my word cloud and I do not know what to do to get rid of them. Any help would be appreciated. Thank you for your time.

```<code>txt <- readLines("11-0.txt")
corpus = VCorpus(VectorSource(txt))
gsub("â€™","â€˜","",txt)

corpus = tm_map(corpus, content_transformer(tolower))
corpus = tm_map(corpus, removeWords, stopwords("english"))
corpus = tm_map(corpus, removePunctuation)
corpus = tm_map(corpus, stripWhitespace)
corpus = tm_map(corpus, removeWords, c("gutenberg","gutenbergtm","â€","project"))

tdm = TermDocumentMatrix(corpus)
m = as.matrix(tdm)
v = sort(rowSums(m),decreasing = TRUE)
d = data.frame(word=names(v),freq=v)

wordcloud(d$word,d$freq,max.words = 20, random.order=FALSE, rot.per=0.2, colors=brewer.pal(8, "Dark2"))
</code>```

Edit: Here is my inconv version

```<code>txt <- readLines("11-0.txt")
Encoding(txt) <- "latin1"
iconv(txt, "latin1", "ASCII", sub="")

corpus = VCorpus(VectorSource(txt))
corpus = tm_map(corpus, content_transformer(tolower))
corpus = tm_map(corpus, removeWords, stopwords("english"))
corpus = tm_map(corpus, removePunctuation)
corpus = tm_map(corpus, stripWhitespace)
corpus = tm_map(corpus, removeWords, c("gutenberg","gutenbergtm","project"))

tdm = TermDocumentMatrix(corpus)
m = as.matrix(tdm)
v = sort(rowSums(m),decreasing = TRUE)
d = data.frame(word=names(v),freq=v)

wordcloud(d$word,d$freq,max.words = 20, random.order=FALSE, rot.per=0.2, colors=brewer.pal(8, "Dark2"))
title(main="Alice in Wonderland word cloud",font.main=1,cex.main =1.5)
</code>```

答:
The signature of gsub is:

gsub(pattern, replacement, x, ignore.case = FALSE, perl = FALSE, fixed = FALSE, useBytes = FALSE)

Not sure what you wanted to do with

gsub("â€™","â€˜","",txt)

but that line is probably not doing what you want it to do...

See here for a previous SO question on gsub and non-ascii symbols.

Edit:

Suggested solution using iconv:

Removing all non-ASCII characters:

```<code>txt <- "â€™xxxâ€˜"

iconv(txt, "latin1", "ASCII", sub="")
</code>```

Returns:

```<code>[1] "xxx"
</code>```

Body:
I need help removing non UTF-8 character from my word cloud. So far this is my code. I've tried gsub and removeWords and they are still there in my word cloud and I do not know what to do to get rid of them. Any help would be appreciated. Thank you for your time.

txt <- readLines("11-0.txt")
corpus = VCorpus(VectorSource(txt))
gsub("â€™","â€˜","",txt)

corpus = tm_map(corpus, content_transformer(tolower))
corpus = tm_map(corpus, removeWords, stopwords("english"))
corpus = tm_map(corpus, removePunctuation)
corpus = tm_map(corpus, stripWhitespace)
corpus = tm_map(corpus, removeWords, c("gutenberg","gutenbergtm","â€","project"))

tdm = TermDocumentMatrix(corpus)
m = as.matrix(tdm)
v = sort(rowSums(m),decreasing = TRUE)
d = data.frame(word=names(v),freq=v)

wordcloud(d$word,d$freq,max.words = 20, random.order=FALSE, rot.per=0.2, colors=brewer.pal(8, "Dark2"))

Edit: Here is my inconv version

txt <- readLines("11-0.txt")
Encoding(txt) <- "latin1"
iconv(txt, "latin1", "ASCII", sub="")

corpus = VCorpus(VectorSource(txt))
corpus = tm_map(corpus, content_transformer(tolower))
corpus = tm_map(corpus, removeWords, stopwords("english"))
corpus = tm_map(corpus, removePunctuation)
corpus = tm_map(corpus, stripWhitespace)
corpus = tm_map(corpus, removeWords, c("gutenberg","gutenbergtm","project"))

tdm = TermDocumentMatrix(corpus)
m = as.matrix(tdm)
v = sort(rowSums(m),decreasing = TRUE)
d = data.frame(word=names(v),freq=v)

wordcloud(d$word,d$freq,max.words = 20, random.order=FALSE, rot.per=0.2, colors=brewer.pal(8, "Dark2"))
title(main="Alice in Wonderland word cloud",font.main=1,cex.main =1.5)

Body:
The signature of gsub is:

gsub(pattern, replacement, x, ignore.case = FALSE, perl = FALSE, fixed = FALSE, useBytes = FALSE)

Not sure what you wanted to do with

gsub("â€™","â€˜","",txt)

but that line is probably not doing what you want it to do...

See here for a previous SO question on gsub and non-ascii symbols.

Edit:

Suggested solution using iconv:

Removing all non-ASCII characters:

txt <- "â€™xxxâ€˜"

iconv(txt, "latin1", "ASCII", sub="")

Returns:

[1] "xxx"


问:
How can I covert AJAX to dictionary in python?
I Have the following AJAX  submission data entry and I want to organize it in report format so I have to covert it to table format
The table should include the field_name as header and the field_value as rows
Can anyone help me?</p>

```<code>[{"field_name":"patientno","field_value":"1"},
{"field_name":"patient_unhcr_id","field_value":"1"},
{"field_name":"jps_file","field_value":"1"},
{"field_name":"patient_individual_id","field_value":"1"},
{"field_name":"name","field_value":"1"},
{"field_name":"name_in_arabic","field_value":"1"},
{"field_name":"age","field_value":"1"},
{"field_name":"age_category","field_value":"U5"},
{"field_name":"gender","field_value":"F"},
{"field_name":"coo","field_value":"Syria"},
{"field_name":"phone_number","field_value":"1"},
{"field_name":"governorate","field_value":"Mafraq"},
{"field_name":"bank_branch","field_value":"\u0641\u0631\u0639 \u0636\u0627\u062d\u064a\u0629 \u0627\u0644\u064a\u0627\u0633\u0645\u064a\u0646"},
{"field_name":"treatment_site","field_value":"Ramtha Governmental Hospital"},
{"field_name":"case_category","field_value":"CS"},
{"field_name":"description","field_value":"a"},
{"field_name":"eligibilities_","field_value":"Eligible Level 2"},{"field_name":"approved_amount_before_rounding","field_value":"21.5"},
{"field_name":"approved_amount","field_value":"20"},
{"field_name":"radio_buttons","field_value":"Yes"},
{"field_name":"recipient_name","field_value":"a"},
{"field_name":"recipient__dob","field_value":"02\/24\/2020"},
{"field_name":"gender_of_recpient_","field_value":"F"},
{"field_name":"recipient_unhcr_id_number","field_value":"1"},
{"field_name":"recipient_individual_id","field_value":"1"},
{"field_name":"relationship_to_patient","field_value":"Daughter-in-law"},
{"field_name":"recepient_phone_no","field_value":"1"},
{"field_name":"date_request_send_to_unhcr","field_value":"02\/17\/2020"},
{"field_name":"approval_date","field_value":"02\/24\/2020"},
{"field_name":"closure_date","field_value":"02\/11\/2020"},
{"field_name":"comment","field_value":"a"},
{"field_name":"attatchment","field_value":"http:\/\/192.168.1.52:9999\/wordpress\/wp-content\/uploads\/2020\/02\/IC-Weekly-Task-List-Template-8624.xlsx"}]
</code>```

Body:
I Have the following AJAX  submission data entry and I want to organize it in report format so I have to covert it to table format
The table should include the field_name as header and the field_value as rows
Can anyone help me?

[{"field_name":"patientno","field_value":"1"},
{"field_name":"patient_unhcr_id","field_value":"1"},
{"field_name":"jps_file","field_value":"1"},
{"field_name":"patient_individual_id","field_value":"1"},
{"field_name":"name","field_value":"1"},
{"field_name":"name_in_arabic","field_value":"1"},
{"field_name":"age","field_value":"1"},
{"field_name":"age_category","field_value":"U5"},
{"field_name":"gender","field_value":"F"},
{"field_name":"coo","field_value":"Syria"},
{"field_name":"phone_number","field_value":"1"},
{"field_name":"governorate","field_value":"Mafraq"},
{"field_name":"bank_branch","field_value":"\u0641\u0631\u0639 \u0636\u0627\u062d\u064a\u0629 \u0627\u0644\u064a\u0627\u0633\u0645\u064a\u0646"},
{"field_name":"treatment_site","field_value":"Ramtha Governmental Hospital"},
{"field_name":"case_category","field_value":"CS"},
{"field_name":"description","field_value":"a"},
{"field_name":"eligibilities_","field_value":"Eligible Level 2"},{"field_name":"approved_amount_before_rounding","field_value":"21.5"},
{"field_name":"approved_amount","field_value":"20"},
{"field_name":"radio_buttons","field_value":"Yes"},
{"field_name":"recipient_name","field_value":"a"},
{"field_name":"recipient__dob","field_value":"02\/24\/2020"},
{"field_name":"gender_of_recpient_","field_value":"F"},
{"field_name":"recipient_unhcr_id_number","field_value":"1"},
{"field_name":"recipient_individual_id","field_value":"1"},
{"field_name":"relationship_to_patient","field_value":"Daughter-in-law"},
{"field_name":"recepient_phone_no","field_value":"1"},
{"field_name":"date_request_send_to_unhcr","field_value":"02\/17\/2020"},
{"field_name":"approval_date","field_value":"02\/24\/2020"},
{"field_name":"closure_date","field_value":"02\/11\/2020"},
{"field_name":"comment","field_value":"a"},
{"field_name":"attatchment","field_value":"http:\/\/192.168.1.52:9999\/wordpress\/wp-content\/uploads\/2020\/02\/IC-Weekly-Task-List-Template-8624.xlsx"}]


问:
how to insert filename as a string variable into mysql database in python
I have extracted filename from a certain directory. Filename is a string variable. Such as: 'LineNo-YYMMDD_hhmm'. Now I want to insert the filename into the existing MySQL DB. But couldn't find how to add as a str variable. This is what I had tried. Have some other way also checking net.

```<code>....

file_prefix = os.path.basename(name)[: - (len(filetail))]
(file_prefix is: LineName-200215_1619)
try:
    connection = mysql.connector.connect(
                host='localhost', db = 'db_name',
                user='usr', password='myPass')
   mysql_insert_query = " INSERT INTO fileHeader (headInfo) VALUES %s" % str(file_prefix)
   cursor = connection.cursor()
   cursor.execute(mysql_insert_query)
   connection.commit()
   cursor.close()

except mysql.connector.Error as error:
    print ("Failed to insert record into fileHeader table {}".format(error))
</code>```

Getting the following error:

```<code>Failed to insert record into fileHeader table 1064 (42000): You have an error in your SQL syntax;
check the manual that corresponds to your MySQL server version for the right syntax to use near
'LineName-200215_1619' at line 1
</code>```

答:
add '' in your mysql_insert_query:</p>

```<code> mysql_insert_query = " INSERT INTO fileHeader (headInfo) VALUES ( '{0}')".format(file_prefix)
</code>```

Body:
I have extracted filename from a certain directory. Filename is a string variable. Such as: 'LineNo-YYMMDD_hhmm'. Now I want to insert the filename into the existing MySQL DB. But couldn't find how to add as a str variable. This is what I had tried. Have some other way also checking net.

....

file_prefix = os.path.basename(name)[: - (len(filetail))]
(file_prefix is: LineName-200215_1619)
try:
    connection = mysql.connector.connect(
                host='localhost', db = 'db_name',
                user='usr', password='myPass')
   mysql_insert_query = " INSERT INTO fileHeader (headInfo) VALUES %s" % str(file_prefix)
   cursor = connection.cursor()
   cursor.execute(mysql_insert_query)
   connection.commit()
   cursor.close()

except mysql.connector.Error as error:
    print ("Failed to insert record into fileHeader table {}".format(error))

Getting the following error:

Failed to insert record into fileHeader table 1064 (42000): You have an error in your SQL syntax;
check the manual that corresponds to your MySQL server version for the right syntax to use near
'LineName-200215_1619' at line 1

Body:
add '' in your mysql_insert_query:

 mysql_insert_query = " INSERT INTO fileHeader (headInfo) VALUES ( '{0}')".format(file_prefix)


问:
Is there any way to prevent a browser from requesting a file directly from IIS while still permitting it to be requested in a
```

But the problem is that if someone happens to punch in the URL of the script into their browser, they get to see the source code of the script! If this is impossible as I expect, how can I secure these scripts without breaking anything? The scripts have all sorts of weirdness going on with variables being declared in pages and referenced in the included scripts, and vice versa. So it's not a simple matter of encapsulating them into ASCX controls or VB classes/modules. Or is there some way to actually prevent direct access to these scripts from a browser while still allowing them to be loaded into a

答:
URL rewrite rule can be used to prevent user access your aspx script directly.

```<code> <rule name="Block browser" stopProcessing="true">
                    <match url="AdditionalField/AdditionalFieldReport.aspx$" />
                    <conditions>
                        <add input="{HTTP_REFERER}" pattern="^$" />
                    </conditions>
                    <action type="CustomResponse" statusCode="401" subStatusCode="0" statusReason="Direct access denied" statusDescription="Not allowed" />
                </rule>
</code>```

Since HTTP_REFERRER can be faked by client side.This rule can only be used to prevent direct access.  You also need to modify and add more restriction on this rule IIS IP restriction/request filter also can help in this case.

You may need some unique identity for request filter to prevent fake request.

Title: Is there any way to prevent a browser from requesting a file directly from IIS while still permitting it to be requested in a <script> tag?

Body:
I kind of doubt it because

But the problem is that if someone happens to punch in the URL of the script into their browser, they get to see the source code of the script! If this is impossible as I expect, how can I secure these scripts without breaking anything? The scripts have all sorts of weirdness going on with variables being declared in pages and referenced in the included scripts, and vice versa. So it's not a simple matter of encapsulating them into ASCX controls or VB classes/modules. Or is there some way to actually prevent direct access to these scripts from a browser while still allowing them to be loaded into a

Body:
URL rewrite rule can be used to prevent user access your aspx script directly.

 <rule name="Block browser" stopProcessing="true">
                    <match url="AdditionalField/AdditionalFieldReport.aspx$" />
                    <conditions>
                        <add input="{HTTP_REFERER}" pattern="^$" />
                    </conditions>
                    <action type="CustomResponse" statusCode="401" subStatusCode="0" statusReason="Direct access denied" statusDescription="Not allowed" />
                </rule>

Since HTTP_REFERRER can be faked by client side.This rule can only be used to prevent direct access.  You also need to modify and add more restriction on this rule IIS IP restriction/request filter also can help in this case.

You may need some unique identity for request filter to prevent fake request.


问:
Google Query Language GQL string manipulation I need a REPLACE or SUBSTITUTE function
I'm using Google Sheets with an imported CSV.

The range I perform queries on is called 'Import'

I need to do simple string functions, like REPLACE() and LEN() to evaluate if the data I have is acceptable.

I've been searching, and cannot find any way to manipulate strings in Google Query Language?

Other sources suggest I can use 'Standard' SQL, but I don't seem to be able to?

Any ideas where to look and learn? This language doesn't seem well supported and I couldn't find any IRC groups either.

What I have:

```<code>=query(Import,"select A,B,F,E,AL where not AL matches '^[A-Za-z]{2}[ ]{0,1}[0-9]{2}[ ]{0,1}[a-zA-Z]{3}$'")
</code>```

What I want: (T-SQL)

```<code>=query(Import,"select A,B,F,E,AL where not REPLACE(AL," ","") matches '^[A-Za-z]{2}[ ]{0,1}[0-9]{2}[ ]{0,1}[a-zA-Z]{3}$'")
</code>```

答:
You can manipulate the data while defining the Query Range in the Query Formula itself, e.g.:</p>

```<code>=query(ArrayFormula(regexreplace('Filters@0'!F1:L250,"Eliminate ","")),"select Col1, Col7 where Col7 matches '(.*\d{1,3}\.){3}.*'"))
</code>```

Body:
I'm using Google Sheets with an imported CSV.

The range I perform queries on is called 'Import'

I need to do simple string functions, like REPLACE() and LEN() to evaluate if the data I have is acceptable.

I've been searching, and cannot find any way to manipulate strings in Google Query Language?

Other sources suggest I can use 'Standard' SQL, but I don't seem to be able to?

Any ideas where to look and learn? This language doesn't seem well supported and I couldn't find any IRC groups either.

What I have:

=query(Import,"select A,B,F,E,AL where not AL matches '^[A-Za-z]{2}[ ]{0,1}[0-9]{2}[ ]{0,1}[a-zA-Z]{3}$'")

What I want: (T-SQL)

=query(Import,"select A,B,F,E,AL where not REPLACE(AL," ","") matches '^[A-Za-z]{2}[ ]{0,1}[0-9]{2}[ ]{0,1}[a-zA-Z]{3}$'")

Body:
You can manipulate the data while defining the Query Range in the Query Formula itself, e.g.:

=query(ArrayFormula(regexreplace('Filters@0'!F1:L250,"Eliminate ","")),"select Col1, Col7 where Col7 matches '(.*\d{1,3}\.){3}.*'"))


问:
Why does Beautiful Soup not return the content?
I am using bs4 to scrape some results. I could see the HTML content in the source but when I try to fetch it using bs4, it does not give rather says "File does not exist"

```<code>from bs4 import BeautifulSoup
import requests

source = requests.get("https://result.smitcs.in/grade.php?subid=BA1106")
soup = BeautifulSoup(source.text, "html.parser")

marks_pre = soup.find("pre")
marks = marks_pre.find("div")

print(marks.prettify())
</code>```

The above code returns

  File does not exist

```

The above code works fine if I copy the source code from the web and save it locally as HTML file and then fetch it.

答:
try this</p>

```<code>from bs4 import BeautifulSoup
import requests

URL = "https://result.smitcs.in/grade.php?subid=BA1106"
PAGE = requests.get(URL)

# get HTML content
SOUP = BeautifulSoup(PAGE.content, 'lxml')

marks = SOUP.find("div")

print(marks.prettify())
</code>```

Body:
I am using bs4 to scrape some results. I could see the HTML content in the source but when I try to fetch it using bs4, it does not give rather says "File does not exist"

from bs4 import BeautifulSoup
import requests

source = requests.get("https://result.smitcs.in/grade.php?subid=BA1106")
soup = BeautifulSoup(source.text, "html.parser")

marks_pre = soup.find("pre")
marks = marks_pre.find("div")

print(marks.prettify())

The above code returns

  File does not exist

The above code works fine if I copy the source code from the web and save it locally as HTML file and then fetch it.

Body:
try this

from bs4 import BeautifulSoup
import requests

URL = "https://result.smitcs.in/grade.php?subid=BA1106"
PAGE = requests.get(URL)

# get HTML content
SOUP = BeautifulSoup(PAGE.content, 'lxml')

marks = SOUP.find("div")

print(marks.prettify())


问:
How do I fix memory allocation problem with pandas?
little background, I've inherited a piece of code in company that is written in Python, which I don't really know, the code is combining few pre-downloaded Excel reports into one using pandas. I keep running into Memory Allocation Error:

MemoryError: Unable to allocate 368. MiB for an array with shape (17, 5668350) and data type object

This is the code that gives me the error:

```<code>dfCC = dfVendNew.merge(dfVendOld[['SAP ID', 'Cost ctr']], on='SAP ID', how='left')
</code>```

I am stuck on this point unable to progress further. I've tried changing the paging size on Windows but it did not help. I suspect that it is related to my computer settings since this script is running on the other machines without a hitch.

I would greatly appreciate any help.

答:
Objects are going to be the fattest way to store lists of things.  But you need to know a bit about how something is stored in order to make it smaller and faster.
Check out the column types using dataframe's df.info()

This is a toy example:

```<code><class 'pandas.core.frame.DataFrame'>
RangeIndex: 3 entries, 0 to 2
Data columns (total 3 columns):
people     3 non-null object
cost_ctr    3 non-null object
number     3 non-null int64
dtypes: int64(1), object(2)
memory usage: 152.0+ bytes
</code>```

In this case people is an object class and so is message.  The other thing to look at is on the last line: memory usage. So now you'll be able to change your data types and watch your memory usage drop. So let's go through how to change some of these types.

Your SAP_ID is probably an int by default.  If it is not and it's all numeric data you can use:

```<code>df['SAP ID']=df['SAP ID'].astype(int)
</code>```

or

```<code>df['SAP ID']=pd.to_numeric(df['SAP ID'])
</code>```

so now that you have changed a type of one column check out the memory with df.info() again.

"Cost ctr" sounds like it's going to be a short list of things that repeats a lot,  but is generally stored as a list of strings.  You could change this column to a pd.categorical and see how much memory you save with this command.

```<code>df['Cost_Ctr'] = df['Cost_Ctr'].astype(pd.Categorical)
</code>```

Check out the documentation for using astype here
a next level move would be importing it correctly in the first place. When you read the excel file use the converters argument in read_excel.

If getting your memory usage down still is a problem (this shouldn't be with only Excel records) there are other distributed technologies you can use for this purpose namely Dask.

Hope this helps.

Body:
little background, I've inherited a piece of code in company that is written in Python, which I don't really know, the code is combining few pre-downloaded Excel reports into one using pandas. I keep running into Memory Allocation Error:

MemoryError: Unable to allocate 368. MiB for an array with shape (17, 5668350) and data type object

This is the code that gives me the error:

dfCC = dfVendNew.merge(dfVendOld[['SAP ID', 'Cost ctr']], on='SAP ID', how='left')

I am stuck on this point unable to progress further. I've tried changing the paging size on Windows but it did not help. I suspect that it is related to my computer settings since this script is running on the other machines without a hitch.

I would greatly appreciate any help.

Body:
Objects are going to be the fattest way to store lists of things.  But you need to know a bit about how something is stored in order to make it smaller and faster.
Check out the column types using dataframe's df.info()

This is a toy example:

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 3 entries, 0 to 2
Data columns (total 3 columns):
people     3 non-null object
cost_ctr    3 non-null object
number     3 non-null int64
dtypes: int64(1), object(2)
memory usage: 152.0+ bytes

In this case people is an object class and so is message.  The other thing to look at is on the last line: memory usage. So now you'll be able to change your data types and watch your memory usage drop. So let's go through how to change some of these types.

Your SAP_ID is probably an int by default.  If it is not and it's all numeric data you can use:

df['SAP ID']=df['SAP ID'].astype(int)

or

df['SAP ID']=pd.to_numeric(df['SAP ID'])

so now that you have changed a type of one column check out the memory with df.info() again.

"Cost ctr" sounds like it's going to be a short list of things that repeats a lot,  but is generally stored as a list of strings.  You could change this column to a pd.categorical and see how much memory you save with this command.

df['Cost_Ctr'] = df['Cost_Ctr'].astype(pd.Categorical)

Check out the documentation for using astype here
a next level move would be importing it correctly in the first place. When you read the excel file use the converters argument in read_excel.

If getting your memory usage down still is a problem (this shouldn't be with only Excel records) there are other distributed technologies you can use for this purpose namely Dask.

Hope this helps.