问:
AWS API Gateway Calls not working for Android
I have a custom API set up on AWS for passing a message to the server. The server then responds to that message in a simple fashion. The API calls work from the browser and Postman, but do not work via the mobile client. I've removed authorization on the API, deployed it, and triple checked my api calls, but I continue to receive the 403 "Forbidden" error. I'm calling the API using Retrofit. Here is my client code:

API Calls:

```<code>import retrofit2.Call;
import retrofit2.http.Body;
import retrofit2.http.Headers;
import retrofit2.http.POST;

public interface MyAPI {
    @Headers({
            "content-type: application/json",
            "Accept: application/json"
    })
    @POST("/chat/new")
    Call<ChatMessageBody> postMessage(@Body ChatMessageBody message);
}
</code>```

ChatMessageBody class:

```<code>public class ChatMessageBody {

    private String messageBody;

    public ChatMessageBody(String body){
        this.messageBody = body;
    }

    public String body() {
        return messageBody;
    }
}
</code>```

Theoretically, the Request Body of my call as a result of the set up should be something like this:

```<code>{
     "messageBody" : "This is my message"
}
</code>```

Am I missing a vital aspect of calling the API from a mobile setting? It doesn't make sense to me that I'd be able to call the API endpoint from my browser and through Postman and have it work, yet I can't seem to do so via Android.

EDIT:
Response Body from a successful call (via postman or API Gateway test)

```<code>{\"messageBody\": \"You said \\\"test\\\"\"}
</code>```

Response Body from the 403 error

```<code>{"message":"Forbidden"}
</code>```

Postman generates code based on your desired client and your calls. Here is the code generated for this call by Postman (with respect to OkHTTP. I'm not using this code, but it could provide insight into why Postman works and my client doesn't):

```<code>OkHttpClient client = new OkHttpClient();

MediaType mediaType = MediaType.parse("application/json");
RequestBody body = RequestBody.create(mediaType, "{\n\t\"messageBody\" : \"test\"\n}");
Request request = new Request.Builder()
  .url("https://qj2wkyi91c.execute-api.us-east-1.amazonaws.com/prototype_mvp/chat/new")
  .post(body)
  .addHeader("content-type", "application/json")
  .addHeader("cache-control", "no-cache")
  .addHeader("postman-token", "dccf1f20-f20e-fd53-dd24-a9f4ae7b5c69")
  .build();

Response response = client.newCall(request).execute();
</code>```

Here is a stack trace of my requests/responses in the Android Environment:

```<code>03-27 12:32:27.835 29985-31080/com.dev.example.debug D/OkHttp: --> POST [MY_API_ENDPOINT] http/1.1
03-27 12:32:27.835 29985-31080/com.dev.example.debug D/OkHttp: Content-Type: application/json
03-27 12:32:27.835 29985-31080/com.dev.example.debug D/OkHttp: Content-Length: 36
03-27 12:32:27.835 29985-31080/com.dev.example.debug D/OkHttp: Accept: application/json
03-27 12:32:27.836 29985-31080/com.dev.example.debug D/OkHttp: {"messageBody":"This is my message"}
03-27 12:32:27.836 29985-31080/com.dev.example.debug D/OkHttp: --> END POST (36-byte body)
03-27 12:32:28.076 29985-31080/com.dev.example.debug D/OkHttp: <-- 403  [MY_API_ENDPOINT] (239ms)
03-27 12:32:28.076 29985-31080/com.dev.example.debug D/OkHttp: content-type: application/json
03-27 12:32:28.076 29985-31080/com.dev.example.debug D/OkHttp: content-length: 24
03-27 12:32:28.076 29985-31080/com.dev.example.debug D/OkHttp: date: Mon, 27 Mar 2017 16:32:28 GMT
03-27 12:32:28.076 29985-31080/com.dev.example.debug D/OkHttp: x-amzn-requestid: [REQUEST_ID]
03-27 12:32:28.076 29985-31080/com.dev.example.debug D/OkHttp: x-amzn-errortype: ForbiddenException
03-27 12:32:28.076 29985-31080/com.dev.example.debug D/OkHttp: x-cache: Error from cloudfront
03-27 12:32:28.076 29985-31080/com.dev.example.debug D/OkHttp: via: 1.1 [CLOUDFRONT_URL] (CloudFront)
03-27 12:32:28.077 29985-31080/com.dev.example.debug D/OkHttp: x-amz-cf-id: [CF_ID]
03-27 12:32:28.077 29985-31080/com.dev.example.debug D/OkHttp: {"message":"Forbidden"}
03-27 12:32:28.077 29985-31080/com.dev.example.debug D/OkHttp: <-- END HTTP (24-byte body)
03-27 12:32:28.079 29985-29985/com.dev.example.debug E/MyAPI: Your request did not receive a successful response.
03-27 12:32:28.079 29985-29985/com.dev.example.debug E/MyAPI: Response Error Code: 403
03-27 12:32:28.079 29985-29985/com.dev.example.debug E/MyAPI: Response Body: {"message":"Forbidden"}
</code>```

答: I ended up solving this issue by abandoning Retrofit and designing an OKHttp client which interfaces with AWS. My requests are now being properly signed for AWS, and I'm no longer receiving 403 errors.</p>

Body:
I have a custom API set up on AWS for passing a message to the server. The server then responds to that message in a simple fashion. The API calls work from the browser and Postman, but do not work via the mobile client. I've removed authorization on the API, deployed it, and triple checked my api calls, but I continue to receive the 403 "Forbidden" error. I'm calling the API using Retrofit. Here is my client code:

API Calls:

import retrofit2.Call;
import retrofit2.http.Body;
import retrofit2.http.Headers;
import retrofit2.http.POST;

public interface MyAPI {
    @Headers({
            "content-type: application/json",
            "Accept: application/json"
    })
    @POST("/chat/new")
    Call<ChatMessageBody> postMessage(@Body ChatMessageBody message);
}

ChatMessageBody class:

public class ChatMessageBody {

    private String messageBody;

    public ChatMessageBody(String body){
        this.messageBody = body;
    }

    public String body() {
        return messageBody;
    }
}

Theoretically, the Request Body of my call as a result of the set up should be something like this:

{
     "messageBody" : "This is my message"
}

Am I missing a vital aspect of calling the API from a mobile setting? It doesn't make sense to me that I'd be able to call the API endpoint from my browser and through Postman and have it work, yet I can't seem to do so via Android.

EDIT:
Response Body from a successful call (via postman or API Gateway test)

{\"messageBody\": \"You said \\\"test\\\"\"}

Response Body from the 403 error

{"message":"Forbidden"}

Postman generates code based on your desired client and your calls. Here is the code generated for this call by Postman (with respect to OkHTTP. I'm not using this code, but it could provide insight into why Postman works and my client doesn't):

OkHttpClient client = new OkHttpClient();

MediaType mediaType = MediaType.parse("application/json");
RequestBody body = RequestBody.create(mediaType, "{\n\t\"messageBody\" : \"test\"\n}");
Request request = new Request.Builder()
  .url("https://qj2wkyi91c.execute-api.us-east-1.amazonaws.com/prototype_mvp/chat/new")
  .post(body)
  .addHeader("content-type", "application/json")
  .addHeader("cache-control", "no-cache")
  .addHeader("postman-token", "dccf1f20-f20e-fd53-dd24-a9f4ae7b5c69")
  .build();

Response response = client.newCall(request).execute();

Here is a stack trace of my requests/responses in the Android Environment:

03-27 12:32:27.835 29985-31080/com.dev.example.debug D/OkHttp: --> POST [MY_API_ENDPOINT] http/1.1
03-27 12:32:27.835 29985-31080/com.dev.example.debug D/OkHttp: Content-Type: application/json
03-27 12:32:27.835 29985-31080/com.dev.example.debug D/OkHttp: Content-Length: 36
03-27 12:32:27.835 29985-31080/com.dev.example.debug D/OkHttp: Accept: application/json
03-27 12:32:27.836 29985-31080/com.dev.example.debug D/OkHttp: {"messageBody":"This is my message"}
03-27 12:32:27.836 29985-31080/com.dev.example.debug D/OkHttp: --> END POST (36-byte body)
03-27 12:32:28.076 29985-31080/com.dev.example.debug D/OkHttp: <-- 403  [MY_API_ENDPOINT] (239ms)
03-27 12:32:28.076 29985-31080/com.dev.example.debug D/OkHttp: content-type: application/json
03-27 12:32:28.076 29985-31080/com.dev.example.debug D/OkHttp: content-length: 24
03-27 12:32:28.076 29985-31080/com.dev.example.debug D/OkHttp: date: Mon, 27 Mar 2017 16:32:28 GMT
03-27 12:32:28.076 29985-31080/com.dev.example.debug D/OkHttp: x-amzn-requestid: [REQUEST_ID]
03-27 12:32:28.076 29985-31080/com.dev.example.debug D/OkHttp: x-amzn-errortype: ForbiddenException
03-27 12:32:28.076 29985-31080/com.dev.example.debug D/OkHttp: x-cache: Error from cloudfront
03-27 12:32:28.076 29985-31080/com.dev.example.debug D/OkHttp: via: 1.1 [CLOUDFRONT_URL] (CloudFront)
03-27 12:32:28.077 29985-31080/com.dev.example.debug D/OkHttp: x-amz-cf-id: [CF_ID]
03-27 12:32:28.077 29985-31080/com.dev.example.debug D/OkHttp: {"message":"Forbidden"}
03-27 12:32:28.077 29985-31080/com.dev.example.debug D/OkHttp: <-- END HTTP (24-byte body)
03-27 12:32:28.079 29985-29985/com.dev.example.debug E/MyAPI: Your request did not receive a successful response.
03-27 12:32:28.079 29985-29985/com.dev.example.debug E/MyAPI: Response Error Code: 403
03-27 12:32:28.079 29985-29985/com.dev.example.debug E/MyAPI: Response Body: {"message":"Forbidden"}


来源: stackexchange

问:
How to replace a string in multiple files in linux command line
I need to replace a string in a lot of files in a folder, with only ssh access to the server. How can I do this?

答:
I found this one from another post (can't remember which) and while not the most elegant, it's simple and as a novice Linux user has given me no trouble

```<code>for i in *old_str* ; do mv -v "$i" "${i/\old_str/new_str}" ; done
</code>```

if you have spaces or other special characters use a \

```<code>for i in *old_str\ * ; do mv -v "$i" "${i/\old_str\ /new_str}" ; done
</code>```

for strings in sub-directories use **

```<code>for i in *\*old_str\ * ; do mv -v "$i" "${i/\old_str\ /new_str}" ; done
</code>```

ContentLicense: CC BY-SA 3.0

Body:
I found this one from another post (can't remember which) and while not the most elegant, it's simple and as a novice Linux user has given me no trouble

for i in *old_str* ; do mv -v "$i" "${i/\old_str/new_str}" ; done

if you have spaces or other special characters use a \

for i in *old_str\ * ; do mv -v "$i" "${i/\old_str\ /new_str}" ; done

for strings in sub-directories use **

for i in *\*old_str\ * ; do mv -v "$i" "${i/\old_str\ /new_str}" ; done


问:
UITableView with variable width cells
I'm curious how applications create this look, specifically the photo section and the 'skinnier' UITableViewCells. I haven't seen a delegate method for cell width so I'm curious how they are dynamically resizing the cells and putting the image view in the UITableView. Is this some common trick/smoke-n-mirrors/workaround?

答:
UITableView has two properties named tableHeaderView and tableFooterView, you can add custom UIView or UIView subclasses for these properties.</p>

```<code>@property(nonatomic, retain) UIView *tableHeaderView
@property(nonatomic, retain) UIView *tableFooterView
</code>```

ContentLicense: CC BY-SA 2.5

Title: UITableView with variable width cells

Body:
UITableView has two properties named tableHeaderView and tableFooterView, you can add custom UIView or UIView subclasses for these properties.

@property(nonatomic, retain) UIView *tableHeaderView
@property(nonatomic, retain) UIView *tableFooterView


问:
Accessing name of flat file from FlatFileItemReader LineMapper
I have a need to be able to access the name of the flat file that the current line being read is from. We are porting an application and some processes that are currently in place require this information.

Here is my current configuration:

```<code>private FlatFileItemReader<Invoice> invoiceFlatFileItemReader() throws Exception {
    FlatFileItemReader<Invoice> reader = new FlatFileItemReader<>();
    reader.setLinesToSkip(1); // header row
    reader.setLineMapper(new InvoiceLineMapper());
    reader.afterPropertiesSet();

    return reader;
}
</code>```

InvoiceLineMapper:

```<code>public class InvoiceLineMapper implements LineMapper<Invoice> {

    @Override
    public Invoice mapLine(String line, int lineNumber) throws Exception {
        String[] fields = line.split(",");
        ArrayList<String> fieldList = new ArrayList<>();

        for (int i = 0; i < fields.length; i++) {
            fieldList.add(fields[i].replaceAll("\"", "");
        }

        Invoice invoice = new Invoice();

        invoice.setCustomerNumber(Integer.parseInt(fieldList.get(0));
        invoice.setCustomerName(fieldList.get(1));

        // set other things and stuff...

        //Need to be able to set the file name on the model here
        invoice.setFileName(theFileName);
    }
}
</code>```

I am delegating to the invoiceFlatFileItemReader() via a MultiResourceItemReader:

```<code>@Bean
public ItemReader<Invoice> invoiceItemReader() throws Exception {
    ResourcePatternResolver resolver = new PathMatchingResourcePatterResolver(this.getClass().getClassLoader());

    // currently being loaded from the classpath, eventually will be read from S3...that comes later
    Resource[] resources = resolver.getResources("classpath*:**/InvoiceList*.csv*");

    MultiResourceItemReader<Invoice> reader = new MultiResourceItemReader<>();
    reader.setResources(resources);
    reader.setDelegate(invoiceFlatFileItemReader());

    return reader;
}
</code>```

How can I get the name of the resource passed down the chain to where I can access it in the InvoiceLineMapper class and add it to the model (which later gets persisted with a RepositoryItemWriter)?

答:
Here's what I ended up doing, just passing the MultiItemResourceReader down the chain so that I could call getCurrentResource() in the line mapper:

```<code>@Bean
public ItemReader<Invoice> invoiceItemReader() throws Exception {
    ResourcePatternResolver resolver = new PathMatchingResourcePatterResolver(this.getClass().getClassLoader());

    // currently being loaded from the classpath, eventually will be read from S3...that comes later
    Resource[] resources = resolver.getResources("classpath*:**/InvoiceList*.csv*");

    MultiResourceItemReader<Invoice> reader = new MultiResourceItemReader<>();
    reader.setResources(resources);
    reader.setDelegate(invoiceFlatFileItemReader(reader));

    return reader;
}

private FlatFileItemReader<Invoice> invoiceFlatFileItemReader(MultiResourceItemReader delegator) throws Exception {
    FlatFileItemReader<Invoice> reader = new FlatFileItemReader<>();
    reader.setLinesToSkip(1); // header row
    reader.setLineMapper(new InvoiceLineMapper(delegator));
    reader.afterPropertiesSet();

    return reader;
}
</code>```

And then in InvoiceLineMapper:

```<code>public class InvoiceLineMapper implements LineMapper<Invoice> {

    private MultiResourceItemReader delegator;

    public InvoiceLineMapper(MultiResourceItemReader delegator) {
         this.delegator = delegator;
    }

    @Override
    public Invoice mapLine(String line, int lineNumber) throws Exception {
        String[] fields = line.split(",");
        ArrayList<String> fieldList = new ArrayList<>();

        for (int i = 0; i < fields.length; i++) {
            fieldList.add(fields[i].replaceAll("\"", "");
        }

        Invoice invoice = new Invoice();

        invoice.setCustomerNumber(Integer.parseInt(fieldList.get(0));
        invoice.setCustomerName(fieldList.get(1));

        // set other things and stuff...

        Resource currentResource = delegator.getCurrentResource();
        String[] fileName = currentResource.getFilename().split("/");

        invoice.setFileName(filename[filename.length - 1]);

        return invoice;
    }
}
</code>```

Body:
I have a need to be able to access the name of the flat file that the current line being read is from. We are porting an application and some processes that are currently in place require this information.

Here is my current configuration:

private FlatFileItemReader<Invoice> invoiceFlatFileItemReader() throws Exception {
    FlatFileItemReader<Invoice> reader = new FlatFileItemReader<>();
    reader.setLinesToSkip(1); // header row
    reader.setLineMapper(new InvoiceLineMapper());
    reader.afterPropertiesSet();

    return reader;
}

InvoiceLineMapper:

public class InvoiceLineMapper implements LineMapper<Invoice> {

    @Override
    public Invoice mapLine(String line, int lineNumber) throws Exception {
        String[] fields = line.split(",");
        ArrayList<String> fieldList = new ArrayList<>();

        for (int i = 0; i < fields.length; i++) {
            fieldList.add(fields[i].replaceAll("\"", "");
        }

        Invoice invoice = new Invoice();

        invoice.setCustomerNumber(Integer.parseInt(fieldList.get(0));
        invoice.setCustomerName(fieldList.get(1));

        // set other things and stuff...

        //Need to be able to set the file name on the model here
        invoice.setFileName(theFileName);
    }
}

I am delegating to the invoiceFlatFileItemReader() via a MultiResourceItemReader:

@Bean
public ItemReader<Invoice> invoiceItemReader() throws Exception {
    ResourcePatternResolver resolver = new PathMatchingResourcePatterResolver(this.getClass().getClassLoader());

    // currently being loaded from the classpath, eventually will be read from S3...that comes later
    Resource[] resources = resolver.getResources("classpath*:**/InvoiceList*.csv*");

    MultiResourceItemReader<Invoice> reader = new MultiResourceItemReader<>();
    reader.setResources(resources);
    reader.setDelegate(invoiceFlatFileItemReader());

    return reader;
}

How can I get the name of the resource passed down the chain to where I can access it in the InvoiceLineMapper class and add it to the model (which later gets persisted with a RepositoryItemWriter)?

Body:
Here's what I ended up doing, just passing the MultiItemResourceReader down the chain so that I could call getCurrentResource() in the line mapper:

@Bean
public ItemReader<Invoice> invoiceItemReader() throws Exception {
    ResourcePatternResolver resolver = new PathMatchingResourcePatterResolver(this.getClass().getClassLoader());

    // currently being loaded from the classpath, eventually will be read from S3...that comes later
    Resource[] resources = resolver.getResources("classpath*:**/InvoiceList*.csv*");

    MultiResourceItemReader<Invoice> reader = new MultiResourceItemReader<>();
    reader.setResources(resources);
    reader.setDelegate(invoiceFlatFileItemReader(reader));

    return reader;
}

private FlatFileItemReader<Invoice> invoiceFlatFileItemReader(MultiResourceItemReader delegator) throws Exception {
    FlatFileItemReader<Invoice> reader = new FlatFileItemReader<>();
    reader.setLinesToSkip(1); // header row
    reader.setLineMapper(new InvoiceLineMapper(delegator));
    reader.afterPropertiesSet();

    return reader;
}

And then in InvoiceLineMapper:

public class InvoiceLineMapper implements LineMapper<Invoice> {

    private MultiResourceItemReader delegator;

    public InvoiceLineMapper(MultiResourceItemReader delegator) {
         this.delegator = delegator;
    }

    @Override
    public Invoice mapLine(String line, int lineNumber) throws Exception {
        String[] fields = line.split(",");
        ArrayList<String> fieldList = new ArrayList<>();

        for (int i = 0; i < fields.length; i++) {
            fieldList.add(fields[i].replaceAll("\"", "");
        }

        Invoice invoice = new Invoice();

        invoice.setCustomerNumber(Integer.parseInt(fieldList.get(0));
        invoice.setCustomerName(fieldList.get(1));

        // set other things and stuff...

        Resource currentResource = delegator.getCurrentResource();
        String[] fileName = currentResource.getFilename().split("/");

        invoice.setFileName(filename[filename.length - 1]);

        return invoice;
    }
}


问:
How to debug why a Gradle build failed?
Information:

Gradle tasks [:app:assembleDebug]

C:\Users\davis\AndroidStudioProjects\tester\app\src\main\res\values\dimens.xml

Error:Error: Found item Dimension/abc_alert_dialog_button_bar_height
  more than one time

Error:Execution failed for task ':app:mergeDebugResources'.
  C:\Users\davis\AndroidStudioProjects\tester\app\src\main\res\values\dimens.xml:

Error: Found item Dimension/abc_alert_dialog_button_bar_height more
  than one time

Information:BUILD FAILED

Information:Total time: 6.044 secs Information:2 errors Information:0
  warnings Information:See complete output in console

Please suggest me any IDE or a solution.

答:
Do what the error says.
The item abc_alert_dialog_button_bar_height is defined twice in dimens.xml. So go to dimens.xml and remove one abc_alert_dialog_button_bar_height

Title: How to debug why a Gradle build failed?


问:
Tesserocr did not recognize text
I want to ask for suggestion on how to solve the problem of tesserocr did not recognize certain line from an image.

This is the image. source is from Simple Digit Recognition OCR in OpenCV-Python

The code

```<code>from PIL import Image
from tesserocr import PyTessBaseAPI, RIL

image = Image.open('test3.png')
with PyTessBaseAPI() as api:
    api.SetImage(image)
    boxes = api.GetComponentImages(RIL.TEXTLINE, True)
    print 'Found {} textline image components.'.format(len(boxes))
    for i, (im, box, _, _) in enumerate(boxes):
        api.SetRectangle(box['x'], box['y'], box['w'], box['h'])
        ocrResult = api.GetUTF8Text()
        conf = api.MeanTextConf()
        result = (u"Box[{0}]: x={x}, y={y}, w={w}, h={h}, "
            "confidence: {1}, text: {2}").format(i, conf, ocrResult, **box)

    print result
</code>```

The result is like this

```<code>Found 5 textline image components.
Box[0]: x=10, y=5, w=582, h=29, confidence: 81, text: 9821480865132823066470938

Box[1]: x=9, y=55, w=581, h=30, confidence: 91, text: 4460955058223172535940812

Box[2]: x=10, y=106, w=575, h=30, confidence: 90, text: 8481117450284102701938521

Box[3]: x=12, y=157, w=580, h=30, confidence: 0, text:
Box[4]: x=11, y=208, w=581, h=30, confidence: 89, text: 6442881097566593344612847
</code>```

It did not recognize the number in box 3. What should I add or modify the script so the box 3 will show the proper result?

Thank you for your help.

答:
It's correctly recognized with Tesseract 4.00.00alpha with default psm 3 and oem 3 modes. Below is the result.

Suggest to upgrade tesseract to v4.0 with your tesserocr if you are still using v3.x.

EDIT:

To upgrade tesserocr to support v4.00.00.alpha, check this "Is any
  plan to porting tesseract 4.0 (alpha)" issue page. There are
  guidelines to make it works.

Title: Tesserocr did not recognize text

Body:
I want to ask for suggestion on how to solve the problem of tesserocr did not recognize certain line from an image.

This is the image. source is from Simple Digit Recognition OCR in OpenCV-Python

The code

from PIL import Image
from tesserocr import PyTessBaseAPI, RIL

image = Image.open('test3.png')
with PyTessBaseAPI() as api:
    api.SetImage(image)
    boxes = api.GetComponentImages(RIL.TEXTLINE, True)
    print 'Found {} textline image components.'.format(len(boxes))
    for i, (im, box, _, _) in enumerate(boxes):
        api.SetRectangle(box['x'], box['y'], box['w'], box['h'])
        ocrResult = api.GetUTF8Text()
        conf = api.MeanTextConf()
        result = (u"Box[{0}]: x={x}, y={y}, w={w}, h={h}, "
            "confidence: {1}, text: {2}").format(i, conf, ocrResult, **box)

    print result

The result is like this

Found 5 textline image components.
Box[0]: x=10, y=5, w=582, h=29, confidence: 81, text: 9821480865132823066470938

Box[1]: x=9, y=55, w=581, h=30, confidence: 91, text: 4460955058223172535940812

Box[2]: x=10, y=106, w=575, h=30, confidence: 90, text: 8481117450284102701938521

Box[3]: x=12, y=157, w=580, h=30, confidence: 0, text:
Box[4]: x=11, y=208, w=581, h=30, confidence: 89, text: 6442881097566593344612847

It did not recognize the number in box 3. What should I add or modify the script so the box 3 will show the proper result?

Thank you for your help.


问:
Comparing a text file to another text file in python?
I need the ability to compare two text files. File 1 being a chat log and file 2 being a wordlist with key words in. I am struggling to get the output I desire which is ideally showing every time one of the key words in File 2 appears in the chat log which is file 1. Any ideas on how I could achieve this output?

edit*

this is the code I'm currently trying to use, however the output i get is that it prints both files to the text box within the gui. The output need is to show what lines the words from file 2 occur within the file 1. Some of the code is taken from a keyword search feature I already have working.

```<code>def wordlistsearch():

filename = tkFileDialog.askopenfile(filetypes=(("Text files", "*.txt") ,)) //file1
mtxt = filename.readline()
i =0
filename2 = tkFileDialog.askopenfile(filetypes=(("Text files", "*.txt") ,)) //file2

while i<10000:
    keystring = filename2.readline()
    print keystring
    participant = mtxt.split("(")[0]
    temppart2 = mtxt.split("(")[-1]
    keyword = temppart2.split(")")[0]
    if mtxt.find(str(keystring)) != -1:
        print i, ": ", mtxt
    i=i+1
    mtxt = filename.readline()
</code>```

答:
If you want to find all the words in File 1 that are also in File2, you can use:

```<code>keywords = set([word for line in open("keyword_file","r") for word in line.split()])

words = set([word for line in open("log_file","r") for word in line.split()])

common = words.intersection(keywords)
</code>```

To find the occurrence of a match while reading File 1 instead:

```<code>keywords = set([word for line in open("keyword_file","r") for word in line.split()])

for line in open("log_file","r"):
    for word in line:
        if word in keywords:
            print "found {0} in line {1}".format(word, line)
</code>```

Body:
I need the ability to compare two text files. File 1 being a chat log and file 2 being a wordlist with key words in. I am struggling to get the output I desire which is ideally showing every time one of the key words in File 2 appears in the chat log which is file 1. Any ideas on how I could achieve this output?

edit*

this is the code I'm currently trying to use, however the output i get is that it prints both files to the text box within the gui. The output need is to show what lines the words from file 2 occur within the file 1. Some of the code is taken from a keyword search feature I already have working.

def wordlistsearch():

filename = tkFileDialog.askopenfile(filetypes=(("Text files", "*.txt") ,)) //file1
mtxt = filename.readline()
i =0
filename2 = tkFileDialog.askopenfile(filetypes=(("Text files", "*.txt") ,)) //file2

while i<10000:
    keystring = filename2.readline()
    print keystring
    participant = mtxt.split("(")[0]
    temppart2 = mtxt.split("(")[-1]
    keyword = temppart2.split(")")[0]
    if mtxt.find(str(keystring)) != -1:
        print i, ": ", mtxt
    i=i+1
    mtxt = filename.readline()

Body:
If you want to find all the words in File 1 that are also in File2, you can use:

keywords = set([word for line in open("keyword_file","r") for word in line.split()])

words = set([word for line in open("log_file","r") for word in line.split()])

common = words.intersection(keywords)

To find the occurrence of a match while reading File 1 instead:

keywords = set([word for line in open("keyword_file","r") for word in line.split()])

for line in open("log_file","r"):
    for word in line:
        if word in keywords:
            print "found {0} in line {1}".format(word, line)


问:
Can an SVN checkout be multi-threaded?
I have a repo with 30k+ small files, and I can only access the server over the internet via https://.  Checkout are very, very slow.  On the order of hours.  Internet connection speed is ~20Mbs and local machine is 3Ghz multi-core & 10k RPM HD.

(svn:// protocol is not an option, sadly.)

So my question:

Can svn do a multi-threaded parallel checkout of a working copy?

It looks like the svn checkout is going file-by-file in a sequential order.  There is a minuscule delay between each file and I suppose that's the http request/reply and possibly my local filesystem's delays.  (Perhaps a little server filesystem latency as well?)

Thanks

答:
Would it not depend a lot on the ping time to the server?

If the ping is long, then you may not be able to do anything about it.

There is also a chance that the router that handles the firewall is falling behind and using something like stateful packet inspection, ie looking at each packet. the router can be changed to let stuff from the svn server escape packet scanning.

Title: Can an SVN checkout be multi-threaded?


问:
How to store variables values in a class i.e "settings"?
OK. After spending literally a few hours trying to get things working I give up.

Basically I am creating a program in which user inputs some values, then after hitting button a new scene is created and depending on the values it was given, different things take place.
My problem - I created "Settings.class" with a few variables with getters and setters. My assumption was to store the values input in there and whenever needed I have easy access to them using getters.
For some reason it doesn't work.

Keep in mind I simplified it as much as I can because It'd look very messy and would be very long if I pasted my original code. I made sure that the core of the problem is the same.

Settings class:

```<code>public class Settings {
private boolean diamonds;

public boolean getDiamonds() {
    return diamonds;
}
public void setDiamonds(boolean diamonds) {
    this.diamonds = diamonds;
}
}
</code>```

Controller class:

```<code>public class Controller implements Initializable {
private Settings settings = new Settings();
private ProblematicOne prob = new ProblematicOne();

public void handleGoAction() throws IOException {

settings.setDiamonds(true);

prob.editText("This shall be set");

/* ..creating new stage and scene here no reason to paste it here, no probs with that.. */

@Override
public void initialize(URL location, ResourceBundle resources) {
}
}
}
</code>```

And crème de la crème, problematic class:

```<code>public class ProblematicOne{

private Setting settings = new Settings();
String toBeEdited = "";  //

public void editText(String text){

if(settings.getDiamonds){      // for some reason it doesn't work; The getter returns false.
    toBeEdited = text;
}else if(!settings.getDiamonds){
    toBeEdited = "getDiamonds is false";
}
}
}
</code>```

答:
Alright, first off, what you are trying to do can be achieved by serializing the object (i.e Settings) and storing it. Or, simpler, just write to a file with values and load from there when you want to instantiate the class.

Look at this line in your "ProblematicOne"
private Setting settings = new Settings();

You just created a new instance of Settings. This instance does not have any idea of your Settings instance in your Controller.

Another way is to make your Settings class a singleton and then just reuse it. Example:

Settings.java

```<code>public class Settings {

    private static Settings instance = null;

    private boolean diamonds;

    public boolean getDiamonds() {
        return diamonds;
    }

    public void setDiamonds(boolean diamonds) {
        this.diamonds = diamonds;
    }

    private Settings() {}

    public static Settings getInstance(){
        return instance == null ? new Settings() : instance;
    }
}
</code>```

Then in your Controller class just get the instance using the getInstance() method;

private Settings settings = Settings.getInstance();

Similarly, when you use it again in your ProblematicOne class, use the getInstance() method

Body:
OK. After spending literally a few hours trying to get things working I give up.

Basically I am creating a program in which user inputs some values, then after hitting button a new scene is created and depending on the values it was given, different things take place.
My problem - I created "Settings.class" with a few variables with getters and setters. My assumption was to store the values input in there and whenever needed I have easy access to them using getters.
For some reason it doesn't work.

Keep in mind I simplified it as much as I can because It'd look very messy and would be very long if I pasted my original code. I made sure that the core of the problem is the same.

Settings class:

public class Settings {
private boolean diamonds;

public boolean getDiamonds() {
    return diamonds;
}
public void setDiamonds(boolean diamonds) {
    this.diamonds = diamonds;
}
}

Controller class:

public class Controller implements Initializable {
private Settings settings = new Settings();
private ProblematicOne prob = new ProblematicOne();

public void handleGoAction() throws IOException {

settings.setDiamonds(true);

prob.editText("This shall be set");

/* ..creating new stage and scene here no reason to paste it here, no probs with that.. */

@Override
public void initialize(URL location, ResourceBundle resources) {
}
}
}

And crème de la crème, problematic class:

public class ProblematicOne{

private Setting settings = new Settings();
String toBeEdited = "";  //

public void editText(String text){

if(settings.getDiamonds){      // for some reason it doesn't work; The getter returns false.
    toBeEdited = text;
}else if(!settings.getDiamonds){
    toBeEdited = "getDiamonds is false";
}
}
}

Body:
Alright, first off, what you are trying to do can be achieved by serializing the object (i.e Settings) and storing it. Or, simpler, just write to a file with values and load from there when you want to instantiate the class.

Look at this line in your "ProblematicOne"
private Setting settings = new Settings();

You just created a new instance of Settings. This instance does not have any idea of your Settings instance in your Controller.

Another way is to make your Settings class a singleton and then just reuse it. Example:

Settings.java

public class Settings {

    private static Settings instance = null;

    private boolean diamonds;

    public boolean getDiamonds() {
        return diamonds;
    }

    public void setDiamonds(boolean diamonds) {
        this.diamonds = diamonds;
    }

    private Settings() {}

    public static Settings getInstance(){
        return instance == null ? new Settings() : instance;
    }
}

Then in your Controller class just get the instance using the getInstance() method;

private Settings settings = Settings.getInstance();

Similarly, when you use it again in your ProblematicOne class, use the getInstance() method


问:
Anorm parsing JSON from PostgreSQL jsonb column
i'm running play 2.3 and have no way to update ...
whil i'm trying to parse jsonb column i get the error play - Cannot invoke the action, eventually got an error: java.lang.RuntimeException: UnexpectedNullableFound(ColumnName(.sports,Some(sports)))

my code is :

```<code>case class Sport(
    id: UUID,
    name: Option[String]
)

implicit val SportWrites: Writes[Sport] = Json.writes[Sport]
implicit val SportReads: Reads[Sport] = (
    (JsPath \ "f1").read[UUID] and
    (JsPath \ "f2").readNullable[String]
)

case class Member(
    id:UUID,
    name:String,
    sports: List[Sport]
)

def memberParser: RowParser[Member] = {
    get[UUID]("member_id") ~
    get[String]("name") ~
    get[JsValue]("sports") map {
        case(id ~ name ~ sports) => Member(
            id, name
            sports.validate[List[Sport]].getOrElse(List())
        )
    }
}

def getMember():List[Member] = DB.WithConnection{ implicit c =>
    SQL("SELECT m.id, m.name, m.sports::json from member m ").as(memberParser.*)
}
</code>```

|Sports| is a jsonb postgresql column, if i dont cast this column as ::json i get a TypeDoesNotMatch error.

Json fields are named "f1", "f2", "f3"(...) because i have populated this column with data from another table with JSON_AGG(... so it's postgresql naming.

also this function is working:

```<code>def getMemberWithSports():List[Member] = DB.withConnection{ implicit c =>
    SQL("""
        SELECT m.member_id, m.name,
        json_agg(row_to_json(row(s.sport_id, s.name))) as sports
        FROM rank_member m
        LEFT JOIN member_sport ms ON m.member_id = ms.member_id
        LEFT JOIN sport s ON ms.sport_id = s.sport_id
        GROUP BY m.member_id, m.fullname
    """).as(memberParser.*)
}
</code>```

the only difference between the results of these two SQL commande (in getMember() & getMemberWithSport() ) is that with the first query i get nothing in sport will the second query (the one that work in play) return JSON populated with null values.

答:
the only workaround i've found for the moment is by adding COALESCE to the SQL query as follow: </p>

```<code>def getMember():List[Member] = DB.WithConnection{ implicit c =>
    SQL("SELECT m.id, m.name, COALESCE(m.sports::json, '[]') as sports from member m ").as(memberParser.*)
}
</code>```

Body:
i'm running play 2.3 and have no way to update ...
whil i'm trying to parse jsonb column i get the error play - Cannot invoke the action, eventually got an error: java.lang.RuntimeException: UnexpectedNullableFound(ColumnName(.sports,Some(sports)))

my code is :

case class Sport(
    id: UUID,
    name: Option[String]
)

implicit val SportWrites: Writes[Sport] = Json.writes[Sport]
implicit val SportReads: Reads[Sport] = (
    (JsPath \ "f1").read[UUID] and
    (JsPath \ "f2").readNullable[String]
)

case class Member(
    id:UUID,
    name:String,
    sports: List[Sport]
)

def memberParser: RowParser[Member] = {
    get[UUID]("member_id") ~
    get[String]("name") ~
    get[JsValue]("sports") map {
        case(id ~ name ~ sports) => Member(
            id, name
            sports.validate[List[Sport]].getOrElse(List())
        )
    }
}

def getMember():List[Member] = DB.WithConnection{ implicit c =>
    SQL("SELECT m.id, m.name, m.sports::json from member m ").as(memberParser.*)
}

|Sports| is a jsonb postgresql column, if i dont cast this column as ::json i get a TypeDoesNotMatch error.

Json fields are named "f1", "f2", "f3"(...) because i have populated this column with data from another table with JSON_AGG(... so it's postgresql naming.

also this function is working:

def getMemberWithSports():List[Member] = DB.withConnection{ implicit c =>
    SQL("""
        SELECT m.member_id, m.name,
        json_agg(row_to_json(row(s.sport_id, s.name))) as sports
        FROM rank_member m
        LEFT JOIN member_sport ms ON m.member_id = ms.member_id
        LEFT JOIN sport s ON ms.sport_id = s.sport_id
        GROUP BY m.member_id, m.fullname
    """).as(memberParser.*)
}

the only difference between the results of these two SQL commande (in getMember() & getMemberWithSport() ) is that with the first query i get nothing in sport will the second query (the one that work in play) return JSON populated with null values.

Body:
the only workaround i've found for the moment is by adding COALESCE to the SQL query as follow:

def getMember():List[Member] = DB.WithConnection{ implicit c =>
    SQL("SELECT m.id, m.name, COALESCE(m.sports::json, '[]') as sports from member m ").as(memberParser.*)
}


问:
Java - throws not working for a subtype of an exception that is handled
Take this as example:

```<code>public class TestClass {

    public static void test() throws SSLHandshakeException {
        throw new SSLHandshakeException("I'm a SSL Exception");
    }

    public static void main(String[] args) throws SSLHandshakeException {
        try {
            test ();
        } catch (IOException ioe) {
            System.out.println("I`m handling IO exception");
        }
    }
}
</code>```

So, I have my test method in which I'm just throwing an SSLHandshakeException which is a subtype of IOException.

The output for this is "I`m handling IO exception".

Why does this happen? I expected that my method will throw the SSLHandshakeException. Is there a rule that catch is more important than throws?

I just want to avoid using

```<code>try {
   test ();
} catch (SSLHandshakeException se) {
   throw se;
} catch (IOException ioe) {
   System.out.println("I`m handling IO exception");
}
</code>```

Because I consider it less readable

答:
That's probably because you printed your custom string instead of message of exception. Try this :</p>

```<code>public static void main(String[] args) throws SSLHandshakeException {
    try {
        test ();
    } catch (IOException ioe) {
        System.out.println("I`m handling IO exception");
        System.out.println(ioe.getMessage());

    }
}
</code>```

Body:
Take this as example:

public class TestClass {

    public static void test() throws SSLHandshakeException {
        throw new SSLHandshakeException("I'm a SSL Exception");
    }

    public static void main(String[] args) throws SSLHandshakeException {
        try {
            test ();
        } catch (IOException ioe) {
            System.out.println("I`m handling IO exception");
        }
    }
}

So, I have my test method in which I'm just throwing an SSLHandshakeException which is a subtype of IOException.

The output for this is "I`m handling IO exception".

Why does this happen? I expected that my method will throw the SSLHandshakeException. Is there a rule that catch is more important than throws?

I just want to avoid using

try {
   test ();
} catch (SSLHandshakeException se) {
   throw se;
} catch (IOException ioe) {
   System.out.println("I`m handling IO exception");
}

Because I consider it less readable

Body:
That's probably because you printed your custom string instead of message of exception. Try this :

public static void main(String[] args) throws SSLHandshakeException {
    try {
        test ();
    } catch (IOException ioe) {
        System.out.println("I`m handling IO exception");
        System.out.println(ioe.getMessage());

    }
}


问:
speeding up php/mysql query/correct usage of joins?
I'm having some issues with some mysql queries taking around 50 seconds to execute some times, but other times its around 1 second. The query does what I need it do but obviously is too slow.

Below are the three tables that are used and showing some of the columns:

products Table (14315 rows)

```<code>id    | title  | image | price | rrp   | upsell
20381 | test   | test  | 24.00 | 30.00 | 1
</code>```

product_to_category Table (26283 rows)

```<code>ptc_id | product_id | category_id
33797  | 20381      | 29
</code>```

category_description Table (29 rows)

```<code>category_id | name     | url_slug
29          | Kitchens | kitchen-products
</code>```

seo_slugs Table (14898 rows)

```<code>seo_slug_id | route         | seo_slug
26987       | product=20381 | test-product-kitchen-area-2000-500
</code>```

Below is the current codeigniter functions I am using.

```<code>public function getUpsells() {
$this->db->select('products.id, products.title, products.image, products.category, products.price, products.rrp', FALSE);
$this->db->from('product_to_category');
$this->db->join('products', 'products.id = product_to_category.product_id');
$this->db->join('category_description', 'category_description.category_id = product_to_category.category_id');
$this->db->join('seo_slugs', 'seo_slugs.route = concat(\'product=\', product_to_category.product_id)');
$this->db->where('products.upsell',1);
$this->db->where('products.status',1);
$this->db->group_by('product_to_category.product_id');
$query = $this->db->get();
return $query->result();
</code>```

}
I'm hoping someone can point me in the right direction and advise where I'm going wrong. It is to be honest the first time i've used joins in a query and although it seems to work, it is generally very slow when first loading the page, then if refreshed etc it seems to load fairly quick.

Below is the query it runs:

```<code>SELECT products.id, products.title, products.image, products.category, products.price, products.rrp
FROM `product_to_category`
JOIN `products` ON `products`.`id` = `product_to_category`.`product_id`
JOIN `category_description` ON `category_description`.`category_id` = `product_to_category`.`category_id`
JOIN `seo_slugs` ON `seo_slugs`.`route` = concat('product=', product_to_category.product_id)
WHERE `products`.`upsell` = 1
AND `products`.`status` = 1
GROUP BY `product_to_category`.`product_id`
</code>```

Profiling result:

```<code>Starting    18 µs
Waiting For Query Cache Lock    5 µs
Checking Query Cache For Query  48 µs
Checking Permissions    5 µs
Checking Permissions    4 µs
Checking Permissions    4 µs
Checking Permissions    5 µs
Opening Tables  20 µs
System Lock 9 µs
Waiting For Query Cache Lock    15 µs
Init    25 µs
Optimizing  13 µs
Statistics  23 µs
Preparing   19 µs
Creating Tmp Table  21 µs
Executing   5 µs
Copying To Tmp Table    51.8 s
Sorting Result  26 µs
Sending Data    24 µs
End 9 µs
Removing Tmp Table  33 µs
End 7 µs
Query End   7 µs
Closing Tables  14 µs
Freeing Items   12 µs
Waiting For Query Cache Lock    6 µs
Freeing Items   11 µs
Waiting For Query Cache Lock    7 µs
Freeing Items   7 µs
Storing Result In Query Cache   162 µs
Logging Slow Query  6 µs
Logging Slow Query  38 µs
Cleaning Up 7 µs
</code>```

EXPLAIN Result:

```+----+-------------+----------------------+--------+---------------+---------+---------+-----------------------------------------------------+-------+---------------------------------+
| id | select_type |        table         |  type  | possible_keys |   key   | key_len |                         ref                         | rows  |              Extra              |
+----+-------------+----------------------+--------+---------------+---------+---------+-----------------------------------------------------+-------+---------------------------------+
|  1 | SIMPLE      | seo_slugs            | ALL    | NULL          | NULL    | NULL    | NULL                                                | 14958 | Using temporary; Using filesort |
|  1 | SIMPLE      | product_to_category  | ALL    | category_id   | NULL    | NULL    | NULL                                                | 26343 | Using where; Using join buffer  |
|  1 | SIMPLE      | category_description | eq_ref | PRIMARY       | PRIMARY | 4       | wilsonar_hcsupplies.product_to_category.category_id |     1 | Using index                     |
|  1 | SIMPLE      | products             | eq_ref | PRIMARY,id    | PRIMARY | 4       | wilsonar_hcsupplies.product_to_category.product_id  |     1 | Using where                     |
+----+-------------+----------------------+--------+---------------+---------+---------+-----------------------------------------------------+-------+---------------------------------+
```

答:
EXPLAIN tells us that product_to_category has an index that MySQL considers using, since possible_keys is "category_id". However, it is not used for some reason.

"rows" is 26343 and "key" is NULL, which means MySQL will have to scan all
26343 rows in product_to_category for each product, which is going to be horrendous even using the join buffer optimization.

```<code>product_to_category Table (26283 rows)
ptc_id | product_id | category_id
33797  | 20381      | 29
</code>```

ptc_id is useless, since the primary key of this table should be (category_id, product_id) with a UNIQUE index on (product_id, category_id) added to optimize searches. This makes me suspicious of the database structure, therefore I am going to bet that the designer used the wrong type (ie, anything except INT) for one of the columns in one of the tables. So, check your column types for tinyint, smallint, bigint, unsigned, whatever...

Other causes would be that the index named category_id does not actually index the column category_id. You never know. Please check.

Now, there is another issue with the query, namely that the GROUP BY makes the entire query useless. I wonder what this query is supposed to do...

Body:
I'm having some issues with some mysql queries taking around 50 seconds to execute some times, but other times its around 1 second. The query does what I need it do but obviously is too slow.

Below are the three tables that are used and showing some of the columns:

products Table (14315 rows)

id    | title  | image | price | rrp   | upsell
20381 | test   | test  | 24.00 | 30.00 | 1

product_to_category Table (26283 rows)

ptc_id | product_id | category_id
33797  | 20381      | 29

category_description Table (29 rows)

category_id | name     | url_slug
29          | Kitchens | kitchen-products

seo_slugs Table (14898 rows)

seo_slug_id | route         | seo_slug
26987       | product=20381 | test-product-kitchen-area-2000-500

Below is the current codeigniter functions I am using.

public function getUpsells() {
$this->db->select('products.id, products.title, products.image, products.category, products.price, products.rrp', FALSE);
$this->db->from('product_to_category');
$this->db->join('products', 'products.id = product_to_category.product_id');
$this->db->join('category_description', 'category_description.category_id = product_to_category.category_id');
$this->db->join('seo_slugs', 'seo_slugs.route = concat(\'product=\', product_to_category.product_id)');
$this->db->where('products.upsell',1);
$this->db->where('products.status',1);
$this->db->group_by('product_to_category.product_id');
$query = $this->db->get();
return $query->result();

}
I'm hoping someone can point me in the right direction and advise where I'm going wrong. It is to be honest the first time i've used joins in a query and although it seems to work, it is generally very slow when first loading the page, then if refreshed etc it seems to load fairly quick.

Below is the query it runs:

SELECT products.id, products.title, products.image, products.category, products.price, products.rrp
FROM `product_to_category`
JOIN `products` ON `products`.`id` = `product_to_category`.`product_id`
JOIN `category_description` ON `category_description`.`category_id` = `product_to_category`.`category_id`
JOIN `seo_slugs` ON `seo_slugs`.`route` = concat('product=', product_to_category.product_id)
WHERE `products`.`upsell` = 1
AND `products`.`status` = 1
GROUP BY `product_to_category`.`product_id`

Profiling result:

Starting    18 µs
Waiting For Query Cache Lock    5 µs
Checking Query Cache For Query  48 µs
Checking Permissions    5 µs
Checking Permissions    4 µs
Checking Permissions    4 µs
Checking Permissions    5 µs
Opening Tables  20 µs
System Lock 9 µs
Waiting For Query Cache Lock    15 µs
Init    25 µs
Optimizing  13 µs
Statistics  23 µs
Preparing   19 µs
Creating Tmp Table  21 µs
Executing   5 µs
Copying To Tmp Table    51.8 s
Sorting Result  26 µs
Sending Data    24 µs
End 9 µs
Removing Tmp Table  33 µs
End 7 µs
Query End   7 µs
Closing Tables  14 µs
Freeing Items   12 µs
Waiting For Query Cache Lock    6 µs
Freeing Items   11 µs
Waiting For Query Cache Lock    7 µs
Freeing Items   7 µs
Storing Result In Query Cache   162 µs
Logging Slow Query  6 µs
Logging Slow Query  38 µs
Cleaning Up 7 µs

EXPLAIN Result:

+----+-------------+----------------------+--------+---------------+---------+---------+-----------------------------------------------------+-------+---------------------------------+
| id | select_type |        table         |  type  | possible_keys |   key   | key_len |                         ref                         | rows  |              Extra              |
+----+-------------+----------------------+--------+---------------+---------+---------+-----------------------------------------------------+-------+---------------------------------+
|  1 | SIMPLE      | seo_slugs            | ALL    | NULL          | NULL    | NULL    | NULL                                                | 14958 | Using temporary; Using filesort |
|  1 | SIMPLE      | product_to_category  | ALL    | category_id   | NULL    | NULL    | NULL                                                | 26343 | Using where; Using join buffer  |
|  1 | SIMPLE      | category_description | eq_ref | PRIMARY       | PRIMARY | 4       | wilsonar_hcsupplies.product_to_category.category_id |     1 | Using index                     |
|  1 | SIMPLE      | products             | eq_ref | PRIMARY,id    | PRIMARY | 4       | wilsonar_hcsupplies.product_to_category.product_id  |     1 | Using where                     |
+----+-------------+----------------------+--------+---------------+---------+---------+-----------------------------------------------------+-------+---------------------------------+

Body:
EXPLAIN tells us that product_to_category has an index that MySQL considers using, since possible_keys is "category_id". However, it is not used for some reason.

"rows" is 26343 and "key" is NULL, which means MySQL will have to scan all
26343 rows in product_to_category for each product, which is going to be horrendous even using the join buffer optimization.

product_to_category Table (26283 rows)
ptc_id | product_id | category_id
33797  | 20381      | 29

ptc_id is useless, since the primary key of this table should be (category_id, product_id) with a UNIQUE index on (product_id, category_id) added to optimize searches. This makes me suspicious of the database structure, therefore I am going to bet that the designer used the wrong type (ie, anything except INT) for one of the columns in one of the tables. So, check your column types for tinyint, smallint, bigint, unsigned, whatever...

Other causes would be that the index named category_id does not actually index the column category_id. You never know. Please check.

Now, there is another issue with the query, namely that the GROUP BY makes the entire query useless. I wonder what this query is supposed to do...


问:
How does a Symfony pass parameters into a yaml config?
A Symfony Config component allows to create and load a configuration. For instance, it allows to use a yml for config.

However, in a Symfony framework configuration uses parameters with a special syntax - %parameter.key%

I've looked at Symfony YamlFileLoader but it only load a content of a yaml-file and handle such sections as imports, parameters and services. The only goal of the parameters is to be stored in a container parameter bag.

So I'm wondering what's magic used to replace placeholders in a yaml config file?

Moreover. I've tried to load a config:

```<code>application:
    aBooleanKey: "true"
</code>```

and I've got an error since aBooleanKey requires a boolean value but got a string.

答:
So I'm wondering what's magic used to replace placeholders in a yaml config file?

This Parameters are part of the Dependency Injection component, not the Config.

The only goal of the parameters is to be stored in a container parameter bag.

Not quite. Special compiler pass resolves the placeholders. It works at the optimization step of the container compilation and inject actual values, so that placeholders can be used in the service definitions and in the semantic configuration files.

... and I've got an error since aBooleanKey requires a boolean value but got a string.

I guess that caused by YAML Component:

Finally, there are other cases when the strings must be quoted, no matter if you're using single or double quotes:

When the string is true or false (otherwise, it would be treated as a boolean value);

...

So, in your example true is quoted and treated as string.

Body:
A Symfony Config component allows to create and load a configuration. For instance, it allows to use a yml for config.

However, in a Symfony framework configuration uses parameters with a special syntax - %parameter.key%

I've looked at Symfony YamlFileLoader but it only load a content of a yaml-file and handle such sections as imports, parameters and services. The only goal of the parameters is to be stored in a container parameter bag.

So I'm wondering what's magic used to replace placeholders in a yaml config file?

Moreover. I've tried to load a config:

application:
    aBooleanKey: "true"

and I've got an error since aBooleanKey requires a boolean value but got a string.


问:
MS Access SQL Server Always Encrypted Parameterization
I'm evaluating whether or not SQL Server 2016 Always Encrypted will work with an existing MS Access 2010 application that I support.

Here's my current roadblock:

My application calls many SQL Server stored procedures that require parameters. I use the following function to make those calls:

```<code>Public Function ExecuteSPWithParamsQuery(poQDFStub As DAO.QueryDef, psParameterString As String) As DAO.Recordset

'-------------------------------------------------------------------------------------------------
' Purpose   : Execute an SQL pass-through query that calls a stored procedures requiring parameters.
'
' Params    : poQDFStub: pass through query with name of SPROC
'                : psParameterString : one or more parameters to be appended to poQDFStub
'
' Returns   : Dao.Recordset(dbOpenSnapshot)
'-------------------------------------------------------------------------------------------------
'

    If G_HANDLE_ERRORS Then On Error GoTo ErrorHandler

    Dim rstResult As DAO.Recordset

    'db interface
    Dim dbs As DAO.Database: Set dbs = CurrentDb
    Dim qdfResult As DAO.QueryDef: Set qdfResult = dbs.CreateQueryDef(vbNullString)

    'setup pass through
    With qdfResult
        .Connect = poQDFStub.Connect
        .SQL = poQDFStub.SQL & " " & psParameterString
        .ODBCTimeout = 0
        .ReturnsRecords = True
    End With

    'setup result
    Set rstResult = qdfResult.OpenRecordset(dbOpenSnapshot, dbSQLPassThrough + dbReadOnly + dbFailOnError)

ExitHere:

    'housekeeping
    On Error Resume Next
    'add cleanup here
    Set qdfResult = Nothing
    Set dbs = Nothing

    'exit protocol
    On Error GoTo 0
    Set ExecuteSPWithParamsQuery = rstResult
    Set rstResult = Nothing
    Exit Function

ErrorHandler:

    Err.Source = "SQLStoredProcedureHelper.ExecuteSPWithParamsQuery"
    HandleError
    Resume ExitHere

End Function
</code>```

Calls to this function will now include parameters that are clear text versions of values encrypted in the database.

When this happens, I get the following error.

206 [Microsoft][ODBC SQL Server Driver][SQL Server] Operand type clash: varchar is incompatible > with nvarchar(255) encrypted with (encryption_type = 'DETERMINISTIC', encryption_algorithm_name
  = 'AEAD_AES_256_CBC_HMAC_SHA_256', column_encryption_key_name = 'CEK_Auto1',
  column_encryption_key_database_name = 'sandbox')

I've done some investigation on Always Encrypted Parameterization. It requires one of two technologies

.NET

ODBC 13.1 For SQL Server

Since this is an MS Access application, .NET is not applicable.

I have ODBC 13.1 installed, but I'm guessing that my pass through query is bypassing parameterization.

Here are my ODBC settings:

```<code>[ODBC]
DRIVER=ODBC Driver 13 for SQL Server
ColumnEncryption=Enabled
TrustServerCertificate=No
DATABASE=sandbox
WSID=********
APP=Microsoft Office 2010
Trusted_Connection=Yes
SERVER=*********
</code>```

Any ideas on how I can resolve this problem or is Always Encrypted not a fit for my application?

答:
The resolution to my problem was to convert my function from DAO to ADO.  Hope the following code helps someone else down the road:</p>

```<code>Public Function ExecuteSPWithParamsQueryADO(pSPROCName As String, ParamArray pParams() As Variant) As ADODB.RecordSet

'---------------------------------------------------------------------------------------------------------------------
' Purpose   : Executes an SQL pass-through query that requires parameters and returns a recordset.
'           : Utilizes ADO rather than DAO.
'
' Author    : M. Minneman
'
' Params    : pSPROCName - (required) name of SPROC to be executed
'           : pParams - (required) one or more parameters required by SPROC
'
' Returns   : ADODB.Recordset - ResultSet
'
' Contract  : Dependencies
'           :   G_HANDLE_ERRORS - Global Boolean Constant
'           :   ImprovedErrorHandler.HandleError - Global error handler
'           :   ADODB - Microsoft AcitveX Data Objects Library
'           :   ADO_CONNECT_STRING - valid connect string
'           :   GeneralFunctions.doCloseAndRelease - CCL Function for cleaning up DAO objects
'           :
'           : Assumptions (routine may still work, but produce unexpected results)
'           :   pParams has one index that is 0-based
'           :
'           : Pre Conditions (must be true before execution)
'           :   pSPROCName - SPROC exists in ADODB.Connection
'           :
'           : Post Conditions (should be true after execution)
'           :   ADODB.Recordset has 0 to many records
'           :
'---------------------------------------------------------------------------------------------------------------------
'
' Change Log:
'
' Date      By              Comment
' 03/17/17  M. Minneman     created
'

    If G_HANDLE_ERRORS Then On Error GoTo ErrorHandler

    Dim oReturn As ADODB.RecordSet

    'db interface
    Dim cnn As New ADODB.Connection
    Dim cmd As New ADODB.Command
    Dim prm As New ADODB.Parameter

    ' Set CommandText equal to the stored procedure name.
    cmd.CommandText = pSPROCName
    cmd.CommandType = adCmdStoredProc

    ' Connect to the data source.
    cnn.Open ADO_CONNECT_STRING

    'validate connection
    If cnn.State <> adStateOpen Then
        Err.Raise vbObjectError, , "ADO Connection failed to open"
    End If

    'assign connection to command
    cmd.ActiveConnection = cnn

    'automatically fill in parameter info from stored procedure.
    cmd.Parameters.Refresh

    'make sure expected parameters and given arguments are equal
    If cmd.Parameters.Count <> UBound(pParams) + 2 Then
        Err.Raise vbObjectError, , "SPROC '" & pSPROCName & "' expects " & cmd.Parameters.Count & " arguments. " & UBound(pParams) & " provided."
    End If

    'set the param values.
    Dim i As Integer
    For i = 1 To cmd.Parameters.Count - 1
        cmd(i) = pParams(i - 1)
    Next i

    'execute SPROC
    Set oReturn = cmd.Execute

ExitHere:

    'housekeeping - failure okay
    On Error Resume Next
    'add cleanup here
    GeneralFunctions.doCloseAndRelease _
        prm, _
        cmd, _
        cnn

    'everything else - failure not okay
    On Error GoTo 0
    Set ExecuteSPWithParamsQueryADO = oReturn
    Exit Function

ErrorHandler:

    'local action
    'add local actions here

    'default action
    Select Case Err.Source
    Case "CONSUMED"
        Call MsgBox("Operation failed!", vbExclamation, "Message")
    Case Else
        Err.Source = "SQLStoredProcedureHelper.ExecuteSPWithParamsQueryADO"
        Select Case Err.Number
        Case Else
            HandleError , , , True         'rethrow
        End Select
    End Select
    Resume ExitHere
    Resume

End Function
</code>```

Body:
I'm evaluating whether or not SQL Server 2016 Always Encrypted will work with an existing MS Access 2010 application that I support.

Here's my current roadblock:

My application calls many SQL Server stored procedures that require parameters. I use the following function to make those calls:

Public Function ExecuteSPWithParamsQuery(poQDFStub As DAO.QueryDef, psParameterString As String) As DAO.Recordset

'-------------------------------------------------------------------------------------------------
' Purpose   : Execute an SQL pass-through query that calls a stored procedures requiring parameters.
'
' Params    : poQDFStub: pass through query with name of SPROC
'                : psParameterString : one or more parameters to be appended to poQDFStub
'
' Returns   : Dao.Recordset(dbOpenSnapshot)
'-------------------------------------------------------------------------------------------------
'

    If G_HANDLE_ERRORS Then On Error GoTo ErrorHandler

    Dim rstResult As DAO.Recordset

    'db interface
    Dim dbs As DAO.Database: Set dbs = CurrentDb
    Dim qdfResult As DAO.QueryDef: Set qdfResult = dbs.CreateQueryDef(vbNullString)

    'setup pass through
    With qdfResult
        .Connect = poQDFStub.Connect
        .SQL = poQDFStub.SQL & " " & psParameterString
        .ODBCTimeout = 0
        .ReturnsRecords = True
    End With

    'setup result
    Set rstResult = qdfResult.OpenRecordset(dbOpenSnapshot, dbSQLPassThrough + dbReadOnly + dbFailOnError)

ExitHere:

    'housekeeping
    On Error Resume Next
    'add cleanup here
    Set qdfResult = Nothing
    Set dbs = Nothing

    'exit protocol
    On Error GoTo 0
    Set ExecuteSPWithParamsQuery = rstResult
    Set rstResult = Nothing
    Exit Function

ErrorHandler:

    Err.Source = "SQLStoredProcedureHelper.ExecuteSPWithParamsQuery"
    HandleError
    Resume ExitHere

End Function

Calls to this function will now include parameters that are clear text versions of values encrypted in the database.

When this happens, I get the following error.

206 [Microsoft][ODBC SQL Server Driver][SQL Server] Operand type clash: varchar is incompatible > with nvarchar(255) encrypted with (encryption_type = 'DETERMINISTIC', encryption_algorithm_name
  = 'AEAD_AES_256_CBC_HMAC_SHA_256', column_encryption_key_name = 'CEK_Auto1',
  column_encryption_key_database_name = 'sandbox')

I've done some investigation on Always Encrypted Parameterization. It requires one of two technologies

.NET

ODBC 13.1 For SQL Server

Since this is an MS Access application, .NET is not applicable.

I have ODBC 13.1 installed, but I'm guessing that my pass through query is bypassing parameterization.

Here are my ODBC settings:

[ODBC]
DRIVER=ODBC Driver 13 for SQL Server
ColumnEncryption=Enabled
TrustServerCertificate=No
DATABASE=sandbox
WSID=********
APP=Microsoft Office 2010
Trusted_Connection=Yes
SERVER=*********

Any ideas on how I can resolve this problem or is Always Encrypted not a fit for my application?

Body:
The resolution to my problem was to convert my function from DAO to ADO.  Hope the following code helps someone else down the road:

Public Function ExecuteSPWithParamsQueryADO(pSPROCName As String, ParamArray pParams() As Variant) As ADODB.RecordSet

'---------------------------------------------------------------------------------------------------------------------
' Purpose   : Executes an SQL pass-through query that requires parameters and returns a recordset.
'           : Utilizes ADO rather than DAO.
'
' Author    : M. Minneman
'
' Params    : pSPROCName - (required) name of SPROC to be executed
'           : pParams - (required) one or more parameters required by SPROC
'
' Returns   : ADODB.Recordset - ResultSet
'
' Contract  : Dependencies
'           :   G_HANDLE_ERRORS - Global Boolean Constant
'           :   ImprovedErrorHandler.HandleError - Global error handler
'           :   ADODB - Microsoft AcitveX Data Objects Library
'           :   ADO_CONNECT_STRING - valid connect string
'           :   GeneralFunctions.doCloseAndRelease - CCL Function for cleaning up DAO objects
'           :
'           : Assumptions (routine may still work, but produce unexpected results)
'           :   pParams has one index that is 0-based
'           :
'           : Pre Conditions (must be true before execution)
'           :   pSPROCName - SPROC exists in ADODB.Connection
'           :
'           : Post Conditions (should be true after execution)
'           :   ADODB.Recordset has 0 to many records
'           :
'---------------------------------------------------------------------------------------------------------------------
'
' Change Log:
'
' Date      By              Comment
' 03/17/17  M. Minneman     created
'

    If G_HANDLE_ERRORS Then On Error GoTo ErrorHandler

    Dim oReturn As ADODB.RecordSet

    'db interface
    Dim cnn As New ADODB.Connection
    Dim cmd As New ADODB.Command
    Dim prm As New ADODB.Parameter

    ' Set CommandText equal to the stored procedure name.
    cmd.CommandText = pSPROCName
    cmd.CommandType = adCmdStoredProc

    ' Connect to the data source.
    cnn.Open ADO_CONNECT_STRING

    'validate connection
    If cnn.State <> adStateOpen Then
        Err.Raise vbObjectError, , "ADO Connection failed to open"
    End If

    'assign connection to command
    cmd.ActiveConnection = cnn

    'automatically fill in parameter info from stored procedure.
    cmd.Parameters.Refresh

    'make sure expected parameters and given arguments are equal
    If cmd.Parameters.Count <> UBound(pParams) + 2 Then
        Err.Raise vbObjectError, , "SPROC '" & pSPROCName & "' expects " & cmd.Parameters.Count & " arguments. " & UBound(pParams) & " provided."
    End If

    'set the param values.
    Dim i As Integer
    For i = 1 To cmd.Parameters.Count - 1
        cmd(i) = pParams(i - 1)
    Next i

    'execute SPROC
    Set oReturn = cmd.Execute

ExitHere:

    'housekeeping - failure okay
    On Error Resume Next
    'add cleanup here
    GeneralFunctions.doCloseAndRelease _
        prm, _
        cmd, _
        cnn

    'everything else - failure not okay
    On Error GoTo 0
    Set ExecuteSPWithParamsQueryADO = oReturn
    Exit Function

ErrorHandler:

    'local action
    'add local actions here

    'default action
    Select Case Err.Source
    Case "CONSUMED"
        Call MsgBox("Operation failed!", vbExclamation, "Message")
    Case Else
        Err.Source = "SQLStoredProcedureHelper.ExecuteSPWithParamsQueryADO"
        Select Case Err.Number
        Case Else
            HandleError , , , True         'rethrow
        End Select
    End Select
    Resume ExitHere
    Resume

End Function


问:
Can Chronicle Queue become a Single Point of Failure?
Is Chronicle Queue resilient?

For example, if I had 50,000 clients all sending data to Chronicle Queue very frequently, how can I guarantee that all messages have been stored on safely in a distributed form?

What if Chronicle Queue goes down?

Would it then become a single point of failure in my messaging fabric?

答:
Is Chronicle Queue resilient?

You can use chronicle-queue-enterprise ( a commercial product that extends the functionality of chronicle queue )  if you want to guarantee a copy to another machine.

What if Chronicle Queue goes down?

Chronicle-Queue-Enterprise will guarantee all your events are replicated

Would it then become a single point of failure in my messaging fabric?

If its replicated to another machine its no longer a single point of failure

If you are interested in chronicle queue enterprise please email sales@chronicle.software


答:
I think a more explicit and elegant way to do this is to simply use apply.

df = df.apply(pd.to_datetime, axis="columns")  # just to make sure values are datetime
df.apply(lambda x: x - df.index.to_series(), axis="rows)

Title: Operation on Pandas Dataframe columns using its Index

Body:
This should be relatively easy.  I have a pandas dataframe (Dates):

    A   B   C
1/8/2017    1/11/2017   1/20/2017   1/25/2017
1/9/2017    1/11/2017   1/20/2017   1/25/2017
1/10/2017   1/11/2017   1/20/2017   1/25/2017
1/11/2017   1/20/2017   1/25/2017   1/31/2017
1/12/2017   1/20/2017   1/25/2017   1/31/2017
1/13/2017   1/20/2017   1/25/2017   1/31/2017

I would like to take the difference between Dates.index and Dates.  The output would be like so:

    A   B   C
1/8/2017     3   12      17
1/9/2017     2   11      16
1/10/2017    1   10      15
1/11/2017    9   14      20
1/12/2017    8   13      19
1/13/2017    7   12      18

Naturally, I tried this:

Dates - Dates.index

But I receive this lovely TypeError:

TypeError: Could not operate DatetimeIndex...with block values ufunc subtract cannot use operands with types dtype('<M8[ns]') and dtype('O')

Instead, I've written a loop to go column by column, but that just seems silly. Can anyone suggest a pythonic way to do this?

EDIT

In [1]: import pandas as pd
import numpy as np
import datetime
dates = pd.date_range('20170108',periods=6)
df = pd.DataFrame(np.empty([len(dates),3]),index=dates,columns=list('ABC'))
df['A'].loc[0:3] = datetime.date(2017, 1, 11)
df['B'].loc[0:3] = datetime.date(2017, 1, 20)
df['C'].loc[0:3] = datetime.date(2017, 1, 25)
df['A'].loc[3:6] = datetime.date(2017, 1, 20)
df['B'].loc[3:6] = datetime.date(2017, 1, 25)
df['C'].loc[3:6] = datetime.date(2017, 1, 31)

In [2]: print(df)
                     A           B           C
2017-01-08  2017-01-11  2017-01-20  2017-01-25
2017-01-09  2017-01-11  2017-01-20  2017-01-25
2017-01-10  2017-01-11  2017-01-20  2017-01-25
2017-01-11  2017-01-20  2017-01-25  2017-01-31
2017-01-12  2017-01-20  2017-01-25  2017-01-31
2017-01-13  2017-01-20  2017-01-25  2017-01-31

In [3]: df = df.sub(df.index.to_series(),axis=0)

ValueError: operands could not be broadcast together with shapes (18,) (6,)

ContentLicense: CC BY-SA 4.0


答:
That's a very good question. Personally I think you can do this:

```<code># I suppose the keywords has non repeated words separated by a space
keywords_file = open('path_to_file_keywords')
keywords_dict = {word: 0 for word in keywords_file.readlines().strip().split(' ')} # Iterate through all the words removing '\n'characters and generate a dict

# Then read the chat log
chat_log_file = open('path_to_file_chat_log')
chat_log_words_generator = (word for word in chat_log_file.readlines().strip().split(' ')) # Create a generator with the words from the chat log

for word in chat_log_words_generator:
    try:
        word_count = keywords_dict[word]
    except KeyError:
        continue # The word is not a keyword
    word_count += 1 # increment the total
    keywords_dict[word] = word_count # override the value of the count in the dict
</code>```

In the end the keywords_dict should have the count of the occurrences of all the keywords.

Title: Comparing a text file to another text file in python?

Body:
That's a very good question. Personally I think you can do this:

# I suppose the keywords has non repeated words separated by a space
keywords_file = open('path_to_file_keywords')
keywords_dict = {word: 0 for word in keywords_file.readlines().strip().split(' ')} # Iterate through all the words removing '\n'characters and generate a dict

# Then read the chat log
chat_log_file = open('path_to_file_chat_log')
chat_log_words_generator = (word for word in chat_log_file.readlines().strip().split(' ')) # Create a generator with the words from the chat log

for word in chat_log_words_generator:
    try:
        word_count = keywords_dict[word]
    except KeyError:
        continue # The word is not a keyword
    word_count += 1 # increment the total
    keywords_dict[word] = word_count # override the value of the count in the dict

In the end the keywords_dict should have the count of the occurrences of all the keywords.


问:
Efficiently store list of matrices
I have a large list of images stored as numpy matrices. The images have different sizes e.g.

```<code>import numpy as np
from numpy.random import rand

data = [rand(100,200), rand(1024, 768)]
</code>```

I am looking for a way to store this list of matrices such that it can be read fast (writing the data can be slow). I tried pickle/numpy.savez, but reading the data was slower than loading the raw images again.

I think hdf5 may be fast, however I cannot figure out how to store this list. Not mandatory, but useful would data format which allows to append data such that the list does not have to in memory as a whole.

Edit:
Based on the answers so far I tried to time some suggestions

```<code>data = [rand(1024, 768) for i in np.arange(100)]
def timenp():
    np.savez("test.npz",*data)
    d=np.load('test.npz')
    loaded = [d[f] for f in d]

def timebinary():
    with file("tmp.bin","wb") as f:
        np.save(f, len(data))
        for img in data:
            np.save(f,img)

    with file("tmp.bin","rb") as f:
        n = np.load(f)
        loaded = []
        for i in np.arange(n):
            loaded.append(np.load(f))

import h5py
def timeh5py():
    with h5py.File('foo.hdf5','w') as f:
        dt = h5py.special_dtype(vlen=np.dtype('float32'))
        dset = f.create_dataset('data', (len(data),), dtype=dt)
        shapes = f.create_dataset('shapes', (len(data), 2), dtype='int32')
        dset[...] = [img.flatten() for img in data]
        shapes[...] = [img.shape for img in data]

    with h5py.File('foo.hdf5','r') as f:
        loaded=[]
        for (img, shape) in zip(f['data'],f['shapes']):
            loaded.append(np.reshape(img,shape))
</code>```

python -m cProfile timenp.py

           452906 function calls (451141 primitive calls) in 9.256 seconds

python -m cProfile timebinary.py
          73085 function calls (71340 primitive calls) in 4.945 seconds

python -m cProfile timeh5py.py

          33151 function calls (32568 primitive calls) in 4.384 seconds

答: Try using the numpy savez function , which comes in both compressed and uncompressed versions.</p>

Title: Efficiently store list of matrices

Body:
I have a large list of images stored as numpy matrices. The images have different sizes e.g.

import numpy as np
from numpy.random import rand

data = [rand(100,200), rand(1024, 768)]

I am looking for a way to store this list of matrices such that it can be read fast (writing the data can be slow). I tried pickle/numpy.savez, but reading the data was slower than loading the raw images again.

I think hdf5 may be fast, however I cannot figure out how to store this list. Not mandatory, but useful would data format which allows to append data such that the list does not have to in memory as a whole.

Edit:
Based on the answers so far I tried to time some suggestions

data = [rand(1024, 768) for i in np.arange(100)]
def timenp():
    np.savez("test.npz",*data)
    d=np.load('test.npz')
    loaded = [d[f] for f in d]

def timebinary():
    with file("tmp.bin","wb") as f:
        np.save(f, len(data))
        for img in data:
            np.save(f,img)

    with file("tmp.bin","rb") as f:
        n = np.load(f)
        loaded = []
        for i in np.arange(n):
            loaded.append(np.load(f))

import h5py
def timeh5py():
    with h5py.File('foo.hdf5','w') as f:
        dt = h5py.special_dtype(vlen=np.dtype('float32'))
        dset = f.create_dataset('data', (len(data),), dtype=dt)
        shapes = f.create_dataset('shapes', (len(data), 2), dtype='int32')
        dset[...] = [img.flatten() for img in data]
        shapes[...] = [img.shape for img in data]

    with h5py.File('foo.hdf5','r') as f:
        loaded=[]
        for (img, shape) in zip(f['data'],f['shapes']):
            loaded.append(np.reshape(img,shape))

python -m cProfile timenp.py

           452906 function calls (451141 primitive calls) in 9.256 seconds

python -m cProfile timebinary.py
          73085 function calls (71340 primitive calls) in 4.945 seconds

python -m cProfile timeh5py.py

          33151 function calls (32568 primitive calls) in 4.384 seconds


问:
Postgres - Lateral join with random values on both sides
I am trying to generate the following table:

```<code>random person a | random utility 1
random person a | random utility 2
random person a | random utility 3
random person b | random utility 4
random person b | random utility 5
random person b | random utility 6
</code>```

So, I want to generate two random persons and for each person, 3 random utilities. Generated utilities are not shared between persons.

I came up with the following query:

```<code>select              gen_random_uuid() person, utility.utility
from                generate_series(1, 2) person
inner join lateral  (select person, gen_random_uuid() utility from generate_series(1, 3)) utility
on                  person.person = utility.person;
</code>```

But this generates the following:

```<code>64bf364b-ff74-4888-b2d9-0ffcf3147dd7 | f8455aa3-73ab-4432-9c49-2c940da28fa7
05f54217-0316-410d-83e8-2b5306eee143 | 68f7b48-2561-438c-a906-34b141c4adc5
07c23fc3-a6b9-4d74-a611-264c4de9a0bd | 5a597190-09f9-44ea-960d-f6657aa10a81
b5903b45-f96a-4eb4-8b67-95340131a29b | 742daba-de73-45c4-8b2b-758663c4af47
6ba75082-77e4-408b-9e19-92595121cf43 | 01635986-98e3-432e-9f4e-0a88edba3d67
48d7ab6f-4a32-4306-b060-f0019aef1d11 | 99f4e91f-c05f-496c-a4a3-7b62f64ed6e1
</code>```

So, as you can see, the result contains 6 random persons, with each person his/her own random utility.

As said, I need 2 random persons with each 3 random utlities. How can I fix this query?

Body:
I am trying to generate the following table:

random person a | random utility 1
random person a | random utility 2
random person a | random utility 3
random person b | random utility 4
random person b | random utility 5
random person b | random utility 6

So, I want to generate two random persons and for each person, 3 random utilities. Generated utilities are not shared between persons.

I came up with the following query:

select              gen_random_uuid() person, utility.utility
from                generate_series(1, 2) person
inner join lateral  (select person, gen_random_uuid() utility from generate_series(1, 3)) utility
on                  person.person = utility.person;

But this generates the following:

64bf364b-ff74-4888-b2d9-0ffcf3147dd7 | f8455aa3-73ab-4432-9c49-2c940da28fa7
05f54217-0316-410d-83e8-2b5306eee143 | 68f7b48-2561-438c-a906-34b141c4adc5
07c23fc3-a6b9-4d74-a611-264c4de9a0bd | 5a597190-09f9-44ea-960d-f6657aa10a81
b5903b45-f96a-4eb4-8b67-95340131a29b | 742daba-de73-45c4-8b2b-758663c4af47
6ba75082-77e4-408b-9e19-92595121cf43 | 01635986-98e3-432e-9f4e-0a88edba3d67
48d7ab6f-4a32-4306-b060-f0019aef1d11 | 99f4e91f-c05f-496c-a4a3-7b62f64ed6e1

So, as you can see, the result contains 6 random persons, with each person his/her own random utility.

As said, I need 2 random persons with each 3 random utlities. How can I fix this query?

Body:
select *
from
    (
        select gen_random_uuid() as person
        from generate_series(1,2)
    ) p
    inner join lateral
    (
        select gen_random_uuid() as utility, p.person
        from generate_series(1,3)
    ) u using (person)
;
                person                |               utility
--------------------------------------+--------------------------------------
 d8655c1d-5051-4f5c-929e-efb92a2e6a93 | 1524b662-f2a3-4bfa-bc47-71b96a8e6121
 d8655c1d-5051-4f5c-929e-efb92a2e6a93 | e38190f4-bf36-49f9-92c0-11fa18113aad
 d8655c1d-5051-4f5c-929e-efb92a2e6a93 | 799a9e39-f954-4d35-8b9a-1fe444d91ccf
 3f581430-75be-4c07-aff1-e1c2c802b463 | 3261f4f9-4c28-4709-9b38-cf3ce287317e
 3f581430-75be-4c07-aff1-e1c2c802b463 | f9bbadaf-7c68-48f2-bb9b-d57e2902e3f1
 3f581430-75be-4c07-aff1-e1c2c802b463 | 158b8e0d-5b22-4f4d-8052-589aaa224b2e


问:
Reverse an array without using Array.Reverse()
How to reverse an array (in C#) without using Array.Reverse() method?

For example,

```<code>int[] arr = {1,3,4,9,8};
// some code here
Console.WriteLine(string.Join(",", arr));
</code>```

should result in

```<code>8,9,4,3,1
</code>```

I got this as an interview task.

答:
Basically, you are asked to reimplement Array.Reverse(Array). If you look at how it is implemented in the framework itself and ignore many technical details around, you’ll find that it just calls its three-parameter version (which reverses specified part of an array) on the whole array.

Array.Reverse(Array,Int32,Int32) is a while-loop that swaps elements and maintains two indexes:

i points to the first element of the reversed part, and

j points to the last element of the reversed part.

Rewritten to be substituted in place of // some code here in the question:

```<code>int i = 0;
int j = arr.Length - 1;
while (i < j)
{
    var temp = arr[i];
    arr[i] = arr[j];
    arr[j] = temp;
    i++;
    j--;
}
</code>```

This is easier to grasp than the implementation using for-loop, does less arithmetic and elegantly evades the gotcha with double reversion.

Body:
How to reverse an array (in C#) without using Array.Reverse() method?

For example,

int[] arr = {1,3,4,9,8};
// some code here
Console.WriteLine(string.Join(",", arr));

should result in

8,9,4,3,1

I got this as an interview task.

Body:
Basically, you are asked to reimplement Array.Reverse(Array). If you look at how it is implemented in the framework itself and ignore many technical details around, you’ll find that it just calls its three-parameter version (which reverses specified part of an array) on the whole array.

Array.Reverse(Array,Int32,Int32) is a while-loop that swaps elements and maintains two indexes:

i points to the first element of the reversed part, and

j points to the last element of the reversed part.

Rewritten to be substituted in place of // some code here in the question:

int i = 0;
int j = arr.Length - 1;
while (i < j)
{
    var temp = arr[i];
    arr[i] = arr[j];
    arr[j] = temp;
    i++;
    j--;
}

This is easier to grasp than the implementation using for-loop, does less arithmetic and elegantly evades the gotcha with double reversion.


问:
Determining an estimator for two parameters in R over a sum, given binary data
I have the following sum (it is a score-vector to a partial likelihood) and want to find the values for beta_0 and beta_1 for which the equation is equal to zero.

The Y_t’s {t=0,...n} are known measurements at different points in time and can only be 0 or 1 (binary data).

Here is the code I came up with so far. I tried to implement the function, which should be minimized:

```<code>    y = c(1,0,1,1,1,1,0)

    f <- function(beta_0, beta_1){

      sum = matrix(c(0, 0), ncol = 1, nrow = 2)

      for(t in 2:length(y))
      {
        sum = sum + matrix(c(1, y[t-1]), ncol = 1, nrow = 2)  %*% c(y[t] - (exp(beta_0 + beta_1 * y[t-1])) / (1 + exp(beta_0 + beta_1 * y[t-1])))

      }

      return(sum)
    }
    #Just making a guess:
    f(0.5,0.5)
</code>```

There is no analytical solution for this and therefor I’d like to use R to determine both parameters (beta_0 and beta_1) given a set of binary data. To find the numerical solution,  I think some kind of iterative solution is the way to go.

So far I have looked at different optimization packages in R but couldn’t find the right one.

One of the problems I'm facing is that the result of the function is an vector and the optimization approaches I have seen so far, all just use an output of one variable.

Could you recommend a package for this problem, or any other way I could solve this? Maybe treat the Vector as one number, by adding the absolute values of it?

Thanks in advance!

答:
I was able to solve the problem by using absolute values and optim:</p>

```<code> estimate_parameter_binary_model  <- function(y = y) {

  #Function to be optimized:
  optimization_function_binary_model <- function(p = c(1,1)){ #to use the optim function, it has to be called by a vector p. Which contains: p[1] = beta_0 and p[2] = beta_1

    #p is an vector and contains both parameters
    beta_0 = p[1]
    beta_1 = p[2]

    sum = matrix(c(0, 0), ncol = 1, nrow = 2)

    for(t in 2:length(y))
    {
      sum = sum + matrix(c(1, y[t-1]), ncol = 1, nrow = 2)  %*% c(y[t] - (exp(beta_0 + beta_1 * y[t-1])) / (1 + exp(beta_0 + beta_1 * y[t-1])))

    }

    return(abs(sum[1]) + abs(sum[2])) #optim just needs one number, so we use absolute values here and add them.
  }

  #Call optim()

  par = optim(c(0,0), optimization_function_binary_model)$par

  Solution<- list(beta_0 = par[1],beta_1 = par[2])

  return(Solution)
}
</code>```

Body:
I have the following sum (it is a score-vector to a partial likelihood) and want to find the values for beta_0 and beta_1 for which the equation is equal to zero.

The Y_t’s {t=0,...n} are known measurements at different points in time and can only be 0 or 1 (binary data).

Here is the code I came up with so far. I tried to implement the function, which should be minimized:

    y = c(1,0,1,1,1,1,0)

    f <- function(beta_0, beta_1){

      sum = matrix(c(0, 0), ncol = 1, nrow = 2)

      for(t in 2:length(y))
      {
        sum = sum + matrix(c(1, y[t-1]), ncol = 1, nrow = 2)  %*% c(y[t] - (exp(beta_0 + beta_1 * y[t-1])) / (1 + exp(beta_0 + beta_1 * y[t-1])))

      }

      return(sum)
    }
    #Just making a guess:
    f(0.5,0.5)

There is no analytical solution for this and therefor I’d like to use R to determine both parameters (beta_0 and beta_1) given a set of binary data. To find the numerical solution,  I think some kind of iterative solution is the way to go.

So far I have looked at different optimization packages in R but couldn’t find the right one.

One of the problems I'm facing is that the result of the function is an vector and the optimization approaches I have seen so far, all just use an output of one variable.

Could you recommend a package for this problem, or any other way I could solve this? Maybe treat the Vector as one number, by adding the absolute values of it?

Thanks in advance!

Body:
I was able to solve the problem by using absolute values and optim:

 estimate_parameter_binary_model  <- function(y = y) {

  #Function to be optimized:
  optimization_function_binary_model <- function(p = c(1,1)){ #to use the optim function, it has to be called by a vector p. Which contains: p[1] = beta_0 and p[2] = beta_1

    #p is an vector and contains both parameters
    beta_0 = p[1]
    beta_1 = p[2]

    sum = matrix(c(0, 0), ncol = 1, nrow = 2)

    for(t in 2:length(y))
    {
      sum = sum + matrix(c(1, y[t-1]), ncol = 1, nrow = 2)  %*% c(y[t] - (exp(beta_0 + beta_1 * y[t-1])) / (1 + exp(beta_0 + beta_1 * y[t-1])))

    }

    return(abs(sum[1]) + abs(sum[2])) #optim just needs one number, so we use absolute values here and add them.
  }

  #Call optim()

  par = optim(c(0,0), optimization_function_binary_model)$par

  Solution<- list(beta_0 = par[1],beta_1 = par[2])

  return(Solution)
}


问:
Height 100% and css issue with footer
I am currently working on a script and in some pages there isn't enough content.In this case I want the page to cover 100% of the browser and put the footer at the bottom.I tried many codes and nothing seems to work I ended up by having a code like this:

```<code><div class="container">
<div id=nav>
NAV
</div>
<div id=core>
    <div id=content>
        <div id=tophea>
           TOP Content
        </div>
        <div id=msgs>
       MSG Content
        </div>
    </div>
    <div id="footer">
    Footer
    </div>
    </div>
</div>
</code>```

and css like this :

```<code>#container{
  height:100%
}
#nav{
  height:55px;
}
#core{
  height:100%
}
#content{
  height:100%;
  background:red;
}
</code>```

Here is my jsfiddle : https://jsfiddle.net/k8k7o36b/

Any help will be appreciate. I'll be more than thankful if you add small explanation so I can understand what were I doing wrong.
Thanks

答:
html,body,#container{
  height:100%
}
#nav{
  height:55px;
}
#core{
  height:100%
}
#content{
  height:100%;
  background:red;
}
#footer{
  position:absolute;bottom:0;
  right:0;left:0
}```
<pre class="snippet-code-html lang-html prettyprint-override"><code><div class="container">
    <div id=nav>
    NAV
    </div>
    <div id=core>
        <div id=content>
            <div id=tophea>
               TOP Content
            </div>
            <div id=msgs>
           MSG Content
            </div>
        </div>
        <div id="footer">
        Footer
        </div>
        </div>
    </div>
</div></code>```

Add these classes

```<code>html,body{
  height:100%
}

#footer{
  position:absolute;bottom:0;
  right:0;left:0
}
</code>```

Title: Height 100% and css issue with footer

Body:
I am currently working on a script and in some pages there isn't enough content.In this case I want the page to cover 100% of the browser and put the footer at the bottom.I tried many codes and nothing seems to work I ended up by having a code like this:

NAV

           TOP Content

       MSG Content

    Footer

and css like this :

#container{
  height:100%
}
#nav{
  height:55px;
}
#core{
  height:100%
}
#content{
  height:100%;
  background:red;
}

Here is my jsfiddle : https://jsfiddle.net/k8k7o36b/

Any help will be appreciate. I'll be more than thankful if you add small explanation so I can understand what were I doing wrong.
Thanks

Body:
html,body,#container{
  height:100%
}
#nav{
  height:55px;
}
#core{
  height:100%
}
#content{
  height:100%;
  background:red;
}
#footer{
  position:absolute;bottom:0;
  right:0;left:0
}

<div class="container">
    <div id=nav>
    NAV
    </div>
    <div id=core>
        <div id=content>
            <div id=tophea>
               TOP Content
            </div>
            <div id=msgs>
           MSG Content
            </div>
        </div>
        <div id="footer">
        Footer
        </div>
        </div>
    </div>
</div>

Add these classes

html,body{
  height:100%
}

#footer{
  position:absolute;bottom:0;
  right:0;left:0
}


问:
How can i add an 'unversioned file' to github?
I am trying to use GitHub for the first time and have managed to upload a small project i am working on.

The problem is that all of my files but one are being added to GitHub - i created this file after committing the project and have tried right clicking and 'ADD' but this does not seem to work.

Does anyone have any ideas of what the problem could be? I understand this is probably a fault on my behalf, i am just not able to figure out what i must do to rectify it.

答:
The most common reason for this is there is no change seen.  Try making another change to the file, even if you just add spaces to the end of one of he lines. Then save and try the commit again.

I had the same problem after creating the repository and committing; the only file committed was the MainActivity.java file, all other files were showing as 'unversioned file'.

Make sure to change your Project Window view to 'Project'.
Then right click your project and select Git --> +Add.
All files will be updated and the "unversioned" list will be gone.


问:
Table rows toggle() with jQuery and DataTable
I have a small table with some rows and some of them with extra information which will be show or hide with jQuery toggle(). Everything works fine but I can't use DataTable in this context. My jQuery looks like this:

```<code>

    $(document).ready(function() {
        $('#serien tbody').on('click', 'td.details-control', function () {

            if ( $(this).hasClass('showDetails') ) {
                $(this).closest('tr').removeClass('shown');
            }
            else {
                $(this).closest('tr').addClass('shown');
            }

            $(this).toggleClass('showDetails');
            $(this).parent().next().toggle();
        });
    });

</code>```

But when I want to use DataTable - in the easiest way like https://datatables.net/examples/advanced_init/events_live.html

```<code>

    $(document).ready(function() {
        var table = $('#serien').DataTable();

        $('#serien tbody').on('click', 'td.details-control', function () {
            var data = table.row( this ).data();
            alert( 'You clicked on '+data[0]+'\'s row' );
        } );
    } );

</code>```

It won't work, can somebody give me a hint?

答:
A part of the table looks like

```</p>

```

      The Avengers

      Name
      Vorname

      Peel
      Emma

            Name	Vorname

            Rigg	Diana

      ....

```

<p></code>```

Body:
I have a small table with some rows and some of them with extra information which will be show or hide with jQuery toggle(). Everything works fine but I can't use DataTable in this context. My jQuery looks like this:

    $(document).ready(function() {
        $('#serien tbody').on('click', 'td.details-control', function () {

            if ( $(this).hasClass('showDetails') ) {
                $(this).closest('tr').removeClass('shown');
            }
            else {
                $(this).closest('tr').addClass('shown');
            }

            $(this).toggleClass('showDetails');
            $(this).parent().next().toggle();
        });
    });

But when I want to use DataTable - in the easiest way like https://datatables.net/examples/advanced_init/events_live.html

    $(document).ready(function() {
        var table = $('#serien').DataTable();

        $('#serien tbody').on('click', 'td.details-control', function () {
            var data = table.row( this ).data();
            alert( 'You clicked on '+data[0]+'\'s row' );
        } );
    } );

It won't work, can somebody give me a hint?

Body:
A part of the table looks like

<table id="serien" class="table table-hover">
  <thead>
    <tr class="success">
      <th colspan="3">The Avengers</th>
    </tr>
    <tr>
      <th></th>
      <th>Name</th>
      <th>Vorname</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td class="details-control"></td>
      <td>Peel</td>
      <td>Emma</td>
    </tr>
    <tr class="showDetails">
      <td colspan="3">
        <table class="table table-bordered">
          <tr>
            <th>Name</th><th>Vorname</th>
          </tr>
          <tr>
            <td>Rigg</td><td>Diana</td>
          </tr>
        </table>
        </td>
      </tr>
      ....
    </tbody>
</table>


问:
How to I assign the ttf font file in the vfs_fonts.js?
I am trying to add the custom fonts into vfs_fonts.js but I didn't understand what needs to be written in the value field of window.pdfMake object?

I have my key as "MyFont.ttf" and I know that value is nothing but ttf file. My ttf is a physical file not a set of characters. How do I fill the value field?

```<code> window.pdfMake = window.pdfMake || {}; window.pdfMake.vfs = {
      "Roboto-Italic.ttf": "AAEAAAASAQAABA",
      "Roboto-Medium.ttf": "AAEAAA",
      "MyFont.ttf":"???????????????????"
    }
</code>```

Thank you

答:
```<code>/*  Read the documenation at https://github.com/bpampuch/pdfmake
specially the readme file */

<!doctype html>
<html lang='en'>
<head>
   <meta charset='utf-8'>
   <title>my first pdfmake example</title>
   <script src='build/pdfmake.min.js'></script>
   <script src='build/vfs_fonts.js'></script>
</head>
<body>

<script>

// works great with the default vfs_fonts.js

   pdfMake.fonts = {
      Roboto: {
         normal: 'Roboto-Regular.ttf',
         bold: 'Roboto-Medium.ttf',
         italics: 'Roboto-Italic.ttf',
         bolditalics: 'Roboto-MediumItalic.ttf'
      }
   };

 // open the PDF in a new window
   pdfMake.createPdf(docDefinition).open();

</script>

</body>
</code>```

Body:
I am trying to add the custom fonts into vfs_fonts.js but I didn't understand what needs to be written in the value field of window.pdfMake object?

I have my key as "MyFont.ttf" and I know that value is nothing but ttf file. My ttf is a physical file not a set of characters. How do I fill the value field?

 window.pdfMake = window.pdfMake || {}; window.pdfMake.vfs = {
      "Roboto-Italic.ttf": "AAEAAAASAQAABA",
      "Roboto-Medium.ttf": "AAEAAA",
      "MyFont.ttf":"???????????????????"
    }

Thank you