问:
Is there a way to change tsconfig to ignore certain libraries?
My codebase is a mix of typescript + javascript. When importing a javascript library into a typescript library, i just use the // @ts-ignore. But i'm using this javascript library quite frequently and it's been tedious to constantly put //@ts-ignore on top of the import. Is there a way to globally ignore the non-ts libraries?</p>

```<code>//@ts-ignore
import jsLibraryThing from "@jsLibraryThing"
</code>```

答:
You can create a declaration file for this library that just exports any. This will allow you to import it without any error and the import will be typed as any so you can use it almost like in JS

```<code>// jsLibraryThing.d.ts
declare module '@jsLibraryThing' {
    const e: any;
    export default e
 }

//usage.ts
import jsLibraryThing from "@jsLibraryThing"
jsLibraryThing.something // all fine
</code>```

Body:
My codebase is a mix of typescript + javascript. When importing a javascript library into a typescript library, i just use the // @ts-ignore. But i'm using this javascript library quite frequently and it's been tedious to constantly put //@ts-ignore on top of the import. Is there a way to globally ignore the non-ts libraries?

//@ts-ignore
import jsLibraryThing from "@jsLibraryThing"

Body:
You can create a declaration file for this library that just exports any. This will allow you to import it without any error and the import will be typed as any so you can use it almost like in JS

// jsLibraryThing.d.ts
declare module '@jsLibraryThing' {
    const e: any;
    export default e
 }

//usage.ts
import jsLibraryThing from "@jsLibraryThing"
jsLibraryThing.something // all fine


来源: stackexchange

问:
How to use curve_fit from scipy.optimize with a shared fit parameter across multiple datasets?
Assuming I have a fit function f with multiple parameters, for example a and b. Now I want to fit multiple datasets to this function and use the same a for all of them (the shared parameter) while b can be individual for each fit.

Example:

import numpy as np

# Fit function
def f(x, a, b):
    return a * x + b

# Datasets
x = np.arange(4)
y = np.array([x + a + np.random.normal(0, 0.5, len(x)) for a in range(3)])
```

So we have 4 x values and 3 datasets of 4 y values each.

答:
One way to do this is to concatenate the datasets and use an adjusted fit function.

In the following example, this happens inside the new fit function g with np.concatenate. The individual fits for each dataset are also done so we can compare their graph to the concatenated fit with the shared parameter.

import matplotlib.pyplot as plt
import numpy as np
from scipy.optimize import curve_fit

# Create example datasets

x = np.arange(4)
y = np.array([x + a + np.random.normal(0, 0.5, len(x)) for a in range(3)])
print("x =", x)
print("y =", y)

# Individual fits to each dataset

def f(x, a, b):
    return a * x + b

for y_i in y:
    (a, b), _ = curve_fit(f, x, y_i)
    plt.plot(x, f(x, a, b), label=f"{a:.1f}x{b:+.1f}")
    plt.plot(x, y_i, linestyle="", marker="x", color=plt.gca().lines[-1].get_color())

plt.legend()
plt.show()

# Fit to concatenated dataset with shared parameter

def g(x, a, b_1, b_2, b_3):
    return np.concatenate((f(x, a, b_1), f(x, a, b_2), f(x, a, b_3)))

(a, *b), _ = curve_fit(g, x, y.ravel())
for b_i, y_i in zip(b, y):
    plt.plot(x, f(x, a, b_i), label=f"{a:.1f}x{b_i:+.1f}")
    plt.plot(x, y_i, linestyle="", marker="x", color=plt.gca().lines[-1].get_color())

plt.legend()
plt.show()
```

<p>Output:</p>

```x = [0 1 2 3]
y = [[0.40162683 0.65320576 1.92549698 2.9759299 ]
 [1.15804251 1.69973973 3.24986941 3.25735249]
 [1.97214167 2.60206217 3.93789235 6.04590999]]
```

Individual fits with three different values for a:

Fit with shared parameter a:

ContentLicense: CC BY-SA 4.0

Body:
Assuming I have a fit function f with multiple parameters, for example a and b. Now I want to fit multiple datasets to this function and use the same a for all of them (the shared parameter) while b can be individual for each fit.

Example:

import numpy as np

# Fit function
def f(x, a, b):
    return a * x + b

# Datasets
x = np.arange(4)
y = np.array([x + a + np.random.normal(0, 0.5, len(x)) for a in range(3)])

So we have 4 x values and 3 datasets of 4 y values each.

Body:
One way to do this is to concatenate the datasets and use an adjusted fit function.

In the following example, this happens inside the new fit function g with np.concatenate. The individual fits for each dataset are also done so we can compare their graph to the concatenated fit with the shared parameter.

import matplotlib.pyplot as plt
import numpy as np
from scipy.optimize import curve_fit

# Create example datasets

x = np.arange(4)
y = np.array([x + a + np.random.normal(0, 0.5, len(x)) for a in range(3)])
print("x =", x)
print("y =", y)

# Individual fits to each dataset

def f(x, a, b):
    return a * x + b

for y_i in y:
    (a, b), _ = curve_fit(f, x, y_i)
    plt.plot(x, f(x, a, b), label=f"{a:.1f}x{b:+.1f}")
    plt.plot(x, y_i, linestyle="", marker="x", color=plt.gca().lines[-1].get_color())

plt.legend()
plt.show()

# Fit to concatenated dataset with shared parameter

def g(x, a, b_1, b_2, b_3):
    return np.concatenate((f(x, a, b_1), f(x, a, b_2), f(x, a, b_3)))

(a, *b), _ = curve_fit(g, x, y.ravel())
for b_i, y_i in zip(b, y):
    plt.plot(x, f(x, a, b_i), label=f"{a:.1f}x{b_i:+.1f}")
    plt.plot(x, y_i, linestyle="", marker="x", color=plt.gca().lines[-1].get_color())

plt.legend()
plt.show()

Output:

x = [0 1 2 3]
y = [[0.40162683 0.65320576 1.92549698 2.9759299 ]
 [1.15804251 1.69973973 3.24986941 3.25735249]
 [1.97214167 2.60206217 3.93789235 6.04590999]]

Individual fits with three different values for a:

Fit with shared parameter a:


问:
How to extract all .tgz files in subdirectories?
I have a list of *tgz files in a ./main/ directory such as

```<code>$ tree -L 3
.
├── 191009-Grace_587_8G_R2
│   └── Grace_587_8G_R2
│       └── output.tgz
├── 191009-Grace_V0G_R2
│   └── Grace_V0G_R2
│       └── output.tgz
├── 191009-Grace_V8G_R2
│   └── Grace_V8G_R2
│       └── output.tgz
├── 191014-Grace_587_0G_R2
│   └── Grace_587_0G_R2
│       └── output.tgz
├── 191014-Grace_587_8G_R2
│   └── Grace_587_8G_R2
│       └── output.tgz
├── 191014-Grace_V0G_R2
│   └── Grace_V0G_R2
│       └── output.tgz
└── 191014-Grace_V8G_R2
    └── Grace_V8G_R2
        └── output.tgz
</code>```

I am wondering how to extract them all together to the directory containing them.

答:
Using tar's  -C / --directory option:

-C, --directory=DIR

    Change to DIR before performing any operations. This option is order-sensitive, i.e. it affects all options that follow.

```<code>for i in *-Grace_*/Grace_*/output.tgz; do
  tar xzf "$i" --directory="${i%/*}"
done
</code>```

The parameter expansion ${i%/*} removes the filename from the path (like the dirname command). To extract the files to the main directory, remove the --directory option.

Using find with the -execdir option:

```<code>find . -type f -name 'output.tgz' -execdir tar xfz {} +
</code>```

Body:
I have a list of *tgz files in a ./main/ directory such as

$ tree -L 3
.
├── 191009-Grace_587_8G_R2
│   └── Grace_587_8G_R2
│       └── output.tgz
├── 191009-Grace_V0G_R2
│   └── Grace_V0G_R2
│       └── output.tgz
├── 191009-Grace_V8G_R2
│   └── Grace_V8G_R2
│       └── output.tgz
├── 191014-Grace_587_0G_R2
│   └── Grace_587_0G_R2
│       └── output.tgz
├── 191014-Grace_587_8G_R2
│   └── Grace_587_8G_R2
│       └── output.tgz
├── 191014-Grace_V0G_R2
│   └── Grace_V0G_R2
│       └── output.tgz
└── 191014-Grace_V8G_R2
    └── Grace_V8G_R2
        └── output.tgz

I am wondering how to extract them all together to the directory containing them.

Body:
Using tar's  -C / --directory option:

-C, --directory=DIR

    Change to DIR before performing any operations. This option is order-sensitive, i.e. it affects all options that follow.

for i in *-Grace_*/Grace_*/output.tgz; do
  tar xzf "$i" --directory="${i%/*}"
done

The parameter expansion ${i%/*} removes the filename from the path (like the dirname command). To extract the files to the main directory, remove the --directory option.

Using find with the -execdir option:

find . -type f -name 'output.tgz' -execdir tar xfz {} +


问:
Failed to use .NET SDK agent in Azure DevOps Build Pipeline
I have a ASP.NET Core 2.1 and added a nuget package of  Microsoft.WindowsAzure.Storage , But after pushing the code to repo, the build pipeline occurs package error while running the build agent and is it necessary to add any other agents in build pipeline other than  BUILD, RESTORE, TEST and PUBLISH.

[error]Error: The process 'C:\hostedtoolcache\windows\dotnet\dotnet.exe' failed with exit code 1

Info: Azure Pipelines hosted agents have been updated to contain .Net Core 3.x SDK/Runtime along with 2.2 & 2.1. Unless you have locked down a SDK version for your project(s), 3.x SDK might be picked up which might have breaking behavior as compared to previous versions.

This is the error it shows while running the pipeline.

答:
I encountered the same issue with version 2.1.505 and now I am using the following configuration for variables and installer step for .NET Core in my yaml pipeline while using version 3.1.101 and it fixed my issue.</p>

```<code>variables:
  buildConfiguration: 'Release'
  dotnetSdkVersion: '3.1.101'

steps:
- task: DotNetCoreInstaller@0
  displayName: 'Use .NET Core SDK $(dotnetSdkVersion)'
  inputs:
    version: '$(dotnetSdkVersion)'
</code>```

Body:
I encountered the same issue with version 2.1.505 and now I am using the following configuration for variables and installer step for .NET Core in my yaml pipeline while using version 3.1.101 and it fixed my issue.

variables:
  buildConfiguration: 'Release'
  dotnetSdkVersion: '3.1.101'

steps:
- task: DotNetCoreInstaller@0
  displayName: 'Use .NET Core SDK $(dotnetSdkVersion)'
  inputs:
    version: '$(dotnetSdkVersion)'


问:
How can you display your logs when testing android apps on real devices?
So I am developing my first app with Android Studio, it's a Shopping app and everything seems to work just fine when I try it out on the built-in Emulator (Android 9). However, when I connect my Smartphone (same Android version), I won't get pass my Login screen. Currently the data is stored through SQLite, which may cause issues.
I'd really like to look up what's the exact problem, but there is no Log feed showing up in Android Studios.

Are there any settings that provide the same Logs as if I was running it on an Emulator?

答: Make sure you have selected right device i.e there is a dropdown option for devices in logcat check that and choose your device because it will be set to the previous device by default</p>


答:
I came up with a solution but you'll have to alter the input spreadsheet slightly. It should look like the table below:

```<code>         Label Operator Condition Value
0       gender                 ==     F
1       gender        |        ==     M
2          Age        &        >=    75
3          Age        &        >=    45
4    Indicator        &        ==     Y
5  X Indicator        &        ==     Y
6        State        &        ==    NJ
</code>```

The table lists the filtering criteria. Conditions applied to the same criteria must be written in consecutive rows for the code to work. In your example, gender and age.

Your table:

```<code>        gender  Age Indicator X Indicator State foreign_ind Eu Resident address1
Cust Id
987685       M   65         Y           N    TX           N           N  XYZ,USA
987686       F   54         Y           N    NJ           N           N  XYZ,USA
987687       M   75         Y           Y    NJ           N           N  XYZ,USA
987688       M   45         N           Y    NY           N           N  XYS,USA
987689       F   45         Y           Y    NJ           N           N  XYS,USA
</code>```

Output:

```<code>        gender  Age Indicator X Indicator State foreign_ind Eu Resident address1
Cust Id
987687       M   75         Y           Y    NJ           N           N  XYZ,USA
987689       F   45         Y           Y    NJ           N           N  XYS,USA
</code>```

Code:

```<code>i = 0
filter_string = 'df'
while i < conditions.shape[0]:
    if i == conditions.shape[0] - 1:
        if conditions.iloc[i,0] == conditions.iloc[i-1,0]:
            label = conditions.iloc[i,0]
            operator = conditions.iloc[i,1]
            condition = conditions.iloc[i,2]
            value = conditions.iloc[i,3]
            if type(value) == 'int':
                string = str(operator + '(df[\'' + label + '\']' + condition + str(value) + '))]')
                filter_string += string
                i += 1
            else:
                string = str(operator + '(df[\'' + label + '\']' + condition + '\''+ str(value) + '\'))]')
                filter_string += string
                i += 1
        else:
            label = conditions.iloc[i,0]
            operator = conditions.iloc[i,1]
            condition = conditions.iloc[i,2]
            value = conditions.iloc[i,3]
            if type(value) == 'int':
                string = str(operator + '(df[\'' + label + '\']' + condition + str(value) + ')]')
                filter_string += string
                i += 1
            else:
                string = str(operator + '(df[\'' + label + '\']' + condition + '\''+ str(value) + '\')]')
                filter_string += string
                i += 1
    elif i == 0:
        if conditions.iloc[i,0] == conditions.iloc[i+1,0]:
            value = conditions.iloc[i,3]
            label = conditions.iloc[i,0]
            operator = conditions.iloc[i,1]
            condition = conditions.iloc[i,2]
            value = conditions.iloc[i,3]
            if type(value) == 'int':
                string = operator + '[((df[\'' + label + '\']' + condition + str(value) + ')'
                filter_string += string
                i += 1
            else:
                string = operator + '[((df[\'' + label + '\']' + condition + '\''+ str(value) + '\')'
                filter_string += string
                i += 1
        else:
            value = conditions.iloc[i,3]
            label = conditions.iloc[i,0]
            operator = conditions.iloc[i,1]
            condition = conditions.iloc[i,2]
            value = conditions.iloc[i,3]
            if type(value) == 'int':
                string = operator + '[(df[\'' + label + '\']' + condition + str(value) + ')'
                filter_string += string
                i += 1
            else:
                string = operator + '[(df[\'' + label + '\']' + condition + '\''+ str(value) + '\')'
                filter_string += string
                i += 1
    else:
        if conditions.iloc[i,0] == conditions.iloc[i+1,0]:
            value = conditions.iloc[i,3]
            label = conditions.iloc[i,0]
            operator = conditions.iloc[i,1]
            condition = conditions.iloc[i,2]
            value = conditions.iloc[i,3]
            if type(value) == 'int':
                string = operator + '((df[\'' + label + '\']' + condition + str(value) + ')'
                filter_string += string
                i += 1
            else:
                string = operator + '((df[\'' + label + '\']' + condition + '\''+ str(value) + '\')'
                filter_string += string
                i += 1
        elif conditions.iloc[i,0] == conditions.iloc[i-1,0]:
            value = conditions.iloc[i,3]
            label = conditions.iloc[i,0]
            operator = conditions.iloc[i,1]
            condition = conditions.iloc[i,2]
            value = conditions.iloc[i,3]
            if type(value) == 'int':
                string = operator + '(df[\'' + label + '\']' + condition + str(value) + '))'
                filter_string += string
                i += 1
            else:
                string = operator + '(df[\'' + label + '\']' + condition + '\''+ str(value) + '\'))'
                filter_string += string
                i += 1
        else:
            value = conditions.iloc[i,3]
            label = conditions.iloc[i,0]
            operator = conditions.iloc[i,1]
            condition = conditions.iloc[i,2]
            value = conditions.iloc[i,3]
            if type(value) == 'int':
                string = operator + '(df[\'' + label + '\']' + condition + str(value) + ')'
                filter_string += string
                i += 1
            else:
                string = operator + '(df[\'' + label + '\']' + condition + '\''+ str(value) + '\')'
                filter_string += string
                i += 1

print(eval(re.sub("'(\d+)'", r'\1', filter_string)))
</code>```

Edit: Note: this will only work for queries where multiple criteria are applied to the same label as in your example. It won't work if you want to query males older than 45 and females older than 60, for example. If needed, it can easily be added by including some alterations.

Title: Apply filtering on dataframe based on the input from other sheet

Body:
I am trying to filter main dataset(Pandas Dataframe) by applying filters (inputs from the another spreadsheet).

Main data set:

+---------+--------+-----+-----------+-------------+-------+-------------+-------------+----------+
| Cust Id | gender | Age | Indicator | X Indicator | State | foreign_ind | Eu Resident | address1 |
+---------+--------+-----+-----------+-------------+-------+-------------+-------------+----------+
|  987685 | M      |  65 | Y         | N           | TX    | N           | N           | XYZ,USA  |
|  987686 | F      |  54 | Y         | N           | NJ    | N           | N           | XYZ,USA  |
|  987687 | M      |  75 | Y         | Y           | NJ    | N           | N           | XYZ,USA  |
|  987688 | M      |  45 | N         | Y           | NY    | N           | N           | XYZ,USA  |
|  987689 | F      |  45 | Y         | Y           | NJ    | N           | N           | XYZ,USA  |
+---------+--------+-----+-----------+-------------+-------+-------------+-------------+----------+

Below is the config list which we are taking inputs from the end user in spreadsheet format and applying this conditions on top of the main dataset.

Where condition inputs from another spreadsheet:

+-------------+-----------+--------+------------------------------+---------+-----------+--------+
|   column1   | operator1 | value1 | Logical Condition(And or OR) | column2 | operator2 | value2 |
+-------------+-----------+--------+------------------------------+---------+-----------+--------+
| gender      | ==        | F      |                              |         |           |        |
| gender      | ==        | M      |                              |         |           |        |
| Age         | >=        | 75     | ||                           | Age     | >=        |     45 |
| Indicator   | ==        | Y      |                              |         |           |        |
| X Idnicator | ==        | Y      |                              |         |           |        |
| State       | ==        | NJ     |                              |         |           |        |
+-------------+-----------+--------+------------------------------+---------+-----------+--------+

Expected output dataframe after applying filters.

+---------+--------+-----+-----------+-------------+-------+-------------+-------------+----------+
| Cust Id | gender | Age | Indicator | X Indicator | State | foreign_ind | Eu Resident | address1 |
+---------+--------+-----+-----------+-------------+-------+-------------+-------------+----------+
|  987687 | M      |  75 | Y         | Y           | NJ    | N           | N           | XYZ,USA  |
|  987689 | F      |  45 | N         | Y           | DL    | N           | N           | XYZ,USA  |
+---------+--------+-----+-----------+-------------+-------+-------------+-------------+----------+

Body:
I came up with a solution but you'll have to alter the input spreadsheet slightly. It should look like the table below:

         Label Operator Condition Value
0       gender                 ==     F
1       gender        |        ==     M
2          Age        &        >=    75
3          Age        &        >=    45
4    Indicator        &        ==     Y
5  X Indicator        &        ==     Y
6        State        &        ==    NJ

The table lists the filtering criteria. Conditions applied to the same criteria must be written in consecutive rows for the code to work. In your example, gender and age.

Your table:

        gender  Age Indicator X Indicator State foreign_ind Eu Resident address1
Cust Id
987685       M   65         Y           N    TX           N           N  XYZ,USA
987686       F   54         Y           N    NJ           N           N  XYZ,USA
987687       M   75         Y           Y    NJ           N           N  XYZ,USA
987688       M   45         N           Y    NY           N           N  XYS,USA
987689       F   45         Y           Y    NJ           N           N  XYS,USA

Output:

        gender  Age Indicator X Indicator State foreign_ind Eu Resident address1
Cust Id
987687       M   75         Y           Y    NJ           N           N  XYZ,USA
987689       F   45         Y           Y    NJ           N           N  XYS,USA

Code:

i = 0
filter_string = 'df'
while i < conditions.shape[0]:
    if i == conditions.shape[0] - 1:
        if conditions.iloc[i,0] == conditions.iloc[i-1,0]:
            label = conditions.iloc[i,0]
            operator = conditions.iloc[i,1]
            condition = conditions.iloc[i,2]
            value = conditions.iloc[i,3]
            if type(value) == 'int':
                string = str(operator + '(df[\'' + label + '\']' + condition + str(value) + '))]')
                filter_string += string
                i += 1
            else:
                string = str(operator + '(df[\'' + label + '\']' + condition + '\''+ str(value) + '\'))]')
                filter_string += string
                i += 1
        else:
            label = conditions.iloc[i,0]
            operator = conditions.iloc[i,1]
            condition = conditions.iloc[i,2]
            value = conditions.iloc[i,3]
            if type(value) == 'int':
                string = str(operator + '(df[\'' + label + '\']' + condition + str(value) + ')]')
                filter_string += string
                i += 1
            else:
                string = str(operator + '(df[\'' + label + '\']' + condition + '\''+ str(value) + '\')]')
                filter_string += string
                i += 1
    elif i == 0:
        if conditions.iloc[i,0] == conditions.iloc[i+1,0]:
            value = conditions.iloc[i,3]
            label = conditions.iloc[i,0]
            operator = conditions.iloc[i,1]
            condition = conditions.iloc[i,2]
            value = conditions.iloc[i,3]
            if type(value) == 'int':
                string = operator + '[((df[\'' + label + '\']' + condition + str(value) + ')'
                filter_string += string
                i += 1
            else:
                string = operator + '[((df[\'' + label + '\']' + condition + '\''+ str(value) + '\')'
                filter_string += string
                i += 1
        else:
            value = conditions.iloc[i,3]
            label = conditions.iloc[i,0]
            operator = conditions.iloc[i,1]
            condition = conditions.iloc[i,2]
            value = conditions.iloc[i,3]
            if type(value) == 'int':
                string = operator + '[(df[\'' + label + '\']' + condition + str(value) + ')'
                filter_string += string
                i += 1
            else:
                string = operator + '[(df[\'' + label + '\']' + condition + '\''+ str(value) + '\')'
                filter_string += string
                i += 1
    else:
        if conditions.iloc[i,0] == conditions.iloc[i+1,0]:
            value = conditions.iloc[i,3]
            label = conditions.iloc[i,0]
            operator = conditions.iloc[i,1]
            condition = conditions.iloc[i,2]
            value = conditions.iloc[i,3]
            if type(value) == 'int':
                string = operator + '((df[\'' + label + '\']' + condition + str(value) + ')'
                filter_string += string
                i += 1
            else:
                string = operator + '((df[\'' + label + '\']' + condition + '\''+ str(value) + '\')'
                filter_string += string
                i += 1
        elif conditions.iloc[i,0] == conditions.iloc[i-1,0]:
            value = conditions.iloc[i,3]
            label = conditions.iloc[i,0]
            operator = conditions.iloc[i,1]
            condition = conditions.iloc[i,2]
            value = conditions.iloc[i,3]
            if type(value) == 'int':
                string = operator + '(df[\'' + label + '\']' + condition + str(value) + '))'
                filter_string += string
                i += 1
            else:
                string = operator + '(df[\'' + label + '\']' + condition + '\''+ str(value) + '\'))'
                filter_string += string
                i += 1
        else:
            value = conditions.iloc[i,3]
            label = conditions.iloc[i,0]
            operator = conditions.iloc[i,1]
            condition = conditions.iloc[i,2]
            value = conditions.iloc[i,3]
            if type(value) == 'int':
                string = operator + '(df[\'' + label + '\']' + condition + str(value) + ')'
                filter_string += string
                i += 1
            else:
                string = operator + '(df[\'' + label + '\']' + condition + '\''+ str(value) + '\')'
                filter_string += string
                i += 1

print(eval(re.sub("'(\d+)'", r'\1', filter_string)))

Edit: Note: this will only work for queries where multiple criteria are applied to the same label as in your example. It won't work if you want to query males older than 45 and females older than 60, for example. If needed, it can easily be added by including some alterations.


问:
Rails 3 - How to pass authentication token to client side framework like Sproutcore
I'm looking into developing a Sproutcore client-side webapp with a Rails 3 backend. One of the problems I'm envisioning is having to pass the authentication token along with the communication between Sproutcore and Rails. </p>

<p>When is the CSRF token regenerated? Is it on a per-session basis? Would it be okay to disable the authenticity check for login requests, then pass the authentication token back from the login and store it on the client side for future requests? </p>

答:
You can load the authenticity token whether the user is logged in or not. It's stored with the session and would not change after a user logs in and  you don't need to disable the authenticity check. I use this snippet to set a javascript variable:</p>

```<code><%= javascript_tag "var AUTH_TOKEN = #{form_authenticity_token.inspect};" if protect_against_forgery? %>
</code>```

ContentLicense: CC BY-SA 3.0

Body:
I'm looking into developing a Sproutcore client-side webapp with a Rails 3 backend. One of the problems I'm envisioning is having to pass the authentication token along with the communication between Sproutcore and Rails.

When is the CSRF token regenerated? Is it on a per-session basis? Would it be okay to disable the authenticity check for login requests, then pass the authentication token back from the login and store it on the client side for future requests?

Body:
You can load the authenticity token whether the user is logged in or not. It's stored with the session and would not change after a user logs in and  you don't need to disable the authenticity check. I use this snippet to set a javascript variable:

<%= javascript_tag "var AUTH_TOKEN = #{form_authenticity_token.inspect};" if protect_against_forgery? %>


问:
NuGet; unable to get package as wrong version reported in Visual Studio 2012
I'm trying to build a solution but I get this error,

he 'Microsoft.Bcl.Build 1.0.14' package requires NuGet client version '2.8.1' or above, but the current NuGet version is '2.6.40619.9041'.

I've just upgraded NuGet package manager in Tools/"Extensions and Updates" which reports the version as 2.8.50313.31.

I've restarted VS2012, restarted the computer, cleaned the project and rebuilt, but I still can't get Visual Studio to recognise the new version and hence can't compile.

Thanks

答:
This worked for me in VS 2017 Enterprise-

Right click your solution and go to "Manage NuGet Packages for Solution..."

In the window that opens up, click the gear icon at the top right corner.

Select "General" and click the "Clear Clear All NuGet Cache(s)" button. This will run for a few seconds depending on the size of your
  cache.

Clean and rebuild your solution.


问:
Why is access to the path denied?
I am having a problem where I am trying to delete my file but I get an exception.

```<code>if (result == "Success")
{
     if (FileUpload.HasFile)
     {
         try
         {
              File.Delete(Request.PhysicalApplicationPath + app_settings.login_images + txtUploadStatus.Text);
              string filename = Path.GetFileName(btnFileUpload.FileName);
              btnFileUpload.SaveAs(Request.PhysicalApplicationPath + app_settings.login_images + filename);
         }
         catch (Exception ex)
         {
               Message(ex.ToString());
         }
      }
}
</code>```

Also I should note that the folder I am trying to delete from has full control to network services.

The full exception message is:

System.UnauthorizedAccessException: Access to the path 'C:\Users\gowdyn\Documents\Visual Studio 2008\Projects\hybrid\hybrid\temp_loginimages\enviromental.jpg' is denied. at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath) at System.IO.File.Delete(String path) at hybrid.User_Controls.Imgloader_Add_Edit_Tbl.btnUpdate_Click(Object sender, EventArgs e) in C:\Users\gowdyn\Documents\Visual Studio 2008\Projects\hybrid\hybrid\User_Controls\Imgloader_Add_Edit_Tbl.ascx.cs:line 242

Any ideas?

答:
In my case the problem was Norton. My in-house program doesn't have the proper digital signature and when it tried to delete a file it gave the UnauthorizedAccessException.

If it give you a notification, you can handle it from there. In my case it didn't give a notification that I noticed. So here's how to keep Norton from blocking the program.

Open Norton

Click the down arrow

Click History

Find activity by program

Click More Options

Click Exclude Process

Title: Why is access to the path denied?

Body:
I am having a problem where I am trying to delete my file but I get an exception.

if (result == "Success")
{
     if (FileUpload.HasFile)
     {
         try
         {
              File.Delete(Request.PhysicalApplicationPath + app_settings.login_images + txtUploadStatus.Text);
              string filename = Path.GetFileName(btnFileUpload.FileName);
              btnFileUpload.SaveAs(Request.PhysicalApplicationPath + app_settings.login_images + filename);
         }
         catch (Exception ex)
         {
               Message(ex.ToString());
         }
      }
}

Also I should note that the folder I am trying to delete from has full control to network services.

The full exception message is:

System.UnauthorizedAccessException: Access to the path 'C:\Users\gowdyn\Documents\Visual Studio 2008\Projects\hybrid\hybrid\temp_loginimages\enviromental.jpg' is denied. at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath) at System.IO.File.Delete(String path) at hybrid.User_Controls.Imgloader_Add_Edit_Tbl.btnUpdate_Click(Object sender, EventArgs e) in C:\Users\gowdyn\Documents\Visual Studio 2008\Projects\hybrid\hybrid\User_Controls\Imgloader_Add_Edit_Tbl.ascx.cs:line 242

Any ideas?


问:
Use pdfplumber to find text in PDF, return page number, then return table
I downloaded 42 PDFs which are each formatted similarly. Each has various tables, one of which is labeled "Campus Reported Incidents." That particular table is on a different page in each PDF. I want to write a function that will search for the page that has "Campus Reported Incidents" and scrape that table so that I can put it into a dataframe.

I figured that I could use PDFPlumber to search for the string "Campus Reported Incidents" and return the page number. I would then write a function that uses the page number to scrape the table I want, and I would loop that function through every PDF. However, I keep on getting the error "argument is not iterable" or "type object is not subscriptable." I looked through the PDFPlumber documentation but it didn't help my problem.

Here is one example of code that I tried:

```<code>url = "pdfs/example.pdf"

import pdfplumber

pdf = pdfplumber.open(url)

for page in range[0:len(pdf.pages)]:
    if 'Total number of physical restraints' in pdf.pages[page]:
        print(pdf.page_number)
</code>```

答:
I see this post was from a while ago but maybe this response will still help you or someone else.

The error looks like it's coming from the way you are looping through the pages. The range object is not a list, which is why you're seeing the "type object is not subscriptable" error message. Instead, try to "Enumerate" through the pages. The "i" will give you access to the index (aka current count in the loop). The "pg", will give you access to the page object in the PDF pages. I didn't use the "pg" variable below, but you could use that instead of "pages[i]" if you want.

The code below should print the tables from each page, as well as give you access to the tables to manipulate them further.

import pdfplumber
pdf_file = "pdfs/example.pdf"
tables=[]
with pdfplumber.open(pdf_file) as pdf:
    pages = pdf.pages
    for i,pg in enumerate(pages):
        tbl = pages[i].extract_tables()
        print(f'{i} --- {tbl}')
```

Body:
I downloaded 42 PDFs which are each formatted similarly. Each has various tables, one of which is labeled "Campus Reported Incidents." That particular table is on a different page in each PDF. I want to write a function that will search for the page that has "Campus Reported Incidents" and scrape that table so that I can put it into a dataframe.

I figured that I could use PDFPlumber to search for the string "Campus Reported Incidents" and return the page number. I would then write a function that uses the page number to scrape the table I want, and I would loop that function through every PDF. However, I keep on getting the error "argument is not iterable" or "type object is not subscriptable." I looked through the PDFPlumber documentation but it didn't help my problem.

Here is one example of code that I tried:

url = "pdfs/example.pdf"

import pdfplumber

pdf = pdfplumber.open(url)

for page in range[0:len(pdf.pages)]:
    if 'Total number of physical restraints' in pdf.pages[page]:
        print(pdf.page_number)


问:
Debugging nest.js application with vscode
I am testing the nest.js framework but I am struggling to run it with VSCode so that I can properly debug my code. This is pretty much the same issue as described here Running nest.js from VS Code. However I made sure I am using the latest packages. I always get this error:

```<code>Error: Cannot find module 'cats/cats.module'
    at Function.Module._resolveFilename (module.js:485:15)
    at Function.Module._load (module.js:437:25)
    at Module.require (module.js:513:17)
    at require (internal/module.js:11:18)
    at Object.<anonymous> (c:\Users\user\Documents\random-api\dist\app.module.js:11:26)
    at Module._compile (module.js:569:30)
    at Object.Module._extensions..js (module.js:580:10)
    at Module.load (module.js:503:32)
    at tryModuleLoad (module.js:466:12)
    at Function.Module._load (module.js:458:3)
</code>```

npm run start works completely perfect, but I want to debug the application with the VSCode IDE.

My package.json dependencies:

```<code>  "dependencies": {
    "@nestjs/common": "^4.6.6",
    "@nestjs/core": "^4.6.6",
    "@nestjs/microservices": "^4.6.6",
    "@nestjs/testing": "^4.6.6",
    "@nestjs/websockets": "^4.6.6",
    "reflect-metadata": "^0.1.12",
    "rxjs": "^5.5.7",
    "typescript": "^2.7.2"
  },
  "devDependencies": {
    "@types/express": "^4.11.1",
    "@types/jest": "^22.2.2",
    "@types/node": "^9.6.0",
    "@types/supertest": "^2.0.4",
    "jest": "^22.4.3",
    "nodemon": "^1.17.2",
    "prettier": "^1.11.1",
    "supertest": "^3.0.0",
    "ts-jest": "^22.4.2",
    "ts-node": "^5.0.1",
    "tsconfig-paths": "^3.1.3",
    "tslint": "5.9.1",
    "tslint-microsoft-contrib": "^5.0.3"
  },
</code>```

My vscode's launch.json:

```<code>{
    "version": "0.2.0",
    "configurations": [
        {
            "type": "node",
            "request": "launch",
            "name": "Launch Program",
            "program": "${workspaceFolder}\\dist\\main.js",
            "smartStep": true,
            "outFiles": [
                "${workspaceFolder}/dist/**/*.js"
            ]
        }
    ]
}
</code>```

I tried the same launch.json with the typescript file as path, but that threw the same exception:

```<code>"program": "${workspaceFolder}\\src\\main.ts",
</code>```

答:
I just have to "promote" this comment by @JWess to a real answer (and update it with the current location of the relevant setting) so it may be found more easily (this worked for me out of the box for a newly generated nest project, without changing any other configuration or file):

If you go to Settings > Extensions > Node debug and look for the setting Debug › Node: Auto Attach and turn it on, VSCode will auto-attach when you run npm run start:debug (i.e. nest start --debug --watch) in the integrated terminal.

Body:
I am testing the nest.js framework but I am struggling to run it with VSCode so that I can properly debug my code. This is pretty much the same issue as described here Running nest.js from VS Code. However I made sure I am using the latest packages. I always get this error:

Error: Cannot find module 'cats/cats.module'
    at Function.Module._resolveFilename (module.js:485:15)
    at Function.Module._load (module.js:437:25)
    at Module.require (module.js:513:17)
    at require (internal/module.js:11:18)
    at Object.<anonymous> (c:\Users\user\Documents\random-api\dist\app.module.js:11:26)
    at Module._compile (module.js:569:30)
    at Object.Module._extensions..js (module.js:580:10)
    at Module.load (module.js:503:32)
    at tryModuleLoad (module.js:466:12)
    at Function.Module._load (module.js:458:3)

npm run start works completely perfect, but I want to debug the application with the VSCode IDE.

My package.json dependencies:

  "dependencies": {
    "@nestjs/common": "^4.6.6",
    "@nestjs/core": "^4.6.6",
    "@nestjs/microservices": "^4.6.6",
    "@nestjs/testing": "^4.6.6",
    "@nestjs/websockets": "^4.6.6",
    "reflect-metadata": "^0.1.12",
    "rxjs": "^5.5.7",
    "typescript": "^2.7.2"
  },
  "devDependencies": {
    "@types/express": "^4.11.1",
    "@types/jest": "^22.2.2",
    "@types/node": "^9.6.0",
    "@types/supertest": "^2.0.4",
    "jest": "^22.4.3",
    "nodemon": "^1.17.2",
    "prettier": "^1.11.1",
    "supertest": "^3.0.0",
    "ts-jest": "^22.4.2",
    "ts-node": "^5.0.1",
    "tsconfig-paths": "^3.1.3",
    "tslint": "5.9.1",
    "tslint-microsoft-contrib": "^5.0.3"
  },

My vscode's launch.json:

{
    "version": "0.2.0",
    "configurations": [
        {
            "type": "node",
            "request": "launch",
            "name": "Launch Program",
            "program": "${workspaceFolder}\\dist\\main.js",
            "smartStep": true,
            "outFiles": [
                "${workspaceFolder}/dist/**/*.js"
            ]
        }
    ]
}

I tried the same launch.json with the typescript file as path, but that threw the same exception:

"program": "${workspaceFolder}\\src\\main.ts",


问:
c# linq: how to retrieve pieces of data from a returned record
So i have the query:

```<code> var query =
                from product in db.LUT_ProductInfos
                where product.flavor == flavor && product.Container == container
                select flavor;
</code>```

which is returning 1 record (it should always return one record). How do i pull individual attribute values from the result? For example, if it returns 1 record with 3 columns like firstName lastName phoneNumber, and i want to do something with each of the values like

```<code>string fName = firstName
string lName = lastName
int pNumber = phoneNumber
</code>```

without getting into data mapping for objects, what is the most direct way to extract those values from the result variable query

答:
The best and easiest way to do this is to recode like this:

```<code>var query =
                (from product in db.LUT_ProductInfos
                where product.flavor == flavor && product.Container == container
                select flavor).FirstOrDefault();
</code>```

it will return the first matched record. If there's no matched record it will retutn null value, otherwise it returns an object of LUT_ProductInfos class. you can also skip records as you wish like this:

```<code>var query =
                (from product in db.LUT_ProductInfos
                where product.flavor == flavor && product.Container == container
                select flavor).Skip(25).FirstOrDefault();
</code>```

Body:
So i have the query:

 var query =
                from product in db.LUT_ProductInfos
                where product.flavor == flavor && product.Container == container
                select flavor;

which is returning 1 record (it should always return one record). How do i pull individual attribute values from the result? For example, if it returns 1 record with 3 columns like firstName lastName phoneNumber, and i want to do something with each of the values like

string fName = firstName
string lName = lastName
int pNumber = phoneNumber

without getting into data mapping for objects, what is the most direct way to extract those values from the result variable query

Body:
The best and easiest way to do this is to recode like this:

var query =
                (from product in db.LUT_ProductInfos
                where product.flavor == flavor && product.Container == container
                select flavor).FirstOrDefault();

it will return the first matched record. If there's no matched record it will retutn null value, otherwise it returns an object of LUT_ProductInfos class. you can also skip records as you wish like this:

var query =
                (from product in db.LUT_ProductInfos
                where product.flavor == flavor && product.Container == container
                select flavor).Skip(25).FirstOrDefault();


问:
Azure Logic App with SharePoint Connection
I am creating my first logic app which connects to SharePoint and adds entries into some sharepoint list.

Whenever I create a SharePoint Connection it adds below resource to my logic app.

```<code>{
  "type": "MICROSOFT.WEB/CONNECTIONS",
  "apiVersion": "2018-07-01-preview",
  "name": "[parameters('sharepointonline_1_Connection_Name')]",
  "location": "[parameters('logicAppLocation')]",
  "properties": {
    "api": {
      "id": "[concat(subscription().id, '/providers/Microsoft.Web/locations/', parameters('logicAppLocation'), '/managedApis/', 'sharepointonline')]"
    },
    "displayName": "[parameters('sharepointonline_1_Connection_DisplayName')]",
    "nonSecretParameterValues": {
      "token:TenantId": "[parameters('sharepointonline_1_token:TenantId')]"
    }
  }
</code>```

Could anyone give an explanation of "token:TenantId". How/where to get this value in my dev tenant. How this can be moved to UAT/PROD environment?

Whenever I recreate my logic app with the SharePoint connection it loses the connection and shows me below screen with a warning icon.

Is there a way we can authenticate this connection via PowerShell or via Azure DevOps deployment?

答:
This seems to the OAuth connection and will need to be re-authorized after the template deployment to obtain valid access token. Some connections support using an Azure Active Directory (Azure AD) service principal to authorize connections for a logic app that's registered in Azure AD.

Documentation here shows how Azure data lake's connection resource definition can be configured to use parameter values of the template and Azure AD service principal to generate the token so you might want to check if SharePoint connection can be configured in same way or not.

```<code>{
   <other-template-objects>
   "type": "MICROSOFT.WEB/CONNECTIONS",
   "apiVersion": "2016-06-01",
   "name": "[parameters('azuredatalake_1_Connection_Name')]",
   "location": "[parameters('LogicAppLocation')]",
   "properties": {
      "api": {
         "id": "[concat(subscription().id, '/providers/Microsoft.Web/locations/', 'resourceGroup().location', '/managedApis/', 'azuredatalake')]"
      },
      "displayName": "[parameters('azuredatalake_1_Connection_DisplayName')]",
      "parameterValues": {
         "token:clientId": "[parameters('azuredatalake_1_token:clientId')]",
         "token:clientSecret": "[parameters('azuredatalake_1_token:clientSecret')]",
         "token:TenantId": "[parameters('azuredatalake_1_token:TenantId')]",
         "token:grantType": "[parameters('azuredatalake_1_token:grantType')]"
      }
   }
}
</code>```

Reference: https://learn.microsoft.com/en-us/azure/logic-apps/logic-apps-azure-resource-manager-templates-overview#authenticate-connections

Body:
I am creating my first logic app which connects to SharePoint and adds entries into some sharepoint list.

Whenever I create a SharePoint Connection it adds below resource to my logic app.

{
  "type": "MICROSOFT.WEB/CONNECTIONS",
  "apiVersion": "2018-07-01-preview",
  "name": "[parameters('sharepointonline_1_Connection_Name')]",
  "location": "[parameters('logicAppLocation')]",
  "properties": {
    "api": {
      "id": "[concat(subscription().id, '/providers/Microsoft.Web/locations/', parameters('logicAppLocation'), '/managedApis/', 'sharepointonline')]"
    },
    "displayName": "[parameters('sharepointonline_1_Connection_DisplayName')]",
    "nonSecretParameterValues": {
      "token:TenantId": "[parameters('sharepointonline_1_token:TenantId')]"
    }
  }

Could anyone give an explanation of "token:TenantId". How/where to get this value in my dev tenant. How this can be moved to UAT/PROD environment?

Whenever I recreate my logic app with the SharePoint connection it loses the connection and shows me below screen with a warning icon.

Is there a way we can authenticate this connection via PowerShell or via Azure DevOps deployment?

Body:
This seems to the OAuth connection and will need to be re-authorized after the template deployment to obtain valid access token. Some connections support using an Azure Active Directory (Azure AD) service principal to authorize connections for a logic app that's registered in Azure AD.

Documentation here shows how Azure data lake's connection resource definition can be configured to use parameter values of the template and Azure AD service principal to generate the token so you might want to check if SharePoint connection can be configured in same way or not.

{
   <other-template-objects>
   "type": "MICROSOFT.WEB/CONNECTIONS",
   "apiVersion": "2016-06-01",
   "name": "[parameters('azuredatalake_1_Connection_Name')]",
   "location": "[parameters('LogicAppLocation')]",
   "properties": {
      "api": {
         "id": "[concat(subscription().id, '/providers/Microsoft.Web/locations/', 'resourceGroup().location', '/managedApis/', 'azuredatalake')]"
      },
      "displayName": "[parameters('azuredatalake_1_Connection_DisplayName')]",
      "parameterValues": {
         "token:clientId": "[parameters('azuredatalake_1_token:clientId')]",
         "token:clientSecret": "[parameters('azuredatalake_1_token:clientSecret')]",
         "token:TenantId": "[parameters('azuredatalake_1_token:TenantId')]",
         "token:grantType": "[parameters('azuredatalake_1_token:grantType')]"
      }
   }
}

Reference: https://learn.microsoft.com/en-us/azure/logic-apps/logic-apps-azure-resource-manager-templates-overview#authenticate-connections


问:
Not asking external storage permission on Android studio
my name is Tom, i am trying to open a file from the external storage. for some reason it is not asking the permission at all?!?!
I will thank for any answer!
Tom.
Here is my code:</p>

```<code>if(Build.VERSION.SDK_INT >= Build.VERSION_CODES.M && checkSelfPermission(Manifest.permission.READ_EXTERNAL_STORAGE) != PackageManager.PERMISSION_GRANTED){
            Toast.makeText(MainActivity.this,"should snd request ",Toast.LENGTH_SHORT).show();

            requestPermissions(new String[]{Manifest.permission.READ_EXTERNAL_STORAGE}, PERMISSION_REQUET_CODE);
        }

public void OnRequestPermissionsResult(int requestCode, @NonNull String[] permissions, @NonNull int[] grantResult){
        if(requestCode == PERMISSION_REQUET_CODE){
            if(grantResult[0] == PackageManager.PERMISSION_GRANTED){
                Toast.makeText(MainActivity.this,"permission granted!",Toast.LENGTH_SHORT).show();
            }
            else{
                Toast.makeText(MainActivity.this,"*** permission  not granted! ***",Toast.LENGTH_LONG).show();
                finish();
            }
        }
    }
</code>```

答:
Make sure you also have added permissions to the Manifest file

```<code><?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
    package="com.example.appname">

    <uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />
...
</manifest>
</code>```

Body:
my name is Tom, i am trying to open a file from the external storage. for some reason it is not asking the permission at all?!?!
I will thank for any answer!
Tom.
Here is my code:

if(Build.VERSION.SDK_INT >= Build.VERSION_CODES.M && checkSelfPermission(Manifest.permission.READ_EXTERNAL_STORAGE) != PackageManager.PERMISSION_GRANTED){
            Toast.makeText(MainActivity.this,"should snd request ",Toast.LENGTH_SHORT).show();

            requestPermissions(new String[]{Manifest.permission.READ_EXTERNAL_STORAGE}, PERMISSION_REQUET_CODE);
        }

public void OnRequestPermissionsResult(int requestCode, @NonNull String[] permissions, @NonNull int[] grantResult){
        if(requestCode == PERMISSION_REQUET_CODE){
            if(grantResult[0] == PackageManager.PERMISSION_GRANTED){
                Toast.makeText(MainActivity.this,"permission granted!",Toast.LENGTH_SHORT).show();
            }
            else{
                Toast.makeText(MainActivity.this,"*** permission  not granted! ***",Toast.LENGTH_LONG).show();
                finish();
            }
        }
    }

Body:
Make sure you also have added permissions to the Manifest file

<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
    package="com.example.appname">

    <uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />
...
</manifest>


问:
Read/Write Parquet with Struct column type
I am trying to write a Dataframe like this to Parquet:

```<code>| foo | bar               |
|-----|-------------------|
|  1  | {"a": 1, "b": 10} |
|  2  | {"a": 2, "b": 20} |
|  3  | {"a": 3, "b": 30} |
</code>```

I am doing it with Pandas and Fastparquet:

df = pd.DataFrame({
    "foo": [1, 2, 3],
    "bar": [{"a": 1, "b": 10}, {"a": 2, "b": 20}, {"a": 3, "b": 30}]
})

import fastparquet
fastparquet.write('/my/parquet/location/toy-fastparquet.parq', df)
```

<p>I would like to load Parquet in (py)Spark, and query the data with Spark SQL, like:</p>

<pre class="lang-py prettyprint-override"><code>df = spark.read.parquet("/my/parquet/location/")
df.registerTempTable('my_toy_table')
result = spark.sql("SELECT * FROM my_toy_table WHERE bar.b > 15")
</code>```

My issue is that, even though fastparquet can read its Parquet file correctly (the bar field is correctly deserialized as a Struct), in Spark, bar is read as a column of type String, that just contains a JSON representation of the original structure:

```<code>In [2]: df.head()
Out[2]: Row(foo=1, bar='{"a": 1, "b": 10}')
</code>```

I tried writing Parquet from PyArrow, but no luck there: ArrowNotImplementedError: Level generation for Struct not supported yet. I have also tried passing file_scheme='hive' to Fastparquet, but I got the same results. Changing Fastparquet serialization to BSON (object_encoding='bson') produced an unreadable binary field.

[EDIT] I see the following approaches:

[answered] Write Parquet from Spark

[open] Find a Python library that implements Parquet's specification for nested types, and that is compatible with the way Spark reads them

[answered] Read Fastparquet files in Spark with specific JSON de-serialization (I suppose this has an impact on performance)

Do not use nested structures altogether

答:
You have at least 3 options here:

Option 1:

You don't need to use any extra libraries like fastparquet since Spark provides that functionality already:

pdf = pd.DataFrame({
    "foo": [1, 2, 3],
    "bar": [{"a": 1, "b": 10}, {"a": 2, "b": 20}, {"a": 3, "b": 30}]
})

df = spark.createDataFrame(pdf)
df.write.mode("overwrite").parquet("/tmp/parquet1")
```

<p>If try to load your data with <code>df = spark.read.parquet("/tmp/parquet1")</code> the schema will be:</p>

```StructType([
            StructField("foo", LongType(), True),
            StructField("bar",MapType(StringType(), LongType(), True), True)])
```

<p>As you can see in this case Spark will retain the correct schema.</p>

<p><strong>Option 2:</strong></p>

<p>If for any reason still need to use <code>fastparquet</code> then <code>bar</code> will be treated as string therefore you can load <code>bar</code> as a string and then convert it to JSON using from_json function. In your case we will handle the json as a dictionary of Map(string, int). This is for our own convenience since the data seems to be a sequence of key/value which can be perfectly represented by a dictionary:</p>

```from pyspark.sql.types import StringType, MapType,LongType
from pyspark.sql.functions import from_json

df = spark.read.parquet("/tmp/parquet1")

# schema should be a Map(string, string)
df.withColumn("bar", from_json("bar", MapType(StringType(), LongType()))).show()

# +---+-----------------+
# |foo|              bar|
# +---+-----------------+
# |  1|[a -> 1, b -> 10]|
# |  2|[a -> 2, b -> 20]|
# |  3|[a -> 3, b -> 30]|
# +---+-----------------+
```

<p><strong>Option 3:</strong></p>

<p>If you your schema does not change and you know that each value of bar will always have the same combination of fields (a, b) you can also convert <code>bar</code> into a struct:</p>

<pre class="lang-py prettyprint-override"><code>schema = StructType([
                    StructField("a", LongType(), True),
                    StructField("b", LongType(), True)
            ])

df = df.withColumn("bar", from_json("bar", schema))

df.printSchema()

# root
#  |-- foo: long (nullable = true)
#  |-- bar: struct (nullable = true)
#  |    |-- a: long (nullable = true)
#  |    |-- b: long (nullable = true)
</code>```

Example:

Then you can run your code with:

```<code>df.registerTempTable('my_toy_table')

spark.sql("SELECT * FROM my_toy_table WHERE bar.b > 20").show()
# or spark.sql("SELECT * FROM my_toy_table WHERE bar['b'] > 20")

# +---+-----------------+
# |foo|              bar|
# +---+-----------------+
# |  3|[a -> 3, b -> 30]|
# +---+-----------------+
</code>```

Body:
I am trying to write a Dataframe like this to Parquet:

| foo | bar               |
|-----|-------------------|
|  1  | {"a": 1, "b": 10} |
|  2  | {"a": 2, "b": 20} |
|  3  | {"a": 3, "b": 30} |

I am doing it with Pandas and Fastparquet:

df = pd.DataFrame({
    "foo": [1, 2, 3],
    "bar": [{"a": 1, "b": 10}, {"a": 2, "b": 20}, {"a": 3, "b": 30}]
})

import fastparquet
fastparquet.write('/my/parquet/location/toy-fastparquet.parq', df)

I would like to load Parquet in (py)Spark, and query the data with Spark SQL, like:

df = spark.read.parquet("/my/parquet/location/")
df.registerTempTable('my_toy_table')
result = spark.sql("SELECT * FROM my_toy_table WHERE bar.b > 15")

My issue is that, even though fastparquet can read its Parquet file correctly (the bar field is correctly deserialized as a Struct), in Spark, bar is read as a column of type String, that just contains a JSON representation of the original structure:

In [2]: df.head()
Out[2]: Row(foo=1, bar='{"a": 1, "b": 10}')

I tried writing Parquet from PyArrow, but no luck there: ArrowNotImplementedError: Level generation for Struct not supported yet. I have also tried passing file_scheme='hive' to Fastparquet, but I got the same results. Changing Fastparquet serialization to BSON (object_encoding='bson') produced an unreadable binary field.

[EDIT] I see the following approaches:

[answered] Write Parquet from Spark

[open] Find a Python library that implements Parquet's specification for nested types, and that is compatible with the way Spark reads them

[answered] Read Fastparquet files in Spark with specific JSON de-serialization (I suppose this has an impact on performance)

Do not use nested structures altogether

Body:
You have at least 3 options here:

Option 1:

You don't need to use any extra libraries like fastparquet since Spark provides that functionality already:

pdf = pd.DataFrame({
    "foo": [1, 2, 3],
    "bar": [{"a": 1, "b": 10}, {"a": 2, "b": 20}, {"a": 3, "b": 30}]
})

df = spark.createDataFrame(pdf)
df.write.mode("overwrite").parquet("/tmp/parquet1")

If try to load your data with df = spark.read.parquet("/tmp/parquet1") the schema will be:

StructType([
            StructField("foo", LongType(), True),
            StructField("bar",MapType(StringType(), LongType(), True), True)])

As you can see in this case Spark will retain the correct schema.

Option 2:

If for any reason still need to use fastparquet then bar will be treated as string therefore you can load bar as a string and then convert it to JSON using from_json function. In your case we will handle the json as a dictionary of Map(string, int). This is for our own convenience since the data seems to be a sequence of key/value which can be perfectly represented by a dictionary:

from pyspark.sql.types import StringType, MapType,LongType
from pyspark.sql.functions import from_json

df = spark.read.parquet("/tmp/parquet1")

# schema should be a Map(string, string)
df.withColumn("bar", from_json("bar", MapType(StringType(), LongType()))).show()

# +---+-----------------+
# |foo|              bar|
# +---+-----------------+
# |  1|[a -> 1, b -> 10]|
# |  2|[a -> 2, b -> 20]|
# |  3|[a -> 3, b -> 30]|
# +---+-----------------+

Option 3:

If you your schema does not change and you know that each value of bar will always have the same combination of fields (a, b) you can also convert bar into a struct:

schema = StructType([
                    StructField("a", LongType(), True),
                    StructField("b", LongType(), True)
            ])

df = df.withColumn("bar", from_json("bar", schema))

df.printSchema()

# root
#  |-- foo: long (nullable = true)
#  |-- bar: struct (nullable = true)
#  |    |-- a: long (nullable = true)
#  |    |-- b: long (nullable = true)

Example:

Then you can run your code with:

df.registerTempTable('my_toy_table')

spark.sql("SELECT * FROM my_toy_table WHERE bar.b > 20").show()
# or spark.sql("SELECT * FROM my_toy_table WHERE bar['b'] > 20")

# +---+-----------------+
# |foo|              bar|
# +---+-----------------+
# |  3|[a -> 3, b -> 30]|
# +---+-----------------+


问:
Unexpected Token = getting while running the following code
I'm getting the following error while running the below. The issue is with the unexpected toke "=". Due to which I'm getting the error.</p>

```<code># ./yarn install
/var/jenkins_home/jobs/in-install/workspace/yarn-bin/yarn-1.15.2.js:1227
var _ref27 = (0, (_asyncToGenerator2 || _load_asyncToGenerator()).default)(function* (dir,relativeDir, ignoreBasenames = new Set()) {

SyntaxError: Unexpected token =
at exports.runInThisContext (vm.js:53:16)
at Module._compile (module.js:373:25)
at Object.Module._extensions..js (module.js:416:10)
at Module.load (module.js:343:32)
at Function.Module._load (module.js:300:12)
at Function.Module.runMain (module.js:441:10)
at startup (node.js:140:18)
at node.js:1043:3

Nodejs version 4.8.3
</code>```

答:
function* (dir,relativeDir, ignoreBasenames = new Set() uses default parameters.

Node.js 4.8.3 does not support default parameters for functions.  You will need to use at least Node.js 6.

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Functions/Default_parameters

Body:
I'm getting the following error while running the below. The issue is with the unexpected toke "=". Due to which I'm getting the error.

# ./yarn install
/var/jenkins_home/jobs/in-install/workspace/yarn-bin/yarn-1.15.2.js:1227
var _ref27 = (0, (_asyncToGenerator2 || _load_asyncToGenerator()).default)(function* (dir,relativeDir, ignoreBasenames = new Set()) {

SyntaxError: Unexpected token =
at exports.runInThisContext (vm.js:53:16)
at Module._compile (module.js:373:25)
at Object.Module._extensions..js (module.js:416:10)
at Module.load (module.js:343:32)
at Function.Module._load (module.js:300:12)
at Function.Module.runMain (module.js:441:10)
at startup (node.js:140:18)
at node.js:1043:3

Nodejs version 4.8.3


问:
Unable to create DB in cosmos emulator
I have been trying to test my changes using cosmos db emulator. I kept getting exception, service unavailable. Tried via the explorer and got the same error:</p>

```<code>{"code":500,"body":{"code":"InternalServerError","message":"Service is currently unavailable, please retry after a while. If this problem persists please contact support.\r\nActivityId: e7204d29-793e-4490-9916-6138fad1b715, Microsoft.Azure.Documents.Common/2.9.2"},"headers":{"content-type":"application/json","date":"Wed, 12 Feb 2020 17:53:44 GMT","server":"Microsoft-HTTPAPI/2.0","x-ms-activity-id":"e7204d29-793e-4490-9916-6138fad1b715","x-ms-gatewayversion":"version=2.9.2","x-ms-throttle-retry-count":0,"x-ms-throttle-retry-wait-time-ms":0},"activityId":"e7204d29-793e-4490-9916-6138fad1b715"}
</code>```

答:
The solution is to apply "Reset Data..." on the context menu of the Emulator tray icon:

This error often occurs if you already filled all the possible data partitions in the Emulator and are trying to create more collections or if an Emulator update might have corrupted data.

Related troubleshooting guide: https://learn.microsoft.com/azure/cosmos-db/local-emulator#troubleshooting

Title: Unable to create DB in cosmos emulator

Body:
I have been trying to test my changes using cosmos db emulator. I kept getting exception, service unavailable. Tried via the explorer and got the same error:

{"code":500,"body":{"code":"InternalServerError","message":"Service is currently unavailable, please retry after a while. If this problem persists please contact support.\r\nActivityId: e7204d29-793e-4490-9916-6138fad1b715, Microsoft.Azure.Documents.Common/2.9.2"},"headers":{"content-type":"application/json","date":"Wed, 12 Feb 2020 17:53:44 GMT","server":"Microsoft-HTTPAPI/2.0","x-ms-activity-id":"e7204d29-793e-4490-9916-6138fad1b715","x-ms-gatewayversion":"version=2.9.2","x-ms-throttle-retry-count":0,"x-ms-throttle-retry-wait-time-ms":0},"activityId":"e7204d29-793e-4490-9916-6138fad1b715"}


问:
When running an Async/Await Experiment, operations do not occur in expected order
I have created an experiment (run in XUnit) to test output. I use three methods - the test method, an asynchronous method that calls await, and an inner Task that sleeps for 5 seconds.

```<code>    [Fact]
    public void TestAsyncRunning()
    {
        _testOutput.WriteLine("Beginning Test");
        _testOutput.WriteLine("Calling RunningAsync");

        var t = RunningAsync();

        _testOutput.WriteLine("In outer method, after calling async method");

        t.Wait(); //IOW, Thread.Join();

        _testOutput.WriteLine("Exiting Test");
    }

    private async Task RunningAsync()
    {
        _testOutput.WriteLine("RunningAsync: Enter Method");
        _testOutput.WriteLine("RunningAsync: Calling and awaiting inner running");
        await InnerRunning();
        _testOutput.WriteLine("RunningAsync: Returning");
    }

    private Task InnerRunning()
    {
        _testOutput.WriteLine("InnerRunning: Begin method");
        _testOutput.WriteLine("InnerRunning: Sleeping For 5 seconds");
        var t = Task.Run(() => Thread.Sleep(5000));
        _testOutput.WriteLine("InnerRunning: Wake up and exit");
        return t;
    }
</code>```

In the following output, the lines with the asterisks is where the unexpected order occurs.

The output I expect is

```<code>Beginning Test
Calling RunningAsync
RunningAsync: Enter Method
RunningAsync: Calling and awaiting inner running
InnerRunning: Begin method
InnerRunning: Sleeping For 5 seconds
*In outer method, after calling async method
*InnerRunning: Wake up and exit
RunningAsync: Returning
Exiting Test
</code>```

The output I get is

```<code>Beginning Test
Calling RunningAsync
RunningAsync: Enter Method
RunningAsync: Calling and awaiting inner running
InnerRunning: Begin method
InnerRunning: Sleeping For 5 seconds
*InnerRunning: Wake up and exit
*In outer method, after calling async method
RunningAsync: Returning
Exiting Test
</code>```

It appears that the async method is actually running async but the method it calls is running synchronously. Does this have to do with the TaskScheduler? I am clearly missing something...….

答:
Adding more to Athanasios Kataras & Stephen Cleary post's

You can even see the change in thread for incoming and returning thread ids and this code will qualify for full asynchronous code. if you use t.Wait() it just makes just immediate async methods to run synchronously :

```<code>    public static async Task Main()
    {
        Console.WriteLine("Beginning Test");
        Console.WriteLine("Calling RunningAsync");
        Console.WriteLine("1 " + Thread.CurrentThread.ManagedThreadId);
        await RunningAsync();
        Console.WriteLine("2 " + Thread.CurrentThread.ManagedThreadId);
        Console.WriteLine("In outer method, after calling async method");
        Console.WriteLine("Exiting Test");
    }

    private static async Task RunningAsync()
    {
        Console.WriteLine("RunningAsync: Enter Method");
        Console.WriteLine("RunningAsync: Calling and awaiting inner running");
        Console.WriteLine("3 " + Thread.CurrentThread.ManagedThreadId);
        await InnerRunning();
        Console.WriteLine("4 " + Thread.CurrentThread.ManagedThreadId);
        Console.WriteLine("RunningAsync: Returning");
    }

    private static async Task InnerRunning()
    {

        Console.WriteLine("InnerRunning: Begin method");
        Console.WriteLine("InnerRunning: Sleeping For 5 seconds");
        Console.WriteLine("5 " + Thread.CurrentThread.ManagedThreadId);
        await Task.Run(() => Thread.Sleep(5000));
        Console.WriteLine("6 " + Thread.CurrentThread.ManagedThreadId);
        Console.WriteLine("InnerRunning: Wake up and exit");
    }
</code>```

Output will be :

```<code>Calling RunningAsync
1 1
RunningAsync: Enter Method
RunningAsync: Calling and awaiting inner running
3 1
InnerRunning: Begin method
InnerRunning: Sleeping For 5 seconds
5 1
6 4
InnerRunning: Wake up and exit
4 4
RunningAsync: Returning
2 4
In outer method, after calling async method
Exiting Test
</code>```

Body:
I have created an experiment (run in XUnit) to test output. I use three methods - the test method, an asynchronous method that calls await, and an inner Task that sleeps for 5 seconds.

    [Fact]
    public void TestAsyncRunning()
    {
        _testOutput.WriteLine("Beginning Test");
        _testOutput.WriteLine("Calling RunningAsync");

        var t = RunningAsync();

        _testOutput.WriteLine("In outer method, after calling async method");

        t.Wait(); //IOW, Thread.Join();

        _testOutput.WriteLine("Exiting Test");
    }

    private async Task RunningAsync()
    {
        _testOutput.WriteLine("RunningAsync: Enter Method");
        _testOutput.WriteLine("RunningAsync: Calling and awaiting inner running");
        await InnerRunning();
        _testOutput.WriteLine("RunningAsync: Returning");
    }

    private Task InnerRunning()
    {
        _testOutput.WriteLine("InnerRunning: Begin method");
        _testOutput.WriteLine("InnerRunning: Sleeping For 5 seconds");
        var t = Task.Run(() => Thread.Sleep(5000));
        _testOutput.WriteLine("InnerRunning: Wake up and exit");
        return t;
    }

In the following output, the lines with the asterisks is where the unexpected order occurs.

The output I expect is

Beginning Test
Calling RunningAsync
RunningAsync: Enter Method
RunningAsync: Calling and awaiting inner running
InnerRunning: Begin method
InnerRunning: Sleeping For 5 seconds
*In outer method, after calling async method
*InnerRunning: Wake up and exit
RunningAsync: Returning
Exiting Test

The output I get is

Beginning Test
Calling RunningAsync
RunningAsync: Enter Method
RunningAsync: Calling and awaiting inner running
InnerRunning: Begin method
InnerRunning: Sleeping For 5 seconds
*InnerRunning: Wake up and exit
*In outer method, after calling async method
RunningAsync: Returning
Exiting Test

It appears that the async method is actually running async but the method it calls is running synchronously. Does this have to do with the TaskScheduler? I am clearly missing something...….

Body:
Adding more to Athanasios Kataras & Stephen Cleary post's

You can even see the change in thread for incoming and returning thread ids and this code will qualify for full asynchronous code. if you use t.Wait() it just makes just immediate async methods to run synchronously :

    public static async Task Main()
    {
        Console.WriteLine("Beginning Test");
        Console.WriteLine("Calling RunningAsync");
        Console.WriteLine("1 " + Thread.CurrentThread.ManagedThreadId);
        await RunningAsync();
        Console.WriteLine("2 " + Thread.CurrentThread.ManagedThreadId);
        Console.WriteLine("In outer method, after calling async method");
        Console.WriteLine("Exiting Test");
    }

    private static async Task RunningAsync()
    {
        Console.WriteLine("RunningAsync: Enter Method");
        Console.WriteLine("RunningAsync: Calling and awaiting inner running");
        Console.WriteLine("3 " + Thread.CurrentThread.ManagedThreadId);
        await InnerRunning();
        Console.WriteLine("4 " + Thread.CurrentThread.ManagedThreadId);
        Console.WriteLine("RunningAsync: Returning");
    }

    private static async Task InnerRunning()
    {

        Console.WriteLine("InnerRunning: Begin method");
        Console.WriteLine("InnerRunning: Sleeping For 5 seconds");
        Console.WriteLine("5 " + Thread.CurrentThread.ManagedThreadId);
        await Task.Run(() => Thread.Sleep(5000));
        Console.WriteLine("6 " + Thread.CurrentThread.ManagedThreadId);
        Console.WriteLine("InnerRunning: Wake up and exit");
    }

Output will be :

Calling RunningAsync
1 1
RunningAsync: Enter Method
RunningAsync: Calling and awaiting inner running
3 1
InnerRunning: Begin method
InnerRunning: Sleeping For 5 seconds
5 1
6 4
InnerRunning: Wake up and exit
4 4
RunningAsync: Returning
2 4
In outer method, after calling async method
Exiting Test


问:
Why does my sequelize query break with ER_BAD_FIELD_ERROR when eager loading a model associated to another with belongsTo/hasMany?

Sequelize version:

4.37.10

Model definitions:

LocalChapters.js

```<code>const Database = require('./database');
const Sequelize = require('sequelize');
const NationalChapters = require('./nationalchapters');

const LocalChapters = Database
.getInstance()
.define('localchapters', {
    idLocalChapter: {
        type: Sequelize.INTEGER(11),
        primaryKey: true,
        allowNull: false,
        autoIncrement: true,
        unique: true
    },
    LocalName: {
        type: Sequelize.STRING(45),
        allowNull: false
    }
}, {
    timestamps: false
});

LocalChapters.belongsTo(NationalChapters, { foreignKey: 'NationalChapter_id', onDelete: 'CASCADE' });
NationalChapters.hasMany(LocalChapters);

module.exports = LocalChapters;
</code>```

NationalChapters.js

```<code>const Database = require('./database');
const Sequelize = require('sequelize');

const NationalChapters = Database
.getInstance()
.define('nationalchapters', {
    idNationalChapter: {
        type: Sequelize.INTEGER(11),
        primaryKey: true,
        allowNull: false,
        autoIncrement: true,
        unique: true
    },
    NationalName: {
        type: Sequelize.STRING(45),
        allowNull: false,
        unique: true
    }
}, {
    timestamps: false
});

module.exports = NationalChapters;
</code>```

Query ran:

```<code>const result = await NationalChapters.findAll({
    include: [ LocalChapters ]
});
</code>```

Expected result:

```<code>[
  {
    idNationalChapter: 21,
    NationalName: 'Some National Chapter Name',
    localchapters: [
      {
        idLocalChapter: 15,
        LocalName: 'Some Local Chapter Name'
      },
      ...
    ]
  }
  ...
]
</code>```

Error received:

```<code>DatabaseError [SequelizeDatabaseError]: Unknown column 'localchapters.nationalchapterIdNationalChapter' in 'field list'
    at Query.formatError (/<file path>/node_modules/sequelize/lib/dialects/mysql/query.js:247:16)
    at Query.handler [as onResult] (/<file path>/node_modules/sequelize/lib/dialects/mysql/query.js:68:23)
    at Query.Command.execute (/<file path>/node_modules/mysql2/lib/commands/command.js:30:12)
    at Connection.handlePacket (/<file path>/node_modules/mysql2/lib/connection.js:502:28)
    at PacketParser.onPacket (/<file path>/node_modules/mysql2/lib/connection.js:81:16)
    at PacketParser.executeStart (/<file path>/node_modules/mysql2/lib/packet_parser.js:77:14)
    at Socket.<anonymous> (/<file path>/node_modules/mysql2/lib/connection.js:89:29)
    at Socket.emit (events.js:210:5)
    at addChunk (_stream_readable.js:308:12)
    at readableAddChunk (_stream_readable.js:289:11)
    at Socket.Readable.push (_stream_readable.js:223:10)
    at TCP.onStreamRead (internal/stream_base_commons.js:182:23) {
  name: 'SequelizeDatabaseError',
  parent: Error: Unknown column 'localchapters.nationalchapterIdNationalChapter' in 'field list'
      at Packet.asError (/<file path>/node_modules/mysql2/lib/packets/packet.js:716:13)
      at Query.Command.execute (/<file path>/node_modules/mysql2/lib/commands/command.js:28:22)
      at Connection.handlePacket (/<file path>/node_modules/mysql2/lib/connection.js:502:28)
      at PacketParser.onPacket (/<file path>/node_modules/mysql2/lib/connection.js:81:16)
      at PacketParser.executeStart (/<file path>/node_modules/mysql2/lib/packet_parser.js:77:14)
      at Socket.<anonymous> (/<file path>/node_modules/mysql2/lib/connection.js:89:29)
      at Socket.emit (events.js:210:5)
      at addChunk (_stream_readable.js:308:12)
      at readableAddChunk (_stream_readable.js:289:11)
      at Socket.Readable.push (_stream_readable.js:223:10)
      at TCP.onStreamRead (internal/stream_base_commons.js:182:23) {
    code: 'ER_BAD_FIELD_ERROR',
    errno: 1054,
    sqlState: '42S22',
    sqlMessage: "Unknown column 'localchapters.nationalchapterIdNationalChapter' in 'field list'",
    sql: 'SELECT `nationalchapters`.`idNationalChapter`,
                `nationalchapters`.`NationalName`,
                `localchapters`.`idLocalChapter` AS `localchapters.idLocalChapter`,
                `localchapters`.`LocalName` AS `localchapters.LocalName`,
                `localchapters`.`NationalChapter_id` AS `localchapters.NationalChapter_id`,
                `localchapters`.`nationalchapterIdNationalChapter` AS `localchapters.nationalchapterIdNationalChapter`
            FROM `nationalchapters` AS `nationalchapters`
            LEFT OUTER JOIN `localchapters` AS `localchapters`
            ON `nationalchapters`.`idNationalChapter` = `localchapters`.`nationalchapterIdNationalChapter`;'
  },
  original: Error: Unknown column 'localchapters.nationalchapterIdNationalChapter' in 'field list'
      at Packet.asError (/<file path>/node_modules/mysql2/lib/packets/packet.js:716:13)
      at Query.Command.execute (/<file path>/node_modules/mysql2/lib/commands/command.js:28:22)
      at Connection.handlePacket (/<file path>/node_modules/mysql2/lib/connection.js:502:28)
      at PacketParser.onPacket (/<file path>/node_modules/mysql2/lib/connection.js:81:16)
      at PacketParser.executeStart (/<file path>/node_modules/mysql2/lib/packet_parser.js:77:14)
      at Socket.<anonymous> (/<file path>/node_modules/mysql2/lib/connection.js:89:29)
      at Socket.emit (events.js:210:5)
      at addChunk (_stream_readable.js:308:12)
      at readableAddChunk (_stream_readable.js:289:11)
      at Socket.Readable.push (_stream_readable.js:223:10)
      at TCP.onStreamRead (internal/stream_base_commons.js:182:23) {
    code: 'ER_BAD_FIELD_ERROR',
    errno: 1054,
    sqlState: '42S22',
    sqlMessage: "Unknown column 'localchapters.nationalchapterIdNationalChapter' in 'field list'",
    sql: 'SELECT `nationalchapters`.`idNationalChapter`,
                `nationalchapters`.`NationalName`,
                `localchapters`.`idLocalChapter` AS `localchapters.idLocalChapter`,
                `localchapters`.`LocalName` AS `localchapters.LocalName`,
                `localchapters`.`NationalChapter_id` AS `localchapters.NationalChapter_id`,
                `localchapters`.`nationalchapterIdNationalChapter` AS `localchapters.nationalchapterIdNationalChapter`
            FROM `nationalchapters` AS `nationalchapters`
            LEFT OUTER JOIN `localchapters` AS `localchapters`
            ON `nationalchapters`.`idNationalChapter` = `localchapters`.`nationalchapterIdNationalChapter`;'
  },
  sql: 'SELECT `nationalchapters`.`idNationalChapter`,
                `nationalchapters`.`NationalName`,
                `localchapters`.`idLocalChapter` AS `localchapters.idLocalChapter`,
                `localchapters`.`LocalName` AS `localchapters.LocalName`,
                `localchapters`.`NationalChapter_id` AS `localchapters.NationalChapter_id`,
                `localchapters`.`nationalchapterIdNationalChapter` AS `localchapters.nationalchapterIdNationalChapter`
            FROM `nationalchapters` AS `nationalchapters`
            LEFT OUTER JOIN `localchapters` AS `localchapters`
            ON `nationalchapters`.`idNationalChapter` = `localchapters`.`nationalchapterIdNationalChapter`;'
}
</code>```

Where I've landed:

This seems like a basic belongsTo/hasMany relationship that is generating broken SQL, yet I can't for the life of me figure out where I'm going wrong... Any ideas?

Edits:
- Fixed typo for model name in "Query ran" section

答:
Resolved

When creating a one-to-many association between models with a custom foreign key, apparently you must specify the foreign key configuration in both the belongsTo and hasMany associations (not just one of them) like so:

```<code>LocalChapters.belongsTo(NationalChapters, { foreignKey: 'NationalChapter_id', onDelete: 'CASCADE' });
NationalChapters.hasMany(LocalChapters, { foreignKey: 'NationalChapter_id', onDelete: 'CASCADE' });
</code>```

Hope this helps.

Body:
Sequelize version:

4.37.10

Model definitions:

LocalChapters.js

const Database = require('./database');
const Sequelize = require('sequelize');
const NationalChapters = require('./nationalchapters');

const LocalChapters = Database
.getInstance()
.define('localchapters', {
    idLocalChapter: {
        type: Sequelize.INTEGER(11),
        primaryKey: true,
        allowNull: false,
        autoIncrement: true,
        unique: true
    },
    LocalName: {
        type: Sequelize.STRING(45),
        allowNull: false
    }
}, {
    timestamps: false
});

LocalChapters.belongsTo(NationalChapters, { foreignKey: 'NationalChapter_id', onDelete: 'CASCADE' });
NationalChapters.hasMany(LocalChapters);

module.exports = LocalChapters;

NationalChapters.js

const Database = require('./database');
const Sequelize = require('sequelize');

const NationalChapters = Database
.getInstance()
.define('nationalchapters', {
    idNationalChapter: {
        type: Sequelize.INTEGER(11),
        primaryKey: true,
        allowNull: false,
        autoIncrement: true,
        unique: true
    },
    NationalName: {
        type: Sequelize.STRING(45),
        allowNull: false,
        unique: true
    }
}, {
    timestamps: false
});

module.exports = NationalChapters;

Query ran:

const result = await NationalChapters.findAll({
    include: [ LocalChapters ]
});

Expected result:

[
  {
    idNationalChapter: 21,
    NationalName: 'Some National Chapter Name',
    localchapters: [
      {
        idLocalChapter: 15,
        LocalName: 'Some Local Chapter Name'
      },
      ...
    ]
  }
  ...
]

Error received:

DatabaseError [SequelizeDatabaseError]: Unknown column 'localchapters.nationalchapterIdNationalChapter' in 'field list'
    at Query.formatError (/<file path>/node_modules/sequelize/lib/dialects/mysql/query.js:247:16)
    at Query.handler [as onResult] (/<file path>/node_modules/sequelize/lib/dialects/mysql/query.js:68:23)
    at Query.Command.execute (/<file path>/node_modules/mysql2/lib/commands/command.js:30:12)
    at Connection.handlePacket (/<file path>/node_modules/mysql2/lib/connection.js:502:28)
    at PacketParser.onPacket (/<file path>/node_modules/mysql2/lib/connection.js:81:16)
    at PacketParser.executeStart (/<file path>/node_modules/mysql2/lib/packet_parser.js:77:14)
    at Socket.<anonymous> (/<file path>/node_modules/mysql2/lib/connection.js:89:29)
    at Socket.emit (events.js:210:5)
    at addChunk (_stream_readable.js:308:12)
    at readableAddChunk (_stream_readable.js:289:11)
    at Socket.Readable.push (_stream_readable.js:223:10)
    at TCP.onStreamRead (internal/stream_base_commons.js:182:23) {
  name: 'SequelizeDatabaseError',
  parent: Error: Unknown column 'localchapters.nationalchapterIdNationalChapter' in 'field list'
      at Packet.asError (/<file path>/node_modules/mysql2/lib/packets/packet.js:716:13)
      at Query.Command.execute (/<file path>/node_modules/mysql2/lib/commands/command.js:28:22)
      at Connection.handlePacket (/<file path>/node_modules/mysql2/lib/connection.js:502:28)
      at PacketParser.onPacket (/<file path>/node_modules/mysql2/lib/connection.js:81:16)
      at PacketParser.executeStart (/<file path>/node_modules/mysql2/lib/packet_parser.js:77:14)
      at Socket.<anonymous> (/<file path>/node_modules/mysql2/lib/connection.js:89:29)
      at Socket.emit (events.js:210:5)
      at addChunk (_stream_readable.js:308:12)
      at readableAddChunk (_stream_readable.js:289:11)
      at Socket.Readable.push (_stream_readable.js:223:10)
      at TCP.onStreamRead (internal/stream_base_commons.js:182:23) {
    code: 'ER_BAD_FIELD_ERROR',
    errno: 1054,
    sqlState: '42S22',
    sqlMessage: "Unknown column 'localchapters.nationalchapterIdNationalChapter' in 'field list'",
    sql: 'SELECT `nationalchapters`.`idNationalChapter`,
                `nationalchapters`.`NationalName`,
                `localchapters`.`idLocalChapter` AS `localchapters.idLocalChapter`,
                `localchapters`.`LocalName` AS `localchapters.LocalName`,
                `localchapters`.`NationalChapter_id` AS `localchapters.NationalChapter_id`,
                `localchapters`.`nationalchapterIdNationalChapter` AS `localchapters.nationalchapterIdNationalChapter`
            FROM `nationalchapters` AS `nationalchapters`
            LEFT OUTER JOIN `localchapters` AS `localchapters`
            ON `nationalchapters`.`idNationalChapter` = `localchapters`.`nationalchapterIdNationalChapter`;'
  },
  original: Error: Unknown column 'localchapters.nationalchapterIdNationalChapter' in 'field list'
      at Packet.asError (/<file path>/node_modules/mysql2/lib/packets/packet.js:716:13)
      at Query.Command.execute (/<file path>/node_modules/mysql2/lib/commands/command.js:28:22)
      at Connection.handlePacket (/<file path>/node_modules/mysql2/lib/connection.js:502:28)
      at PacketParser.onPacket (/<file path>/node_modules/mysql2/lib/connection.js:81:16)
      at PacketParser.executeStart (/<file path>/node_modules/mysql2/lib/packet_parser.js:77:14)
      at Socket.<anonymous> (/<file path>/node_modules/mysql2/lib/connection.js:89:29)
      at Socket.emit (events.js:210:5)
      at addChunk (_stream_readable.js:308:12)
      at readableAddChunk (_stream_readable.js:289:11)
      at Socket.Readable.push (_stream_readable.js:223:10)
      at TCP.onStreamRead (internal/stream_base_commons.js:182:23) {
    code: 'ER_BAD_FIELD_ERROR',
    errno: 1054,
    sqlState: '42S22',
    sqlMessage: "Unknown column 'localchapters.nationalchapterIdNationalChapter' in 'field list'",
    sql: 'SELECT `nationalchapters`.`idNationalChapter`,
                `nationalchapters`.`NationalName`,
                `localchapters`.`idLocalChapter` AS `localchapters.idLocalChapter`,
                `localchapters`.`LocalName` AS `localchapters.LocalName`,
                `localchapters`.`NationalChapter_id` AS `localchapters.NationalChapter_id`,
                `localchapters`.`nationalchapterIdNationalChapter` AS `localchapters.nationalchapterIdNationalChapter`
            FROM `nationalchapters` AS `nationalchapters`
            LEFT OUTER JOIN `localchapters` AS `localchapters`
            ON `nationalchapters`.`idNationalChapter` = `localchapters`.`nationalchapterIdNationalChapter`;'
  },
  sql: 'SELECT `nationalchapters`.`idNationalChapter`,
                `nationalchapters`.`NationalName`,
                `localchapters`.`idLocalChapter` AS `localchapters.idLocalChapter`,
                `localchapters`.`LocalName` AS `localchapters.LocalName`,
                `localchapters`.`NationalChapter_id` AS `localchapters.NationalChapter_id`,
                `localchapters`.`nationalchapterIdNationalChapter` AS `localchapters.nationalchapterIdNationalChapter`
            FROM `nationalchapters` AS `nationalchapters`
            LEFT OUTER JOIN `localchapters` AS `localchapters`
            ON `nationalchapters`.`idNationalChapter` = `localchapters`.`nationalchapterIdNationalChapter`;'
}

Where I've landed:

This seems like a basic belongsTo/hasMany relationship that is generating broken SQL, yet I can't for the life of me figure out where I'm going wrong... Any ideas?

Edits:
- Fixed typo for model name in "Query ran" section

Body:
Resolved

When creating a one-to-many association between models with a custom foreign key, apparently you must specify the foreign key configuration in both the belongsTo and hasMany associations (not just one of them) like so:

LocalChapters.belongsTo(NationalChapters, { foreignKey: 'NationalChapter_id', onDelete: 'CASCADE' });
NationalChapters.hasMany(LocalChapters, { foreignKey: 'NationalChapter_id', onDelete: 'CASCADE' });

Hope this helps.


问:
Python conditional using regex with multiline string
I need help with very simples question, a conditional using regex with multiline string. No make sense to me why this not work:

```<code>if(re.match(r"\w", " \n\n\n  aaaaaaaaaaaa\n\n", re.MULTILINE)):
    print('ok')
else:
    print('fail')

fail
</code>```

I expected that result be ok, but no match any data. I trying using https://regex101.com/r/BsdymE/1, but there works and in my code not works.

答:
From pydoc re.match:

Try to apply the pattern at the start of the string, returning a
  Match object, or None if no match was found.

(emphasis mine). So, the problem is not with the string being multiline, but rather with it not beginning with a word-class character. If you want to check if a string contains something anywhere, use re.search instead.

Body:
I need help with very simples question, a conditional using regex with multiline string. No make sense to me why this not work:

if(re.match(r"\w", " \n\n\n  aaaaaaaaaaaa\n\n", re.MULTILINE)):
    print('ok')
else:
    print('fail')

fail

I expected that result be ok, but no match any data. I trying using https://regex101.com/r/BsdymE/1, but there works and in my code not works.