问:
Keras Input shape of a pretrained model not changing
I am trying to use transfer learning in Keras. I have trained a model for a different task, but now I want to use it for a similar task, but the input and output shapes are different.
I loaded the trained model using load_model. My original model is:

```<code>model = Sequential()
model.add(Conv2D(32, (5,5), input_shape=(28,28,1), padding='same', activation='relu'))
model.add(Conv2D(32, (5,5), padding='same', activation='relu'))
model.add(BatchNormalization())
model.add(Dropout(0.25))
model.add(MaxPool2D(padding='same', strides=2))
model.add(Conv2D(128, (5, 5), padding='same', activation='relu'))
model.add(BatchNormalization())
model.add(Dropout(0.25))
model.add(MaxPool2D(padding='same', strides=2))
model.add(Conv2D(64, (4,4), padding='same', activation='relu'))
model.add(Conv2D(64, (4,4), padding='same', activation='relu'))
model.add(BatchNormalization())
model.add(Dropout(0.25))
model.add(MaxPool2D(padding='same', strides=2))
model.add(Flatten())
model.add(Dense(256, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(26, activation='softmax'))
rmsdrp = optimizers.rmsprop(lr=0.001, epsilon=1e-08)

model.compile( loss = "categorical_crossentropy",
               optimizer = rmsdrp,
               metrics=['accuracy']
             )
</code>```

Then, for output I did the following:

```<code>    model.pop()
    model.add(Dense(3*168,activation='softmax'))
    model.add(Reshape((3,168)))
</code>```

This is working. For input, I did this:

```<code>model.layers[0] = Input(shape=(137,236))
</code>```

But when I printout the model summary, it still gives the previous input shape of the model. What am I doing wrong? How else should I change the input shape? This is the model summary in the end:

```<code>Model: "sequential_1"
_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
conv2d_1 (Conv2D)            (None, 28, 28, 32)        832
_________________________________________________________________
conv2d_2 (Conv2D)            (None, 28, 28, 32)        25632
_________________________________________________________________
batch_normalization_1 (Batch (None, 28, 28, 32)        128
_________________________________________________________________
dropout_1 (Dropout)          (None, 28, 28, 32)        0
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 14, 14, 32)        0
_________________________________________________________________
conv2d_3 (Conv2D)            (None, 14, 14, 128)       102528
_________________________________________________________________
batch_normalization_2 (Batch (None, 14, 14, 128)       512
_________________________________________________________________
dropout_2 (Dropout)          (None, 14, 14, 128)       0
_________________________________________________________________
max_pooling2d_2 (MaxPooling2 (None, 7, 7, 128)         0
_________________________________________________________________
conv2d_4 (Conv2D)            (None, 7, 7, 64)          131136
_________________________________________________________________
conv2d_5 (Conv2D)            (None, 7, 7, 64)          65600
_________________________________________________________________
batch_normalization_3 (Batch (None, 7, 7, 64)          256
_________________________________________________________________
dropout_3 (Dropout)          (None, 7, 7, 64)          0
_________________________________________________________________
max_pooling2d_3 (MaxPooling2 (None, 4, 4, 64)          0
_________________________________________________________________
flatten_1 (Flatten)          (None, 1024)              0
_________________________________________________________________
dense_1 (Dense)              (None, 256)               262400
_________________________________________________________________
dropout_4 (Dropout)          (None, 256)               0
_________________________________________________________________
dense_2 (Dense)              (None, 504)               129528
_________________________________________________________________
reshape_1 (Reshape)          (None, 3, 168)            0
=================================================================
Total params: 595,706
Trainable params: 595,258
Non-trainable params: 448
_________________________________________________________________
</code>```

答:
It looks like the problem is with the use of Input(shape=(137,236)) which is usually used for the functional model, not the sequential. You can change the input layer by essentially changing the model:

input = Input(shape=(137,236))
x = model.layers[1](input) #assuming you are ignoring the first conv layer as implied in your code
for layer in model.layers[2:]:
    x = layer(x)
model = Model(inputs=input, outputs=x)
model.compile(*args, **kwargs)
```
<h2>Edit:</h2>
<p>To make sure the weights do not get, you need to add a for loop to set them to un-trainable.</p>
<pre class="lang-py prettyprint-override"><code>for layer in model.layers[1:-2]:
    layer.trainable=False
</code>```

Body:
I am trying to use transfer learning in Keras. I have trained a model for a different task, but now I want to use it for a similar task, but the input and output shapes are different.
I loaded the trained model using load_model. My original model is:

model = Sequential()
model.add(Conv2D(32, (5,5), input_shape=(28,28,1), padding='same', activation='relu'))
model.add(Conv2D(32, (5,5), padding='same', activation='relu'))
model.add(BatchNormalization())
model.add(Dropout(0.25))
model.add(MaxPool2D(padding='same', strides=2))
model.add(Conv2D(128, (5, 5), padding='same', activation='relu'))
model.add(BatchNormalization())
model.add(Dropout(0.25))
model.add(MaxPool2D(padding='same', strides=2))
model.add(Conv2D(64, (4,4), padding='same', activation='relu'))
model.add(Conv2D(64, (4,4), padding='same', activation='relu'))
model.add(BatchNormalization())
model.add(Dropout(0.25))
model.add(MaxPool2D(padding='same', strides=2))
model.add(Flatten())
model.add(Dense(256, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(26, activation='softmax'))
rmsdrp = optimizers.rmsprop(lr=0.001, epsilon=1e-08)

model.compile( loss = "categorical_crossentropy",
               optimizer = rmsdrp,
               metrics=['accuracy']
             )

Then, for output I did the following:

    model.pop()
    model.add(Dense(3*168,activation='softmax'))
    model.add(Reshape((3,168)))

This is working. For input, I did this:

model.layers[0] = Input(shape=(137,236))

But when I printout the model summary, it still gives the previous input shape of the model. What am I doing wrong? How else should I change the input shape? This is the model summary in the end:

Model: "sequential_1"
_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
conv2d_1 (Conv2D)            (None, 28, 28, 32)        832
_________________________________________________________________
conv2d_2 (Conv2D)            (None, 28, 28, 32)        25632
_________________________________________________________________
batch_normalization_1 (Batch (None, 28, 28, 32)        128
_________________________________________________________________
dropout_1 (Dropout)          (None, 28, 28, 32)        0
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 14, 14, 32)        0
_________________________________________________________________
conv2d_3 (Conv2D)            (None, 14, 14, 128)       102528
_________________________________________________________________
batch_normalization_2 (Batch (None, 14, 14, 128)       512
_________________________________________________________________
dropout_2 (Dropout)          (None, 14, 14, 128)       0
_________________________________________________________________
max_pooling2d_2 (MaxPooling2 (None, 7, 7, 128)         0
_________________________________________________________________
conv2d_4 (Conv2D)            (None, 7, 7, 64)          131136
_________________________________________________________________
conv2d_5 (Conv2D)            (None, 7, 7, 64)          65600
_________________________________________________________________
batch_normalization_3 (Batch (None, 7, 7, 64)          256
_________________________________________________________________
dropout_3 (Dropout)          (None, 7, 7, 64)          0
_________________________________________________________________
max_pooling2d_3 (MaxPooling2 (None, 4, 4, 64)          0
_________________________________________________________________
flatten_1 (Flatten)          (None, 1024)              0
_________________________________________________________________
dense_1 (Dense)              (None, 256)               262400
_________________________________________________________________
dropout_4 (Dropout)          (None, 256)               0
_________________________________________________________________
dense_2 (Dense)              (None, 504)               129528
_________________________________________________________________
reshape_1 (Reshape)          (None, 3, 168)            0
=================================================================
Total params: 595,706
Trainable params: 595,258
Non-trainable params: 448
_________________________________________________________________

Body:
It looks like the problem is with the use of Input(shape=(137,236)) which is usually used for the functional model, not the sequential. You can change the input layer by essentially changing the model:

input = Input(shape=(137,236))
x = model.layers[1](input) #assuming you are ignoring the first conv layer as implied in your code
for layer in model.layers[2:]:
    x = layer(x)
model = Model(inputs=input, outputs=x)
model.compile(*args, **kwargs)

Edit:

To make sure the weights do not get, you need to add a for loop to set them to un-trainable.

for layer in model.layers[1:-2]:
    layer.trainable=False


来源: stackexchange

问:
Problem in Handling the Mouse click of CListCtrl
I have a listctrl with CheckBox in it(LVS_EX_CHECKBOXES) .It is a single column List Control . My Problem is when I Click on the CheckBox the particular item is getting selected/UnSelected. But when I click on the Item text the corresponding Checkbox is not getting Selected/UnSelected . How to handle both the Scenarios.</p>

答:
To check the item when the user clicks on the item text, you'll have to handle the NM_CLICK message, which is sent whenever the user clicks on the item.

Something along the lines of:

```<code>CYourListCtrl::OnNMClick(NMHDR* pNMHDR, LRESULT* pResult)
{

LPNMITEMACTIVATE pNMItemActivate = reinterpret_cast<LPNMITEMACTIVATE>(pNMHDR);

int nItemIndex = pNMItemActivate->iItem;

BOOL bCurrentCheckState = GetCheck(nItemIndex);

SetCheck(nItemIndex, !bCurrentCheckState);

*pResult = 0;

}
</code>```

I'm writing this without testing though, so you'll have to make sure that it doesn't conflict with the handler for clicks on the check box iteself.

ContentLicense: CC BY-SA 2.5

Body:
To check the item when the user clicks on the item text, you'll have to handle the NM_CLICK message, which is sent whenever the user clicks on the item.

Something along the lines of:

CYourListCtrl::OnNMClick(NMHDR* pNMHDR, LRESULT* pResult)
{

LPNMITEMACTIVATE pNMItemActivate = reinterpret_cast<LPNMITEMACTIVATE>(pNMHDR);

int nItemIndex = pNMItemActivate->iItem;

BOOL bCurrentCheckState = GetCheck(nItemIndex);

SetCheck(nItemIndex, !bCurrentCheckState);

*pResult = 0;

}

I'm writing this without testing though, so you'll have to make sure that it doesn't conflict with the handler for clicks on the check box iteself.

ContentLicense: CC BY-SA 3.0


问:
Uploading .xlsx file and processing it with .py file when we click submit button and store data to mysql
I am developing program where I can upload excel invoice .xlsx file using Flask and then store that file to mysql when user click submit button. There are total three files involved. getFile.py which is Flask to upload file using index.html. exlTomySQL.py store data to mysql from file received by index.html. I am new to Flask and don't know how to process data after clicking submit button on index.html to process .xlsx file and return response for user.

getFile.py

```<code>from flask import Flask, render_template, request

app = Flask(__name__)

@app.route('/', methods=['GET', 'POST'])
def index():
    if request.method == 'POST':

        fileDetails = request.form
        myFile = fileDetails['myfile']

        return myFile

    return render_template('index.html')

if __name__ == "__main__":
    app.run(debug=True)
</code>```

Index.html

```<code><!DOCTYPE html>
<html lang="en">

<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>File Page</title>
</head>

<body>
    <form method="POST" action="">
        <label for="myfile">Select a file:</label>
        <br>
        <input type="file" id="myfile" name="myfile">
        <br>
        <input type="submit">
    </form>
</body>

</html>
</code>```

exlToSQL.py

```<code>import openpyxl
import math
import datetime
import mysql.connector

wb = openpyxl.load_workbook('myFile')
sheet = wb.active

# count rows of data for column 2 as it represent description

def countRows():
    count = 0
    for i in range(1, 100, 1):
        if sheet.cell(row=i, column=2).value != None:
            count = i
    return count

# list each row description, (not counting quantity), gst amount and last column(total excluding GST)

def readRows():
    description = []
    # quantity = []
    amount = []
    gstAmount = []

    for i in range(3, countRows() + 1, 1):
        description.append(sheet.cell(row=i, column=2).value)
        # quantity.append(sheet.cell(row=i, column=5).value)
        gstAmount.append(sheet.cell(row=i, column=10).value)
        amount.append(sheet.cell(row=i, column=11).value)
    uperCaseDescription = [desc.upper()
                           for desc in description]
    return uperCaseDescription, gstAmount, amount

# count all list totals for GST and Amount

def countTotals():
    uperCaseDescription, gstAmount, amount = readRows()

    totalGST = round(sum(gstAmount), 2)
    subTotalAmount = round(sum(amount), 2)
    totalAmountOfInvoice = round((totalGST + subTotalAmount), 2)

    return totalGST, subTotalAmount, totalAmountOfInvoice

# get static data of invoice e.g. invoice number,
# invoice date, sub total, total gst, total inc gst

print(countRows())
print(countTotals())
invoice_number = sheet.cell(row=1, column=1).value
print("Invoice Number:", invoice_number)
invoice_date = sheet.cell(row=1, column=2).value
print("Invoice Date:", invoice_date)

sun_offset = (invoice_date.weekday() - 6) % 7

weekEnd = invoice_date - datetime.timedelta(days=sun_offset)
print("Week End Date:", weekEnd)

print(readRows())

# merge all list into tuple

def mergeList():
    uperCaseDescription, gstAmount, amount = readRows()

    mergeTuple = tuple(zip(uperCaseDescription, gstAmount, amount))
    return mergeTuple

# MySql connection and transfer data to mysql

mydb = mysql.connector.connect(
    host="localhost",
    user="root",
    password="Vrishab2145",
    database="testdbOne"
)
# creating database_cursor to perform SQL operation

mycursor = mydb.cursor()
sqlFormula = "INSERT INTO invoice (description, gstAmount, totalExGst) VALUES (%s,%s,%s)"
mycursor.executemany(sqlFormula, mergeList())
mydb.commit()

</code>```

答:
You can save the file being uploaded by making some changes in your code as shown below.

Once you have saved the file, you can deal with the file processing to store the data to MySQL. You can also delete the file previously saved once the processing is done.

getFile.py

```<code>from flask import Flask, render_template, request
import os

app = Flask(__name__)
app.config["UPLOADS"] = "C:/FlaskSandbox/uploads" # path where uploads are saved. Can also be UNIX path

@app.route('/', methods=['GET', 'POST'])
def index():
    if request.method == 'POST':
        userfile = request.files['myfile']
        userfile.save(os.path.join(app.config["UPLOADS"], userfile.filename))
        print('File Saved!')

        # call your file processing functions below

    return render_template('index.html')

if __name__ == "__main__":
    app.run(debug=True)
</code>```

index.html

```<code><!DOCTYPE html>
<html lang="en">

<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>File Page</title>
</head>

<body>
    <form method="POST" action="" enctype=multipart/form-data>
        <label for="myfile">Select a file:</label>
        <br>
        <input type="file" id="myfile" name="myfile">
        <br>
        <input type="submit">
    </form>
</body>

</html>
</code>```

ContentLicense: CC BY-SA 4.0

Body:
I am developing program where I can upload excel invoice .xlsx file using Flask and then store that file to mysql when user click submit button. There are total three files involved. getFile.py which is Flask to upload file using index.html. exlTomySQL.py store data to mysql from file received by index.html. I am new to Flask and don't know how to process data after clicking submit button on index.html to process .xlsx file and return response for user.

getFile.py

from flask import Flask, render_template, request

app = Flask(__name__)

@app.route('/', methods=['GET', 'POST'])
def index():
    if request.method == 'POST':

        fileDetails = request.form
        myFile = fileDetails['myfile']

        return myFile

    return render_template('index.html')

if __name__ == "__main__":
    app.run(debug=True)

Index.html

<!DOCTYPE html>

    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>File Page</title>

    <form method="POST" action="">
        Select a file:

        <input type="file" id="myfile" name="myfile">

        <input type="submit">
    </form>

exlToSQL.py

import openpyxl
import math
import datetime
import mysql.connector

wb = openpyxl.load_workbook('myFile')
sheet = wb.active

# count rows of data for column 2 as it represent description

def countRows():
    count = 0
    for i in range(1, 100, 1):
        if sheet.cell(row=i, column=2).value != None:
            count = i
    return count

# list each row description, (not counting quantity), gst amount and last column(total excluding GST)

def readRows():
    description = []
    # quantity = []
    amount = []
    gstAmount = []

    for i in range(3, countRows() + 1, 1):
        description.append(sheet.cell(row=i, column=2).value)
        # quantity.append(sheet.cell(row=i, column=5).value)
        gstAmount.append(sheet.cell(row=i, column=10).value)
        amount.append(sheet.cell(row=i, column=11).value)
    uperCaseDescription = [desc.upper()
                           for desc in description]
    return uperCaseDescription, gstAmount, amount

# count all list totals for GST and Amount

def countTotals():
    uperCaseDescription, gstAmount, amount = readRows()

    totalGST = round(sum(gstAmount), 2)
    subTotalAmount = round(sum(amount), 2)
    totalAmountOfInvoice = round((totalGST + subTotalAmount), 2)

    return totalGST, subTotalAmount, totalAmountOfInvoice

# get static data of invoice e.g. invoice number,
# invoice date, sub total, total gst, total inc gst

print(countRows())
print(countTotals())
invoice_number = sheet.cell(row=1, column=1).value
print("Invoice Number:", invoice_number)
invoice_date = sheet.cell(row=1, column=2).value
print("Invoice Date:", invoice_date)

sun_offset = (invoice_date.weekday() - 6) % 7

weekEnd = invoice_date - datetime.timedelta(days=sun_offset)
print("Week End Date:", weekEnd)

print(readRows())

# merge all list into tuple

def mergeList():
    uperCaseDescription, gstAmount, amount = readRows()

    mergeTuple = tuple(zip(uperCaseDescription, gstAmount, amount))
    return mergeTuple

# MySql connection and transfer data to mysql

mydb = mysql.connector.connect(
    host="localhost",
    user="root",
    password="Vrishab2145",
    database="testdbOne"
)
# creating database_cursor to perform SQL operation

mycursor = mydb.cursor()
sqlFormula = "INSERT INTO invoice (description, gstAmount, totalExGst) VALUES (%s,%s,%s)"
mycursor.executemany(sqlFormula, mergeList())
mydb.commit()

Body:
You can save the file being uploaded by making some changes in your code as shown below.

Once you have saved the file, you can deal with the file processing to store the data to MySQL. You can also delete the file previously saved once the processing is done.

getFile.py

from flask import Flask, render_template, request
import os

app = Flask(__name__)
app.config["UPLOADS"] = "C:/FlaskSandbox/uploads" # path where uploads are saved. Can also be UNIX path

@app.route('/', methods=['GET', 'POST'])
def index():
    if request.method == 'POST':
        userfile = request.files['myfile']
        userfile.save(os.path.join(app.config["UPLOADS"], userfile.filename))
        print('File Saved!')

        # call your file processing functions below

    return render_template('index.html')

if __name__ == "__main__":
    app.run(debug=True)

index.html

<!DOCTYPE html>
<html lang="en">

<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>File Page</title>
</head>

<body>
    <form method="POST" action="" enctype=multipart/form-data>
        <label for="myfile">Select a file:</label>
        <br>
        <input type="file" id="myfile" name="myfile">
        <br>
        <input type="submit">
    </form>
</body>

</html>


问:
auto_arima returning the best model as SARIMAX even when time series is stationary and has no seasonal component in Python
I have a time series dataset having frequency as daily. I have checked that my dataset is stationary using augmented dickey-fuller test.

After which, when i am trying to determine the values of p,d,q using:

```<code>from pmdarima import auto_arima
stepwise_fit = auto_arima(df2['Births'],start_p=0,max_p=6, start_q=0, max_q=3, seasonal=False,trace=True)
</code>```

Moreover, I have mentioned seasonal=False in auto_arima argument, but when i did:

```<code>stepwise_fit.summary()
</code>```

Its returning:

```<code>SARIMAX Results
Dep. Variable:  y   No. Observations:   365
Model:  SARIMAX(1, 1, 1)    Log Likelihood  -1226.077
Date:   Mon, 17 Feb 2020    AIC 2460.154
Time:   20:02:17    BIC 2475.743
Sample: 0   HQIC    2466.350
- 365
Covariance Type:    opg
             coef   std err   z     P>|z|   [0.025  0.975]
intercept   0.0132  0.014   0.975   0.330   -0.013  0.040
ar.L1       0.1299  0.059   2.217   0.027   0.015   0.245
ma.L1      -0.9694  0.016   -62.235 0.000   -1.000  -0.939
sigma2      48.9989 3.432   14.279  0.000   42.273  55.725
Ljung-Box (Q):  36.69   Jarque-Bera (JB):   26.17
Prob(Q):        0.62    Prob(JB):   0.00
Heteroskedasticity (H): 0.97    Skew:   0.58
Prob(H) (two-sided):    0.85    Kurtosis:   3.62
</code>```

We can see, its returning Model: SARIMAX(1, 1, 1). What can we infer from it? Any suggestion is helpful, or if i am missing something.

答: I found the reason its showing SARIMAX(1, 1, 1). It simply means ARIMA only because the format for SARIMAX is basically SARIMAX(p,d,q)(P,D,Q) where P,D,Q are seasonal parameters, so, in our case SARIMAX(1,1,1)(0,0,0) seasonal components are zero only.

Body:
I have a time series dataset having frequency as daily. I have checked that my dataset is stationary using augmented dickey-fuller test.

After which, when i am trying to determine the values of p,d,q using:

from pmdarima import auto_arima
stepwise_fit = auto_arima(df2['Births'],start_p=0,max_p=6, start_q=0, max_q=3, seasonal=False,trace=True)

Moreover, I have mentioned seasonal=False in auto_arima argument, but when i did:

stepwise_fit.summary()

Its returning:

SARIMAX Results
Dep. Variable:  y   No. Observations:   365
Model:  SARIMAX(1, 1, 1)    Log Likelihood  -1226.077
Date:   Mon, 17 Feb 2020    AIC 2460.154
Time:   20:02:17    BIC 2475.743
Sample: 0   HQIC    2466.350
- 365
Covariance Type:    opg
             coef   std err   z     P>|z|   [0.025  0.975]
intercept   0.0132  0.014   0.975   0.330   -0.013  0.040
ar.L1       0.1299  0.059   2.217   0.027   0.015   0.245
ma.L1      -0.9694  0.016   -62.235 0.000   -1.000  -0.939
sigma2      48.9989 3.432   14.279  0.000   42.273  55.725
Ljung-Box (Q):  36.69   Jarque-Bera (JB):   26.17
Prob(Q):        0.62    Prob(JB):   0.00
Heteroskedasticity (H): 0.97    Skew:   0.58
Prob(H) (two-sided):    0.85    Kurtosis:   3.62

We can see, its returning Model: SARIMAX(1, 1, 1). What can we infer from it? Any suggestion is helpful, or if i am missing something.


问:
How do you add multiple unique Excel files in one long file? (Python)
There are many posts on how to add xlsx files, this is not the same.

I have 189 excel files each representing unique data. I want to add all this into one xlsx file as one long thread, for easier readability. There are no rows/columns to merge/concat, each column/row is unique. Each file has multiple rows and columns.

It can all be in one sheet or each file can be in its own sheet where the sheet name == file name.

I need this for easier readability so I dont have to open 189 different files, but scroll through one.
Thanks!

答:
For that you should consider Pandas Library.

First,

```<code>python -m pip install pandas
</code>```

The you should import Pandas and Glob

```<code>import pandas as pd
import glob
</code>```

Then you should start mapping your necessities.

```<code>def All_XLSX_in_One():
    Excel_path = 'Your_Worksheets_Path'
    all_files = glob.glob(Excel_path + "/*.xlsx")
    li = []
    for filename in all_files:
        df = pd.read_excel(filename, index_col=None, header=0)
        li.append(df)
    frame = pd.concat(li, axis=0, ignore_index=True)
    Excel_path_final = Excel_path + '/' + ' ALL_EXCEL.xlsx'
    frame.to_excel(Excel_path_final, header=True, encoding='utf-8-sig')
</code>```

The code above should save an Excel worksheet with all the xlsx files in 'Your_Worksheets_Path' merged.

It works better if they all have the same columns.

Run it by calling your def.

```<code>All_XLSX_in_One()
</code>```

Body:
For that you should consider Pandas Library.

First,

python -m pip install pandas

The you should import Pandas and Glob

import pandas as pd
import glob

Then you should start mapping your necessities.

def All_XLSX_in_One():
    Excel_path = 'Your_Worksheets_Path'
    all_files = glob.glob(Excel_path + "/*.xlsx")
    li = []
    for filename in all_files:
        df = pd.read_excel(filename, index_col=None, header=0)
        li.append(df)
    frame = pd.concat(li, axis=0, ignore_index=True)
    Excel_path_final = Excel_path + '/' + ' ALL_EXCEL.xlsx'
    frame.to_excel(Excel_path_final, header=True, encoding='utf-8-sig')

The code above should save an Excel worksheet with all the xlsx files in 'Your_Worksheets_Path' merged.

It works better if they all have the same columns.

Run it by calling your def.

All_XLSX_in_One()


问:
Flask redirect from a different class
I'm trying to call a function that has a flask redirect which exists in a certain class.
My files are like so:

```<code>--- Lib/config.py
--- Lib/lib.py
--- auth.py
</code>```

My code inside lib.py

```<code>    import os
    import json
    import time
    import requests
    from Lib.config import APP_UID, REDIRECT_URI
    from flask import redirect

    class PyLi:
        def __init__(self):
            self.app_uid = APP_UID
            self.redirect_uri = REDIRECT_URI

       def code_redirect(self):
        d = {'client_id=' + self.app_uid, 'redirect_uri=' + self.redirect_uri,
            'response_type=' + 'code'}
        return redirect("https://website.com/oauth/authorize?%s" %
            ("&".join(d)))
</code>```

My config.py is just used to define the env variables I'm calling.

```<code>from decouple import config

APP_UID = config('APP_UID', cast=str, default=None)
...
</code>```

And in auth.py I am calling it like so:

```<code>from Lib import lib

@token.route("/authorize", methods=['GET'])
def redirect():
    auth = lib.PyLi()
    auth.code_redirect()
</code>```

But I'm getting TypeError: The view function did not return a valid response. The function either returned None or ended without a return statement.

What am I doing wrong?

答:
You're missing a return on the last line of redirect() view function. Add it as follows:

```<code>@token.route("/authorize", methods=['GET'])
def redirect():
    auth = lib.PyLi()
    return auth.code_redirect()
</code>```

Title: Flask redirect from a different class

Body:
I'm trying to call a function that has a flask redirect which exists in a certain class.
My files are like so:

--- Lib/config.py
--- Lib/lib.py
--- auth.py

My code inside lib.py

    import os
    import json
    import time
    import requests
    from Lib.config import APP_UID, REDIRECT_URI
    from flask import redirect

    class PyLi:
        def __init__(self):
            self.app_uid = APP_UID
            self.redirect_uri = REDIRECT_URI

       def code_redirect(self):
        d = {'client_id=' + self.app_uid, 'redirect_uri=' + self.redirect_uri,
            'response_type=' + 'code'}
        return redirect("https://website.com/oauth/authorize?%s" %
            ("&".join(d)))

My config.py is just used to define the env variables I'm calling.

from decouple import config

APP_UID = config('APP_UID', cast=str, default=None)
...

And in auth.py I am calling it like so:

from Lib import lib

@token.route("/authorize", methods=['GET'])
def redirect():
    auth = lib.PyLi()
    auth.code_redirect()

But I'm getting TypeError: The view function did not return a valid response. The function either returned None or ended without a return statement.

What am I doing wrong?

Body:
You're missing a return on the last line of redirect() view function. Add it as follows:

@token.route("/authorize", methods=['GET'])
def redirect():
    auth = lib.PyLi()
    return auth.code_redirect()


问:
AWS Cognito Users with Authorization (how to add a user to a user group from the application?)
I'm building an app with react on the front end and aws dynamodb + labmda on the back end to save data.  I'm using amplify to handle user authentication but I'm struggling with a good way to handle authorization.

I'd like to use the Cognito userpool groups but there does not appear to be a way to manage a user from my app because amplify does not support updating a user to place them in a group.  I'd really rather not expose the aws console login to the app administrator so it would seem i need to find a way in the app itself to handle this.

I could create a user table in dynamodb itself to hold a user profile which could have a field to hold the roles I want the user to be able to perform and then add a CRUD page for an admin user to update the profile of a non-admin user.  However, I don't particularly like this option because its really a duplication of data and because it doesn't also support a way to (easily) disable a user.

It does appear that a dynamodb table update can trigger a lambda function so I'd also considered somehow doing this to trigger the cognito userpool role addition for the user when the table is updated by the admin user on the CRUD page but I'm having a little trouble connecting all the dots on the trigger, the function and the permissions needed.

Any assistance would be greatly appreciated.

Thanks.

答:
Thanks Dylan, this is exactly what I've done.  I had mistakenly thought, that this could not be done with my user but after attempting this and tracing the failures, I was able to determine how to apply the correct policy to the role that was doing the addToGroup and removeFromGroup.

Thanks again,
Mike


问:
Google Cloud Run same URL on different containers
I have two containers (backend and frontend) that I want to access on the same URL. Looking at the Cloud Run documentation, I can't see how I can configure the following two containers.

frontend => example.com/

backend  => example.com/api

Is this possible?

答:
According to the official documentation for Cloud Run link:

Base path mapping: not supported

The term base path refers to the URL
path name that is after the domain name. For example, users is the
base path of example.com/users. Cloud Run only allows you to map a
domain to /, not to a specific base path. So any path routing has to
be handled by using a router inside the service's container or by
using Firebase Hosting.


问:
Normalization: how to avoid zero standard deviation
Have the following task:

Normalize the matrix by columns. From each value in column subtract average (in column) and divide it by standard deviation (in the column). Your output should not contain nan (caused by division by zero). Replace Nans with 1. Don't use if and while/for.

I an working with numpy, so I wrote the following code:

```<code>def normalize(matrix: np.array) -> np.array:
    res = (matrix - np.mean(matrix, axis = 0)) / np.std(matrix, axis = 0, dtype=np.float64)
    return res
matrix = np.array([[1, 4, 4200], [0, 10, 5000], [1, 2, 1000]])
assert np.allclose(
    normalize(matrix),
    np.array([[ 0.7071, -0.39223,  0.46291],
              [-1.4142,  1.37281,  0.92582],
              [ 0.7071, -0.98058, -1.38873]])
)
</code>```

The answer is right.

However, my question is: how do I avoid division by zero? If i have a column of similar numbers, I'll have standard deviation = 0 and the Nan value in result. How do I solve it? Would be grateful!

答:
Your task specifies to avoid nan in the output and replace nan that occur with 1. It does not specify that intermediate results may not contain nan. A valid solution can be to use numpy.nan_to_num on res before returning:

```<code>import numpy as np
def normalize(matrix: np.array) -> np.array:
    res = (matrix - np.mean(matrix, axis = 0)) / np.std(matrix, axis = 0, dtype=np.float64)
    return np.nan_to_num(res, False, 1.0)
matrix = np.array([[2, 4, 4200], [2, 10, 5000], [2, 2, 1000]])
print(normalize(matrix))
</code>```

yields:

```<code>[[ 1.         -0.39223227  0.46291005]
 [ 1.          1.37281295  0.9258201 ]
 [ 1.         -0.98058068 -1.38873015]]
</code>```

Body:
Have the following task:

Normalize the matrix by columns. From each value in column subtract average (in column) and divide it by standard deviation (in the column). Your output should not contain nan (caused by division by zero). Replace Nans with 1. Don't use if and while/for.

I an working with numpy, so I wrote the following code:

def normalize(matrix: np.array) -> np.array:
    res = (matrix - np.mean(matrix, axis = 0)) / np.std(matrix, axis = 0, dtype=np.float64)
    return res
matrix = np.array([[1, 4, 4200], [0, 10, 5000], [1, 2, 1000]])
assert np.allclose(
    normalize(matrix),
    np.array([[ 0.7071, -0.39223,  0.46291],
              [-1.4142,  1.37281,  0.92582],
              [ 0.7071, -0.98058, -1.38873]])
)

The answer is right.

However, my question is: how do I avoid division by zero? If i have a column of similar numbers, I'll have standard deviation = 0 and the Nan value in result. How do I solve it? Would be grateful!

Body:
Your task specifies to avoid nan in the output and replace nan that occur with 1. It does not specify that intermediate results may not contain nan. A valid solution can be to use numpy.nan_to_num on res before returning:

import numpy as np
def normalize(matrix: np.array) -> np.array:
    res = (matrix - np.mean(matrix, axis = 0)) / np.std(matrix, axis = 0, dtype=np.float64)
    return np.nan_to_num(res, False, 1.0)
matrix = np.array([[2, 4, 4200], [2, 10, 5000], [2, 2, 1000]])
print(normalize(matrix))

yields:

[[ 1.         -0.39223227  0.46291005]
 [ 1.          1.37281295  0.9258201 ]
 [ 1.         -0.98058068 -1.38873015]]


问:
Return type dependent on input type in Python
The following function in Python can returns a list no matter if the input is a list, a numpy.array or a pandas.Series.

What is the pythonic way to write it so that the output type is the same as the input type?

```<code>def foo(input):
    output = []

    output.append(input[0])

    for i in range(1, len(input)-1):
        if (some condition):
            output.append(input[i])

    output.append(input[-1])

    return output
</code>```

答:
In general, you can't do this without making a lot of assumptions about what the input is.

The first step would be to make sure that output is the right type, not a list.

```<code>output = type(self)()
</code>```

However, this assumes that whatever type your input is, you can create an instance by calling it with no arguments.

Next, you have to restrict yourself to operations on output that are supported by all expected inputs. Not all iterables support an append method (set, for instance, uses add, not append), and not all iterables support __getitem__ (a generator, for instance). This means that you can't generalize your function too much; you always have to keep in mind which types of input you will support.

Alternatively, if the set of types you want to support can create an instance from a list, you can let output = [] stand, but convert it just before returning:

```<code>return type(self)(output)
</code>```

Body:
The following function in Python can returns a list no matter if the input is a list, a numpy.array or a pandas.Series.

What is the pythonic way to write it so that the output type is the same as the input type?

def foo(input):
    output = []

    output.append(input[0])

    for i in range(1, len(input)-1):
        if (some condition):
            output.append(input[i])

    output.append(input[-1])

    return output

Body:
In general, you can't do this without making a lot of assumptions about what the input is.

The first step would be to make sure that output is the right type, not a list.

output = type(self)()

However, this assumes that whatever type your input is, you can create an instance by calling it with no arguments.

Next, you have to restrict yourself to operations on output that are supported by all expected inputs. Not all iterables support an append method (set, for instance, uses add, not append), and not all iterables support __getitem__ (a generator, for instance). This means that you can't generalize your function too much; you always have to keep in mind which types of input you will support.

Alternatively, if the set of types you want to support can create an instance from a list, you can let output = [] stand, but convert it just before returning:

return type(self)(output)


问:
How is Go executable created in Docker?
I'm pretty new to development Golang & Docker.  I'm following the instructions in the official Golang DockerHub image.  Here's the part I'm a bit confused:

The part I really don't get is the last line of the Dockerfile:

```<code>CMD ["app"]
</code>```

My question is, how is the "app" executable created in the first place?  I created a standard hello-world.go file and added this Docker file to a directory.  I don't get how building the Docker image would generate an executable called "app".  Can someone explain?

答:
Excerpt of the go command   https://golang.org/cmd/go/#hdr-Compile_and_install_packages_and_dependencies

Compile and install packages and dependencies

Usage:

go install [-i] [build flags] [packages]

Install compiles and installs the packages named by the import paths.

Executables are installed in the directory named by the GOBIN
  environment variable, which defaults to $GOPATH/bin or $HOME/go/bin if
  the GOPATH environment variable is not set. Executables in $GOROOT are
  installed in $GOROOT/bin or $GOTOOLDIR instead of $GOBIN.

When module-aware mode is disabled, other packages are installed in
  the directory $GOPATH/pkg/$GOOS_$GOARCH. When module-aware mode is
  enabled, other packages are built and cached but not installed.

The -i flag installs the dependencies of the named packages as well.

For more about the build flags, see 'go help build'. For more about
  specifying packages, see 'go help packages'.

See also: go build, go get, go clean.

This makes an executable out of your go code.

Title: How is Go executable created in Docker?

Body:
I'm pretty new to development Golang & Docker.  I'm following the instructions in the official Golang DockerHub image.  Here's the part I'm a bit confused:

The part I really don't get is the last line of the Dockerfile:

CMD ["app"]

My question is, how is the "app" executable created in the first place?  I created a standard hello-world.go file and added this Docker file to a directory.  I don't get how building the Docker image would generate an executable called "app".  Can someone explain?


问:
Sorting string in Access DB
I have a DB with field Version. There might be many records with the same version string, so to pull out versions I have, I do the following:

```<code>SELECT DISTINCT tblSample.Version
FROM tblSample
ORDER BY tblSample.Version DESC;
</code>```

Here is sample data:

```<code>tblSample.Version
1.1.1000
1.2.1654
1.15.1223
1.2.1654
1.1.1000
</code>```

What I get after executing query:

```<code>1.2.1654
1.15.1223
1.1.1000
</code>```

What I want to get is(because 15 is obviously > 2 and 1):

```<code>1.15.1223
1.2.1654
1.1.1000
</code>```

Respectively other octets should be considered as well.

答:
One solution would be: parse the strings that you get so you extract the numerical values, place leading 0s in front when required, and output a string.
E.g.
1.2.1654 becomes 0001.0002.1654 (if you're sure the numbering won't exceed 4 characters).
Code:

```<code>Function NormalizeVersion(Inputstring As String) As String
' Creates sortable strings out of version numbers such as 1.6.222
Dim Elements() As String
Dim Counter As Integer
Dim Result As String
Elements = Split(Inputstring, ".")
For Counter = 0 To UBound(Elements)
    Select Case Counter
    Case 0 'First element
        Result = Format(Elements(Counter), "00000")
    Case Else 'Followups
        Result = Result & "." & Right("0000" & Elements(Counter), 5)
    End Select
Next Counter
NormalizeVersion = Result
End Function
</code>```

(incorporated HansUp's performance improvement)
Then you can sort on this.

Write a VBA function that does this for you, called e.g. NormalizeVersion, then call it in your query, such as (air code):

```<code>SELECT DISTINCT NormalizeVersion(tblSample.Version)
FROM tblSample
ORDER BY NormalizeVersion(tblSample.Version) DESC;
</code>```

(code fixed after comment)

Title: Sorting string in Access DB

Body:
I have a DB with field Version. There might be many records with the same version string, so to pull out versions I have, I do the following:

SELECT DISTINCT tblSample.Version
FROM tblSample
ORDER BY tblSample.Version DESC;

Here is sample data:

tblSample.Version
1.1.1000
1.2.1654
1.15.1223
1.2.1654
1.1.1000

What I get after executing query:

1.2.1654
1.15.1223
1.1.1000

What I want to get is(because 15 is obviously > 2 and 1):

1.15.1223
1.2.1654
1.1.1000

Respectively other octets should be considered as well.

Body:
One solution would be: parse the strings that you get so you extract the numerical values, place leading 0s in front when required, and output a string.
E.g.
1.2.1654 becomes 0001.0002.1654 (if you're sure the numbering won't exceed 4 characters).
Code:

Function NormalizeVersion(Inputstring As String) As String
' Creates sortable strings out of version numbers such as 1.6.222
Dim Elements() As String
Dim Counter As Integer
Dim Result As String
Elements = Split(Inputstring, ".")
For Counter = 0 To UBound(Elements)
    Select Case Counter
    Case 0 'First element
        Result = Format(Elements(Counter), "00000")
    Case Else 'Followups
        Result = Result & "." & Right("0000" & Elements(Counter), 5)
    End Select
Next Counter
NormalizeVersion = Result
End Function

(incorporated HansUp's performance improvement)
Then you can sort on this.

Write a VBA function that does this for you, called e.g. NormalizeVersion, then call it in your query, such as (air code):

SELECT DISTINCT NormalizeVersion(tblSample.Version)
FROM tblSample
ORDER BY NormalizeVersion(tblSample.Version) DESC;

(code fixed after comment)


问:
What's the difference between tilde(~) and caret(^) in package.json?
After I upgraded to the latest stable node and npm, I tried npm install moment --save. It saves the entry in the package.json with the caret ^ prefix. Previously, it was a tilde ~ prefix.

Why are these changes made in npm?

What is the difference between tilde ~ and caret ^?

What are the advantages over others?

答:
Related to this question you can review Composer documentation on versions, but here in short:

Tilde Version Range (~) - ~1.2.3 is equivalent to >=1.2.3 <1.3.0

Caret Version Range (^) - ~1.2.3 is equivalent to >=1.2.3 <2.0.0

So, with Tilde you will get automatic updates of patches but minor and major versions will not be updated. However, if you use Caret you will get patches and minor versions, but you will not get major (breaking changes) versions.

Tilde Version is considered "safer" approach, but if you are using reliable dependencies (well-maintained libraries) you should not have any problems with Caret Version (because minor changes should not be breaking changes.

You should probably review this stackoverflow post about differences between composer install and composer update.


问:
How to return NAME of the cell (not address, content, etc.)
Let's say I name cell C4 as "TestCell", using NameManager or typing into the Name Box.

I want C5 to have a formula to return "TestCell" without the quotes.

I'll take VBA answer if that's necessary for this.

答:
This is what I was looking for, about a few hours of digging:
How do you get a Range to return its Name?

use the .Name.Name method worked for me (using one .name gives only address).

Good luck!


问:
I need to create a finite automata
Consider the language L of all strings made of the symbols 0, 1 and 2 (Σ = {0, 1, 2}) where the last symbol is not smaller than the first symbol. E.g., the strings 0, 2012, 01231 and 102 are in the language, but 10, 2021 and 201 are not in the language.

As 0 is in the language and I don't understand why, I can't figure out if strings 1 and 2 are in the language?

So could someone please tell me if 1 and 2 as a string themselves are in the language and why?

Thank you

答:
'0' is in the language because the first symbol and last symbol in '0' are both '0'. This satisfies the requirements that the last is not smaller than the first.

This means that the same applies for '1', '2', and for the empty string ''.

Title: I need to create a finite automata


问:
How to do backward pagination in dynamoDB with LastEvaluatedKey?
```<code>  param.FilterExpression = "#black_listed = :black_listed_val";
  param.Limit = 5
  param.ExpressionAttributeValues = {":black_listed_val": body.blacklisted};
  param.ExpressionAttributeNames = {"#black_listed": "black_listed"}
  param.ScanIndexForward = true
  param.ExclusiveStartKey = {"id": "11931258-b582-4d2d-98d9-aa8ae0fe2e43"}
</code>```

I can do forward pagination with this code, but how could I go back?

答: To page backward you need to set ScanIndexForward to false AND the ExclusiveStartKey to the key of the first item of the second page. As an example, say your first page is items 1, 2 & 3. The response from that will include a LastEvaluatedKey for 3. You can use that to go forward by passing that to the ExclusiveStartKey, resulting in 4, 5 & 6. Going backward you can't do that, because using that same key would result in just 2 & 1. Instead you need to use the key of 4 as your ExclusiveStartKey.

Body:
  param.FilterExpression = "#black_listed = :black_listed_val";
  param.Limit = 5
  param.ExpressionAttributeValues = {":black_listed_val": body.blacklisted};
  param.ExpressionAttributeNames = {"#black_listed": "black_listed"}
  param.ScanIndexForward = true
  param.ExclusiveStartKey = {"id": "11931258-b582-4d2d-98d9-aa8ae0fe2e43"}

I can do forward pagination with this code, but how could I go back?


问:
Setting Incremental Refresh for Every 30 Minutes

I have a question about incremental refresh procedure. I have a table and I'm setting up incremental refresh on it. But I can only choose refresh rows as minimum 1 day. I want to set this specialty as for every 30 minutes. Is that possible? I couldn't find any answer to this question. I've only seen that I can set refresh for every 30 minutes in dataflows.

Is there anybody who will give me detailed information about this subject?

Thanks.

答: According to my researches the incremental refresh for every 30 minutes can be set via dataflows. </p>


问:
NodeJS/mySQL - ER_ACCESS_DENIED_ERROR Access denied for user 'root'@'localhost' (using password: YES)
I am attempting to connect to mySQL through a NodeJS file, but I receive the following error:

```<code>{ Error: ER_ACCESS_DENIED_ERROR: Access denied for user 'root'@'localhost' (using password: YES)
    at Handshake.Sequence._packetToError (/home/matthew/Node/mySqlTest/node_modules/mysql/lib/protocol/sequences/Sequence.js:30:14)
    at Handshake.ErrorPacket (/home/matthew/Node/mySqlTest/node_modules/mysql/lib/protocol/sequences/Handshake.js:67:18)
    at Protocol._parsePacket (/home/matthew/Node/mySqlTest/node_modules/mysql/lib/protocol/Protocol.js:197:24)
    at Parser.write (/home/matthew/Node/mySqlTest/node_modules/mysql/lib/protocol/Parser.js:62:12)
    at Protocol.write (/home/matthew/Node/mySqlTest/node_modules/mysql/lib/protocol/Protocol.js:37:16)
    at Socket.ondata (_stream_readable.js:555:20)
    at emitOne (events.js:101:20)
    at Socket.emit (events.js:188:7)
    at readableAddChunk (_stream_readable.js:176:18)
    at Socket.Readable.push (_stream_readable.js:134:10)
    --------------------
    at Protocol._enqueue (/home/matthew/Node/mySqlTest/node_modules/mysql/lib/protocol/Protocol.js:110:26)
    at Protocol.handshake (/home/matthew/Node/mySqlTest/node_modules/mysql/lib/protocol/Protocol.js:42:41)
    at Connection.connect (/home/matthew/Node/mySqlTest/node_modules/mysql/lib/Connection.js:81:18)
    at Connection._implyConnect (/home/matthew/Node/mySqlTest/node_modules/mysql/lib/Connection.js:222:10)
    at Connection.query (/home/matthew/Node/mySqlTest/node_modules/mysql/lib/Connection.js:137:8)
    at Object.<anonymous> (/home/matthew/Node/mySqlTest/index.js:11:12)
    at Module._compile (module.js:570:32)
    at Object.Module._extensions..js (module.js:579:10)
    at Module.load (module.js:487:32)
    at tryModuleLoad (module.js:446:12)
  code: 'ER_ACCESS_DENIED_ERROR',
  errno: 1045,
  sqlState: '28000',
  fatal: true }
</code>```

The weird thing is that I can connect fine through the terminal by running mysql -u root -p. I only get this error when running my javascript. I have been all over Google and StackOverflow, but still have not found a solution that works. I am using MySQL 5.7.16 on Ubuntu 16.04.1 on a VIRTUAL MACHINE. Not sure if a VM makes a difference here. My Javascript code is below:

```<code>'use strict';

var mysql = require('mysql');

var connection = mysql.createConnection({
    host: 'localhost',
    user: 'root',
    password: 'password'
});

connection.query(
    'SELECT "foo" AS first_field, "bar" AS second_field',
    function(err, results, fields) {
        console.log(err);
        console.log(results);
        connection.end();
    }
);
</code>```

I have tried using 'locahost' as well as '127.0.0.1' in my javascript. I have a 'root' user for both 'localhost' and '127.0.0.1' in mySql.user table and I am able to see this by executing SELECT user, host FROM mysql.user WHERE user='root';

I have added privileges to 'root' user by executing this:

```<code>GRANT ALL PRIVILEGES ON *.* TO 'root'@'localhost' IDENTIFIED BY 'password';
</code>```

I ran the above on 127.0.0.1 as well. I have also tried this:

```<code>GRANT ALL PRIVILEGES ON *.* TO 'root'@'localhost' WITH GRANT OPTION
</code>```

I have attempted to reset the root password like this:
https://help.ubuntu.com/community/MysqlPasswordReset

I have run FLUSH PRIVILEGES after each attempt. I've stopped and restarted mySQL. I have uninstalled mySQL completely and reinstalled.

All to no avail. I receive the access denied error every time I try to run the javascript, but I have absolutely no issues when I connect to mySQL via the terminal.

Any ideas?

答:
Login into your mysql using mysql -u root -p password

Create new user z (MySQL console)

```<code>CREATE USER 'z'@'localhost' IDENTIFIED BY '';
GRANT ALL PRIVILEGES ON * . * TO 'z'@'localhost';
</code>```

Node.js script

```<code>var mysql = require('mysql');

var con = mysql.createConnection({
    host: "localhost",
    user: "z",
    password: ""
});

con.connect(function(err) {

    if (err) throw err;
    console.log("Connected!");

    con.query("use mysql;", function(err, result) {
        if (err) throw err;
        console.log(result);
    });

    con.query("select * from user limit 1;", function(err, result) {
        if (err) throw err;
        console.log(result);
    });

});
</code>```

Body:
I am attempting to connect to mySQL through a NodeJS file, but I receive the following error:

{ Error: ER_ACCESS_DENIED_ERROR: Access denied for user 'root'@'localhost' (using password: YES)
    at Handshake.Sequence._packetToError (/home/matthew/Node/mySqlTest/node_modules/mysql/lib/protocol/sequences/Sequence.js:30:14)
    at Handshake.ErrorPacket (/home/matthew/Node/mySqlTest/node_modules/mysql/lib/protocol/sequences/Handshake.js:67:18)
    at Protocol._parsePacket (/home/matthew/Node/mySqlTest/node_modules/mysql/lib/protocol/Protocol.js:197:24)
    at Parser.write (/home/matthew/Node/mySqlTest/node_modules/mysql/lib/protocol/Parser.js:62:12)
    at Protocol.write (/home/matthew/Node/mySqlTest/node_modules/mysql/lib/protocol/Protocol.js:37:16)
    at Socket.ondata (_stream_readable.js:555:20)
    at emitOne (events.js:101:20)
    at Socket.emit (events.js:188:7)
    at readableAddChunk (_stream_readable.js:176:18)
    at Socket.Readable.push (_stream_readable.js:134:10)
    --------------------
    at Protocol._enqueue (/home/matthew/Node/mySqlTest/node_modules/mysql/lib/protocol/Protocol.js:110:26)
    at Protocol.handshake (/home/matthew/Node/mySqlTest/node_modules/mysql/lib/protocol/Protocol.js:42:41)
    at Connection.connect (/home/matthew/Node/mySqlTest/node_modules/mysql/lib/Connection.js:81:18)
    at Connection._implyConnect (/home/matthew/Node/mySqlTest/node_modules/mysql/lib/Connection.js:222:10)
    at Connection.query (/home/matthew/Node/mySqlTest/node_modules/mysql/lib/Connection.js:137:8)
    at Object.<anonymous> (/home/matthew/Node/mySqlTest/index.js:11:12)
    at Module._compile (module.js:570:32)
    at Object.Module._extensions..js (module.js:579:10)
    at Module.load (module.js:487:32)
    at tryModuleLoad (module.js:446:12)
  code: 'ER_ACCESS_DENIED_ERROR',
  errno: 1045,
  sqlState: '28000',
  fatal: true }

The weird thing is that I can connect fine through the terminal by running mysql -u root -p. I only get this error when running my javascript. I have been all over Google and StackOverflow, but still have not found a solution that works. I am using MySQL 5.7.16 on Ubuntu 16.04.1 on a VIRTUAL MACHINE. Not sure if a VM makes a difference here. My Javascript code is below:

'use strict';

var mysql = require('mysql');

var connection = mysql.createConnection({
    host: 'localhost',
    user: 'root',
    password: 'password'
});

connection.query(
    'SELECT "foo" AS first_field, "bar" AS second_field',
    function(err, results, fields) {
        console.log(err);
        console.log(results);
        connection.end();
    }
);

I have tried using 'locahost' as well as '127.0.0.1' in my javascript. I have a 'root' user for both 'localhost' and '127.0.0.1' in mySql.user table and I am able to see this by executing SELECT user, host FROM mysql.user WHERE user='root';

I have added privileges to 'root' user by executing this:

GRANT ALL PRIVILEGES ON *.* TO 'root'@'localhost' IDENTIFIED BY 'password';

I ran the above on 127.0.0.1 as well. I have also tried this:

GRANT ALL PRIVILEGES ON *.* TO 'root'@'localhost' WITH GRANT OPTION

I have attempted to reset the root password like this:
https://help.ubuntu.com/community/MysqlPasswordReset

I have run FLUSH PRIVILEGES after each attempt. I've stopped and restarted mySQL. I have uninstalled mySQL completely and reinstalled.

All to no avail. I receive the access denied error every time I try to run the javascript, but I have absolutely no issues when I connect to mySQL via the terminal.

Any ideas?

Body:
Login into your mysql using mysql -u root -p password

Create new user z (MySQL console)

CREATE USER 'z'@'localhost' IDENTIFIED BY '';
GRANT ALL PRIVILEGES ON * . * TO 'z'@'localhost';

Node.js script

var mysql = require('mysql');

var con = mysql.createConnection({
    host: "localhost",
    user: "z",
    password: ""
});

con.connect(function(err) {

    if (err) throw err;
    console.log("Connected!");

    con.query("use mysql;", function(err, result) {
        if (err) throw err;
        console.log(result);
    });

    con.query("select * from user limit 1;", function(err, result) {
        if (err) throw err;
        console.log(result);
    });

});


问:
JetBrains WebStorm npm modules autocompletion not working
I am new to Node.js development and I have installed WebStorm from JetBrains to use it as my JavaScript IDE.

So I am following a tutorial in Udemy and I have come to notice the following problem.

I installed some modules with npm from WebStorm Console and although my JS script is working as intended, WebStorm autocompletion for the npm modules is not working.

Coding assistance for Node.js in my settings is marked! What do I do wrong?

答:
The problems occur because of the weird way properties are defined.
For example, in chalk package they are generated dynamically using Object.defineProperty(this, styleName, {value: builder});, where the styleName is a color name you use in your code. There is no way to resolve them when analyzing file statically.

Normally installing Typescript stubs can be used as a workaround. But this doesn't work for validator due to WEB-43528.
chulk typings are included in package distribution, but not resolved because the required fields in package.json are missing. As a workaround, open node_modules/chalk/package.json and add "types": "index.d.ts", to it:

this should help:


问:
How do I implement threaded comments?
I am developing a web application that can support threaded comments. I need the ability to rearrange the comments based on the number of votes received. (Identical to how threaded comments work in reddit)

I would love to hear the inputs from the SO community on how to do it.

How should I design the comments table?
Here is the structure I am using now:

```<code>Comment
    id
    parent_post
    parent_comment
    author
    points
</code>```

What changes should be done to this structure?

How should I get the details from this table to display them in the correct manner?
(Implementation in any language is welcome. I just want to know how to do it in the best possible manner)

What are the stuff I need to take care while implementing this feature so that there is less load on the CPU/Database?

Thanks in advance.

答:
Couple things to also consider...

1) When you say "sort like reddit" based on rank or date, do you mean the top-level or the whole thing?

2) When you delete a node, what happens to the branches?  Do you re-parent them?  In my implementation, I'm thinking that the editors will decide--either hide the node and display it as "comment hidden" along with the visible children, hide the comment and it's children, or nuke the whole tree.  Re-parenting should be easy (just set the chidren's parent to the deleted's parent), but it anything involving the whole tree seems to be tricky to implement in the database.

I've been looking at the ltree module for PostgreSQL.  It should make database operations involving parts of the tree a bit faster.  It basically lets you set up a field in the table that looks like:

```<code>ltreetest=# select path from test where path <@ 'Top.Science';
                path
------------------------------------
 Top.Science
 Top.Science.Astronomy
 Top.Science.Astronomy.Astrophysics
 Top.Science.Astronomy.Cosmology
</code>```

However, it doesn't ensure any kind of referential integrity on its own.  In other words, you can have a records for "Top.Science.Astronomy" without having a record for "Top.Science" or "Top".  But what it does let you do is stuff like:

```<code>-- hide the children of Top.Science
UPDATE test SET hide_me=true WHERE path @> 'Top.Science';
</code>```

or

```<code>-- nuke the cosmology branch
DELETE FROM test WHERE path @> 'Top.Science.Cosmology';
</code>```

If combined with the traditional "comment_id"/"parent_id" approach using stored procedures, I'm thinking you can get the best of both worlds.  You can quickly traverse the comment tree in the database using your "path" and still ensure referential integrity via "comment_id"/"parent_id".  I'm envisioning something like:

```<code>CREATE TABLE comments (
comment_id SERIAL PRIMARY KEY,
parent_comment_id int REFERENCES comments(comment_id) ON UPDATE CASCADE ON DELETE CASCADE,
thread_id int NOT NULL  REFERENCES threads(thread_id) ON UPDATE CASCADE ON DELETE CASCADE,
path ltree NOT NULL,
comment_body text NOT NULL,
hide boolean not null default false
);
</code>```

The path string for a comment look like be

```<code><thread_id>.<parent_id_#1>.<parent_id_#2>.<parent_id_#3>.<my_comment_id>
</code>```

Thus a root comment of thread "102" with a comment_id of "1" would have a path of:

102.1

And a child whose comment_id is "3" would be:

102.1.3

A some children of "3" having id's of "31" and "54" would be:

```<code>102.1.3.31
102.1.3.54
</code>```

To hide the node "3" and its kids, you'd issue this:

```<code>UPDATE comments SET hide=true WHERE path @> '102.1.3';
</code>```

I dunno though--it might add needless overhead.  Plus I don't know how well maintained ltree is.

OwnerDisplayName: Niyaz

Title: How do I implement threaded comments?

Body:
I am developing a web application that can support threaded comments. I need the ability to rearrange the comments based on the number of votes received. (Identical to how threaded comments work in reddit)

I would love to hear the inputs from the SO community on how to do it.

How should I design the comments table?
Here is the structure I am using now:

Comment
    id
    parent_post
    parent_comment
    author
    points

What changes should be done to this structure?

How should I get the details from this table to display them in the correct manner?
(Implementation in any language is welcome. I just want to know how to do it in the best possible manner)

What are the stuff I need to take care while implementing this feature so that there is less load on the CPU/Database?

Thanks in advance.

Body:
Couple things to also consider...

1) When you say "sort like reddit" based on rank or date, do you mean the top-level or the whole thing?

2) When you delete a node, what happens to the branches?  Do you re-parent them?  In my implementation, I'm thinking that the editors will decide--either hide the node and display it as "comment hidden" along with the visible children, hide the comment and it's children, or nuke the whole tree.  Re-parenting should be easy (just set the chidren's parent to the deleted's parent), but it anything involving the whole tree seems to be tricky to implement in the database.

I've been looking at the ltree module for PostgreSQL.  It should make database operations involving parts of the tree a bit faster.  It basically lets you set up a field in the table that looks like:

ltreetest=# select path from test where path <@ 'Top.Science';
                path
------------------------------------
 Top.Science
 Top.Science.Astronomy
 Top.Science.Astronomy.Astrophysics
 Top.Science.Astronomy.Cosmology

However, it doesn't ensure any kind of referential integrity on its own.  In other words, you can have a records for "Top.Science.Astronomy" without having a record for "Top.Science" or "Top".  But what it does let you do is stuff like:

-- hide the children of Top.Science
UPDATE test SET hide_me=true WHERE path @> 'Top.Science';

or

-- nuke the cosmology branch
DELETE FROM test WHERE path @> 'Top.Science.Cosmology';

If combined with the traditional "comment_id"/"parent_id" approach using stored procedures, I'm thinking you can get the best of both worlds.  You can quickly traverse the comment tree in the database using your "path" and still ensure referential integrity via "comment_id"/"parent_id".  I'm envisioning something like:

CREATE TABLE comments (
comment_id SERIAL PRIMARY KEY,
parent_comment_id int REFERENCES comments(comment_id) ON UPDATE CASCADE ON DELETE CASCADE,
thread_id int NOT NULL  REFERENCES threads(thread_id) ON UPDATE CASCADE ON DELETE CASCADE,
path ltree NOT NULL,
comment_body text NOT NULL,
hide boolean not null default false
);

The path string for a comment look like be

<thread_id>.<parent_id_#1>.<parent_id_#2>.<parent_id_#3>.<my_comment_id>

Thus a root comment of thread "102" with a comment_id of "1" would have a path of:

102.1

And a child whose comment_id is "3" would be:

102.1.3

A some children of "3" having id's of "31" and "54" would be:

102.1.3.31
102.1.3.54

To hide the node "3" and its kids, you'd issue this:

UPDATE comments SET hide=true WHERE path @> '102.1.3';

I dunno though--it might add needless overhead.  Plus I don't know how well maintained ltree is.

OwnerDisplayName: Cory R. King


问:
Which win32 API can be used to listen on a port in windows?
I'm reverse engineering a software,and I set a bp on listen,doesn't work.

What other win32 APi are there to listen on a specific port?

And I know the port it's listenning on is 8888.

UPDATE
What I'm doing is for this task:

How can I implement a http proxy server myself? Do I need to need to bother with TCP/IP protocols?

答:
OK, as well as my smart-arse comment (sorry about that, my attempt at humour) I'll do my best to point you to where you can start.

Sounds like what you are trying to do is packet "sniffing", you probably need to use the WinPCap library in order to filter and capture the packets before the listening app (i.e. your app as a proxy, intercepts and passes on / or not to the listening app) you will need to be down at the kernel level as far as I understand it. This library comes with a device driver that will allow you to do that.


问:
How to plot the relationship between any error and the exact solution of a problem involving summation of real number
I am taking Numerical Methods class and we were asked this problem.
The problem is as follows,

Add the real number 10^-N to itself 10^N times. Run the program for N = 1 to 12.
So I've written the following C++ code

```<code>#include <iostream>
#include <cmath>        // std::pow for power of numbers
#include <iomanip>      // std::setprecision
using namespace std;

int main(){
double i, sum, Nx; // Initialize the variables

for(i = 1; i < 3; i++ ){

Nx = pow(10,i); // Here Nx is the 10^n term.

cout << "Nx: " << Nx << endl;  // Printing Nx before starting the loop

sum = 0;  // Initializing the Total sum to zero

while(Nx){
     sum = sum + pow(10,-i); // Adds every 1/10^i term to itself till 10^i becomes 0.
     Nx--;
     cout << "For i = " << i << " the sum is: " << setprecision(10) << sum << endl; // Printing the summation value
  }
 }
return 0;
}
</code>```

The summation is 1.

But the second part of the question is Plot the relationship between any error and the exact solution in a meaningful way.

I get that I need a log scale to plot it. But how do I generate the error?
Any help would be greatly appreciated.

答:
The number dx=10^-N itself has a rounding error in its binary representation of at most one half of an ULP, or about 0.5*mu*dx where mu=2^-52 which is about 2e-16. The sum of these errors is at most 0.5*mu, thus does not influence the result.

Let's look at the summation steps. At first the sum is small, so the rounding error will be small accordingly. With increasing sum the error of the single addition will also increase. Halfway through the summation, one encounters the average of the extremes, so look at the addition of dx=10^-N to 0.5. The addition again has a rounding error of at most one half of an ULP, but now that is relative to the result which is again around 0.5, giving an error of 0.25*mu. Taking this as the average error, the whole computation will give an error of around 10^N*0.25*mu which is about 0.5*10^(N-16). Which means that about the last N digits are "dirty". Note that the rounding errors have a somewhat random distribution in size and direction, so that they can also cancel, giving a smaller error in the end than this estimate.

In the cases you observed, the error will be in the last 3 of 16 digits. By rounding to 10 digits, you eliminate that error in the printed numbers. There is two things you can do:

increase the number of digits in the output,

print also the difference to 1.

Body:
I am taking Numerical Methods class and we were asked this problem.
The problem is as follows,

Add the real number 10^-N to itself 10^N times. Run the program for N = 1 to 12.
So I've written the following C++ code

#include <iostream>
#include <cmath>        // std::pow for power of numbers
#include <iomanip>      // std::setprecision
using namespace std;

int main(){
double i, sum, Nx; // Initialize the variables

for(i = 1; i < 3; i++ ){

Nx = pow(10,i); // Here Nx is the 10^n term.

cout << "Nx: " << Nx << endl;  // Printing Nx before starting the loop

sum = 0;  // Initializing the Total sum to zero

while(Nx){
     sum = sum + pow(10,-i); // Adds every 1/10^i term to itself till 10^i becomes 0.
     Nx--;
     cout << "For i = " << i << " the sum is: " << setprecision(10) << sum << endl; // Printing the summation value
  }
 }
return 0;
}

The summation is 1.

But the second part of the question is Plot the relationship between any error and the exact solution in a meaningful way.

I get that I need a log scale to plot it. But how do I generate the error?
Any help would be greatly appreciated.


问:
Routing exception after upgrading to TYPO3 v9.5.14
After upgrading to TYPO3 v9.5.14 our detail pages for news crash with the exception

Symfony\Component\Routing\Exception\InvalidParameterException

Parameter "p88bd715a41119d0e8087a5d19cb049" for route "tx_news_pi1_1" must match "[^/]++" ("" given) to generate a corresponding URL.

What's going on?

The site used this configuration:

```<code>  NewsTagPlugin:
    type: Extbase
    limitToPages: [14]
    extension: News
    plugin: Pi1
    routes:
      - routePath: '/{tag-name}'
        _controller: 'News::list'
        _arguments:
          tag-name: 'overwriteDemand/tags'
      - routePath: '/{tag-name}/page/{page}'
        _controller: 'News::list'
        _arguments:
          tag-name: 'overwriteDemand/tags'
          page: '@widget_0/currentPage'
          requirements:
            page: '\d+'
    defaultController: 'News::list'
    defaults:
      page: ''
    aspects:
      page:
        type: IntegerMapper
        start: 1
        end: 5000
      tag-name:
        type: PersistedAliasMapper
        tableName: tx_news_domain_model_tag
        routeFieldName: slug
</code>```

答:
1) Superfluous mapping

```<code>  NewsTagPlugin:
    ...
    routes:
      ...
      - routePath: '/{tag-name}/page/{page}'
        _controller: 'News::list'
        _arguments:
          tag-name: 'overwriteDemand/tags'
          page: '@widget_0/currentPage'
          requirements:
            page: '\d+'
</code>```

_arguments defines a mapping for route parameter and internal variables (e.g. as query parameter`

requirements is wrong here, since it shall not be used as argument mapping

parameter requirements need to be defined on the root level of NewsTagPlugin

2) Invalid empty default value

```<code>  NewsTagPlugin:
    ...
    routes:
      ...
      - routePath: '/{tag-name}/page/{page}'
      ...
    defaults:
      page: ''
    aspects:
      ...
</code>```

defaults was not applied prior to TYPO3 v9.5.14 and addressed in https://review.typo3.org/c/Packages/TYPO3.CMS/+/60361

the empty default value for parameter page does not make much sense and would lead to an URL like /some-tag/page/ which is causing the error message shown in the answer

the default value should be page: 1

in case the parameter should be omitted in the URL (e.g. having /some-tag/page/) this needs to be defined explicitly using {!page} in the route path

References

https://symfony.com/doc/4.3/routing.html#optional-parameters

https://docs.typo3.org/c/typo3/cms-core/master/en-us/Changelog/9.5.x/Important-86895-RouteAspectsTakePrecedenceOverRequirements.html

Adjusted enhancer configuration

```<code>  NewsTagPlugin:
    type: Extbase
    limitToPages: [14]
    extension: News
    plugin: Pi1
    routes:
      - routePath: '/{tag-name}'
        _controller: 'News::list'
        _arguments:
          tag-name: 'overwriteDemand/tags'
      - routePath: '/{tag-name}/page/{!page}'
        _controller: 'News::list'
        _arguments:
          tag-name: 'overwriteDemand/tags'
          page: '@widget_0/currentPage'
    defaultController: 'News::list'
    defaults:
      page: 1
    aspects:
      page:
        type: IntegerMapper
        start: 1
        end: 5000
      tag-name:
        type: PersistedAliasMapper
        tableName: tx_news_domain_model_tag
        routeFieldName: slug
</code>```

(untested) since IntegerMapper seems to be a custom aspect implementation - not being available to the public

Body:
After upgrading to TYPO3 v9.5.14 our detail pages for news crash with the exception

Symfony\Component\Routing\Exception\InvalidParameterException

Parameter "p88bd715a41119d0e8087a5d19cb049" for route "tx_news_pi1_1" must match "[^/]++" ("" given) to generate a corresponding URL.

What's going on?

The site used this configuration:

  NewsTagPlugin:
    type: Extbase
    limitToPages: [14]
    extension: News
    plugin: Pi1
    routes:
      - routePath: '/{tag-name}'
        _controller: 'News::list'
        _arguments:
          tag-name: 'overwriteDemand/tags'
      - routePath: '/{tag-name}/page/{page}'
        _controller: 'News::list'
        _arguments:
          tag-name: 'overwriteDemand/tags'
          page: '@widget_0/currentPage'
          requirements:
            page: '\d+'
    defaultController: 'News::list'
    defaults:
      page: ''
    aspects:
      page:
        type: IntegerMapper
        start: 1
        end: 5000
      tag-name:
        type: PersistedAliasMapper
        tableName: tx_news_domain_model_tag
        routeFieldName: slug

Body:
1) Superfluous mapping

  NewsTagPlugin:
    ...
    routes:
      ...
      - routePath: '/{tag-name}/page/{page}'
        _controller: 'News::list'
        _arguments:
          tag-name: 'overwriteDemand/tags'
          page: '@widget_0/currentPage'
          requirements:
            page: '\d+'

_arguments defines a mapping for route parameter and internal variables (e.g. as query parameter`

requirements is wrong here, since it shall not be used as argument mapping

parameter requirements need to be defined on the root level of NewsTagPlugin

2) Invalid empty default value

  NewsTagPlugin:
    ...
    routes:
      ...
      - routePath: '/{tag-name}/page/{page}'
      ...
    defaults:
      page: ''
    aspects:
      ...

defaults was not applied prior to TYPO3 v9.5.14 and addressed in https://review.typo3.org/c/Packages/TYPO3.CMS/+/60361

the empty default value for parameter page does not make much sense and would lead to an URL like /some-tag/page/ which is causing the error message shown in the answer

the default value should be page: 1

in case the parameter should be omitted in the URL (e.g. having /some-tag/page/) this needs to be defined explicitly using {!page} in the route path

References

https://symfony.com/doc/4.3/routing.html#optional-parameters

https://docs.typo3.org/c/typo3/cms-core/master/en-us/Changelog/9.5.x/Important-86895-RouteAspectsTakePrecedenceOverRequirements.html

Adjusted enhancer configuration

  NewsTagPlugin:
    type: Extbase
    limitToPages: [14]
    extension: News
    plugin: Pi1
    routes:
      - routePath: '/{tag-name}'
        _controller: 'News::list'
        _arguments:
          tag-name: 'overwriteDemand/tags'
      - routePath: '/{tag-name}/page/{!page}'
        _controller: 'News::list'
        _arguments:
          tag-name: 'overwriteDemand/tags'
          page: '@widget_0/currentPage'
    defaultController: 'News::list'
    defaults:
      page: 1
    aspects:
      page:
        type: IntegerMapper
        start: 1
        end: 5000
      tag-name:
        type: PersistedAliasMapper
        tableName: tx_news_domain_model_tag
        routeFieldName: slug

(untested) since IntegerMapper seems to be a custom aspect implementation - not being available to the public


问:
looping through a object which contains an object php
I have a PHP Object which contains other objects

i.e

```<code>$obj->sec_obj->some_var;
</code>```

I want to use a foreach loop to loop through the object and all objects objects. I think the max level is 3, so

```<code>$obj->sec_obj->third_obj->fourth_obj
</code>```

Any ideas?

答:
Added support for arrays to slikts solution

```<code>function loop($obj, $path = [])
{
    $parent = $path;

    if (is_object($obj)) {
        foreach ($obj as $key => $value) {
            loop($value, array_merge($parent, [$key]));
        }
    } elseif (is_array($obj)) {
        foreach ($obj as $index => $value) {
            loop($value, array_merge($parent, [$index]));
        }
    } else {
        echo join('.', $path) . ": $obj" . PHP_EOL;
    }
}
</code>```

Example:

```<code>$o = (object) [
    "name" => "sample document",
    "items" => [
        (object) [
            "type" => "text",
            "placeholder" => "type it"
        ]
    ]
];

loop($o);
</code>```

Outputs:

```<code>name: sample document
items.0.type: text
items.0.placeholder: type it
</code>```

Body:
I have a PHP Object which contains other objects

i.e

$obj->sec_obj->some_var;

I want to use a foreach loop to loop through the object and all objects objects. I think the max level is 3, so

$obj->sec_obj->third_obj->fourth_obj

Any ideas?

Body:
Added support for arrays to slikts solution

function loop($obj, $path = [])
{
    $parent = $path;

    if (is_object($obj)) {
        foreach ($obj as $key => $value) {
            loop($value, array_merge($parent, [$key]));
        }
    } elseif (is_array($obj)) {
        foreach ($obj as $index => $value) {
            loop($value, array_merge($parent, [$index]));
        }
    } else {
        echo join('.', $path) . ": $obj" . PHP_EOL;
    }
}

Example:

$o = (object) [
    "name" => "sample document",
    "items" => [
        (object) [
            "type" => "text",
            "placeholder" => "type it"
        ]
    ]
];

loop($o);

Outputs:

name: sample document
items.0.type: text
items.0.placeholder: type it


问:
Error when "import requests" - "No module named requests"
N00b Altert.

So I tried to call "import requests" through Python and got the error:

```<code>>>> import requests
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named requests
</code>```

I do not think I have pip installed correctly or at all?

easy_install requests returns:
The following error occurred while trying to add or remove files in the
installation directory:

```<code>[Errno 13] Permission denied: '/Library/Python/2.7/site-packages/test-easy-install-6488.pth'
</code>```

The installation directory you specified (via --install-dir, --prefix, or
the distutils default setting) was:

```<code>/Library/Python/2.7/site-packages/
</code>```

Any help on this would be greatly appreciated... I have seen the other posts with users mentioning the same but it doesn't seem to help.

答:
Type

```<code>import requests
</code>```

Guide your cursor to the command (Donot press space) You will see a red bulb.There will be an option to install requests. Click it and done.
This worked for me on Pycharm. Installing it through pip didnot help me.

Body:
N00b Altert.

So I tried to call "import requests" through Python and got the error:

>>> import requests
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named requests

I do not think I have pip installed correctly or at all?

easy_install requests returns:
The following error occurred while trying to add or remove files in the
installation directory:

[Errno 13] Permission denied: '/Library/Python/2.7/site-packages/test-easy-install-6488.pth'

The installation directory you specified (via --install-dir, --prefix, or
the distutils default setting) was:

/Library/Python/2.7/site-packages/

Any help on this would be greatly appreciated... I have seen the other posts with users mentioning the same but it doesn't seem to help.

Body:
Type

import requests

Guide your cursor to the command (Donot press space) You will see a red bulb.There will be an option to install requests. Click it and done.
This worked for me on Pycharm. Installing it through pip didnot help me.


问:
Apache Knox for SAML2 authentication keeps using NameIDFormat entity instead of what is configured
I am trying to enable SSO capabilities for Apache Zeppelin, using Apache Knox, which is configured to redirect auth requests to a Siteminder IdP.

The issue I am having is related to the NameID format configuration, and the signing configuration.

No matter what I configure in the sp/idp metadata, the NameID format used is

urn:oasis:names:tc:SAML:2.0:nameid-format:entity

And the requests are always being sent with Signed requests set to true.

My SP configuration is as follows:

```<code><EntityDescriptor xmlns="urn:oasis:names:tc:SAML:2.0:metadata" entityID="https://knox.test.com/gateway/knoxsso/api/v1/websso?pac4jCallback=true%26client_name=SAML2Client">
<SPSSODescriptor AuthnRequestsSigned="false" WantAssertionsSigned="false" protocolSupportEnumeration="urn:oasis:names:tc:SAML:2.0:protocol urn:oasis:names:tc:SAML:1.1:protocol http://schemas.xmlsoap.org/ws/2003/07/secext">
<NameIDFormat>
  urn:oasis:names:tc:SAML:1.1:nameid-format:unspecified
</NameIDFormat>
    <SingleLogoutService Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST" Location="https://knox.test.com/gateway/knoxsso/api/v1/websso?pac4jCallback=true%26client_name=SAML2Client"/>
   <AssertionConsumerService
            Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST" index="1" isDefault="true"  Location="https://knox.test.com/gateway/knoxsso/api/v1/websso?pac4jCallback=true%26client_name=SAML2Client"/>
</SPSSODescriptor>
</code>```

I activated a SAML tracer and attempted the logon user journey. The AuthNRequest being sent to the Siteminder IdP based on this configuration looks like this:

```<code><saml2p:AuthnRequest xmlns:saml2p="urn:oasis:names:tc:SAML:2.0:protocol"
                 AssertionConsumerServiceURL="https://knox.test.com/gateway/knoxsso/api/v1/websso?pac4jCallback=true%26client_name=SAML2Client"
                 Destination="https://test-siteminder.com/test/saml2sso"
                 ForceAuthn="false"
                 ID="_yp52mio0oj4ho2niijmnnaikgbkid9tnc5h5ear"
                 IsPassive="false"
                 IssueInstant="2020-02-17T10:19:24.279Z"
                 ProtocolBinding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST"
                 ProviderName="pac4j-saml"
                 Version="2.0"
                 >
<saml2:Issuer xmlns:saml2="urn:oasis:names:tc:SAML:2.0:assertion"
              Format="urn:oasis:names:tc:SAML:2.0:nameid-format:entity"
              NameQualifier="https://knox.test.com/gateway/knoxsso/api/v1/websso?pac4jCallback=true%26client_name=SAML2Client"
              >https://knox.test.com/gateway/knoxsso/api/v1/websso?pac4jCallback=true%26client_name=SAML2Client</saml2:Issuer>
</code>```

I can see a signature value in the Parameters section of the request, which is why I'm assuming that the AuthNRequest is signed (though my understanding of this is minimal, so that could be a wrong assumption!).

Can anyone help explain why the NameIDFormat is coming through as entity, as opposed to unspecified?
Does Apache knox support SAML1 protocols?

Thanks in advance!

答: You mentioned NameID format to be urn:oasis:names:tc:SAML:2.0:nameid-format:entity in your post but in the code you pasted it is urn:oasis:names:tc:SAML:1.1:nameid-format:unspecified, just a copy paste error ? Looks like protocolSupportEnumeration is also referencing SAML 1 protocol. Knox uses Pac4J under the hood which does not support SAML 1, this might be the reason.

Body:
I am trying to enable SSO capabilities for Apache Zeppelin, using Apache Knox, which is configured to redirect auth requests to a Siteminder IdP.

The issue I am having is related to the NameID format configuration, and the signing configuration.

No matter what I configure in the sp/idp metadata, the NameID format used is

urn:oasis:names:tc:SAML:2.0:nameid-format:entity

And the requests are always being sent with Signed requests set to true.

My SP configuration is as follows:

<EntityDescriptor xmlns="urn:oasis:names:tc:SAML:2.0:metadata" entityID="https://knox.test.com/gateway/knoxsso/api/v1/websso?pac4jCallback=true%26client_name=SAML2Client">
<SPSSODescriptor AuthnRequestsSigned="false" WantAssertionsSigned="false" protocolSupportEnumeration="urn:oasis:names:tc:SAML:2.0:protocol urn:oasis:names:tc:SAML:1.1:protocol http://schemas.xmlsoap.org/ws/2003/07/secext">
<NameIDFormat>
  urn:oasis:names:tc:SAML:1.1:nameid-format:unspecified
</NameIDFormat>
    <SingleLogoutService Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST" Location="https://knox.test.com/gateway/knoxsso/api/v1/websso?pac4jCallback=true%26client_name=SAML2Client"/>
   <AssertionConsumerService
            Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST" index="1" isDefault="true"  Location="https://knox.test.com/gateway/knoxsso/api/v1/websso?pac4jCallback=true%26client_name=SAML2Client"/>
</SPSSODescriptor>

I activated a SAML tracer and attempted the logon user journey. The AuthNRequest being sent to the Siteminder IdP based on this configuration looks like this:

<saml2p:AuthnRequest xmlns:saml2p="urn:oasis:names:tc:SAML:2.0:protocol"
                 AssertionConsumerServiceURL="https://knox.test.com/gateway/knoxsso/api/v1/websso?pac4jCallback=true%26client_name=SAML2Client"
                 Destination="https://test-siteminder.com/test/saml2sso"
                 ForceAuthn="false"
                 ID="_yp52mio0oj4ho2niijmnnaikgbkid9tnc5h5ear"
                 IsPassive="false"
                 IssueInstant="2020-02-17T10:19:24.279Z"
                 ProtocolBinding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST"
                 ProviderName="pac4j-saml"
                 Version="2.0"
                 >
<saml2:Issuer xmlns:saml2="urn:oasis:names:tc:SAML:2.0:assertion"
              Format="urn:oasis:names:tc:SAML:2.0:nameid-format:entity"
              NameQualifier="https://knox.test.com/gateway/knoxsso/api/v1/websso?pac4jCallback=true%26client_name=SAML2Client"
              >https://knox.test.com/gateway/knoxsso/api/v1/websso?pac4jCallback=true%26client_name=SAML2Client</saml2:Issuer>

I can see a signature value in the Parameters section of the request, which is why I'm assuming that the AuthNRequest is signed (though my understanding of this is minimal, so that could be a wrong assumption!).

Can anyone help explain why the NameIDFormat is coming through as entity, as opposed to unspecified?
Does Apache knox support SAML1 protocols?

Thanks in advance!


问:
How to filter a query on the results of another one in django?
Lets say that i have 2 models :

```<code>Class OrderEvent(models.Model):
    isPaid = models.Booleanfield()

Class Participant(models.Model):
    orderEvent = models.ForeignKey(OrderEvent)
    participantFirstName = models.CharField()
    participantLastName = models.CharField()
    #etc...
</code>```

And i want to get all the participants where Orderevent.isPaid = True.

I think that i struggle to do something very simple...

答:
It is very simple;

```<code>Participant.objects.filter(orderEvent__isPaid=True)
</code>```

As a suggestion you can follow,

naming conventions from here.

Django making queries from here.

Body:
Lets say that i have 2 models :

Class OrderEvent(models.Model):
    isPaid = models.Booleanfield()

Class Participant(models.Model):
    orderEvent = models.ForeignKey(OrderEvent)
    participantFirstName = models.CharField()
    participantLastName = models.CharField()
    #etc...

And i want to get all the participants where Orderevent.isPaid = True.

I think that i struggle to do something very simple...

Body:
It is very simple;

Participant.objects.filter(orderEvent__isPaid=True)

As a suggestion you can follow,

naming conventions from here.

Django making queries from here.


问:
how to hit the API by using CURL?
i try to hit the API by using curl
in this case i try to hit : https://jsonplaceholder.typicode.com/posts

```<code>  <?php

class process extends CI_Controller {

    $headers = 'Content-Type:application/json';

    $process = curl_init('https://jsonplaceholder.typicode.com/users'); //your API url
    curl_setopt($process, CURLOPT_HTTPHEADER, $headers);
    curl_setopt($process, CURLOPT_HEADER, 1);
    curl_setopt($process, CURLOPT_TIMEOUT, 20);
    curl_setopt($process, CURLOPT_POST, 1);

    curl_setopt($process, CURLOPT_RETURNTRANSFER, TRUE);
    $return = curl_exec($process);
    curl_close($process);

    //finally print your API response
    print_r($return);

}
</code>```

but it shows some errors like this

An uncaught Exception was encountered Type: ParseError

Message: syntax error, unexpected '$headers' (T_VARIABLE), expecting
  function (T_FUNCTION) or const (T_CONST)

Filename: E:\XAMPP\htdocs\Crudview\application\controllers\process.php

Line Number: 6

Backtrace:

File: E:\XAMPP\htdocs\Crudview\index.php Line: 315 Function:
  require_once

答:
Please use like that:- </p>

```<code><?php

class process extends CI_Controller {

    function curlAPI(){
        $headers = 'Content-Type:application/json';
        $process = curl_init('https://jsonplaceholder.typicode.com/users'); //your API url
        curl_setopt($process, CURLOPT_HTTPHEADER, $headers);
        curl_setopt($process, CURLOPT_HEADER, 1);
        curl_setopt($process, CURLOPT_TIMEOUT, 20);
        curl_setopt($process, CURLOPT_POST, 1);

        curl_setopt($process, CURLOPT_RETURNTRANSFER, TRUE);
        $return = curl_exec($process);
        curl_close($process);
        print_r($return);
    }
}

?>
</code>```

Title: how to hit the API by using CURL?

Body:
i try to hit the API by using curl
in this case i try to hit : https://jsonplaceholder.typicode.com/posts

  <?php

class process extends CI_Controller {

    $headers = 'Content-Type:application/json';

    $process = curl_init('https://jsonplaceholder.typicode.com/users'); //your API url
    curl_setopt($process, CURLOPT_HTTPHEADER, $headers);
    curl_setopt($process, CURLOPT_HEADER, 1);
    curl_setopt($process, CURLOPT_TIMEOUT, 20);
    curl_setopt($process, CURLOPT_POST, 1);

    curl_setopt($process, CURLOPT_RETURNTRANSFER, TRUE);
    $return = curl_exec($process);
    curl_close($process);

    //finally print your API response
    print_r($return);

}

but it shows some errors like this

An uncaught Exception was encountered Type: ParseError

Message: syntax error, unexpected '$headers' (T_VARIABLE), expecting
  function (T_FUNCTION) or const (T_CONST)

Filename: E:\XAMPP\htdocs\Crudview\application\controllers\process.php

Line Number: 6

Backtrace:

File: E:\XAMPP\htdocs\Crudview\index.php Line: 315 Function:
  require_once

Body:
Please use like that:-

<?php

class process extends CI_Controller {

    function curlAPI(){
        $headers = 'Content-Type:application/json';
        $process = curl_init('https://jsonplaceholder.typicode.com/users'); //your API url
        curl_setopt($process, CURLOPT_HTTPHEADER, $headers);
        curl_setopt($process, CURLOPT_HEADER, 1);
        curl_setopt($process, CURLOPT_TIMEOUT, 20);
        curl_setopt($process, CURLOPT_POST, 1);

        curl_setopt($process, CURLOPT_RETURNTRANSFER, TRUE);
        $return = curl_exec($process);
        curl_close($process);
        print_r($return);
    }
}

?>


问:
SQL Server archiving table partitions
I am using SQL Server 2008 R2 which has a limitation when it comes to table partitions. Max limit is 1000 partition. SQL Server 2008 R2 does not support expanding the partition limit. I need to archive partitions to free up the production table. I will move partitions to a linked server where I will keep all archived data to save storage space from the D.W. I need some advice on the method to be used.

My thought is to create an un partitioned table on the linked server used for archiving, then move data and then delete partitions from the source table. Would this work using switch instead of copy - delete?

Would it be better to use partitions in the archive table too?

答:
You can switch a partition from the partition table to a standalone table, which happens in seconds. So, your solution can be:

```<code> ALTER TABLE YourTable SWITCH PARTITION x TO ArchivingTable;

 -- Move data from the local ArchivingTable to your linked server (INSERT
 SELECT or bcp..)

 DROP TABLE ArchivingTable;
</code>```

You an also merge existing partitions with aged data so free up partitions for new data.

Title: SQL Server archiving table partitions

Body:
You can switch a partition from the partition table to a standalone table, which happens in seconds. So, your solution can be:

 ALTER TABLE YourTable SWITCH PARTITION x TO ArchivingTable;

 -- Move data from the local ArchivingTable to your linked server (INSERT
 SELECT or bcp..)

 DROP TABLE ArchivingTable;

You an also merge existing partitions with aged data so free up partitions for new data.


问:
Looping within a coroutine
I have written the following snippet in order to play five clicking sounds resembling a rotating wheel.

How can I make a loop out of it, so that the clicks keep on playing until I call the stop function at the bottom?

I toggled on the loop feature on each sound file, but they still each play only once...

public void PlayClickSounds()
{
    StartCoroutine( ClickSounds(0.5f) );
}

private IEnumerator ClickSounds(float clipsGapDuration)
{
    Play(SpinningSoundClick1);
    yield return new WaitForSeconds(clipsGapDuration);
    Play(SpinningSoundClick2);
    yield return new WaitForSeconds(clipsGapDuration);
    Play(SpinningSoundClick3);
    yield return new WaitForSeconds(clipsGapDuration);
    Play(SpinningSoundClick4);
    yield return new WaitForSeconds(clipsGapDuration);
    Play(SpinningSoundClick5);
    yield return new WaitForSeconds(clipsGapDuration);
}

public void StopClickSounds()
{
    StopCoroutine( SpinningClickSounds(0.5f) );
}
```

答:
there are various options for how you can achieve that. You can use while loop, you can call a coroutine from a coroutine.

This is an example while.

```<code>private bool _isClipPlaying;
private IEnumerator ClickSounds(float clipsGapDuration)
{
    while(_isClipPlaying){
        Play(boostWheelSpinningSoundClick1);
        yield return new WaitForSeconds(clipsGapDuration);
        Play(boostWheelSpinningSoundClick2);
        yield return new WaitForSeconds(clipsGapDuration);
        Play(boostWheelSpinningSoundClick3);
        yield return new WaitForSeconds(clipsGapDuration);
        Play(boostWheelSpinningSoundClick4);
        yield return new WaitForSeconds(clipsGapDuration);
        Play(boostWheelSpinningSoundClick5);
        yield return new WaitForSeconds(clipsGapDuration);
    }
}

public void PlayClickSounds()
{
    _isClipPlaying = true;
    StartCoroutine( ClickSounds(0.5f) );
}

public void StopClickSounds()
{
    _isClipPlaying = false;
}
</code>```

Another option is to call coroutine after that.

```<code>private IEnumerator ClickSounds(float clipsGapDuration)
{
    Play(boostWheelSpinningSoundClick1);
    yield return new WaitForSeconds(clipsGapDuration);
    Play(boostWheelSpinningSoundClick2);
    yield return new WaitForSeconds(clipsGapDuration);
    Play(boostWheelSpinningSoundClick3);
    yield return new WaitForSeconds(clipsGapDuration);
    Play(boostWheelSpinningSoundClick4);
    yield return new WaitForSeconds(clipsGapDuration);
    Play(boostWheelSpinningSoundClick5);
    yield return new WaitForSeconds(clipsGapDuration);
    StartCoroutine(ClickSounds(clipsGapDuration));
}
</code>```

I would suggest using list for click sounds as it will be more readable.

Title: Looping within a coroutine

Body:
there are various options for how you can achieve that. You can use while loop, you can call a coroutine from a coroutine.

This is an example while.

private bool _isClipPlaying;
private IEnumerator ClickSounds(float clipsGapDuration)
{
    while(_isClipPlaying){
        Play(boostWheelSpinningSoundClick1);
        yield return new WaitForSeconds(clipsGapDuration);
        Play(boostWheelSpinningSoundClick2);
        yield return new WaitForSeconds(clipsGapDuration);
        Play(boostWheelSpinningSoundClick3);
        yield return new WaitForSeconds(clipsGapDuration);
        Play(boostWheelSpinningSoundClick4);
        yield return new WaitForSeconds(clipsGapDuration);
        Play(boostWheelSpinningSoundClick5);
        yield return new WaitForSeconds(clipsGapDuration);
    }
}

public void PlayClickSounds()
{
    _isClipPlaying = true;
    StartCoroutine( ClickSounds(0.5f) );
}

public void StopClickSounds()
{
    _isClipPlaying = false;
}

Another option is to call coroutine after that.

private IEnumerator ClickSounds(float clipsGapDuration)
{
    Play(boostWheelSpinningSoundClick1);
    yield return new WaitForSeconds(clipsGapDuration);
    Play(boostWheelSpinningSoundClick2);
    yield return new WaitForSeconds(clipsGapDuration);
    Play(boostWheelSpinningSoundClick3);
    yield return new WaitForSeconds(clipsGapDuration);
    Play(boostWheelSpinningSoundClick4);
    yield return new WaitForSeconds(clipsGapDuration);
    Play(boostWheelSpinningSoundClick5);
    yield return new WaitForSeconds(clipsGapDuration);
    StartCoroutine(ClickSounds(clipsGapDuration));
}

I would suggest using list for click sounds as it will be more readable.