问:
How to create CMYK Bitmap using C#/.Net
System.Drawing only accepts RGB values. So, I'm converting CMYK values to RGB using the below method.

```<code>red = ((100 - cyan) * (100 - black)) / 100;
green = ((100 - magenta) * (100 - black)) / 100;
blue = ((100 - yellow) * (100 - black)) / 100;

System.Drawing.Color convertedcolor = System.Drawing.Color.FromArgb(255,
                                  (byte)(((float)red) / 100f * 255f),
                                  (byte)(((float)green) / 100f * 255f),
                                  (byte)(((float)blue) / 100f * 255f));
</code>```

Unfortunately, the above formula only offers approximate conversion.
For example, CMYK value of (98,98,98,98) or (100,100,100,100) will lead to same RGB value of (0,0,0). Is there any more accurate way of converting these values?

I tried using an ICC profile for the conversion process as below.

```<code>float[] colorValues = new float[4];
colorValues[0] = cyan / 255f;
colorValues[1] = magenta / 255f;
colorValues[2] = yellow / 255f;
colorValues[3] = black / 255f;

System.Windows.Media.Color color2 = System.Windows.Media.Color.FromValues(colorValues,
   new Uri(@"\AdobeICCProfilesCS4Win_end-user\Adobe ICC Profiles (end-user)\CMYK\USWebCoatedSWOP.icc"));
System.Drawing.Color convertedcolor2 = System.Drawing.Color.FromArgb(color2.R, color2.G, color2.B);
</code>```

With the above code, RGB values seems to be way off from the formula which I mentioned earlier.

Note: The created bitmap will be sent to printer. So, the input CMYK color should be maintained as specified to have the desired output.

Title: How to create CMYK Bitmap using C#/.Net

Body:
System.Drawing only accepts RGB values. So, I'm converting CMYK values to RGB using the below method.

red = ((100 - cyan) * (100 - black)) / 100;
green = ((100 - magenta) * (100 - black)) / 100;
blue = ((100 - yellow) * (100 - black)) / 100;

System.Drawing.Color convertedcolor = System.Drawing.Color.FromArgb(255,
                                  (byte)(((float)red) / 100f * 255f),
                                  (byte)(((float)green) / 100f * 255f),
                                  (byte)(((float)blue) / 100f * 255f));

Unfortunately, the above formula only offers approximate conversion.
For example, CMYK value of (98,98,98,98) or (100,100,100,100) will lead to same RGB value of (0,0,0). Is there any more accurate way of converting these values?

I tried using an ICC profile for the conversion process as below.

float[] colorValues = new float[4];
colorValues[0] = cyan / 255f;
colorValues[1] = magenta / 255f;
colorValues[2] = yellow / 255f;
colorValues[3] = black / 255f;

System.Windows.Media.Color color2 = System.Windows.Media.Color.FromValues(colorValues,
   new Uri(@"\AdobeICCProfilesCS4Win_end-user\Adobe ICC Profiles (end-user)\CMYK\USWebCoatedSWOP.icc"));
System.Drawing.Color convertedcolor2 = System.Drawing.Color.FromArgb(color2.R, color2.G, color2.B);

With the above code, RGB values seems to be way off from the formula which I mentioned earlier.

Note: The created bitmap will be sent to printer. So, the input CMYK color should be maintained as specified to have the desired output.


来源: stackexchange

问:
How to stay logged in as a user in webview on a site
How can I mage a user stay logged in even if he exits the app and whenever he opens the app again and opens that site via webview he should stay logged in !</p>

ContentLicense: CC BY-SA 4.0


问:
Merging Datasets (Right Join) in VBA
I'm in the process of creating a VBA script where I import two files into separate worksheets. Sheet 1's data includes masked account numbers, actual account numbers, account type and sub-account type. Sheet 2 includes the account number, account type, sub-account type and other columns that are important. What I need to do is merge sheet 1 with sheet 2 so that Sheet 2 will also include the masked account numbers (after the merge I will delete the account numbers and just keep the masked account numbers). Below is an example of what my datasets look like:

I've been doing this process in R with ease, I can simply just merge the datasets with a right_join function but now I need to replicate this process using VBA. I'm trying to search how to merge worksheets (datasets) but what I've found so far are examples of merging substrings which is not what I want. I believe I can do this with a nested vlookup but I was hoping to find a better way to do this in VBA. Any advice and or literature that would help me better understand how to create a script to merge the data would be appreciated. Thanks!

Title: Merging Datasets (Right Join) in VBA


问:
How to set the maxfun limit of the lbfgs solver on scikit-learn LogisticRegression model?
My scikit-learn LogisticRegression model, which uses the lbfgs solver,  is stopping early as shown in the logs bellow. The data is standardized.

```<code>(...)

At iterate13150    f=  4.05397D+03    |proj g|=  2.41194D+04

At iterate13200    f=  4.05213D+03    |proj g|=  1.36863D+04
.venv/lib/python3.8/site-packages/sklearn/linear_model/_logistic.py:444: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of f AND g EVALUATIONS EXCEEDS LIMIT.

Increase the number of iterations (max_iter) or scale the data as shown in:
    https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
  n_iter_i = _check_optimize_result(
[Parallel(n_jobs=-1)]: Done   1 out of   1 | elapsed:    5.5s finished

           * * *

Tit   = total number of iterations
Tnf   = total number of function evaluations
Tnint = total number of segments explored during Cauchy searches
Skip  = number of BFGS updates skipped
Nact  = number of active bounds at final generalized Cauchy point
Projg = norm of the final projected gradient
F     = final function value

           * * *

   N    Tit     Tnf  Tnint  Skip  Nact     Projg        F
   62  13240  15001      1     0     0   4.800D+04   4.051D+03
  F =   4051.0211050375365
</code>```

sklearn uses the scipy implementation of the lbfgs solver. The function scipy/optimize/_lbfgsb_py.py:_minimize_lbfgsb has the following early stop conditions

```<code>            if n_iterations >= maxiter:
                task[:] = 'STOP: TOTAL NO. of ITERATIONS REACHED LIMIT'
            elif sf.nfev > maxfun:
                task[:] = ('STOP: TOTAL NO. of f AND g EVALUATIONS '
                           'EXCEEDS LIMIT')
</code>```

I am indeed hitting the sf.nfev > maxfun limit. Unfortunatly, sklearn fixes the value of maxfun to 15_000 when it instanciates the scipy solver (`sklearn/linear_model/_logistic.py:442).

When I hotfix the sklearn package to set maxfun to 100_000, the solver converges. But this is not a real solution (since I do not want to carry arround a custom sklearn dist with one different constant).

Any ideas on how to set the maxfun parameter in another way?

Body:
My scikit-learn LogisticRegression model, which uses the lbfgs solver,  is stopping early as shown in the logs bellow. The data is standardized.

(...)

At iterate13150    f=  4.05397D+03    |proj g|=  2.41194D+04

At iterate13200    f=  4.05213D+03    |proj g|=  1.36863D+04
.venv/lib/python3.8/site-packages/sklearn/linear_model/_logistic.py:444: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of f AND g EVALUATIONS EXCEEDS LIMIT.

Increase the number of iterations (max_iter) or scale the data as shown in:
    https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
  n_iter_i = _check_optimize_result(
[Parallel(n_jobs=-1)]: Done   1 out of   1 | elapsed:    5.5s finished

           * * *

Tit   = total number of iterations
Tnf   = total number of function evaluations
Tnint = total number of segments explored during Cauchy searches
Skip  = number of BFGS updates skipped
Nact  = number of active bounds at final generalized Cauchy point
Projg = norm of the final projected gradient
F     = final function value

           * * *

   N    Tit     Tnf  Tnint  Skip  Nact     Projg        F
   62  13240  15001      1     0     0   4.800D+04   4.051D+03
  F =   4051.0211050375365

sklearn uses the scipy implementation of the lbfgs solver. The function scipy/optimize/_lbfgsb_py.py:_minimize_lbfgsb has the following early stop conditions

            if n_iterations >= maxiter:
                task[:] = 'STOP: TOTAL NO. of ITERATIONS REACHED LIMIT'
            elif sf.nfev > maxfun:
                task[:] = ('STOP: TOTAL NO. of f AND g EVALUATIONS '
                           'EXCEEDS LIMIT')

I am indeed hitting the sf.nfev > maxfun limit. Unfortunatly, sklearn fixes the value of maxfun to 15_000 when it instanciates the scipy solver (`sklearn/linear_model/_logistic.py:442).

When I hotfix the sklearn package to set maxfun to 100_000, the solver converges. But this is not a real solution (since I do not want to carry arround a custom sklearn dist with one different constant).

Any ideas on how to set the maxfun parameter in another way?


问:
Python - error importing Nasdaq stock list
Im preparing a python code , to screen stocks from the SP500 , DOW and Nasdaq.
SP500 and DOW importing data stocks is working properly , but when I try to import Nasdaq always get similar error, related to timestamp.

See below:
My code:

```<code>import talib
from yahoo_fin.stock_info import get_data
import yahoo_fin.stock_info as si
from datetime import datetime
list = si.tickers_nasdaq()

# Get current date and time
now = datetime.now().strftime("%m_%d_%Y_%I_%M_%S")
end_date = datetime.now().strftime('%Y-%m-%d')
start_date = (datetime.now() - timedelta(days=365)).strftime('%Y-%m-%d')

# Create file to save results
f = open(f'C:/Users/fco_j/OneDrive/Escritorio/CHAT GPT/Python/Reports/dow_results_{now}.csv', 'w')
# print table header to file
f.write('Ticker, ClosePrice, SMA200, SMA20, RSI, RelVol\n')

# Define cache_data function
def cache_data(data, stock):
    data.to_pickle(f'C:/Users/fco_j/OneDrive/Escritorio/CHAT GPT/Python/Pickle/{stock}.pkl')

for stock in list:
    # Download historical data for past year
    data = si.get_data(stock, start_date=start_date, end_date=end_date)
    last_price = data["close"][-1]
    # Get 150 and 20 simple moving averages using Talib
    sma150 = talib.SMA(data['close'], timeperiod=150)[-1]
    sma20 = talib.SMA(data['close'], timeperiod=20)[-1]
    rsi = talib.RSI(data['close'], timeperiod=14)
    # Calculate Relative Volume
    rel_vol = data['volume'] / talib.SMA(data['volume'].values.astype(float), timeperiod = 50)
    # Cache data
    cache_data(data, stock)
    # Filter stocks with relative volume (time period 20) over 1
    if last_price > sma150 and last_price > sma20 and rsi[-1] > 50 and rel_vol[-1] > 1:
        # Print results to file
        f.write(f"{stock},{last_price},{sma150},{sma20},{rsi[-1]},{rel_vol[-1]}\n")

f.close()
</code>```

The error:

KeyError                                  Traceback (most recent call last)
~\AppData\Local\Temp/ipykernel_11208/2663596324.py in
26 for stock in dow_list:
27     # Download historical data for past year
---> 28     data = si.get_data(stock, start_date=start_date, end_date=end_date)
29     last_price = data["close"][-1]
30     # Get 150 and 20 simple moving averages using Talib

~\anaconda3\envs\PyFinance\lib\site-packages\yahoo_fin\stock_info.py in get_data(ticker, start_date, end_date, index_as_date, interval, headers)
98
99     # get the date info
--> 100     temp_time = data["chart"]["result"][0]["timestamp"]
101
102     if interval != "1m":

KeyError: 'timestamp'

The code is working with  si.tickers_dow() and  si.tickers_sp500() , but not with si.tickers_nasdaq() .
Not sure if a dataframe issue.

Body:
Im preparing a python code , to screen stocks from the SP500 , DOW and Nasdaq.
SP500 and DOW importing data stocks is working properly , but when I try to import Nasdaq always get similar error, related to timestamp.

See below:
My code:

import talib
from yahoo_fin.stock_info import get_data
import yahoo_fin.stock_info as si
from datetime import datetime
list = si.tickers_nasdaq()

# Get current date and time
now = datetime.now().strftime("%m_%d_%Y_%I_%M_%S")
end_date = datetime.now().strftime('%Y-%m-%d')
start_date = (datetime.now() - timedelta(days=365)).strftime('%Y-%m-%d')

# Create file to save results
f = open(f'C:/Users/fco_j/OneDrive/Escritorio/CHAT GPT/Python/Reports/dow_results_{now}.csv', 'w')
# print table header to file
f.write('Ticker, ClosePrice, SMA200, SMA20, RSI, RelVol\n')

# Define cache_data function
def cache_data(data, stock):
    data.to_pickle(f'C:/Users/fco_j/OneDrive/Escritorio/CHAT GPT/Python/Pickle/{stock}.pkl')

for stock in list:
    # Download historical data for past year
    data = si.get_data(stock, start_date=start_date, end_date=end_date)
    last_price = data["close"][-1]
    # Get 150 and 20 simple moving averages using Talib
    sma150 = talib.SMA(data['close'], timeperiod=150)[-1]
    sma20 = talib.SMA(data['close'], timeperiod=20)[-1]
    rsi = talib.RSI(data['close'], timeperiod=14)
    # Calculate Relative Volume
    rel_vol = data['volume'] / talib.SMA(data['volume'].values.astype(float), timeperiod = 50)
    # Cache data
    cache_data(data, stock)
    # Filter stocks with relative volume (time period 20) over 1
    if last_price > sma150 and last_price > sma20 and rsi[-1] > 50 and rel_vol[-1] > 1:
        # Print results to file
        f.write(f"{stock},{last_price},{sma150},{sma20},{rsi[-1]},{rel_vol[-1]}\n")

f.close()

The error:

KeyError                                  Traceback (most recent call last)
~\AppData\Local\Temp/ipykernel_11208/2663596324.py in
26 for stock in dow_list:
27     # Download historical data for past year
---> 28     data = si.get_data(stock, start_date=start_date, end_date=end_date)
29     last_price = data["close"][-1]
30     # Get 150 and 20 simple moving averages using Talib

~\anaconda3\envs\PyFinance\lib\site-packages\yahoo_fin\stock_info.py in get_data(ticker, start_date, end_date, index_as_date, interval, headers)
98
99     # get the date info
--> 100     temp_time = data["chart"]["result"][0]["timestamp"]
101
102     if interval != "1m":

KeyError: 'timestamp'

The code is working with  si.tickers_dow() and  si.tickers_sp500() , but not with si.tickers_nasdaq() .
Not sure if a dataframe issue.


问:
Recursive Function to get all files from main folder and subdirectories inside it in Python
I have a file directory that looks something like this. I have a larger directory, but showing this one just for explanation purposes:

```<code>.
├── a.txt
├── b.txt
├── foo
│   └── w.txt
│   └── a.txt
└── moo
    └── cool.csv
    └── bad.csv
    └── more
        └── wow.csv
</code>```

I want to write a recursive function to get year counts for files within each subdirectory within this directory.

I want the code to basically check if it's a directory or file. If it's a directory then I want to call the function again and get counts until there's no more subdirectories.

I have the following code (which keeps breaking my kernel when I test it). There's probably some logic error as well I would think..

```<code>import os
import pandas as pd

dir_path = 'S:\\Test'

def getFiles(dir_path):
     contents = os.listdir(dir_path)
     # check if content is directory or not
     for file in contents:
          if os.path.isdir(os.path.join(dir_path, file)):
               # get everything inside subdirectory
               getFiles(dir_path = os.path.join(dir_path, file))
          # it's a file
          else:
               # do something to get the year of the file and put it in a list or something
     # at the end create pandas data frame and return
</code>```

Expected output would be a pandas dataframe that looks something like this..

```<code>Subdir 2020 2021 2022
foo    0    1    1
moo    0    2    0
more   1    0    0
</code>```

How can I do this in Python?

EDIT:

Just realized os.walk() is probably extremely useful for my case here.

Trying to figure out a solution with os.walk() instead of doing it the long way..

Body:
I have a file directory that looks something like this. I have a larger directory, but showing this one just for explanation purposes:

.
├── a.txt
├── b.txt
├── foo
│   └── w.txt
│   └── a.txt
└── moo
    └── cool.csv
    └── bad.csv
    └── more
        └── wow.csv

I want to write a recursive function to get year counts for files within each subdirectory within this directory.

I want the code to basically check if it's a directory or file. If it's a directory then I want to call the function again and get counts until there's no more subdirectories.

I have the following code (which keeps breaking my kernel when I test it). There's probably some logic error as well I would think..

import os
import pandas as pd

dir_path = 'S:\\Test'

def getFiles(dir_path):
     contents = os.listdir(dir_path)
     # check if content is directory or not
     for file in contents:
          if os.path.isdir(os.path.join(dir_path, file)):
               # get everything inside subdirectory
               getFiles(dir_path = os.path.join(dir_path, file))
          # it's a file
          else:
               # do something to get the year of the file and put it in a list or something
     # at the end create pandas data frame and return

Expected output would be a pandas dataframe that looks something like this..

Subdir 2020 2021 2022
foo    0    1    1
moo    0    2    0
more   1    0    0

How can I do this in Python?

EDIT:

Just realized os.walk() is probably extremely useful for my case here.

Trying to figure out a solution with os.walk() instead of doing it the long way..


问:
How to integrate R script into PowerBI web
Is it possible to integrate R script (function, package) into the PowerBI interactive dashboard and publish it on the web?
Like when the user adjusts some inputs, R script automatically re-computes the output.</p>


问:
Neural Network predicting the same kind of outputs for different inputs (Implemented using python)
I tried to implement my own Neural Network for predicting the classes for MNIST Dataset. As I was just getting started, I used sigmoid as the activation function instead of softmax to check if the general structure I implemented was right. My output layer consists of 10 nodes which I intended to produce an output such as this:

```<code>[0.01213, 0.23432, 0.1412 .... 0.8123, 0.02323]
</code>```

where the argmax will give me the prediction of which number the image represents.

Instead, the model predicted very similar values across different inputs

```<code>>> pred= model.feedforward(testX.T, model.sigmoid)
>> pred.argmax(axis=0)
Output: array([5, 5, 5, ..., 5, 5, 5])
</code>```
```<code>>> print(testY[0])
>> print(pred[:, 0])

Output: [0 0 0 0 0 0 0 1 0 0]
[0.37764784 0.28244381 0.2299524  0.54792815 0.73187166 0.86594748
 0.09725093 0.45846115 0.65113219 0.22196521]

>> print(testY[1])
>> print(pred[:, 1])

Output: [0 1 0 0 0 0 0 0 0 0]
[0.37654529 0.28523675 0.23014656 0.54476446 0.72874274 0.86488462
 0.09627901 0.45961867 0.65288013 0.22308308]
</code>```

I used Andrew NG's course as a reference while making the model. Below is the code I implemented:

```<code>class Sequential():

  def __init__(self):
    self.nodes= []
    self.weights= []
    self.biases= []
    self.total_layers= -1
    self.lr= None
    self.loss= None

  def Dense(self, n, input_features=None):

    if self.total_layers==-1 and input_features== None:
      raise Exception("Input shape of Sequence not defined")

    if input_features != None:
      #if first layer and input_shape given, append it to shapes
      if self.total_layers==-1:
        r= input_features
        self.nodes.append(r)
        self.total_layers=0 #input layer done
      elif not (self.nodes[-1]==input_features):
        raise Exception("Previous layer output and given input shape doesnt match")
      else:
        r= self.nodes[-1]
    else:
      r= self.nodes[-1] #output shape of the previous layer

    w= np.random.randn(r,n)* np.sqrt(1/r) #number of features, number of nodes
    b= np.random.randn(n,1)

    return (w, b)

  def add(self, layer):

    #getting the weights and biases from the layer
    w, b= layer

    #Increase the number of layers
    self.total_layers+=1

    #add the weights and biases to the network
    self.weights.append(w)
    self.biases.append(b)
    self.nodes.append(w.shape[1])

  def sigmoid(self, x):
    return 1.0/(1.0+np.exp(-x))

  def sigmoid_bp(self, x):
    return (x)*(1-x)

  def feedforward(self, A, activation_fn):

    if A.shape[0] != self.nodes[0]:
      raise Exception("Given X doesnt match input shape")

    for i in range(0, len(self.nodes)-1):
      A= activation_fn((np.dot(self.weights[i].T, A)+ self.biases[i]))

    return A

  def compile(self, loss= "crossentropy", lr=0.001):
    self.loss= loss
    self.lr=lr

  def train(self, X, y, epochs, activation_fn=sigmoid):

    if not self.lr:
      raise Exception("Compile the Model before running")

    for i in range(epochs):
        self.SGD(X,y, activation_fn)

  def SGD(self, X, Y, activation_fn):

      A=[np.atleast_2d(X)]
      l= X.shape[1]

      for i in range(0, len(self.nodes)-1):
        A.append(activation_fn((np.dot(self.weights[i].T, A[i])+ self.biases[i])))

      #output layer alone
      dZ= (A[-1]- Y) * self.sigmoid_bp(A[-1])
      dW= np.dot(A[-2], dZ.T)
      db= np.sum(dZ, axis=1).reshape(-1,1)
      self.weights[-1]+= -(self.lr/l)*dW
      self.biases[-1]+= -(self.lr/l)*db

      #number of values in A= 1 + (input, hidden, output (number of weights/ biases is 1 less than nodes)
      #output layer is taken care by the previous

      for i in range(2, len(A)):
        dA= np.dot(self.weights[-i+1], dZ)
        dZ= dA* self.sigmoid_bp(A[-i])
        dW= np.dot(A[-i-1], dZ.T)
        db= np.sum(dZ, axis=1).reshape(-1,1)
        self.weights[-i]-= (self.lr/l)*dW
        self.biases[-i]-= (self.lr/l)*db

from sklearn.preprocessing import LabelBinarizer
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report
from sklearn import datasets

digits = datasets.load_digits()
data = digits.data.astype("float")
data = (data - data.min()) / (data.max() - data.min())

(trainX, testX, trainY, testY) = train_test_split(data,
    digits.target, test_size=0.25)

trainY = LabelBinarizer().fit_transform(trainY)
testY = LabelBinarizer().fit_transform(testY)

import numpy as np

model= Sequential()
model.add(model.Dense(16, input_features= 8**2))
model.add(model.Dense(32))
model.add(model.Dense(10))

model.compile(1e-04)
model.train(trainX.T, trainY.T, 500, model.sigmoid)
</code>```

On checking other solutions, I tried varying the learning rate between 0.1 to 1e-05, but the outputs are all similar regardless. I have also initialized the weights (as per the formula meant for sigmoid fn), varied the number of nodes in the hidden layers, altered the number of epochs, verified whether the dimensions were right, but none of it seemed to correct the output. I have also trained the model on randomly generated inputs and outputs, but the results are similar to one another.

I believe that there is an error in my implementation. I havent been able to figure out where I am going wrong. I hope you can point me in the right direction

Body:
I tried to implement my own Neural Network for predicting the classes for MNIST Dataset. As I was just getting started, I used sigmoid as the activation function instead of softmax to check if the general structure I implemented was right. My output layer consists of 10 nodes which I intended to produce an output such as this:

[0.01213, 0.23432, 0.1412 .... 0.8123, 0.02323]

where the argmax will give me the prediction of which number the image represents.

Instead, the model predicted very similar values across different inputs

>> pred= model.feedforward(testX.T, model.sigmoid)
>> pred.argmax(axis=0)
Output: array([5, 5, 5, ..., 5, 5, 5])

>> print(testY[0])
>> print(pred[:, 0])

Output: [0 0 0 0 0 0 0 1 0 0]
[0.37764784 0.28244381 0.2299524  0.54792815 0.73187166 0.86594748
 0.09725093 0.45846115 0.65113219 0.22196521]

>> print(testY[1])
>> print(pred[:, 1])

Output: [0 1 0 0 0 0 0 0 0 0]
[0.37654529 0.28523675 0.23014656 0.54476446 0.72874274 0.86488462
 0.09627901 0.45961867 0.65288013 0.22308308]

I used Andrew NG's course as a reference while making the model. Below is the code I implemented:

class Sequential():

  def __init__(self):
    self.nodes= []
    self.weights= []
    self.biases= []
    self.total_layers= -1
    self.lr= None
    self.loss= None

  def Dense(self, n, input_features=None):

    if self.total_layers==-1 and input_features== None:
      raise Exception("Input shape of Sequence not defined")

    if input_features != None:
      #if first layer and input_shape given, append it to shapes
      if self.total_layers==-1:
        r= input_features
        self.nodes.append(r)
        self.total_layers=0 #input layer done
      elif not (self.nodes[-1]==input_features):
        raise Exception("Previous layer output and given input shape doesnt match")
      else:
        r= self.nodes[-1]
    else:
      r= self.nodes[-1] #output shape of the previous layer

    w= np.random.randn(r,n)* np.sqrt(1/r) #number of features, number of nodes
    b= np.random.randn(n,1)

    return (w, b)

  def add(self, layer):

    #getting the weights and biases from the layer
    w, b= layer

    #Increase the number of layers
    self.total_layers+=1

    #add the weights and biases to the network
    self.weights.append(w)
    self.biases.append(b)
    self.nodes.append(w.shape[1])

  def sigmoid(self, x):
    return 1.0/(1.0+np.exp(-x))

  def sigmoid_bp(self, x):
    return (x)*(1-x)

  def feedforward(self, A, activation_fn):

    if A.shape[0] != self.nodes[0]:
      raise Exception("Given X doesnt match input shape")

    for i in range(0, len(self.nodes)-1):
      A= activation_fn((np.dot(self.weights[i].T, A)+ self.biases[i]))

    return A

  def compile(self, loss= "crossentropy", lr=0.001):
    self.loss= loss
    self.lr=lr

  def train(self, X, y, epochs, activation_fn=sigmoid):

    if not self.lr:
      raise Exception("Compile the Model before running")

    for i in range(epochs):
        self.SGD(X,y, activation_fn)

  def SGD(self, X, Y, activation_fn):

      A=[np.atleast_2d(X)]
      l= X.shape[1]

      for i in range(0, len(self.nodes)-1):
        A.append(activation_fn((np.dot(self.weights[i].T, A[i])+ self.biases[i])))

      #output layer alone
      dZ= (A[-1]- Y) * self.sigmoid_bp(A[-1])
      dW= np.dot(A[-2], dZ.T)
      db= np.sum(dZ, axis=1).reshape(-1,1)
      self.weights[-1]+= -(self.lr/l)*dW
      self.biases[-1]+= -(self.lr/l)*db

      #number of values in A= 1 + (input, hidden, output (number of weights/ biases is 1 less than nodes)
      #output layer is taken care by the previous

      for i in range(2, len(A)):
        dA= np.dot(self.weights[-i+1], dZ)
        dZ= dA* self.sigmoid_bp(A[-i])
        dW= np.dot(A[-i-1], dZ.T)
        db= np.sum(dZ, axis=1).reshape(-1,1)
        self.weights[-i]-= (self.lr/l)*dW
        self.biases[-i]-= (self.lr/l)*db

from sklearn.preprocessing import LabelBinarizer
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report
from sklearn import datasets

digits = datasets.load_digits()
data = digits.data.astype("float")
data = (data - data.min()) / (data.max() - data.min())

(trainX, testX, trainY, testY) = train_test_split(data,
    digits.target, test_size=0.25)

trainY = LabelBinarizer().fit_transform(trainY)
testY = LabelBinarizer().fit_transform(testY)

import numpy as np

model= Sequential()
model.add(model.Dense(16, input_features= 8**2))
model.add(model.Dense(32))
model.add(model.Dense(10))

model.compile(1e-04)
model.train(trainX.T, trainY.T, 500, model.sigmoid)

On checking other solutions, I tried varying the learning rate between 0.1 to 1e-05, but the outputs are all similar regardless. I have also initialized the weights (as per the formula meant for sigmoid fn), varied the number of nodes in the hidden layers, altered the number of epochs, verified whether the dimensions were right, but none of it seemed to correct the output. I have also trained the model on randomly generated inputs and outputs, but the results are similar to one another.

I believe that there is an error in my implementation. I havent been able to figure out where I am going wrong. I hope you can point me in the right direction


问:
How to add multiple filters to the same document key in mongoose?
I currently have an array of objects in a document that looks something like this:

users: [
    {name: "Bob", age: 23},
    {name: "Tom", age: 23),
    {name: "Sam", age: 26),
]
```
<p>I am trying to find the document that has the name <code>Bob</code> and the age <code>23</code>. while filtering out any documents that do not match this criteria, or do not yet exist. I was able to find the matching object with:</p>
<pre class="lang-js prettyprint-override"><code>User.find({users: { $elemMatch: { name: "Bob", age: 23}}})
</code>```

and I was able to filter out the documents that do not exist with:

User.find({users: {$ne: []}})
```
<p>but when I try to add the two together, like so, the program stops working:</p>
<pre class="lang-js prettyprint-override"><code>User.find({users: {$ne: [], $elemMatch: { name: "Bob", age: 23}}})
</code>```

I want to know how to can add two filters for the same key inside the document.

Body:
I currently have an array of objects in a document that looks something like this:

users: [
    {name: "Bob", age: 23},
    {name: "Tom", age: 23),
    {name: "Sam", age: 26),
]

I am trying to find the document that has the name Bob and the age 23. while filtering out any documents that do not match this criteria, or do not yet exist. I was able to find the matching object with:

User.find({users: { $elemMatch: { name: "Bob", age: 23}}})

and I was able to filter out the documents that do not exist with:

User.find({users: {$ne: []}})

but when I try to add the two together, like so, the program stops working:

User.find({users: {$ne: [], $elemMatch: { name: "Bob", age: 23}}})

I want to know how to can add two filters for the same key inside the document.


问:
how to overlay modal over side nav
The modal is hiding behind my sidebar even after giving the modal a greater z-index, I have fixed the position for the sidebar. The popup works perfectly when sidebar is not active in both desktop and responsive view.

I wanted my popup to come over my sidenav in responsive view.

Title: how to overlay modal over side nav


问:
What is the best way to store large amounts of data on client-side in Angular (to persist on refresh)
I am creating an Angular app where a user will upload a large CSV (could be 10k+ rows). I need to store and process this data on client-side. What is the best way to achieve this?

I have already tried using BehaviorSubject, but data is deleted upon refresh.

Service:

```<code>export class ProcessCsvService {
    private rows = new BehaviorSubject<any>([]);

    constructor() { }

    setRows(rows: any[]): void {
        this.rows.next(rows);
    }

    getRows(): any {
        return this.rows.asObservable();
    }
}
</code>```

Component:

```<code>export class TableViewComponent implements OnInit {
    rows: any[] = [];

    constructor(private csvService: ProcessCsvService) {}

    ngOnInit(): void {
        this.csvService.getRows().subscribe((data: any[]) => (this.rows = data));
    }
}
</code>```

Body:
I am creating an Angular app where a user will upload a large CSV (could be 10k+ rows). I need to store and process this data on client-side. What is the best way to achieve this?

I have already tried using BehaviorSubject, but data is deleted upon refresh.

Service:

export class ProcessCsvService {
    private rows = new BehaviorSubject<any>([]);

    constructor() { }

    setRows(rows: any[]): void {
        this.rows.next(rows);
    }

    getRows(): any {
        return this.rows.asObservable();
    }
}

Component:

export class TableViewComponent implements OnInit {
    rows: any[] = [];

    constructor(private csvService: ProcessCsvService) {}

    ngOnInit(): void {
        this.csvService.getRows().subscribe((data: any[]) => (this.rows = data));
    }
}


问:
A question about spark distributied aggregation
I am reading up on spark from here

At one point the blog says:

consider an app that wants to count the occurrences of each word in a corpus and pull the results into the driver as a map.  One approach, which can be accomplished with the aggregate action, is to compute a local map at each partition and then merge the maps at the driver. The alternative approach, which can be accomplished with aggregateByKey, is to perform the count in a fully distributed way, and then simply collectAsMap the results to the driver.

So, as I understand this, the two approaches described are:

Approach 1:

Create a hash map for within each executor

Collect key 1 from all the executors on the driver and aggregate

Collect key 2 from all the executors on the driver and aggregate

and so on and so forth

This is where the problem is. I do not think this approach 1 ever happens in spark unless the user was hell-bent on doing it and start using collect along with filter to get the data key by key on the driver and then writing code on the driver to merge the results

Approach 2 (I think this is what usually happens in spark unless you use groupBy wherein the combiner is not run. This is typical reduceBy mechanism):

Compute first level of aggregation on map side

Shuffle

Compute second level of aggregation from all the partially aggregated results from the step 1

Which leads me to believe that I am misunderstanding the approach 1 and what the author is trying to say. Can you please help me understand what the approach 1 in the quoted text is?


问:
Send data with angular dialog when closingOnNavigation
I have a dialog with two buttons, one of them send the information to another dialog and the second just close when it check that the data returned is not a certain key word. When you click out of the dialog it get closed (that is good is the behaviour i want) but does not send the data back.

When you close a dialog with the command:

this.dialogRef.close(this.farmForm.value);

You are sending this information that can be used for example with:

dialogRef.afterClosed().subscribe( (result) => {

I want to configure it to obtain the same data when the dialog is close on navigation. Because i have the dialog configured as dialogConfig.data = {closeOnNavigation: true};

I also try to add a function called closeOnNavigation to return the data but that is not a good approach.


问:
I'm getting a cairo library error while uploading my json file to my project
I have a code block and a style.json file in this way. To see an error in the code, I called it with JsonStyle(self, self.ui) during load and added the library from Custom_Widgets.Widgets import *, but even though I got an error in my code, I keep getting it.

```<code>import sys
import os
from interface import *
from Custom_Widgets.Widgets import *

class MainWindow(QMainWindow):
    def __init__(self, parent=None):
        super(MainWindow, self).__init__(parent)
        self.ui = Ui_MainWindow()
        self.ui.setupUi(self)
        loadJsonStyle(self, self.ui)
        self.show()
        # Expand Center Menu Widget Size
        self.ui.settingsBtn.clicked.connect(lambda: self.ui.centerMenuContainer.expandMenu())
        self.ui.infoBtn.clicked.connect(lambda: self.ui.centerMenuContainer.expandMenu())
        self.ui.helpBtn.clicked.connect(lambda: self.ui.centerMenuContainer.expandMenu())
        # Close Center Menu Widget Size
        self.ui.closeCenterMenuBtn.clicked.connect(lambda: self.ui.centerMenuContainer.collapseMenu())
        # Expand Right Menu Widget
        self.ui.moreMenuBtn.clicked.connect(lambda: self.ui.rightMenuContainer.expandMenu())
        self.ui.profileMenuBtn.clicked.connect(lambda: self.ui.rightMenuContainer.expandMenu())
        # Close Right Menu Widget
        self.ui.closeRightMenuBtn.clicked.connect(lambda: self.ui.rightMenuContainer.collapseMenu())
        # Close Notification Menu Widget
        self.ui.closeNotificationBtn.clicked.connect(lambda: self.ui.popupNotificationContainer.collapseMenu())

if __name__ == '__main__':
    app = QApplication(sys.argv)
    window = MainWindow()
    sys.exit(app.exec_())
</code>```

error output:

OSError: no library called "cairo-2" was found no library called "cairo" was found no library called "libcairo-2" was found cannot load library 'libcairo.so.2': error 0x7e cannot load library 'libcairo.2.dylib': error 0x7e cannot load library 'libcairo-2.dll': error 0x7e

note: I installed the library pip install QT-PyQt-PySide-Custom-Widgets

I came across dozens of people on the internet who got an error about this issue, but I couldn't solve my mistake even though I tried all the solutions they found.
I also downloaded the dll files shown in the error and installed the necessary libraries with pip.

Body:
I have a code block and a style.json file in this way. To see an error in the code, I called it with JsonStyle(self, self.ui) during load and added the library from Custom_Widgets.Widgets import *, but even though I got an error in my code, I keep getting it.

import sys
import os
from interface import *
from Custom_Widgets.Widgets import *

class MainWindow(QMainWindow):
    def __init__(self, parent=None):
        super(MainWindow, self).__init__(parent)
        self.ui = Ui_MainWindow()
        self.ui.setupUi(self)
        loadJsonStyle(self, self.ui)
        self.show()
        # Expand Center Menu Widget Size
        self.ui.settingsBtn.clicked.connect(lambda: self.ui.centerMenuContainer.expandMenu())
        self.ui.infoBtn.clicked.connect(lambda: self.ui.centerMenuContainer.expandMenu())
        self.ui.helpBtn.clicked.connect(lambda: self.ui.centerMenuContainer.expandMenu())
        # Close Center Menu Widget Size
        self.ui.closeCenterMenuBtn.clicked.connect(lambda: self.ui.centerMenuContainer.collapseMenu())
        # Expand Right Menu Widget
        self.ui.moreMenuBtn.clicked.connect(lambda: self.ui.rightMenuContainer.expandMenu())
        self.ui.profileMenuBtn.clicked.connect(lambda: self.ui.rightMenuContainer.expandMenu())
        # Close Right Menu Widget
        self.ui.closeRightMenuBtn.clicked.connect(lambda: self.ui.rightMenuContainer.collapseMenu())
        # Close Notification Menu Widget
        self.ui.closeNotificationBtn.clicked.connect(lambda: self.ui.popupNotificationContainer.collapseMenu())

if __name__ == '__main__':
    app = QApplication(sys.argv)
    window = MainWindow()
    sys.exit(app.exec_())

error output:

OSError: no library called "cairo-2" was found no library called "cairo" was found no library called "libcairo-2" was found cannot load library 'libcairo.so.2': error 0x7e cannot load library 'libcairo.2.dylib': error 0x7e cannot load library 'libcairo-2.dll': error 0x7e

note: I installed the library pip install QT-PyQt-PySide-Custom-Widgets

I came across dozens of people on the internet who got an error about this issue, but I couldn't solve my mistake even though I tried all the solutions they found.
I also downloaded the dll files shown in the error and installed the necessary libraries with pip.


问:
Django Exception Hook
is possible to handle all traceback/exception error code in django ?

i try using sys.excepthook = myfunction in the first line manage.py, which my function/method/class provides python functions/methods like send log to syslog and other functions, here's an example of my exception hook function:

```<code> #!/usr/bin/env python
 # ehook.py

 from __future__ import print_function

 from make_colors import make_colors
 from xnotify import notify
 import os, sys
 import socket
 from configset import configset
 import syslogx
 import datetime
 import re
 import traceback
 from pydebugger.debug import debug

 PID = os.getpid()

 class ehook:

    CONFIGNAME = os.path.join(os.path.dirname(os.path.realpath(__file__)), 'ehook.ini')
    CONFIG = configset(CONFIGNAME)

    def __init__(self, exc_type, exc_value, tb, **kwargs):

        #local_vars = {}
        #super(ehook, self)
        self.exc_type = exc_type
        self.exc_value = exc_value
        self.tb = tb
        data, data_color, raw_data, raw_data_color = self.tracert()

        app = "ehook"
        title = "Traceback"
        event = 'Error'
        icon = list(filter(lambda k: os.path.isfile(k), [os.path.splitext(os.path.realpath(__file__))[0] + "." + i for  i in ['png', 'jpg']] + [os.path.join(os.path.dirname(os.path.realpath(__file__)), 'traceback') + "." + i for  i in ['png', 'jpg']]))
        #print("icon:", icon)
        kwargs.update({'app': app,})
        kwargs.update({'title': title,})
        kwargs.update({'event': event,})
        kwargs.update({'sticky': True,})
        if self.CONFIG.get_config('syslog', 'severity'):
            kwargs.update({'syslog_severity': self.CONFIG.get_config('syslog', 'severity').lower(),})
        if self.CONFIG.get_config('syslog', 'facility'):
            kwargs.update({'syslog_facility': self.CONFIG.get_config('syslog', 'facility').lower(),})
        if icon: kwargs.update({'icon': icon[0],})
        #icon = os.path.join(os.path.dirname(os.path.realpath(__file__)), icon)

        syslog_server = re.split("\n|\t|,", self.CONFIG.get_config('syslog', 'host')) or kwargs.get('syslog_server')
        syslog_server = [i.strip() for i in syslog_server]
        debug(syslog_server = syslog_server)
        if isinstance(syslog_server, list):
            for i in syslog_server:
                self.debug_server_client(data, data_color, i.split(":")[0])
        else:
            self.debug_server_client(data, data_color, syslog_server)
        kwargs.update({'syslog_server': syslog_server,})
        notify.send(message_color = data_color, message = "\n".join(raw_data[1:]), **kwargs)

    def sent_to_syslog(self, message, severity=None, facility=None, host = None, port = None):

        severity = severity or 3
        facility = facility or 3
        host = host or self.CONFIG.get_config('syslog', 'host') or '127.0.0.1'
        port = port or self.CONFIG.get_config('syslog', 'port') or 514
        if "," in host:
            host = [i.strip() for i in host.split(",")]
        else:
            if not isinstance(host, list): host = [host]
        if hasattr(message, 'decode'):
            message = message.decode('utf-8')
        for i in host:
            if ":" in i:
                HOST, PORT = str(i).split(":")
                HOST = HOST.strip()
                PORT = PORT.strip()
                PORT = PORT or 514
                print("sent to %s:%s" % (HOST, str(PORT)))
                syslogx.syslog(message, severity, facility, HOST, int(PORT))
            else:
                syslogx.syslog(message, severity, facility, i, port)

         def debug_server_client(self, msg, msg_color = None, host = None, port = None):

        dtime = make_colors(datetime.datetime.strftime(datetime.datetime.now(), '%Y:%m:%d'), 'white', 'red') + \
        make_colors('~', 'white') + \
        make_colors(datetime.datetime.strftime(datetime.datetime.now(), '%H:%M:%S'), 'white', 'green') + \
        make_colors(datetime.datetime.strftime(datetime.datetime.now(), '%f'), 'white', 'magenta') + \
            "[%s]" % PID

        if sys.version_info.major == 3:
            msg = msg.encode('utf-8')
        s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
        host = host or self.CONFIG.get_config('server', 'host') or '127.0.0.1'
        port = port or self.CONFIG.get_config('server', 'port') or 50000
        if not host:
            try:
                if sys.version_info.major == 3:
                    msg_color = bytes(dtime + " " + msg_color, encoding = 'utf-8')
                else:
                    msg_color = dtime + " " + msg_color
                s.sendto((msg_color or msg), (host, port))
            except:
                traceback.format_exc()
                print(msg)
                self.sent_to_syslog(msg)
            s.close()
            #return False
        if "," in host:
            host = [i.strip() for i in host.split(",")]
        else:
            host = [host]
        for i in host:
            if ":" in i:
                host1, port1 = str(i).strip().split(":")
                port1 = int(port1.strip()) or port
                host1 = host1.strip()
                try:
                    if sys.version_info.major == 3:
                        msg_color = bytes(dtime + " " + msg_color, encoding = 'utf-8')
                    else:
                        msg_color = dtime + " " + msg_color
                    s.sendto((msg_color or msg), (host1, port1))
                except:
                    traceback.format_exc()
                    print(msg)
                    self.sent_to_syslog(msg)
                s.close()
                #print "%s:%s 0" % (host, str(port))
                break
            else:
                host = i.strip()
                #print "%s:%s 1" % (host, str(port))
                if sys.version_info.major == 3:
                    msg_color = bytes(dtime + " " + msg_color, encoding = 'utf-8')
                else:
                    msg_color = dtime + " " + msg_color
                s.sendto((msg_color or msg), (host, port))
                s.close()
                break

    def tracert(self):
        trace = ["Traceback: "]
        trace_color = []
        while self.tb:
            filename = self.tb.tb_frame.f_code.co_filename
            name = self.tb.tb_frame.f_code.co_name
            line_no = self.tb.tb_lineno
            data_color = make_colors("Traceback:", 'b', 'y') + " " + \
                make_colors(self.exc_type.__name__, 'lw', 'r') + " (" + \
                make_colors(self.exc_value, 'lw', 'm') + "), File " + \
                make_colors(filename, 'b', 'lg') + ", line " + \
                make_colors(line_no, 'lw', 'bl') + ", in " + \
                make_colors(name, 'b', 'lc')
            print(data_color)
            trace_color.append(data_color)
            data = str(self.exc_type.__name__) + ": " + str(self.exc_value) + "), File " + filename + ", line " + str(line_no) + ", in " + name
            trace.append(data)
            #print(f"File {filename} line {line_no}, in {name}")

            #local_vars = tb.tb_frame.f_locals
            #print(f"{exc_type.__name__}, Message: {exc_value}")
            self.tb = self.tb.tb_next
        #print(f"Local variables in top frame: {local_vars}")
        return "\n".join(trace), "\n".join(trace_color), trace, trace_color

</code>```

in manage.py

```<code>from ehook import ehook
sys.excepthook = ehook
import sys
import os

def main():
    """Run administrative tasks."""
    os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'tobi.settings')
    try:
        from django.core.management import execute_from_command_line
    except ImportError as exc:
        raise ImportError(
            "Couldn't import Django. Are you sure it's installed and "
            "available on your PYTHONPATH environment variable? Did you "
            "forget to activate a virtual environment?"
        ) from exc

    execute_from_command_line(sys.argv)

</code>```

but it doesn't work like what I expected

how to handle all exceptions in django ?

Title: Django Exception Hook

Body:
is possible to handle all traceback/exception error code in django ?

i try using sys.excepthook = myfunction in the first line manage.py, which my function/method/class provides python functions/methods like send log to syslog and other functions, here's an example of my exception hook function:

 #!/usr/bin/env python
 # ehook.py

 from __future__ import print_function

 from make_colors import make_colors
 from xnotify import notify
 import os, sys
 import socket
 from configset import configset
 import syslogx
 import datetime
 import re
 import traceback
 from pydebugger.debug import debug

 PID = os.getpid()

 class ehook:

    CONFIGNAME = os.path.join(os.path.dirname(os.path.realpath(__file__)), 'ehook.ini')
    CONFIG = configset(CONFIGNAME)

    def __init__(self, exc_type, exc_value, tb, **kwargs):

        #local_vars = {}
        #super(ehook, self)
        self.exc_type = exc_type
        self.exc_value = exc_value
        self.tb = tb
        data, data_color, raw_data, raw_data_color = self.tracert()

        app = "ehook"
        title = "Traceback"
        event = 'Error'
        icon = list(filter(lambda k: os.path.isfile(k), [os.path.splitext(os.path.realpath(__file__))[0] + "." + i for  i in ['png', 'jpg']] + [os.path.join(os.path.dirname(os.path.realpath(__file__)), 'traceback') + "." + i for  i in ['png', 'jpg']]))
        #print("icon:", icon)
        kwargs.update({'app': app,})
        kwargs.update({'title': title,})
        kwargs.update({'event': event,})
        kwargs.update({'sticky': True,})
        if self.CONFIG.get_config('syslog', 'severity'):
            kwargs.update({'syslog_severity': self.CONFIG.get_config('syslog', 'severity').lower(),})
        if self.CONFIG.get_config('syslog', 'facility'):
            kwargs.update({'syslog_facility': self.CONFIG.get_config('syslog', 'facility').lower(),})
        if icon: kwargs.update({'icon': icon[0],})
        #icon = os.path.join(os.path.dirname(os.path.realpath(__file__)), icon)

        syslog_server = re.split("\n|\t|,", self.CONFIG.get_config('syslog', 'host')) or kwargs.get('syslog_server')
        syslog_server = [i.strip() for i in syslog_server]
        debug(syslog_server = syslog_server)
        if isinstance(syslog_server, list):
            for i in syslog_server:
                self.debug_server_client(data, data_color, i.split(":")[0])
        else:
            self.debug_server_client(data, data_color, syslog_server)
        kwargs.update({'syslog_server': syslog_server,})
        notify.send(message_color = data_color, message = "\n".join(raw_data[1:]), **kwargs)

    def sent_to_syslog(self, message, severity=None, facility=None, host = None, port = None):

        severity = severity or 3
        facility = facility or 3
        host = host or self.CONFIG.get_config('syslog', 'host') or '127.0.0.1'
        port = port or self.CONFIG.get_config('syslog', 'port') or 514
        if "," in host:
            host = [i.strip() for i in host.split(",")]
        else:
            if not isinstance(host, list): host = [host]
        if hasattr(message, 'decode'):
            message = message.decode('utf-8')
        for i in host:
            if ":" in i:
                HOST, PORT = str(i).split(":")
                HOST = HOST.strip()
                PORT = PORT.strip()
                PORT = PORT or 514
                print("sent to %s:%s" % (HOST, str(PORT)))
                syslogx.syslog(message, severity, facility, HOST, int(PORT))
            else:
                syslogx.syslog(message, severity, facility, i, port)

         def debug_server_client(self, msg, msg_color = None, host = None, port = None):

        dtime = make_colors(datetime.datetime.strftime(datetime.datetime.now(), '%Y:%m:%d'), 'white', 'red') + \
        make_colors('~', 'white') + \
        make_colors(datetime.datetime.strftime(datetime.datetime.now(), '%H:%M:%S'), 'white', 'green') + \
        make_colors(datetime.datetime.strftime(datetime.datetime.now(), '%f'), 'white', 'magenta') + \
            "[%s]" % PID

        if sys.version_info.major == 3:
            msg = msg.encode('utf-8')
        s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
        host = host or self.CONFIG.get_config('server', 'host') or '127.0.0.1'
        port = port or self.CONFIG.get_config('server', 'port') or 50000
        if not host:
            try:
                if sys.version_info.major == 3:
                    msg_color = bytes(dtime + " " + msg_color, encoding = 'utf-8')
                else:
                    msg_color = dtime + " " + msg_color
                s.sendto((msg_color or msg), (host, port))
            except:
                traceback.format_exc()
                print(msg)
                self.sent_to_syslog(msg)
            s.close()
            #return False
        if "," in host:
            host = [i.strip() for i in host.split(",")]
        else:
            host = [host]
        for i in host:
            if ":" in i:
                host1, port1 = str(i).strip().split(":")
                port1 = int(port1.strip()) or port
                host1 = host1.strip()
                try:
                    if sys.version_info.major == 3:
                        msg_color = bytes(dtime + " " + msg_color, encoding = 'utf-8')
                    else:
                        msg_color = dtime + " " + msg_color
                    s.sendto((msg_color or msg), (host1, port1))
                except:
                    traceback.format_exc()
                    print(msg)
                    self.sent_to_syslog(msg)
                s.close()
                #print "%s:%s 0" % (host, str(port))
                break
            else:
                host = i.strip()
                #print "%s:%s 1" % (host, str(port))
                if sys.version_info.major == 3:
                    msg_color = bytes(dtime + " " + msg_color, encoding = 'utf-8')
                else:
                    msg_color = dtime + " " + msg_color
                s.sendto((msg_color or msg), (host, port))
                s.close()
                break

    def tracert(self):
        trace = ["Traceback: "]
        trace_color = []
        while self.tb:
            filename = self.tb.tb_frame.f_code.co_filename
            name = self.tb.tb_frame.f_code.co_name
            line_no = self.tb.tb_lineno
            data_color = make_colors("Traceback:", 'b', 'y') + " " + \
                make_colors(self.exc_type.__name__, 'lw', 'r') + " (" + \
                make_colors(self.exc_value, 'lw', 'm') + "), File " + \
                make_colors(filename, 'b', 'lg') + ", line " + \
                make_colors(line_no, 'lw', 'bl') + ", in " + \
                make_colors(name, 'b', 'lc')
            print(data_color)
            trace_color.append(data_color)
            data = str(self.exc_type.__name__) + ": " + str(self.exc_value) + "), File " + filename + ", line " + str(line_no) + ", in " + name
            trace.append(data)
            #print(f"File {filename} line {line_no}, in {name}")

            #local_vars = tb.tb_frame.f_locals
            #print(f"{exc_type.__name__}, Message: {exc_value}")
            self.tb = self.tb.tb_next
        #print(f"Local variables in top frame: {local_vars}")
        return "\n".join(trace), "\n".join(trace_color), trace, trace_color

in manage.py

from ehook import ehook
sys.excepthook = ehook
import sys
import os

def main():
    """Run administrative tasks."""
    os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'tobi.settings')
    try:
        from django.core.management import execute_from_command_line
    except ImportError as exc:
        raise ImportError(
            "Couldn't import Django. Are you sure it's installed and "
            "available on your PYTHONPATH environment variable? Did you "
            "forget to activate a virtual environment?"
        ) from exc

    execute_from_command_line(sys.argv)

but it doesn't work like what I expected

how to handle all exceptions in django ?


问:
Vite local host link in vscode opens their homepage and not my blank html file
I have set up a new vscode project to build a website and after having installed node.js and Vite. When it comes to clicking on the local host link generated, it should open my html file as a blank webpage but it opens their brands homepage instead.

I have tried changing the localhost number to the same 5173 in the Launch.json - x

I updated windows to the latest win10 version (can't upgrade to 11) - x

I tried the LTS and current versions of node.js - x

I tried opening it in different browsers and incognito mode - x

Using Powershell in the terminal

Not sure what to try next really it should just be a blank html page that updates automatically when I write my code.


问:
POJO to Entity mapping with ModelMapper
My POJO has a variable annotated like this

```<code>@JsonFormat(pattern = "yyyy-MM-dd'T'HH:mm:ss.SSSxxx")
@JsonDeserialize(using = ZonedDateTimeDeserializer.class)
private ZonedDateTime startTs;

//other private variables
</code>```

My goal is to map this to an Entity for which I'm using ModelMapper and the sample code goes like this

```<code>public POJOEntity mapToPOJOEntity(POJO pojo) {
    return modelMapper.map(pojo, POJOEntity.class);
}
</code>```

After mapping when I look at the startTs variable in debug mode it has the desired format but when I check in the db startTs is saving as a whole ZonedDateTime object rather than the desired format I specified in POJO. Did anyone face this issue if yes can someone please help?
Value saved in cosmosDB:

Expected output: "startTs": "2023-01-12T08:58:32.452-06:00"

PS: CosmosDB, spring-boot, Java8, ModelMapper version: 2.3.8

Title: POJO to Entity mapping with ModelMapper

Body:
My POJO has a variable annotated like this

@JsonFormat(pattern = "yyyy-MM-dd'T'HH:mm:ss.SSSxxx")
@JsonDeserialize(using = ZonedDateTimeDeserializer.class)
private ZonedDateTime startTs;

//other private variables

My goal is to map this to an Entity for which I'm using ModelMapper and the sample code goes like this

public POJOEntity mapToPOJOEntity(POJO pojo) {
    return modelMapper.map(pojo, POJOEntity.class);
}

After mapping when I look at the startTs variable in debug mode it has the desired format but when I check in the db startTs is saving as a whole ZonedDateTime object rather than the desired format I specified in POJO. Did anyone face this issue if yes can someone please help?
Value saved in cosmosDB:

Expected output: "startTs": "2023-01-12T08:58:32.452-06:00"

PS: CosmosDB, spring-boot, Java8, ModelMapper version: 2.3.8


问:
Is it possible to change the database type of an enum in gorm (using postgres)?
I wonder if its possible to change the datatype of an enum in gorm when it has the same underlying type?

So say this is my current model

type A

```<code>type typeA int32

const (
    field1         typeA = 0
    field2         typeA = 1
    field3         typeA = 2
)
</code>```
```<code>type typeB int32

const (
    field1         typeB = 0
    field2         typeB = 1
    field3         typeB = 2
)
</code>```

And I have my gorm model like this

```<code>type MyModel struct {
    gorm.Model

    SomeField        typeA
}
</code>```

I wonder if I could easily change my DB model to this (change in type, but same underlying type)
Or if this will cause some migration issues?

```<code>type MyModel struct {
    gorm.Model

    SomeField        typeB
}
</code>```

Body:
I wonder if its possible to change the datatype of an enum in gorm when it has the same underlying type?

So say this is my current model

type A

type typeA int32

const (
    field1         typeA = 0
    field2         typeA = 1
    field3         typeA = 2
)

type typeB int32

const (
    field1         typeB = 0
    field2         typeB = 1
    field3         typeB = 2
)

And I have my gorm model like this

type MyModel struct {
    gorm.Model

    SomeField        typeA
}

I wonder if I could easily change my DB model to this (change in type, but same underlying type)
Or if this will cause some migration issues?

type MyModel struct {
    gorm.Model

    SomeField        typeB
}


问:
Why do I fail to submit data to textarea with python requests.post()
I want to use the requests.post tool to automatically query domain name attribution on this websitea website,But the return value is always empty, I guess it is because the post method failed to transfer the data to the textarea

```<code>url  = 'http://icp.chinaz.com/searchs'
data = {
        'hosts':'qq.com',
        'btn_search':'%E6%9F%A5%E8%AF%A2',        '__RequestVerificationToken=':'CfDJ8KmGV0zny1FLtrcRZkGCczG2owvCRigVmimqp33mw5t861kI7zU2VfQBri65hIwu_4VXbmtqImMmTeZzqxE7NwwvAtwWcZSMnk6UDuzP3Ymzh9YrHPJkj_cy8vSLXvUYXrFO0HnCb0pSweHIL9pkReY',
    }

requests.post(url=url,data=data,headers=headers).content.decode('utf-8')
</code>```

I'd be very grateful if you could point out where I'm going wrong

I have tried to replace headers and so on.

Body:
I want to use the requests.post tool to automatically query domain name attribution on this websitea website,But the return value is always empty, I guess it is because the post method failed to transfer the data to the textarea

url  = 'http://icp.chinaz.com/searchs'
data = {
        'hosts':'qq.com',
        'btn_search':'%E6%9F%A5%E8%AF%A2',        '__RequestVerificationToken=':'CfDJ8KmGV0zny1FLtrcRZkGCczG2owvCRigVmimqp33mw5t861kI7zU2VfQBri65hIwu_4VXbmtqImMmTeZzqxE7NwwvAtwWcZSMnk6UDuzP3Ymzh9YrHPJkj_cy8vSLXvUYXrFO0HnCb0pSweHIL9pkReY',
    }

requests.post(url=url,data=data,headers=headers).content.decode('utf-8')

I'd be very grateful if you could point out where I'm going wrong

I have tried to replace headers and so on.


问:
Which is the best way to work with 3rd party repositories in Odoo.sh?
I got an Odoo 15 Enterprise running with the Odoo.sh platform.

When I add an OCA repository, I add it with the Submodule option, and this generates a OCA directory inside src/user. Which is the best way to update that code? If I go to src/user/OCA/repository-name, would doing a git pull there be ok?

Can anyone share their method? There is documentation on how to create your own modules, but not much about adding external repos and keep them updated.


问:
Suppress timeout exceptions in EF Core request by own try-catch?
I'm using EF Core 6 in a .NET 6 project.

Some of my SQL requests leads to a SQL Server timeout. This is not a big deal and I want to catch this exceptions to just produce a warning instead of an error in the logs.

```<code>try
{
    List<SomeClass> results = dbContext.TableName
                                       .Where ....
                                       .ToList();
}
catch (Exception ex)
{
    // log some warning ...
}
</code>```

The exception is caught and the warning is logged.

BUT: additionally the EF call itself produces an error that is logged/printed out in console.

How can I suppress the exception within the EF database request to let "my" catch be the only one to handle all exceptions?

Body:
I'm using EF Core 6 in a .NET 6 project.

Some of my SQL requests leads to a SQL Server timeout. This is not a big deal and I want to catch this exceptions to just produce a warning instead of an error in the logs.

try
{
    List<SomeClass> results = dbContext.TableName
                                       .Where ....
                                       .ToList();
}
catch (Exception ex)
{
    // log some warning ...
}

The exception is caught and the warning is logged.

BUT: additionally the EF call itself produces an error that is logged/printed out in console.

How can I suppress the exception within the EF database request to let "my" catch be the only one to handle all exceptions?


问:
How to map animation properties of "react-spring" or "framer-motion" values to Qt "SpringAnimation"?
Physics-based animations on the Web use properties like stiffness, damping, and mass to define the spring animations. While values from react-spring and framer-motion are interchangeable (same numerical values), it is much harder to precisely map those onto the range that Qt SpringAnimation uses.

Relevant Properties (with typical values)

react-spring
framer-motion
Qt SpringAnimation

tension 150
stiffness 150
spring 0 - 5.0

friction 20
damping 20
damping 0 - 1.0

mass 1
mass 1
mass 1.0 ✅

Does anybody have experience with translating values from design to development?

Is it a simple factor or more complex (like C to F)?

I could just try different combinations. While spring animations might look similar, they can actually be quite different in fine detail. A mapping would allow me to design animations in something like Figma and use those values directly in Qt, which usually takes much longer to compile and "play around" with...


问:
R Telegram.bot. Polling error: Error in if (handler$check_update(update)) { : argument is of length zero
I'm trying to write telegram bot with payment processing function. This methods are implemented in the telegram API, but unfortunately these methods are not implemented in R package telegram.bot.

My code:

```<code>library(telegram.bot)
library(glue)
library(httr)

bot_token <- "your_bot_token"

provider_token <- "your:TEST:token"

updater <- Updater(token = bot_token)

currency <- "your_currency_code"

## Start
start <- function(bot, update) {

    # create keyboard
    RKM <- ReplyKeyboardMarkup(
        keyboard = list(
            list(
                KeyboardButton(text = "donate")
            )
        ),
        resize_keyboard = TRUE,
        one_time_keyboard = TRUE
        )

    # send keyboard
    bot$sendMessage(update$message$chat_id,
                    text = 'Command',
                    reply_markup = RKM)
    }

## Send invoice
send_invoice <- function(bot, update) {

    chat_id <- update$from_chat_id()

    title <- "Title"

    desc <- "Detail"

    payload <- "specialItem-001"

    prices <- '[{"label": "Payment", "amount": 24900}]'

    invoice <- glue("https://api.telegram.org/bot{bot_token}/sendInvoice?chat_id={chat_id}&title={title}&description={desc}&payload={payload}&provider_token={provider_token}&currency={currency}&prices={prices}")

    httr::POST(
        url = invoice
        )
    }

## Accept pre checkout query
pre_checkout <- function(bot, update) {

    chat_id <- update$pre_checkout_query$from$id

    invoice_id <- update$pre_checkout_query$id

    accept_invoice <- glue("https://api.telegram.org/bot{bot_token}/answerPreCheckoutQuery?pre_checkout_query_id={invoice_id}&ok=TRUE")

    httr::POST(
        url = accept_invoice
        )
    }

## View payment info
success_pay <- function(bot, update) {
    str(update)
}

## Message filter
MessageFilters$invoice <- BaseFilter(function(message) {
    message$text == "donate"
    }
)

## Send RKM
h_start    <- CommandHandler('start', start)

## Invoice hendler
invoice_hendler <- MessageHandler(send_invoice, filters = MessageFilters$invoice)

## Pre checkout hendler
check_update <- function(update) {
    TRUE
}

handle_update <- function(update, dispatcher) {
    self$callback(dispatcher$bot, update)
}

pre_checkout_handler <- Handler(pre_checkout,
                                check_update = check_update,
                                handle_update = handle_update,
                                handlername = "FooHandler")

## Successful payment hendler
payment_handler <- MessageHandler(success_pay, filters = MessageFilters$successful_payment)

## add hendler to dispatcher
updater <- updater +
    h_start +
    invoice_hendler +
    pre_checkout_handler +
    payment_handler

## Start pooling
updater$start_polling(verbose = TRUE, clean = TRUE)
</code>```

So I have to write a custom handler and the payment goes through, but the bot crashes when bot processing Update with successful payment message. Error:

handler$check_update(update)) { : argument is of length zero

I thick problem with my handler config (check_update and handle_update). How can i fix it?

Body:
I'm trying to write telegram bot with payment processing function. This methods are implemented in the telegram API, but unfortunately these methods are not implemented in R package telegram.bot.

My code:

library(telegram.bot)
library(glue)
library(httr)

bot_token <- "your_bot_token"

provider_token <- "your:TEST:token"

updater <- Updater(token = bot_token)

currency <- "your_currency_code"

## Start
start <- function(bot, update) {

    # create keyboard
    RKM <- ReplyKeyboardMarkup(
        keyboard = list(
            list(
                KeyboardButton(text = "donate")
            )
        ),
        resize_keyboard = TRUE,
        one_time_keyboard = TRUE
        )

    # send keyboard
    bot$sendMessage(update$message$chat_id,
                    text = 'Command',
                    reply_markup = RKM)
    }

## Send invoice
send_invoice <- function(bot, update) {

    chat_id <- update$from_chat_id()

    title <- "Title"

    desc <- "Detail"

    payload <- "specialItem-001"

    prices <- '[{"label": "Payment", "amount": 24900}]'

    invoice <- glue("https://api.telegram.org/bot{bot_token}/sendInvoice?chat_id={chat_id}&title={title}&description={desc}&payload={payload}&provider_token={provider_token}&currency={currency}&prices={prices}")

    httr::POST(
        url = invoice
        )
    }

## Accept pre checkout query
pre_checkout <- function(bot, update) {

    chat_id <- update$pre_checkout_query$from$id

    invoice_id <- update$pre_checkout_query$id

    accept_invoice <- glue("https://api.telegram.org/bot{bot_token}/answerPreCheckoutQuery?pre_checkout_query_id={invoice_id}&ok=TRUE")

    httr::POST(
        url = accept_invoice
        )
    }

## View payment info
success_pay <- function(bot, update) {
    str(update)
}

## Message filter
MessageFilters$invoice <- BaseFilter(function(message) {
    message$text == "donate"
    }
)

## Send RKM
h_start    <- CommandHandler('start', start)

## Invoice hendler
invoice_hendler <- MessageHandler(send_invoice, filters = MessageFilters$invoice)

## Pre checkout hendler
check_update <- function(update) {
    TRUE
}

handle_update <- function(update, dispatcher) {
    self$callback(dispatcher$bot, update)
}

pre_checkout_handler <- Handler(pre_checkout,
                                check_update = check_update,
                                handle_update = handle_update,
                                handlername = "FooHandler")

## Successful payment hendler
payment_handler <- MessageHandler(success_pay, filters = MessageFilters$successful_payment)

## add hendler to dispatcher
updater <- updater +
    h_start +
    invoice_hendler +
    pre_checkout_handler +
    payment_handler

## Start pooling
updater$start_polling(verbose = TRUE, clean = TRUE)

So I have to write a custom handler and the payment goes through, but the bot crashes when bot processing Update with successful payment message. Error:

handler$check_update(update)) { : argument is of length zero

I thick problem with my handler config (check_update and handle_update). How can i fix it?


问:
Building a QGIS web app with Python and Flask
I am building a QGIS web application with Python and Flask and would like advice on implementing QGIS.

Currently the application takes Shapefiles uploaded by the user from the browser, adds these to a QGIS project, creates a QGIS layout, exports this to PDF and serves the PDF back to the user.

I have managed to get this to work with the code below but I believe I am implementing QGIS incorrectly.

Currently the QGIS application is initiated in a Flask route but gives a warning about threading:

"WARNING: QApplication was not created in the main() thread."

I believe the QGIS application should be initiated elsewhere and I have studied Davidism's solution given here: Store large data or a service connection per Flask session.

Unfortunately I have been unable to build a working solution. I would stress that my background is as a GIS practitioner with limited software development experience and therefore I would appreciate any direction towards resources or tutorials that may help bridge the gap.

The project layout is based on the one detailed in the Flask tutorial.

```<code>@bp.route('/', methods=['GET', 'POST'])
def index():
    if request.method == 'POST':
        project_name = request.form['project_name']  # accept user input from browser

        # Initiate the QGIS application. I suspect this should be implemented elsewhere.
        enable_gui = False
        qgis_application = QgsApplication([], enable_gui)
        qgis_application.initQgis()

        # Check if shapefilefiles were provided by user, save the files and return shapefile name.
        if request.files['files']:
            shapefile_name = save_shapefiles(request.files.getlist('files'))

        # Functions that create QGIS project and layout, add layers and export to PDF.
        project = qgis_project(project_name)
        layout = qgis_print_layout(project, shapefile_name)
        export_to_pdf(layout, f'flaskr\\pdfs\\{project_name}.pdf')

        # Serve PDF to user
        return serve_pdf(project_name)

    return render_template('index.html')
</code>```

You can access the entire application on GitHub here.

Body:
I am building a QGIS web application with Python and Flask and would like advice on implementing QGIS.

Currently the application takes Shapefiles uploaded by the user from the browser, adds these to a QGIS project, creates a QGIS layout, exports this to PDF and serves the PDF back to the user.

I have managed to get this to work with the code below but I believe I am implementing QGIS incorrectly.

Currently the QGIS application is initiated in a Flask route but gives a warning about threading:

"WARNING: QApplication was not created in the main() thread."

I believe the QGIS application should be initiated elsewhere and I have studied Davidism's solution given here: Store large data or a service connection per Flask session.

Unfortunately I have been unable to build a working solution. I would stress that my background is as a GIS practitioner with limited software development experience and therefore I would appreciate any direction towards resources or tutorials that may help bridge the gap.

The project layout is based on the one detailed in the Flask tutorial.

@bp.route('/', methods=['GET', 'POST'])
def index():
    if request.method == 'POST':
        project_name = request.form['project_name']  # accept user input from browser

        # Initiate the QGIS application. I suspect this should be implemented elsewhere.
        enable_gui = False
        qgis_application = QgsApplication([], enable_gui)
        qgis_application.initQgis()

        # Check if shapefilefiles were provided by user, save the files and return shapefile name.
        if request.files['files']:
            shapefile_name = save_shapefiles(request.files.getlist('files'))

        # Functions that create QGIS project and layout, add layers and export to PDF.
        project = qgis_project(project_name)
        layout = qgis_print_layout(project, shapefile_name)
        export_to_pdf(layout, f'flaskr\\pdfs\\{project_name}.pdf')

        # Serve PDF to user
        return serve_pdf(project_name)

    return render_template('index.html')

You can access the entire application on GitHub here.


问:
How can I get AWS CLI to recognize my config file?
I'm trying to spin up a s3 bucket via the AWS CLI and I'm getting the error "upload failed: .\mylocalfile.txt to s3://testBucket/mylocalfile.txt Unable to locate credentials".

I've ran aws configure and aws configure set on a default profile and i've done aws configure on a specific profile as well. In either case when I run AWS configure list I get the following

```<code>
 Name                    Value             Type    Location
      ----                    -----             ----    --------
   profile              TestProfile           manual    --profile
access_key                <not set>             None    None
secret_key                <not set>             None    None
    region                us-east-1      config-file    ~/.aws/config
</code>```

the secret key is set in the same config file as the region but they are being located for some reason.

When I run AWS configure list --profile TestProfile I expect the credentials set in AWS configure to appear properly.

Body:
I'm trying to spin up a s3 bucket via the AWS CLI and I'm getting the error "upload failed: .\mylocalfile.txt to s3://testBucket/mylocalfile.txt Unable to locate credentials".

I've ran aws configure and aws configure set on a default profile and i've done aws configure on a specific profile as well. In either case when I run AWS configure list I get the following

 Name                    Value             Type    Location
      ----                    -----             ----    --------
   profile              TestProfile           manual    --profile
access_key                <not set>             None    None
secret_key                <not set>             None    None
    region                us-east-1      config-file    ~/.aws/config

the secret key is set in the same config file as the region but they are being located for some reason.

When I run AWS configure list --profile TestProfile I expect the credentials set in AWS configure to appear properly.


问:
Return search result while the user is typing with MongoDB
Create such feature where user can see the results based on the text in the input field

I want to create a feature where user can type in an input field and see the result as the user types. I did implemented it and the code is there below. Now, I know this code is not at all scalable, or even efficient (I am sending DB request on every keystroke). I want to know how I can make this more efficient and possibly scalable.

```<code>const search = async (req, res) => {
  try {
    const input = req.query.text.replace(/ /g,'');
    if(!input){return res.end()}

    const result = await Promise.all([
        User.find({ username: new RegExp(input, "i") }),
        Post.find({ description: new RegExp(input, "i") }),
      ]);

    // const users = await User.find({ username: new RegExp(input, 'i') });

    res.json(result)

  } catch (error) {
    console.log(error);
    res.status(500).json(error);
  }
};
</code>```

Body:
Create such feature where user can see the results based on the text in the input field

I want to create a feature where user can type in an input field and see the result as the user types. I did implemented it and the code is there below. Now, I know this code is not at all scalable, or even efficient (I am sending DB request on every keystroke). I want to know how I can make this more efficient and possibly scalable.

const search = async (req, res) => {
  try {
    const input = req.query.text.replace(/ /g,'');
    if(!input){return res.end()}

    const result = await Promise.all([
        User.find({ username: new RegExp(input, "i") }),
        Post.find({ description: new RegExp(input, "i") }),
      ]);

    // const users = await User.find({ username: new RegExp(input, 'i') });

    res.json(result)

  } catch (error) {
    console.log(error);
    res.status(500).json(error);
  }
};


问:
Camelot pdf extraction has an issue while copying texts among span cells
I am extracting data from PDFs using camelot and am faced with the following issue on 3. page of this datasheet. The problematic table is shown below:

The issue is inconsistency during the copying content of span cells. As you can see on the following picture span cells are correctly detected.

Even if the cells are detected correctly in the 3. column the content is copied to one of two spanned cells and in the 4. column the content is copied to two of three spanned cells. You can see the data I extracted as follow. There is always one missing cell per both columns.

And here is the code I used if you want to try it out;

```<code>table_areas=['86, 697, 529, 95'] # To ignore page borders
tables = camelot.read_pdf(single_source, pages='all',
                          flavor = 'lattice',
                          copy_text=['v'],
                          line_scale = 110,
                          table_regions=table_areas,
                          flag_size = False,
                          process_background=False)
</code>```

Code (Colab):

```<code>!pip install "camelot-py[cv]" -q
!pip install PyPDF2==2.12.1
!apt-get install ghostscript
</code>```
```<code>import camelot
import pandas as pd
from tabulate import tabulate
import re
import fitz
</code>```
```<code>single_source = '/content/FDB9406_F085-D.PDF'
print("Extracting ", single_source, "...")

table_areas=['86, 697, 529, 95']
tables = camelot.read_pdf(single_source, pages='all', flavor = 'lattice', copy_text=['v'], line_scale = 110, table_regions=table_areas, flag_size = False, process_background=False)

print("Extracting ", single_source, "is finished!")
</code>```

to visualize the tables:

```<code>for table in accurate_tables:
  print(table.parsing_report, table.shape, table._bbox)
  print(tabulate(table.df, headers='keys', tablefmt='psql'))
  camelot.plot(table, kind='grid').show()

print("Extracting ", single_source, "is finished!")
</code>```

Body:
I am extracting data from PDFs using camelot and am faced with the following issue on 3. page of this datasheet. The problematic table is shown below:

The issue is inconsistency during the copying content of span cells. As you can see on the following picture span cells are correctly detected.

Even if the cells are detected correctly in the 3. column the content is copied to one of two spanned cells and in the 4. column the content is copied to two of three spanned cells. You can see the data I extracted as follow. There is always one missing cell per both columns.

And here is the code I used if you want to try it out;

table_areas=['86, 697, 529, 95'] # To ignore page borders
tables = camelot.read_pdf(single_source, pages='all',
                          flavor = 'lattice',
                          copy_text=['v'],
                          line_scale = 110,
                          table_regions=table_areas,
                          flag_size = False,
                          process_background=False)

Code (Colab):

!pip install "camelot-py[cv]" -q
!pip install PyPDF2==2.12.1
!apt-get install ghostscript

import camelot
import pandas as pd
from tabulate import tabulate
import re
import fitz

single_source = '/content/FDB9406_F085-D.PDF'
print("Extracting ", single_source, "...")

table_areas=['86, 697, 529, 95']
tables = camelot.read_pdf(single_source, pages='all', flavor = 'lattice', copy_text=['v'], line_scale = 110, table_regions=table_areas, flag_size = False, process_background=False)

print("Extracting ", single_source, "is finished!")

to visualize the tables:

for table in accurate_tables:
  print(table.parsing_report, table.shape, table._bbox)
  print(tabulate(table.df, headers='keys', tablefmt='psql'))
  camelot.plot(table, kind='grid').show()

print("Extracting ", single_source, "is finished!")


问:
VSCode formatting of command options
I am using VScode for writting shell scripts.

I specify the language of the script on VS Code and I am already enjoying the rest of the formatting.

I would like the options of my commands (that would be everything starting with one dash "-" or two dashes "--", including the dashes) to be formatted in a peculiar color for reading convenience.

Example:

```<code>example -in $file -out output.txt -mode 5 --force
</code>```

I would like -in, -out, -mode and --force to all have a similar color, not a color already used in the usual shell formatting in VScode.

Screenshot of the current code in VS Code:

Title: VSCode formatting of command options

Body:
I am using VScode for writting shell scripts.

I specify the language of the script on VS Code and I am already enjoying the rest of the formatting.

I would like the options of my commands (that would be everything starting with one dash "-" or two dashes "--", including the dashes) to be formatted in a peculiar color for reading convenience.

Example:

example -in $file -out output.txt -mode 5 --force

I would like -in, -out, -mode and --force to all have a similar color, not a color already used in the usual shell formatting in VScode.

Screenshot of the current code in VS Code:


问:
Django template isn't displaying some model object attributes on the page but will display others
I'm following a tutorial on Real Python to create a very simple portfolio website.
https://realpython.com/get-started-with-django-1/

I've created a Project model:

```<code>from django.db import models

class Project(models.Model):
    title = models.CharField(max_length=100)
    description = models.TextField()
    technology = models.CharField(max_length=20)
    image = models.FilePathField(path="/img")
</code>```

I've added a number of different objects/projects to this model in the shell and saved them. Here is the HTML for the web page I'm displaying them on:

```<code>{% extends "base.html" %}
{% load static %}

{% block page_content %}
<h1>Projects</h1>
<div class="row">
{% for project in projects %}
    <div class="col-md-4">
        <div class="card mb-2">
            <img class="card-img-top" src="{% static project.image %}">
            <div class="card-body">
                <h5 class="card-title">{{ project.title }}</h5>
                <p class="card-text">{{ project.description }}</p>
                <a href="{% url 'project_detail' project.pk %}" class="btn-primary">
                    Read More
                </a>
            </div>
        </div>
    </div>
    {% endfor %}
</div>
{% endblock %}
</code>```

As you can see, I'm displaying them on cards that are added using a for loop that iterates every time a new object is created. On this page, the project title, description and read more button are displaying correctly but for some reason the image is not. I have the images placed in a /static/img folder in the same directory as the app 'projects'. And I've tried re-entering the object data, using PNGs, JPEGs etc. but none of these seem to get the images to display correctly.

Next, the 'in detail' page is loaded when the read more button is clicked. Here is the HTML for that:

```<code>{% extends "base.html" %}
{% load static %}

{% block page_content %}
<h1>{{ project.title }}</h1>
<div class="row">
    <div class="col-md-8">
        <img src="{% static project.image %}" alt="" width="100%">
    </div>
    <div class="col-md-4">
        <h5>About the project:</h5>
        <p>{{ project.description }}</p>
        <br>
        <h5>Technology used:</h5>
        <p>{{ project.technology }}</p>
    </div>
</div>
{% endblock %}
</code>```

On this page, not only does the image not load, but the project title, description and technology used don't display either. Only a blank space where the image should be and the h5 text shown above is displayed.

I've been following the tutorial as exact as I can and only making small aesthetic changes here and there. I've been through the source code on github and compared my code directly to the authors code and the core components all seem to be exactly the same. Am I missing something?

ADDITIONAL INFO

settings.py static files:

```<code># Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/4.1/howto/static-files/

STATIC_URL = 'static/img/'
</code>```

views.py for the 'projects' app:

```<code>from django.shortcuts import render
from projects.models import Project

def project_index(request):
    projects = Project.objects.all()
    context = {
        'projects': projects
    }
    return render(request, 'project_index.html', context)

def project_detail(request, pk):
    projects = Project.objects.get(pk=pk)
    context = {
        'projects': projects
    }
    return render(request, 'project_detail.html', context)
</code>```

Body:
I'm following a tutorial on Real Python to create a very simple portfolio website.
https://realpython.com/get-started-with-django-1/

I've created a Project model:

from django.db import models

class Project(models.Model):
    title = models.CharField(max_length=100)
    description = models.TextField()
    technology = models.CharField(max_length=20)
    image = models.FilePathField(path="/img")

I've added a number of different objects/projects to this model in the shell and saved them. Here is the HTML for the web page I'm displaying them on:

{% extends "base.html" %}
{% load static %}

{% block page_content %}

Projects

{% for project in projects %}

{{ project.title }}

{{ project.description }}

                    Read More

    {% endfor %}

{% endblock %}

As you can see, I'm displaying them on cards that are added using a for loop that iterates every time a new object is created. On this page, the project title, description and read more button are displaying correctly but for some reason the image is not. I have the images placed in a /static/img folder in the same directory as the app 'projects'. And I've tried re-entering the object data, using PNGs, JPEGs etc. but none of these seem to get the images to display correctly.

Next, the 'in detail' page is loaded when the read more button is clicked. Here is the HTML for that:

{% extends "base.html" %}
{% load static %}

{% block page_content %}

{{ project.title }}

About the project:

{{ project.description }}

Technology used:

{{ project.technology }}

{% endblock %}

On this page, not only does the image not load, but the project title, description and technology used don't display either. Only a blank space where the image should be and the h5 text shown above is displayed.

I've been following the tutorial as exact as I can and only making small aesthetic changes here and there. I've been through the source code on github and compared my code directly to the authors code and the core components all seem to be exactly the same. Am I missing something?

ADDITIONAL INFO

settings.py static files:

# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/4.1/howto/static-files/

STATIC_URL = 'static/img/'

views.py for the 'projects' app:

from django.shortcuts import render
from projects.models import Project

def project_index(request):
    projects = Project.objects.all()
    context = {
        'projects': projects
    }
    return render(request, 'project_index.html', context)

def project_detail(request, pk):
    projects = Project.objects.get(pk=pk)
    context = {
        'projects': projects
    }
    return render(request, 'project_detail.html', context)


问:
Google Sheet API with Protected Range for DATA SOURCE sheets
I am trying to protect a sheet with sheetType as 'DATA_SOURCE'.

I create a new spreadsheet using a service account and the Sheets API v4 with Python. In that spreadsheet I add a new sheet with sheetType as 'DATA_SOURCE' and I add a protected range for the whole sheet.

When I access the Spreadsheet in the browser the sheet is not protected.

This is the code that I use to create the spreadsheet, add the data source, and protect the sheet

```<code>from googleapiclient import discovery
from google.oauth2.service_account import Credentials

scopes = [
    'https://www.googleapis.com/auth/drive',
    'https://www.googleapis.com/auth/drive.file',
    'https://www.googleapis.com/auth/spreadsheets',
    'https://www.googleapis.com/auth/bigquery.readonly']

creds = Credentials.from_service_account_file(SERVICE_ACCOUNT_FILE_PATH, scopes=scopes)

sheets_service = discovery.build('sheets', 'v4', credentials=creds)

request_body = {
                "sheets": [
                    {
                    "properties" : {
                        "sheetId": 0,
                        "title": "data_souce_sheet",
                        "index": 0,
                        "sheetType": "DATA_SOURCE",
                        "dataSourceSheetProperties": {"dataSourceId": "data_souce_sheet"},
                        "hidden": False
                        },
                    "protectedRanges": [{"range": {"sheetId": 0}, "warningOnly": False, "editors": {"users": [SERVICE_ACCOUNT_EMAIL]}}],
                    },
                    ],
                "dataSources": [
                    {"dataSourceId":'data_souce_sheet',
                    "sheetId":0,
                    "spec": {
                        "parameters": [],
                        "bigQuery": {
                            "projectId": PROJECT_ID,
                            "querySpec": {
                                "rawQuery": "SELECT CURRENT_TIMESTAMP() AS NOW"
                                }
                            }
                        }
                    }]
                }
creation_request = sheets_service.spreadsheets().create(body=request_body)
creation_response = creation_request.execute()
</code>```

In the creation_response there are no protected ranges. Even when I go to the spreadsheet using the browser the sheet is not protected.
I can manually protect the sheet, and in the UI I see the "lock", but when I use the API to see the properties I still don't see a protected range

```<code>request = sheets_service.spreadsheets().get(spreadsheetId=SHEET_ID, includeGridData=False)
response = request.execute()
</code>```

The output:

```<code>{'dataSources': [{'dataSourceId': 'data_souce_sheet',
                  'spec': {'bigQuery': {'projectId': '...',
                                        'querySpec': {'rawQuery': 'SELECT '
                                                                  'CURRENT_TIMESTAMP() '
                                                                  'AS NOW'}}}}],
 'properties': {'autoRecalc': '...',
                'defaultFormat': {...},
                'locale': '...',
                'spreadsheetTheme': {...},
 'sheets': [{'properties': {'dataSourceSheetProperties': {'columns': [{'reference': {'name': 'NOW'}}],
                                                          'dataExecutionStatus': {'lastRefreshTime': '2023-01-12T15:30:17.681Z',
                                                                                'state': 'SUCCEEDED'},
                                                          'dataSourceId': 'data_souce_sheet'},
                            'gridProperties': {'columnCount': 1, 'rowCount': 1},
                            'index': 0,
                            'sheetId': 0,
                            'sheetType': 'DATA_SOURCE',
                            'title': 'data_souce_sheet'}}],
 'spreadsheetId': '...',
 'spreadsheetUrl': '...'}
</code>```

Body:
I am trying to protect a sheet with sheetType as 'DATA_SOURCE'.

I create a new spreadsheet using a service account and the Sheets API v4 with Python. In that spreadsheet I add a new sheet with sheetType as 'DATA_SOURCE' and I add a protected range for the whole sheet.

When I access the Spreadsheet in the browser the sheet is not protected.

This is the code that I use to create the spreadsheet, add the data source, and protect the sheet

from googleapiclient import discovery
from google.oauth2.service_account import Credentials

scopes = [
    'https://www.googleapis.com/auth/drive',
    'https://www.googleapis.com/auth/drive.file',
    'https://www.googleapis.com/auth/spreadsheets',
    'https://www.googleapis.com/auth/bigquery.readonly']

creds = Credentials.from_service_account_file(SERVICE_ACCOUNT_FILE_PATH, scopes=scopes)

sheets_service = discovery.build('sheets', 'v4', credentials=creds)

request_body = {
                "sheets": [
                    {
                    "properties" : {
                        "sheetId": 0,
                        "title": "data_souce_sheet",
                        "index": 0,
                        "sheetType": "DATA_SOURCE",
                        "dataSourceSheetProperties": {"dataSourceId": "data_souce_sheet"},
                        "hidden": False
                        },
                    "protectedRanges": [{"range": {"sheetId": 0}, "warningOnly": False, "editors": {"users": [SERVICE_ACCOUNT_EMAIL]}}],
                    },
                    ],
                "dataSources": [
                    {"dataSourceId":'data_souce_sheet',
                    "sheetId":0,
                    "spec": {
                        "parameters": [],
                        "bigQuery": {
                            "projectId": PROJECT_ID,
                            "querySpec": {
                                "rawQuery": "SELECT CURRENT_TIMESTAMP() AS NOW"
                                }
                            }
                        }
                    }]
                }
creation_request = sheets_service.spreadsheets().create(body=request_body)
creation_response = creation_request.execute()

In the creation_response there are no protected ranges. Even when I go to the spreadsheet using the browser the sheet is not protected.
I can manually protect the sheet, and in the UI I see the "lock", but when I use the API to see the properties I still don't see a protected range

request = sheets_service.spreadsheets().get(spreadsheetId=SHEET_ID, includeGridData=False)
response = request.execute()

The output:

{'dataSources': [{'dataSourceId': 'data_souce_sheet',
                  'spec': {'bigQuery': {'projectId': '...',
                                        'querySpec': {'rawQuery': 'SELECT '
                                                                  'CURRENT_TIMESTAMP() '
                                                                  'AS NOW'}}}}],
 'properties': {'autoRecalc': '...',
                'defaultFormat': {...},
                'locale': '...',
                'spreadsheetTheme': {...},
 'sheets': [{'properties': {'dataSourceSheetProperties': {'columns': [{'reference': {'name': 'NOW'}}],
                                                          'dataExecutionStatus': {'lastRefreshTime': '2023-01-12T15:30:17.681Z',
                                                                                'state': 'SUCCEEDED'},
                                                          'dataSourceId': 'data_souce_sheet'},
                            'gridProperties': {'columnCount': 1, 'rowCount': 1},
                            'index': 0,
                            'sheetId': 0,
                            'sheetType': 'DATA_SOURCE',
                            'title': 'data_souce_sheet'}}],
 'spreadsheetId': '...',
 'spreadsheetUrl': '...'}


问:
Hyperlink in Task Dialog footer to CHM?
I know that I can format a hyperlink like this in the footer area of a task dialog:

```<code><a href="https://help-msa.publictalksoftware.co.uk/msa-revision-history.html">Revision History</a>
</code>```

But, my application supports local CHM or online HTML. For CHM:

```<code>HtmlHelp(ptr_cast<DWORD_PTR>(L"msa-revision-history.htm"), HH_DISPLAY_TOPIC);
</code>```

Is there any way to support a hyperlink in the footer that would somehow execute the HtmlHelp equivalent?

Title: Hyperlink in Task Dialog footer to CHM?

Body:
I know that I can format a hyperlink like this in the footer area of a task dialog:

Revision History

But, my application supports local CHM or online HTML. For CHM:

HtmlHelp(ptr_cast<DWORD_PTR>(L"msa-revision-history.htm"), HH_DISPLAY_TOPIC);

Is there any way to support a hyperlink in the footer that would somehow execute the HtmlHelp equivalent?