问:
IronPython: Message: expected c_double, got c_double_Array_3
I’m currently developing a script using the python script editor in Rhino. As I’m currently working in a Windows machine, the script editor uses IronPython as language.

In the same script, I want to interact with an FE software (Straus7) which has a Python API. When doing so, I have experienced some problems as the ctypes module does not seem to work in IronPython the same way it does in regular Python. Especially, I’m finding problems when initializing arrays using the command:

```<code>ctypes.c_double*3
</code>```

For example, if I want to obtain the XYZ coordinates of a node #1 in the FE model, I regular Python I would write the following:

```<code>XYZType = ctypes.c_double*3
XYZ = XYZType()
node_num = 1
st.St7GetNodeXYZ(1,node_num,XYZ)
</code>```

And this returns me a variable XYZ which is a 3D array such that:

```<code>XYZ -> <straus_userfunctions.c_double_Array_3 at 0xc5787b0>
XYZ[0] = -0.7xxxxx -> (X_coord)
XYZ[1] = -0.8xxxxx -> (Y_coord)
XYZ[2] = -0.9xxxxx -> (Z_coord)
</code>```

On the other side, I copy the same exact script in IronPython, the following error message appears

```<code>Message: expected c_double, got c_double_Array_3
</code>```

Obviously, If I change the variable XYZ to c_double; then it becomes a double variable which contains only a single entry, which corresponds to the first element of the array (in this case, the X-coordinate)

This situation is quite annoying as all FEM softwares, the usage of matrices and arrays is widely used. Consequently, I wanted to ask if anyone nows a simple fix to this situation.

I was thinking of using the memory allocation of the first element of the array to obtain the rest but I’m not so sure how to do so.

Thanks a lot. Gerard

答:
I've found when working with IronPython you need to explicitly cast the "Array of three doubles" to a "Pointer to double". So if you're using Grasshopper with the Strand7 / Straus7 API you will need to add an extra bit like this:

```<code>import St7API
import ctypes

# Make the pointer conversion functions
PI = ctypes.POINTER(ctypes.c_long)
PD = ctypes.POINTER(ctypes.c_double)

XYZType = ctypes.c_double*3
XYZ = XYZType()
node_num = 1

# Cast arrays whenever you pass them to St7API from IronPython
St7API.St7GetNodeXYZ(1, node_num, PD(XYZ))
</code>```

I don't have access to IronPython or Strand7 / Straus7 at the moment but from memory that will do it. If that doesn't work for you you can email Strand7 Support - you would typically get feedback on something like this within a day or so.

Body:
I’m currently developing a script using the python script editor in Rhino. As I’m currently working in a Windows machine, the script editor uses IronPython as language.

In the same script, I want to interact with an FE software (Straus7) which has a Python API. When doing so, I have experienced some problems as the ctypes module does not seem to work in IronPython the same way it does in regular Python. Especially, I’m finding problems when initializing arrays using the command:

ctypes.c_double*3

For example, if I want to obtain the XYZ coordinates of a node #1 in the FE model, I regular Python I would write the following:

XYZType = ctypes.c_double*3
XYZ = XYZType()
node_num = 1
st.St7GetNodeXYZ(1,node_num,XYZ)

And this returns me a variable XYZ which is a 3D array such that:

XYZ -> <straus_userfunctions.c_double_Array_3 at 0xc5787b0>
XYZ[0] = -0.7xxxxx -> (X_coord)
XYZ[1] = -0.8xxxxx -> (Y_coord)
XYZ[2] = -0.9xxxxx -> (Z_coord)

On the other side, I copy the same exact script in IronPython, the following error message appears

Message: expected c_double, got c_double_Array_3

Obviously, If I change the variable XYZ to c_double; then it becomes a double variable which contains only a single entry, which corresponds to the first element of the array (in this case, the X-coordinate)

This situation is quite annoying as all FEM softwares, the usage of matrices and arrays is widely used. Consequently, I wanted to ask if anyone nows a simple fix to this situation.

I was thinking of using the memory allocation of the first element of the array to obtain the rest but I’m not so sure how to do so.

Thanks a lot. Gerard

Body:
I've found when working with IronPython you need to explicitly cast the "Array of three doubles" to a "Pointer to double". So if you're using Grasshopper with the Strand7 / Straus7 API you will need to add an extra bit like this:

import St7API
import ctypes

# Make the pointer conversion functions
PI = ctypes.POINTER(ctypes.c_long)
PD = ctypes.POINTER(ctypes.c_double)

XYZType = ctypes.c_double*3
XYZ = XYZType()
node_num = 1

# Cast arrays whenever you pass them to St7API from IronPython
St7API.St7GetNodeXYZ(1, node_num, PD(XYZ))

I don't have access to IronPython or Strand7 / Straus7 at the moment but from memory that will do it. If that doesn't work for you you can email Strand7 Support - you would typically get feedback on something like this within a day or so.


来源: stackexchange

问:
Importing Excel Data Seems to Randomly Give Null Values
Using SSIS for Visual Studio 2017 for some excel file imports.

I've created a package with several loop containers that call to specific packages to handle some files. I have an issue with one particular package being executed in that it seemingly randomly decides the data for columns is NULL per excel file. I was/am under the impression that this is part of the registry setting for TypeGuessRows (changed initially to 0 then to 1000 as a test) located at

```<code>HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Office\14.0\Access Connectivity Engine\Engines\Excel
</code>```

The reason I think this is because the various files being brought in generally have the same data, but it seems that if the first few rows of columns in the source data contains only numbers, that the data with mixed values will not be brought in correctly. All other columns aside from this seems fine.

Looking at the source files, all have the same datatype.
I've tried changing the registry TypeGuessRows value and ensured that the output column property was string-based instead of numerical.
The connection string has IMEX=1

答:
This issue is related to the OLEDB provider used to read excel files: Since excel is not a database where each column has a specific data type, OLEDB provider tries to identify the dominant data types found in each column and replace all other data types that cannot be parsed with NULLs.

There are many articles found online discussing this issue and giving several workarounds (links listed below).

But after using SSIS for years, i can say that best practice is to convert excel files to csv files and read them using Flat File components.

Or, if you don't have the choice to convert excel to flat files then you can force excel connection manager to ignore headers from the first row bu adding HDR=NO to the connection string and adding IMEX=1 to tell the OLEDB provider to specify data types from the first row (which is the header - all string most of the time), in this case all columns are imported as string and no values are replaced with NULLs but you will lose the headers and a additional row (header row is imported).

If you cannot ignore the header row, just add a dummy row that contains dummy string values (example: aaa) after the header row and add IMEX=1 to the connection string.

Helpful links

SSIS Excel Data Import - Mixed data type in Rows

Mixed data types in Excel column

Importing data from Excel having Mixed Data Types in a column (SSIS)

Why SSIS always gets Excel data types wrong, and how to fix it!

EXCEL IN SSIS: FIXING THE WRONG DATA TYPES

IMEX= 1 extended properties in ssis

ContentLicense: CC BY-SA 4.0

Body:
Using SSIS for Visual Studio 2017 for some excel file imports.

I've created a package with several loop containers that call to specific packages to handle some files. I have an issue with one particular package being executed in that it seemingly randomly decides the data for columns is NULL per excel file. I was/am under the impression that this is part of the registry setting for TypeGuessRows (changed initially to 0 then to 1000 as a test) located at

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Office\14.0\Access Connectivity Engine\Engines\Excel

The reason I think this is because the various files being brought in generally have the same data, but it seems that if the first few rows of columns in the source data contains only numbers, that the data with mixed values will not be brought in correctly. All other columns aside from this seems fine.

Looking at the source files, all have the same datatype.
I've tried changing the registry TypeGuessRows value and ensured that the output column property was string-based instead of numerical.
The connection string has IMEX=1


问:
replace a string with special " ' "
I want to find and replace string with special symbol " ' " using gsub of awk.

I tried escape the symbol with \ and even \\ but my code returns > .

```<code>awk '{ gsub("%27","'",$1); print $1 }' file.txt
</code>```

file.txt:

```<code>%27SWAY%27
</code>```

I want above to be replaced to 'SWAY'

答:
You can't use a single quote in an awk program delineated by single quotes.

One way around that is to assign the quote character to an awk variable with -v:

```<code>echo '%27SWAY%27'| awk -v q="'" '{ gsub("%27",q,$1); print $1 }'
'SWAY'
</code>```

Another is to use the octal character escape sequence \047:

```<code>$ echo '%27SWAY%27'| awk '{ gsub("%27","\047",$1); print $1 }'
'SWAY'
</code>```

Finally you can put the awk program in a separate file, in which case the single quote doesn't cause a problem:

```<code>$ cat a.awk
{ gsub("%27","'",$1); print $1 }

$ echo '%27SWAY%27'| awk -f a.awk
'SWAY'
</code>```

Title: replace a string with special " ' "

Body:
I want to find and replace string with special symbol " ' " using gsub of awk.

I tried escape the symbol with \ and even \\ but my code returns > .

awk '{ gsub("%27","'",$1); print $1 }' file.txt

file.txt:

%27SWAY%27

I want above to be replaced to 'SWAY'

Body:
You can't use a single quote in an awk program delineated by single quotes.

One way around that is to assign the quote character to an awk variable with -v:

echo '%27SWAY%27'| awk -v q="'" '{ gsub("%27",q,$1); print $1 }'
'SWAY'

Another is to use the octal character escape sequence \047:

$ echo '%27SWAY%27'| awk '{ gsub("%27","\047",$1); print $1 }'
'SWAY'

Finally you can put the awk program in a separate file, in which case the single quote doesn't cause a problem:

$ cat a.awk
{ gsub("%27","'",$1); print $1 }

$ echo '%27SWAY%27'| awk -f a.awk
'SWAY'


问:
Evaluate sum of values without clicking a button
I really don't know how to program in Javascript, yet I've tried to do some stuff. I have values to be summed if some are selected. The following function does sum operation and show result at once without clicking a button only for checkbox fields, not for the dropdown list and text fields.

However, I need to involve the selected values as well from the dropdown lists and entered both text fields in the same manner of the checkboxes and their immediate responses(output).

I think change method can be used in the same manner.

@Edit:

Sorry, I forgot the tell but, I need to multiply the m2 entered value by before summing with 3134,204499 and Bina Yasi with  -566,2202888.

function calcAndShowTotal() {
  var total = 55435.04798; // it is constant value which must be the beginning value.
  $('#housepriceguesslist :checkbox:checked').each(function() {
    total += parseFloat($(this).attr('price')) || 0;
  });
  $('#total').val(total);
}

$('#housepriceguesslist :checkbox').change(calcAndShowTotal).change();```
<pre class="snippet-code-html lang-html prettyprint-override"><code><script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.3.1/jquery.min.js"></script>
<div id="housepriceguesslist">

  <select id="ilce">
    <option value="0">Choose a state</option>
    <option value="63999.50712318567">Adalar</option>
    <option value="6259.918435398593">Ataşehir</option>
    <option value="46285.35729304834">Beykoz</option>
    <option value="54258.92953697798">Kadıköy</option>
    <option value="-3500.620905653726">Kartal</option>
    <option value="12889.41219909602">Maltepe</option>
    <option value="-33711.83156847153">Pendik</option>
    <option value="-34577.42485251657">Sancaktepe</option>
    <option value="-52183.1608811019">Sultanbeyli</option>
    <option value="-34760.71735007784">Tuzla</option>
    <option value="-27379.87966213786">Çekmeköy</option>
    <option value="-5332.203990418937">Ümraniye</option>
    <option value="26485.33532820411">Üsküdar</option>
    <option value="-18732.62070553167">Şile</option>
  </select>

  <select id="odasalon">
    <option value="0">Choose room number</option>
    <option value="5940.027623576261">1+0</option>
    <option value="-1399.625767475593">1+1</option>
    <option value="-3033.638999060459">2+1</option>
    <option value="11519.83954756076">3+1</option>
    <option value="-7018.52116545078">4+1</option>
    <option value="-6008.081239150166">5+1</option>
  </select>

  m2: <input type="text" id="m2" /> Bina Yasi: <input type="text" id="binayasi" />

  <br/>
  <br/> Features:

  <br/>
  <br/>
  <input type="checkbox" id="Bati" value="Bati" price="-783.4452683566533" /> <label for="Bati">Bati</label>
  <br/>
  <input type="checkbox" id="Dogu" value="Dogu" price="-4895.732009084263" /> <label for="Dogu">Dogu</label>
  <br/>
  <input type="checkbox" id="Guney" value="Guney" price="-2599.001349652765" /> <label for="Guney">Guney</label>
  <br/>
  <input type="checkbox" id="Kuzey" value="Kuzey" price="3832.013070771234" /> <label for="Kuzey">Kuzey</label>
  <br/>
  <input type="checkbox" id="Amerikan Mutfak" value="Amerikan Mutfak" price="2346.347889828301" /> <label for="Amerikan Mutfak">Amerikan Mutfak</label>
  <br/>
  <input type="checkbox" id="Dusakabin" value="Dusakabin" price="-176.5092353740256" /> <label for="Dusakabin">Dusakabin</label>
  <br/>
  <input type="checkbox" id="Akilli Ev" value="Akilli Ev" price="-4756.206384200719" /> <label for="Akilli Ev">Akilli Ev</label>
  <br/>
  <input type="checkbox" id="Ebeveyn Banyosu" value="Ebeveyn Banyosu" price="10135.50644456019" /> <label for="Ebeveyn Banyosu">Ebeveyn Banyosu</label>
  <br/>
  <input type="checkbox" id="Kiler" value="Kiler" price="-10693.49979821809" /> <label for="Kiler">Kiler</label>
  <br/>
  <input type="checkbox" id="Kuvet" value="Kuvet" price="-941.0629741243247" /> <label for="Kuvet">Kuvet</label>
  <br/>
  <input type="checkbox" id="Parke Zemin" value="Parke Zemin" price="-8453.847199707092" /> <label for="Parke Zemin">Parke Zemin</label>
  <br/>
  <input type="checkbox" id="Seramik Zemin" value="Seramik Zemin" price="1545.025601000643" /> <label for="Seramik Zemin">Seramik Zemin</label>
  <br/>
  <input type="checkbox" id="Sauna" value="Sauna" price="8677.708176289747" /> <label for="Sauna">Sauna</label>
  <br/>
  <input type="checkbox" id="Spor Salonu" value="Spor Salonu" price="1159.884679563385" /> <label for="Spor Salonu">Spor Salonu</label>
  <br/>
  <input type="checkbox" id="Tenis Kortu" value="Tenis Kortu" price="16065.26794172257" /> <label for="Tenis Kortu">Tenis Kortu</label>
  <br/>
</div>
<input type="text" id="total" value="0" /></code>```

答:
Similar to Barmar's, I added all elements to the .each() loop but also added the .change() handler to all elements:

```<code>var context = $('#housepriceguesslist');
var elems = $('input:checkbox, input:text, select', context);

function calcAndShowTotal() {
  var total = 55435.04798; // it is constant value which must be the beginning value.
  elems.each(function() {
    if ( $(this).is('input:checkbox:checked') ) {
      total += parseFloat($(this).attr('price')) || 0;
    } else if ( $(this).is('#m2') ) {
      total += parseFloat($(this).val()) * 3134.204499 || 0;
    } else if ( $(this).is('#binayasi') ) {
      total += parseFloat($(this).val()) * -566.2202888 || 0;
    } else if ( $(this).is('select') ) {
      total += parseFloat($('option:selected',this).val()) || 0;
    }
  });
  $('#total').val(total);
}

elems.change(calcAndShowTotal); // handlers
calcAndShowTotal(); // first run
</code>```

See example here: https://jsfiddle.net/qgvsern4/1/

Body:
I really don't know how to program in Javascript, yet I've tried to do some stuff. I have values to be summed if some are selected. The following function does sum operation and show result at once without clicking a button only for checkbox fields, not for the dropdown list and text fields.

However, I need to involve the selected values as well from the dropdown lists and entered both text fields in the same manner of the checkboxes and their immediate responses(output).

I think change method can be used in the same manner.

@Edit:

Sorry, I forgot the tell but, I need to multiply the m2 entered value by before summing with 3134,204499 and Bina Yasi with  -566,2202888.

function calcAndShowTotal() {
  var total = 55435.04798; // it is constant value which must be the beginning value.
  $('#housepriceguesslist :checkbox:checked').each(function() {
    total += parseFloat($(this).attr('price')) || 0;
  });
  $('#total').val(total);
}

$('#housepriceguesslist :checkbox').change(calcAndShowTotal).change();

  <select id="ilce">
    <option value="0">Choose a state</option>
    <option value="63999.50712318567">Adalar</option>
    <option value="6259.918435398593">Ataşehir</option>
    <option value="46285.35729304834">Beykoz</option>
    <option value="54258.92953697798">Kadıköy</option>
    <option value="-3500.620905653726">Kartal</option>
    <option value="12889.41219909602">Maltepe</option>
    <option value="-33711.83156847153">Pendik</option>
    <option value="-34577.42485251657">Sancaktepe</option>
    <option value="-52183.1608811019">Sultanbeyli</option>
    <option value="-34760.71735007784">Tuzla</option>
    <option value="-27379.87966213786">Çekmeköy</option>
    <option value="-5332.203990418937">Ümraniye</option>
    <option value="26485.33532820411">Üsküdar</option>
    <option value="-18732.62070553167">Şile</option>
  </select>

  <select id="odasalon">
    <option value="0">Choose room number</option>
    <option value="5940.027623576261">1+0</option>
    <option value="-1399.625767475593">1+1</option>
    <option value="-3033.638999060459">2+1</option>
    <option value="11519.83954756076">3+1</option>
    <option value="-7018.52116545078">4+1</option>
    <option value="-6008.081239150166">5+1</option>
  </select>

  m2: <input type="text" id="m2" /> Bina Yasi: <input type="text" id="binayasi" />

 Features:

  <input type="checkbox" id="Bati" value="Bati" price="-783.4452683566533" /> Bati

  <input type="checkbox" id="Dogu" value="Dogu" price="-4895.732009084263" /> Dogu

  <input type="checkbox" id="Guney" value="Guney" price="-2599.001349652765" /> Guney

  <input type="checkbox" id="Kuzey" value="Kuzey" price="3832.013070771234" /> Kuzey

  <input type="checkbox" id="Amerikan Mutfak" value="Amerikan Mutfak" price="2346.347889828301" /> Amerikan Mutfak

  <input type="checkbox" id="Dusakabin" value="Dusakabin" price="-176.5092353740256" /> Dusakabin

  <input type="checkbox" id="Akilli Ev" value="Akilli Ev" price="-4756.206384200719" /> Akilli Ev

  <input type="checkbox" id="Ebeveyn Banyosu" value="Ebeveyn Banyosu" price="10135.50644456019" /> Ebeveyn Banyosu

  <input type="checkbox" id="Kiler" value="Kiler" price="-10693.49979821809" /> Kiler

  <input type="checkbox" id="Kuvet" value="Kuvet" price="-941.0629741243247" /> Kuvet

  <input type="checkbox" id="Parke Zemin" value="Parke Zemin" price="-8453.847199707092" /> Parke Zemin

  <input type="checkbox" id="Seramik Zemin" value="Seramik Zemin" price="1545.025601000643" /> Seramik Zemin

  <input type="checkbox" id="Sauna" value="Sauna" price="8677.708176289747" /> Sauna

  <input type="checkbox" id="Spor Salonu" value="Spor Salonu" price="1159.884679563385" /> Spor Salonu

  <input type="checkbox" id="Tenis Kortu" value="Tenis Kortu" price="16065.26794172257" /> Tenis Kortu

<input type="text" id="total" value="0" />

Body:
Similar to Barmar's, I added all elements to the .each() loop but also added the .change() handler to all elements:

var context = $('#housepriceguesslist');
var elems = $('input:checkbox, input:text, select', context);

function calcAndShowTotal() {
  var total = 55435.04798; // it is constant value which must be the beginning value.
  elems.each(function() {
    if ( $(this).is('input:checkbox:checked') ) {
      total += parseFloat($(this).attr('price')) || 0;
    } else if ( $(this).is('#m2') ) {
      total += parseFloat($(this).val()) * 3134.204499 || 0;
    } else if ( $(this).is('#binayasi') ) {
      total += parseFloat($(this).val()) * -566.2202888 || 0;
    } else if ( $(this).is('select') ) {
      total += parseFloat($('option:selected',this).val()) || 0;
    }
  });
  $('#total').val(total);
}

elems.change(calcAndShowTotal); // handlers
calcAndShowTotal(); // first run

See example here: https://jsfiddle.net/qgvsern4/1/


问:
Url/Link validation
Is there a way for link/url validation in Fabric when a user inputs some value in a TextField without Regex?

```<code>  public render(): JSX.Element {
    // tslint:disable:jsx-no-lambda
    return (
      <div style={{ padding: '2px' }}>

        <TextField
          label="Url"
          onGetErrorMessage={this._checkIfValid}
        />
      </div>
    );
  }

  private _checkIfValid(value: string) {
    alert("Test");
  }

</code>```

Link to codepen: https://codepen.io/lipalath/pen/oRaYvJ?editors=1111

答:
Using regular expressions should help you validate for URLs.

  public render(): JSX.Element {
    // tslint:disable:jsx-no-lambda
    return (

        <TextField
          label="Url"
          onGetErrorMessage={this._checkIfValid}
        />

    );
  }

  private _checkIfValid(value: string) {
    const regex = new Regexp(/^(?:http(s)?:\/\/)?[\w.-]+(?:\.[\w\.-]+)+[\w\-\._~:/?#[\]@!\$&'\(\)\*\+,;=.]+$/, 'g');
    return regex.test(value);
  }
```

Title: Url/Link validation

Body:
Is there a way for link/url validation in Fabric when a user inputs some value in a TextField without Regex?

  public render(): JSX.Element {
    // tslint:disable:jsx-no-lambda
    return (

        <TextField
          label="Url"
          onGetErrorMessage={this._checkIfValid}
        />

    );
  }

  private _checkIfValid(value: string) {
    alert("Test");
  }

Link to codepen: https://codepen.io/lipalath/pen/oRaYvJ?editors=1111

Body:
Using regular expressions should help you validate for URLs.

  public render(): JSX.Element {
    // tslint:disable:jsx-no-lambda
    return (
      <div style={{ padding: '2px' }}>

        <TextField
          label="Url"
          onGetErrorMessage={this._checkIfValid}
        />
      </div>
    );
  }

  private _checkIfValid(value: string) {
    const regex = new Regexp(/^(?:http(s)?:\/\/)?[\w.-]+(?:\.[\w\.-]+)+[\w\-\._~:/?#[\]@!\$&'\(\)\*\+,;=.]+$/, 'g');
    return regex.test(value);
  }


问:
How to make python functions bind enclosing variable to value rather than to name
In python functions its perfectly legal to reference variables from enclosing scopes. These variables refer to whatever value they posses at the time the function is called. So, for instance

x = 1 # unused
def foo(y):
    return x+y

x=10
print(foo(1)) # prints 11
x=11
print(foo(1)) # prints 12
```

<p>is it possible to make the variable <code>x</code> be frozen to whatever the value of <code>x</code> was at creation time? That is, I want the <code>x</code> in <code>foo</code> to bind to the value of the outer <code>x</code> rather than its name. Basically the behavior I want is</p>

<pre class="lang-py prettyprint-override"><code>x = 1
def foo(y):
    return 1+y # I'd rather not hard code the value of x here

x=10
print(foo(1)) # prints 2
x=11
print(foo(1)) # prints 2
</code>```

I currently have a working solution, but It's not very readable, I think there might be a better solution (a decorator perhaps?)

my solution

x = 1
def wrapper():
    local_x = x
    def foo(y):
        return local_x+y
    return foo
foo = wrapper()

x=10
print(foo(1)) # prints 2
x=11
print(foo(1)) # prints 2
```

答:
Use functools.partial to apply the first argument and get a new function accepting one argument. I chose not to decorate the add function, as it is very general and useful elsewhere.

```<code>from functools import partial
from operator import add       # does same thing as your foo

x = 1
foo = partial(add, x)
print(foo(1)) # prints 2

x = 11

print(foo(1)) # prints 2
</code>```

In keeping with your original solution, you could also do something like.

```<code>def wrapper(localx):
    def foo(y):
        return localx + y
    return foo

foo2 = wrapper(x)
print(foo2(1)) # prints 12
</code>```

Note that now the dependency on a global is cut and you could even pass a literal value to wrapper (e.g. foo3 = wrapper(5)).

Body:
In python functions its perfectly legal to reference variables from enclosing scopes. These variables refer to whatever value they posses at the time the function is called. So, for instance

x = 1 # unused
def foo(y):
    return x+y

x=10
print(foo(1)) # prints 11
x=11
print(foo(1)) # prints 12

is it possible to make the variable x be frozen to whatever the value of x was at creation time? That is, I want the x in foo to bind to the value of the outer x rather than its name. Basically the behavior I want is

x = 1
def foo(y):
    return 1+y # I'd rather not hard code the value of x here

x=10
print(foo(1)) # prints 2
x=11
print(foo(1)) # prints 2

I currently have a working solution, but It's not very readable, I think there might be a better solution (a decorator perhaps?)

my solution

x = 1
def wrapper():
    local_x = x
    def foo(y):
        return local_x+y
    return foo
foo = wrapper()

x=10
print(foo(1)) # prints 2
x=11
print(foo(1)) # prints 2

Body:
Use functools.partial to apply the first argument and get a new function accepting one argument. I chose not to decorate the add function, as it is very general and useful elsewhere.

from functools import partial
from operator import add       # does same thing as your foo

x = 1
foo = partial(add, x)
print(foo(1)) # prints 2

x = 11

print(foo(1)) # prints 2

In keeping with your original solution, you could also do something like.

def wrapper(localx):
    def foo(y):
        return localx + y
    return foo

foo2 = wrapper(x)
print(foo2(1)) # prints 12

Note that now the dependency on a global is cut and you could even pass a literal value to wrapper (e.g. foo3 = wrapper(5)).


问:
adb server version doesn't match this client
Whenever I try to run adb devices:

```<code>            $ adb devices
            * daemon not running. starting it now *
            * daemon started successfully *
            List of devices attached
            HT0ANRV05740    device
</code>```

It says daemon is not running and restarts the daemon.

Then, if I run adb devices again, it does the same thing -

```<code>            $ adb devices
            adb server is out of date.  killing...
            * daemon started successfully *
            List of devices attached
            HT0ANRV05740    device
</code>```

Then, if I run it again, it again does the exact same thing -

```<code>            $ adb devices
            adb server is out of date.  killing...
            * daemon started successfully *
            List of devices attached
            HT0ANRV05740    device
</code>```

Please help!!

Also, my DDMS keeps giving me the following message -

```<code>[2011-02-23 16:17:05 - DeviceMonitor]Adb connection Error:An existing connection was forcibly closed by the remote host
</code>```

Here are the logs from just before adb restarts -

```<code>              1291        locapi_rpc_glue  V  loc_ioctl
              1291                    RPC  D  written RPC packet size: [480]
              1291                    RPC  D  read RPC packet
              1291                    RPC  D  read RPC packet size: [28]
              1291             lib_locapi  V  qct_loc_eng_inject_xtra_data, inject part = 100, len = 167, len = 167
              1291             lib_locapi  V  qct_loc_eng_inject_xtra_data, total part = 100, len = 167
              1291        locapi_rpc_glue  V  loc_ioctl
              1291                    RPC  D  written RPC packet size: [248]
              1291                    RPC  D  read RPC packet
              1291                    RPC  D  read RPC packet size: [28]
              1291             lib_locapi  D  qct_loc_eng_inject_xtra_data: injected 39767, SUCCESS
              1291             lib_locapi  V  inject_xtra_waiting = flase
              1291    GpsLocationProvider  D  Releasing wakelock
              1291            WifiService  D  releaseWifiLockLocked: WifiLock{NetworkLocationProvider type=2 binder=android.os.Binder@47aead50}
              1291            WifiService  D  enable and start wifi due to updateWifiState
              1218            rmt_storage  I  rmt_storage open event
              1218            rmt_storage  I  rmt_storage client thread started
              1218            rmt_storage  I  rmt_storage events processing done
              1218            rmt_storage  I  rmt_storage register cb event
              1218            rmt_storage  I  rmt_storage status id = 2
              1218            rmt_storage  I  rmt_storage events processing done
              1218            rmt_storage  I  rmt_storage open event
              1218            rmt_storage  I  rmt_storage client thread started
              1218            rmt_storage  I  rmt_storage events processing done
              1218            rmt_storage  I  rmt_storage register cb event
              1218            rmt_storage  I  rmt_storage status id = 2
              1218            rmt_storage  I  rmt_storage events processing done
              1218            rmt_storage  I  rmt_storage write event
              1218            rmt_storage  I  unblock rmt_storage client thread
              1218            rmt_storage  I  rmt_storage events processing done
              1218            rmt_storage  I  rmt_storage write: bytes written = 3145216
              1218            rmt_storage  I  rmt_storage write: bytes written = 512
              1218            rmt_storage  I  rmt_storage status handle = 1
              1291       BatteryStatsImpl  I  notePhoneSignalStrengthLocked: 4->3
              1218            rmt_storage  I  rmt_storage write event
              1218            rmt_storage  I  rmt_storage events processing done
              1218            rmt_storage  I  unblock rmt_storage client thread
              1291    PowerManagerService  D  New lightsensor value:40, lcdValue:143
              1291    PowerManagerService  D  lightSensorChangedLocked, buttonValue >= 0, mPowerState = 3
              1218            rmt_storage  I  rmt_storage write: bytes written = 3145216
              1218            rmt_storage  I  rmt_storage write: bytes written = 512
              1218            rmt_storage  I  rmt_storage status handle = 2
              1291                    RPC  D  read RPC packet
              1291                    RPC  D  read RPC packet size: [80]
              1291        locapi_rpc_glue  V  Callback received: 80 (cb_id=0x50B0000 handle=1)
              1291                    RPC  D  written RPC packet size: [28]
              1470   usicWidgetController  D  unbindMusicPlaybackService()
              1707   MediaPlaybackService  E  BadQueue mPlayListLen : 0 mAlbumListLen : 0 mShrinkAlbumListLen : 0
              1291    NotificationService  V  Battery Full, Charging
              1291                 lights  E  write ok string=0,len=1
              1291                 lights  E  write ok string=0,len=1
              1291                 lights  E  write ok string=0 0,len=3
              1291                 lights  E  write ok string=1,len=1
              1707       MediaPlayer.java  D  setOnCompletionListener being cleaned to null
              1291                 lights  E  write ok string=0,len=1
              1291                 lights  E  write ok string=0 0,len=3
              1291    NotificationService  V  Turn off Jogball/OJ LED
              1291    NotificationService  D  cancelNotification, ACTION_NOTIFICATION_REMOVE,pkg=com.htc.music,id=1
              1291           AudioService  I   AudioFocus  abandonAudioFocus() from android.media.AudioManager@476ddee0com.htc.music.MediaPlaybackService$5@476dd838
</code>```

答:
I have seen the following error twice in the past week or so:

```<code>adb server version (40) doesn't match this client (41)
</code>```

I fixed it by rebooting my PC every time.  I do not know how to explain it.

I tried killing and starting adb via command line multiple timea, restarting Android Studio multiple times to no avail.  Task Manager does not show adb.exe.

I am running Android Studio 3.4 on Windows 10 (V1809, Build:17763.503)

Edit (2019-05-30):
When this happened for the 3rd time, I closed all applications, then restarted Android Studio, and the problem was solved.  It seems that another application causes this.   I will report back once I find the culprit.

Edit (2019-05-30): I have finally found the culprit - Visual Studio paired with a Macbook Pro for building and debugging an iOS app.  Once VS is closed or the pairding is disabled, adb becomes normal.

ContentLicense: CC BY-SA 3.0

Body:
Whenever I try to run adb devices:

            $ adb devices
            * daemon not running. starting it now *
            * daemon started successfully *
            List of devices attached
            HT0ANRV05740    device

It says daemon is not running and restarts the daemon.

Then, if I run adb devices again, it does the same thing -

            $ adb devices
            adb server is out of date.  killing...
            * daemon started successfully *
            List of devices attached
            HT0ANRV05740    device

Then, if I run it again, it again does the exact same thing -

            $ adb devices
            adb server is out of date.  killing...
            * daemon started successfully *
            List of devices attached
            HT0ANRV05740    device

Please help!!

Also, my DDMS keeps giving me the following message -

[2011-02-23 16:17:05 - DeviceMonitor]Adb connection Error:An existing connection was forcibly closed by the remote host

Here are the logs from just before adb restarts -

              1291        locapi_rpc_glue  V  loc_ioctl
              1291                    RPC  D  written RPC packet size: [480]
              1291                    RPC  D  read RPC packet
              1291                    RPC  D  read RPC packet size: [28]
              1291             lib_locapi  V  qct_loc_eng_inject_xtra_data, inject part = 100, len = 167, len = 167
              1291             lib_locapi  V  qct_loc_eng_inject_xtra_data, total part = 100, len = 167
              1291        locapi_rpc_glue  V  loc_ioctl
              1291                    RPC  D  written RPC packet size: [248]
              1291                    RPC  D  read RPC packet
              1291                    RPC  D  read RPC packet size: [28]
              1291             lib_locapi  D  qct_loc_eng_inject_xtra_data: injected 39767, SUCCESS
              1291             lib_locapi  V  inject_xtra_waiting = flase
              1291    GpsLocationProvider  D  Releasing wakelock
              1291            WifiService  D  releaseWifiLockLocked: WifiLock{NetworkLocationProvider type=2 binder=android.os.Binder@47aead50}
              1291            WifiService  D  enable and start wifi due to updateWifiState
              1218            rmt_storage  I  rmt_storage open event
              1218            rmt_storage  I  rmt_storage client thread started
              1218            rmt_storage  I  rmt_storage events processing done
              1218            rmt_storage  I  rmt_storage register cb event
              1218            rmt_storage  I  rmt_storage status id = 2
              1218            rmt_storage  I  rmt_storage events processing done
              1218            rmt_storage  I  rmt_storage open event
              1218            rmt_storage  I  rmt_storage client thread started
              1218            rmt_storage  I  rmt_storage events processing done
              1218            rmt_storage  I  rmt_storage register cb event
              1218            rmt_storage  I  rmt_storage status id = 2
              1218            rmt_storage  I  rmt_storage events processing done
              1218            rmt_storage  I  rmt_storage write event
              1218            rmt_storage  I  unblock rmt_storage client thread
              1218            rmt_storage  I  rmt_storage events processing done
              1218            rmt_storage  I  rmt_storage write: bytes written = 3145216
              1218            rmt_storage  I  rmt_storage write: bytes written = 512
              1218            rmt_storage  I  rmt_storage status handle = 1
              1291       BatteryStatsImpl  I  notePhoneSignalStrengthLocked: 4->3
              1218            rmt_storage  I  rmt_storage write event
              1218            rmt_storage  I  rmt_storage events processing done
              1218            rmt_storage  I  unblock rmt_storage client thread
              1291    PowerManagerService  D  New lightsensor value:40, lcdValue:143
              1291    PowerManagerService  D  lightSensorChangedLocked, buttonValue >= 0, mPowerState = 3
              1218            rmt_storage  I  rmt_storage write: bytes written = 3145216
              1218            rmt_storage  I  rmt_storage write: bytes written = 512
              1218            rmt_storage  I  rmt_storage status handle = 2
              1291                    RPC  D  read RPC packet
              1291                    RPC  D  read RPC packet size: [80]
              1291        locapi_rpc_glue  V  Callback received: 80 (cb_id=0x50B0000 handle=1)
              1291                    RPC  D  written RPC packet size: [28]
              1470   usicWidgetController  D  unbindMusicPlaybackService()
              1707   MediaPlaybackService  E  BadQueue mPlayListLen : 0 mAlbumListLen : 0 mShrinkAlbumListLen : 0
              1291    NotificationService  V  Battery Full, Charging
              1291                 lights  E  write ok string=0,len=1
              1291                 lights  E  write ok string=0,len=1
              1291                 lights  E  write ok string=0 0,len=3
              1291                 lights  E  write ok string=1,len=1
              1707       MediaPlayer.java  D  setOnCompletionListener being cleaned to null
              1291                 lights  E  write ok string=0,len=1
              1291                 lights  E  write ok string=0 0,len=3
              1291    NotificationService  V  Turn off Jogball/OJ LED
              1291    NotificationService  D  cancelNotification, ACTION_NOTIFICATION_REMOVE,pkg=com.htc.music,id=1
              1291           AudioService  I   AudioFocus  abandonAudioFocus() from android.media.AudioManager@476ddee0com.htc.music.MediaPlaybackService$5@476dd838

Body:
I have seen the following error twice in the past week or so:

adb server version (40) doesn't match this client (41)

I fixed it by rebooting my PC every time.  I do not know how to explain it.

I tried killing and starting adb via command line multiple timea, restarting Android Studio multiple times to no avail.  Task Manager does not show adb.exe.

I am running Android Studio 3.4 on Windows 10 (V1809, Build:17763.503)

Edit (2019-05-30):
When this happened for the 3rd time, I closed all applications, then restarted Android Studio, and the problem was solved.  It seems that another application causes this.   I will report back once I find the culprit.

Edit (2019-05-30): I have finally found the culprit - Visual Studio paired with a Macbook Pro for building and debugging an iOS app.  Once VS is closed or the pairding is disabled, adb becomes normal.


问:
Pure GWT, events converting
So, I have a ‘FlexTable’ in GWT and I need to drop down list of actions on right click on some cell. On left mouse click to retrieve ‘rowIndex’ of my cell I simply use ClickEvent method ‘getCellForEvent(event).getRowIndex()’. But there is no right click handler in pure GWT. So I decided to use ContextMenuHandler which requires ContextMenuEvent. And, of course, I can’t put ContextMenuEvent into ClickEvent method ‘getCellForEvent’. Is there any solution to this case? Or maybe someone knows easier way to drop down list on rightClick on ‘FlexTable’.</p>

答:
If you use a CellTable or DataGrid Widget, you can handle the CellPreviewEvent that gets fired when a cell is clicked, and that can also tell you which row/cell the event happened in.  Here is some code:

```<code>private void makeItemRightClickListener()
{
    grid.addCellPreviewHandler(new CellPreviewEvent.Handler<T>() {
        @Override
        public void onCellPreview(CellPreviewEvent<T> event)
        {
            if (event.getNativeEvent().getButton() !=
                        NativeEvent.BUTTON_RIGHT)
                return;

            // Prevent browser's own context menu from appearing
            event.getNativeEvent().preventDefault();
            event.getNativeEvent().stopPropagation();

            handleItemRightClickEvent(event);
        }
    });
}

private void handleItemRightClickEvent(CellPreviewEvent<T> event)
{
    NativeEvent nativeClickEvent = event.getNativeEvent();
    // Get the data (type T) that is being displayed in the cell
    // by the CellTable or DataGrid widget.
    T rowClicked = event.getValue();

    // Create PopupPanel for menu
    PopupPanel popup = ...

    // Show the popup menu at the click position
    UIObject target = new MousePointUIObject(nativelickEvent);
    popup.showRelativeTo(target);
}

private static class MousePointUIObject extends UIObject
{
    private NativeEvent mouseEvent;

    public MousePointUIObject(NativeEvent mouseEvent)
    {
        this.mouseEvent = mouseEvent;
    }

    @Override
    public int getAbsoluteLeft()
    {
        return mouseEvent.getClientX() + Window.getScrollLeft();
    }

    @Override
    public int getAbsoluteTop()
    {
        return mouseEvent.getClientY() + Window.getScrollTop();
    }

    @Override
    public int getOffsetWidth()
    {
        return 0;
    }

    @Override
    public int getOffsetHeight()
    {
        return 0;
    }
}
</code>```

Title: Pure GWT, events converting

Body:
If you use a CellTable or DataGrid Widget, you can handle the CellPreviewEvent that gets fired when a cell is clicked, and that can also tell you which row/cell the event happened in.  Here is some code:

private void makeItemRightClickListener()
{
    grid.addCellPreviewHandler(new CellPreviewEvent.Handler<T>() {
        @Override
        public void onCellPreview(CellPreviewEvent<T> event)
        {
            if (event.getNativeEvent().getButton() !=
                        NativeEvent.BUTTON_RIGHT)
                return;

            // Prevent browser's own context menu from appearing
            event.getNativeEvent().preventDefault();
            event.getNativeEvent().stopPropagation();

            handleItemRightClickEvent(event);
        }
    });
}

private void handleItemRightClickEvent(CellPreviewEvent<T> event)
{
    NativeEvent nativeClickEvent = event.getNativeEvent();
    // Get the data (type T) that is being displayed in the cell
    // by the CellTable or DataGrid widget.
    T rowClicked = event.getValue();

    // Create PopupPanel for menu
    PopupPanel popup = ...

    // Show the popup menu at the click position
    UIObject target = new MousePointUIObject(nativelickEvent);
    popup.showRelativeTo(target);
}

private static class MousePointUIObject extends UIObject
{
    private NativeEvent mouseEvent;

    public MousePointUIObject(NativeEvent mouseEvent)
    {
        this.mouseEvent = mouseEvent;
    }

    @Override
    public int getAbsoluteLeft()
    {
        return mouseEvent.getClientX() + Window.getScrollLeft();
    }

    @Override
    public int getAbsoluteTop()
    {
        return mouseEvent.getClientY() + Window.getScrollTop();
    }

    @Override
    public int getOffsetWidth()
    {
        return 0;
    }

    @Override
    public int getOffsetHeight()
    {
        return 0;
    }
}


问:
How to drop column in your dataset with pandas?
Now I'm taking a machine learning course. Now I should import my dataset after I read a "CSV" file to import my dataset I want to drop a column but I don't know how to do it.</p>

答:
Let's assume LIKES as column, how to drop it?</p>
```<code>df.drop('LIKES', axis=1, inplace=True)
</code>```

Body:
Let's assume LIKES as column, how to drop it?

df.drop('LIKES', axis=1, inplace=True)


问:
Incrementing counter in Stream findfirst Java 8
There is a raw text stored in an ArrayList. To clean away unwanted lines from the top, I need to find the index of a string containing a certain tag.

The tag is not a complete String, only a substring. This means I cannot use myArrayList.indexOf("start marker")

(indexOf only looks for complete strings)

I tried to put a counter on a stream, counting the Strings it processes until it finds the marker, but it is not working.

Using a for-loop i can find the index, but this is slow and ugly (also gave some memory errors).

Streaming the ArrayList through a filter, and having the stream return the String that contains the marker gives me the full String that I'm looking for.
Having the full string enables the ArrayList.indexOf method.

All in all it works now, but I still want to know how to set a counter on a stream.

For-loop (works but ugly):

```<code>    for(String s : myArrayList)
    {
        if(s.contains("marker text"))
        {
            break;
        }
        else counter++;
    }
</code>```

Finding the full string (decent work-around):

```<code>    String markerText = myArrayList.stream().filter(s -> s.contains("marker text")).
    findFirst().orElse("marker not found");
    int x = myArrayList.indexOf(markerText);
</code>```

This doesn't work:

```<code>    int counter = 0;
    myArrayList.stream().peek(counter++).
    filter(s -> s.contains("marker text")).findFirst();
</code>```

This does not compile as peek() refuses to operate on integer x.

I have also tried casting x as an AtomicInteger and calling the getAndIncrement() method. This does not work either.

I see a lot of similar topics, but they all use the foreach method. Can someone tell me what I'm doing wrong?

答:
You could get the index in one line using IntStream :

```<code>OptionalInt index = IntStream.range(0, list.size()).filter(i -> list.get(i).contains("marker text")).findFirst();
</code>```

Then work with it:

```<code>if(index.isPresent())
    System.out.println("I found " + list.get(index.getAsInt()) + "!");
else
    System.out.println("I didn't find any match!");
</code>```

Body:
There is a raw text stored in an ArrayList. To clean away unwanted lines from the top, I need to find the index of a string containing a certain tag.

The tag is not a complete String, only a substring. This means I cannot use myArrayList.indexOf("start marker")

(indexOf only looks for complete strings)

I tried to put a counter on a stream, counting the Strings it processes until it finds the marker, but it is not working.

Using a for-loop i can find the index, but this is slow and ugly (also gave some memory errors).

Streaming the ArrayList through a filter, and having the stream return the String that contains the marker gives me the full String that I'm looking for.
Having the full string enables the ArrayList.indexOf method.

All in all it works now, but I still want to know how to set a counter on a stream.

For-loop (works but ugly):

    for(String s : myArrayList)
    {
        if(s.contains("marker text"))
        {
            break;
        }
        else counter++;
    }

Finding the full string (decent work-around):

    String markerText = myArrayList.stream().filter(s -> s.contains("marker text")).
    findFirst().orElse("marker not found");
    int x = myArrayList.indexOf(markerText);

This doesn't work:

    int counter = 0;
    myArrayList.stream().peek(counter++).
    filter(s -> s.contains("marker text")).findFirst();

This does not compile as peek() refuses to operate on integer x.

I have also tried casting x as an AtomicInteger and calling the getAndIncrement() method. This does not work either.

I see a lot of similar topics, but they all use the foreach method. Can someone tell me what I'm doing wrong?

Body:
You could get the index in one line using IntStream :

OptionalInt index = IntStream.range(0, list.size()).filter(i -> list.get(i).contains("marker text")).findFirst();

Then work with it:

if(index.isPresent())
    System.out.println("I found " + list.get(index.getAsInt()) + "!");
else
    System.out.println("I didn't find any match!");


问:
How to stop a Airflow DAG run from task
so I have an airflow DAG task which performs several checks on a file (e.g., if it was already processed in the past) and I want the whole DAG run to stop completely if one of the checks fails. How can I do that?

Basically the checks are done in a python script.

Thx :

答:
Assuming you have no downstream tasks that ignore the failure of a previous task, raising an exception from within the task will cause the DAG to stop processing and be marked as failed.

So in your case, you can raise an exception when you find that the file has already been processed.

Title: How to stop a Airflow DAG run from task


问:
React Hook useEffect for fetching and posting to Backend (API)?
That’s more a question to know more of how you guys commonly do it as a problem. I’m currently having no problem with the code, but I’m not pretty sure that’s the right way to do that.

In my code I have a fetch (with Axios) for the data in the Backend (a simple list with objects). And for that I’m using the new React hooks with useEffect, works like a charm:

```<code>useEffect(() => {
    axios
      .get("http://localhost:5000/api/list")
      .then(result => {
        setExampleList(result.data);
      })
      .catch(error => console.log(error));
  }, [setExampleList]);
</code>```

But I also have a screen where the process to add to that list in Backend comes. And for that I’m handling the form submit also with axios in Backend. And that’s the “problem”. It works and goes to Backend but my fetch was not updating, because it was not rerendered or something. Then I came up with the ideia to call the setData() where the fetching lives  inside the function for adding a new item (after posting it):

```<code>function handleSubmit(event) {
    event.preventDefault();
    let obj = {
      name: name,
      city: city
    };
    axios
      .post("http://localhost:5000/api/add", obj)
      .then(response => {
        console.log(response.data);
        obj.id = response.data.inserted_id;
        setExampleList([...setExampleList, obj]);
      })
      .catch(error => {
        console.log(error);
      });
    handleCloseModal();
  }
</code>```

And that works, but as you can imagine, a little “fake” because the data is not really what in Backend is until I reload the page. So, have you guys done something in this way? What would be the best solution in that case?

答:
It's common practice to "fake" update a list even if you didn't have a refresh problem. This is because you want your app to seem fast to the user. If the backend request take 5 sec to complete and then updates the screen the user is going to think your app is broken.

In chat apps it is extremely common to show a message as sent before it actual gets sent. The same concept is used a lot for deleting items. The item gets removed from the screen immediately before the backend is updated and a refresh is not needed.

You may want to add an UI message that tells the user if the submit failed so they don't refresh the page and wonder why the list is missing new items.

So to answer your questions

Have you guys done something in this way? Yep!

What would be the best solution in that case? If the list is for a single user then your solution is great.

That being said if you are expecting multiple users to be updating the list at the same time then you are going to want to sync the list before a page refresh.

Body:
That’s more a question to know more of how you guys commonly do it as a problem. I’m currently having no problem with the code, but I’m not pretty sure that’s the right way to do that.

In my code I have a fetch (with Axios) for the data in the Backend (a simple list with objects). And for that I’m using the new React hooks with useEffect, works like a charm:

useEffect(() => {
    axios
      .get("http://localhost:5000/api/list")
      .then(result => {
        setExampleList(result.data);
      })
      .catch(error => console.log(error));
  }, [setExampleList]);

But I also have a screen where the process to add to that list in Backend comes. And for that I’m handling the form submit also with axios in Backend. And that’s the “problem”. It works and goes to Backend but my fetch was not updating, because it was not rerendered or something. Then I came up with the ideia to call the setData() where the fetching lives  inside the function for adding a new item (after posting it):

function handleSubmit(event) {
    event.preventDefault();
    let obj = {
      name: name,
      city: city
    };
    axios
      .post("http://localhost:5000/api/add", obj)
      .then(response => {
        console.log(response.data);
        obj.id = response.data.inserted_id;
        setExampleList([...setExampleList, obj]);
      })
      .catch(error => {
        console.log(error);
      });
    handleCloseModal();
  }

And that works, but as you can imagine, a little “fake” because the data is not really what in Backend is until I reload the page. So, have you guys done something in this way? What would be the best solution in that case?


问:
Pandas: Faster way to do complex column transposition with pivot function
Simply put, I need to convert the following input dataframe to the output below.

After a few hours struggling to figure out how, by combining multiple previous stackoverflow questions, I could transform dataframe, but it takes so much time for large dataframe to convert since I use pivot and apply method.

```<code>import numpy as np
import pandas as pd
df = pd.DataFrame({"id":[1,2,3,4,5],
                  "day":pd.Timestamp('20190529'),
                  "subject":"math,english,economics",
                  "score":pd.Categorical(["68,62,49","58,72,87","28,32,46","48,72,66","46,25,93"]),
                  "Department":pd.Categorical(["Economics","Computer Science","Sociology","Business","Math"])})

---Input DataFrame---

    id    day           subject                score       Department
0   1   2019-05-29  math,english,economics    68,62,49    Economics
1   2   2019-05-29  math,economics             58,87       Computer Science
2   3   2019-05-29  philosophy,english,business 28,32,46    Sociology
3   4   2019-05-29  physics,sociology           72,66      Business
4   5   2019-05-29  Math                          93          Math
</code>```

And output is like the one below

```<code>---Output DataFrame---

id  day         Department          Math business economics english math philosophy physics sociology
1   2019-05-29  Economics           NaN NaN 49  62  68  NaN NaN NaN
2   2019-05-29  Computer Science    NaN NaN 87  NaN 58  NaN NaN NaN
3   2019-05-29  Sociology           NaN 46  NaN 32  NaN 28  NaN NaN
4   2019-05-29  Business            NaN NaN NaN NaN NaN NaN 72  66
5   2019-05-29  Math                93  NaN NaN NaN NaN NaN NaN NaN
</code>```

And my approach is

I split subject and score columns by ','.

Explode the list of elements in subject and score columns to separate row as pandas.Series

Join each pandas.Series to make new dataframe

Pivot new dataframe created in step 3

Drop subject and score column in original dataframe

Join each dataframe made in step 4 and step 5

And my code is as follows

```<code>df["subject"] = df["subject"].str.split(",")
df["score"] = df["score"].str.split(",")

subject = df.apply(lambda x: pd.Series(x['subject']),axis=1).stack().reset_index(level=1, drop=True)
score = df.apply(lambda x: pd.Series(x['score']),axis=1).stack().reset_index(level=1, drop=True)
subject.name = 'subject'
score.name = 'score'

subject_score = pd.concat([subject, score],join='outer', axis=1)
pdf = df.drop('subject', axis=1).drop("score", axis=1).join(subject_score)

pivot = pdf.pivot(columns="subject",values="score")
concate_table = df.drop("subject",axis = 1).drop("score", axis=1)
output = concate_table.join(pivot)
</code>```

I just started learning pandas recently and I am sure this wouldn't be best way to column transposition.

If you could give me some advice how to optimize this code, I appreciate.

Thank you in advance.

答:
I am using unnesting then with pivot and concat back

```<code>s=df[['subject','score']]
s=unnesting(s.apply(lambda x :x.str.split(','),1),['subject','score']).pivot(columns='subject',values='score')
df=pd.concat([df,s],axis=1)
</code>```

```<code>def unnesting(df, explode):
    idx = df.index.repeat(df[explode[0]].str.len())
    df1 = pd.concat([
        pd.DataFrame({x: np.concatenate(df[x].values)}) for x in explode], axis=1)
    df1.index = idx

    return df1.join(df.drop(explode, 1), how='left')
</code>```

Body:
Simply put, I need to convert the following input dataframe to the output below.

After a few hours struggling to figure out how, by combining multiple previous stackoverflow questions, I could transform dataframe, but it takes so much time for large dataframe to convert since I use pivot and apply method.

import numpy as np
import pandas as pd
df = pd.DataFrame({"id":[1,2,3,4,5],
                  "day":pd.Timestamp('20190529'),
                  "subject":"math,english,economics",
                  "score":pd.Categorical(["68,62,49","58,72,87","28,32,46","48,72,66","46,25,93"]),
                  "Department":pd.Categorical(["Economics","Computer Science","Sociology","Business","Math"])})

---Input DataFrame---

    id    day           subject                score       Department
0   1   2019-05-29  math,english,economics    68,62,49    Economics
1   2   2019-05-29  math,economics             58,87       Computer Science
2   3   2019-05-29  philosophy,english,business 28,32,46    Sociology
3   4   2019-05-29  physics,sociology           72,66      Business
4   5   2019-05-29  Math                          93          Math

And output is like the one below

---Output DataFrame---

id  day         Department          Math business economics english math philosophy physics sociology
1   2019-05-29  Economics           NaN NaN 49  62  68  NaN NaN NaN
2   2019-05-29  Computer Science    NaN NaN 87  NaN 58  NaN NaN NaN
3   2019-05-29  Sociology           NaN 46  NaN 32  NaN 28  NaN NaN
4   2019-05-29  Business            NaN NaN NaN NaN NaN NaN 72  66
5   2019-05-29  Math                93  NaN NaN NaN NaN NaN NaN NaN

And my approach is

I split subject and score columns by ','.

Explode the list of elements in subject and score columns to separate row as pandas.Series

Join each pandas.Series to make new dataframe

Pivot new dataframe created in step 3

Drop subject and score column in original dataframe

Join each dataframe made in step 4 and step 5

And my code is as follows

df["subject"] = df["subject"].str.split(",")
df["score"] = df["score"].str.split(",")

subject = df.apply(lambda x: pd.Series(x['subject']),axis=1).stack().reset_index(level=1, drop=True)
score = df.apply(lambda x: pd.Series(x['score']),axis=1).stack().reset_index(level=1, drop=True)
subject.name = 'subject'
score.name = 'score'

subject_score = pd.concat([subject, score],join='outer', axis=1)
pdf = df.drop('subject', axis=1).drop("score", axis=1).join(subject_score)

pivot = pdf.pivot(columns="subject",values="score")
concate_table = df.drop("subject",axis = 1).drop("score", axis=1)
output = concate_table.join(pivot)

I just started learning pandas recently and I am sure this wouldn't be best way to column transposition.

If you could give me some advice how to optimize this code, I appreciate.

Thank you in advance.

Body:
I am using unnesting then with pivot and concat back

s=df[['subject','score']]
s=unnesting(s.apply(lambda x :x.str.split(','),1),['subject','score']).pivot(columns='subject',values='score')
df=pd.concat([df,s],axis=1)

def unnesting(df, explode):
    idx = df.index.repeat(df[explode[0]].str.len())
    df1 = pd.concat([
        pd.DataFrame({x: np.concatenate(df[x].values)}) for x in explode], axis=1)
    df1.index = idx

    return df1.join(df.drop(explode, 1), how='left')


问:
Is std::push_back relatively expensive to use?
I want to improve the performance of the following code. What aspect might affect the performance of the code when it's executed?

Also, considering that there is no limit to how many objects you can add to the container, what improvements could be made to “Object” or “addToContainer” to improve the performance of the program?

I was wondering if std::push_back in C++ affects performance of the code in any way? Especially if there is no limit to adding to list.

```<code>struct Object{
    string name;
    string description;
};

vector<Object> container;
void addToContainer(Object object) {
    container.push_back(object);
}

int main() {
    addToContainer({ "Fira", "+5 ATTACK" });
    addToContainer({ "Potion", "+10 HP" });
}
</code>```

答:
push_back can be extremely expensive, but as with everything, it depends on the context. Take for example this terrible code:

```<code>std::vector<float> slow_func(const float* ptr)
{
   std::vector<float> v;
   for(size_t i = 0; i < 256; ++i)
     v.push_back(ptr[i]);
   return v;
}
</code>```

each call to push_back has to do the following:

Check to see if there is enough space in the vector

If not, allocate new memory, and copy the old values into the new vector

copy the new item to the end of the vector

increment end

Now there are two big problems here wrt performance. Firstly each push_back operation depends upon the previous operation (since the previous operation modified end, and possibly the entire contents of the array if it had to be resized). This pretty much destroys any vectorisation possibilities in the code. Take a look here:

https://godbolt.org/z/RU2tM0

The func that uses push_back does not make for very pretty asm. It's effectively hamstrung into being forced to copy a single float at a time. Now if you compare that to an alternative approach where you resize first, and then assign; the compiler just replaces the whole lot with a call to new, and a call to memcpy. This will be a few orders of magnitude faster than the previous method.

```<code>std::vector<float> fast_func(const float* ptr)
{
   std::vector<float> v(256);
   for(size_t i = 0; i < 256; ++i)
     v[i] = ptr[i];
   return v;
}
</code>```

BUT, and it's a big but, the relative performance of push_back very much depends on whether the items in the array can be trivially copied (or moved). If you example you do something silly like:

```<code>struct Vec3 {
   float x = 0;
   float y = 0;
   float z = 0;
};
</code>```

Well now when we did this:

```<code>std::vector<Vec3> v(256);
</code>```

The compiler will allocate memory, but also be forced to set all the values to zero (which is pointless if you are about to overwrite them again!). The obvious way around this is to use a different constructor:

```<code>std::vector<Vec3> v(ptr, ptr + 256);
</code>```

So really, only use push_back (well, really you should prefer emplace_back in most cases) when either:

additional elements are added to your vector occasionally

or, The objects you are adding are complex to construct (in which case, use emplace_back!)

Body:
I want to improve the performance of the following code. What aspect might affect the performance of the code when it's executed?

Also, considering that there is no limit to how many objects you can add to the container, what improvements could be made to “Object” or “addToContainer” to improve the performance of the program?

I was wondering if std::push_back in C++ affects performance of the code in any way? Especially if there is no limit to adding to list.

struct Object{
    string name;
    string description;
};

vector<Object> container;
void addToContainer(Object object) {
    container.push_back(object);
}

int main() {
    addToContainer({ "Fira", "+5 ATTACK" });
    addToContainer({ "Potion", "+10 HP" });
}

Body:
push_back can be extremely expensive, but as with everything, it depends on the context. Take for example this terrible code:

std::vector<float> slow_func(const float* ptr)
{
   std::vector<float> v;
   for(size_t i = 0; i < 256; ++i)
     v.push_back(ptr[i]);
   return v;
}

each call to push_back has to do the following:

Check to see if there is enough space in the vector

If not, allocate new memory, and copy the old values into the new vector

copy the new item to the end of the vector

increment end

Now there are two big problems here wrt performance. Firstly each push_back operation depends upon the previous operation (since the previous operation modified end, and possibly the entire contents of the array if it had to be resized). This pretty much destroys any vectorisation possibilities in the code. Take a look here:

https://godbolt.org/z/RU2tM0

The func that uses push_back does not make for very pretty asm. It's effectively hamstrung into being forced to copy a single float at a time. Now if you compare that to an alternative approach where you resize first, and then assign; the compiler just replaces the whole lot with a call to new, and a call to memcpy. This will be a few orders of magnitude faster than the previous method.

std::vector<float> fast_func(const float* ptr)
{
   std::vector<float> v(256);
   for(size_t i = 0; i < 256; ++i)
     v[i] = ptr[i];
   return v;
}

BUT, and it's a big but, the relative performance of push_back very much depends on whether the items in the array can be trivially copied (or moved). If you example you do something silly like:

struct Vec3 {
   float x = 0;
   float y = 0;
   float z = 0;
};

Well now when we did this:

std::vector<Vec3> v(256);

The compiler will allocate memory, but also be forced to set all the values to zero (which is pointless if you are about to overwrite them again!). The obvious way around this is to use a different constructor:

std::vector<Vec3> v(ptr, ptr + 256);

So really, only use push_back (well, really you should prefer emplace_back in most cases) when either:

additional elements are added to your vector occasionally

or, The objects you are adding are complex to construct (in which case, use emplace_back!)


问:
Export HTML table to CSV using vanilla javascript
I am trying to add a feature of csv download option in my website. It should convert the html table present in the website in to csv content and make it downloadable. Ive been searching through internet for a good plugin and found some usefule ones like http://www.dev-skills.com/export-html-table-to-csv-file/  But it uses php script to do the download part. I was wondering if there is a pure javascript library available to do this feature using server side softwares like node.js without the use of php??</p>

答:
Should work on every modern browser and without jQuery or any dependency, here my implementation :

```<code>// Quick and simple export target #table_id into a csv
function download_table_as_csv(table_id, separator = ',') {
    // Select rows from table_id
    var rows = document.querySelectorAll('table#' + table_id + ' tr');
    // Construct csv
    var csv = [];
    for (var i = 0; i < rows.length; i++) {
        var row = [], cols = rows[i].querySelectorAll('td, th');
        for (var j = 0; j < cols.length; j++) {
            // Clean innertext to remove multiple spaces and jumpline (break csv)
            var data = cols[j].innerText.replace(/(\r\n|\n|\r)/gm, '').replace(/(\s\s)/gm, ' ')
            // Escape double-quote with double-double-quote (see https://stackoverflow.com/questions/17808511/properly-escape-a-double-quote-in-csv)
            data = data.replace(/"/g, '""');
            // Push escaped string
            row.push('"' + data + '"');
        }
        csv.push(row.join(separator));
    }
    var csv_string = csv.join('\n');
    // Download it
    var filename = 'export_' + table_id + '_' + new Date().toLocaleDateString() + '.csv';
    var link = document.createElement('a');
    link.style.display = 'none';
    link.setAttribute('target', '_blank');
    link.setAttribute('href', 'data:text/csv;charset=utf-8,' + encodeURIComponent(csv_string));
    link.setAttribute('download', filename);
    document.body.appendChild(link);
    link.click();
    document.body.removeChild(link);
}
</code>```

Then add your download button/link :

```<code><a href="#" onclick="download_table_as_csv('my_id_table_to_export');">Download as CSV</a>
</code>```

CSV file is timedated and compatible with default Excel format.

Update after comments: Added second parameter "separator", it can be used to configure another character like ;, it's useful if you have user downloading your csv in different region of the world because they can use another default separator for Excel, for more information see : https://superuser.com/a/606274/908273

Body:
Should work on every modern browser and without jQuery or any dependency, here my implementation :

// Quick and simple export target #table_id into a csv
function download_table_as_csv(table_id, separator = ',') {
    // Select rows from table_id
    var rows = document.querySelectorAll('table#' + table_id + ' tr');
    // Construct csv
    var csv = [];
    for (var i = 0; i < rows.length; i++) {
        var row = [], cols = rows[i].querySelectorAll('td, th');
        for (var j = 0; j < cols.length; j++) {
            // Clean innertext to remove multiple spaces and jumpline (break csv)
            var data = cols[j].innerText.replace(/(\r\n|\n|\r)/gm, '').replace(/(\s\s)/gm, ' ')
            // Escape double-quote with double-double-quote (see https://stackoverflow.com/questions/17808511/properly-escape-a-double-quote-in-csv)
            data = data.replace(/"/g, '""');
            // Push escaped string
            row.push('"' + data + '"');
        }
        csv.push(row.join(separator));
    }
    var csv_string = csv.join('\n');
    // Download it
    var filename = 'export_' + table_id + '_' + new Date().toLocaleDateString() + '.csv';
    var link = document.createElement('a');
    link.style.display = 'none';
    link.setAttribute('target', '_blank');
    link.setAttribute('href', 'data:text/csv;charset=utf-8,' + encodeURIComponent(csv_string));
    link.setAttribute('download', filename);
    document.body.appendChild(link);
    link.click();
    document.body.removeChild(link);
}

Then add your download button/link :

<a href="#" onclick="download_table_as_csv('my_id_table_to_export');">Download as CSV</a>

CSV file is timedated and compatible with default Excel format.

Update after comments: Added second parameter "separator", it can be used to configure another character like ;, it's useful if you have user downloading your csv in different region of the world because they can use another default separator for Excel, for more information see : https://superuser.com/a/606274/908273


问:
sql stores different value then I store
I am using sql server 2005 and .NET.
I have a table with float column named "price".
When I stores 1890 it store 1889.99966796875.

What is the reason?

答: Any number that you want to store an exact number for or do math calculations on should be stored as decimal or number not float. This is an inexact data type and it will cause rounding errors and incorrect calculations. In general, float should be avoided.</p>

Title: sql stores different value then I store


问:
Using Dictionary to Cut-Out Parts of Strings in a List
I have a long list that is generated using Beautiful Soup in Python 3.

Right now, the list is generated as so..

```<code>mylist = [a['href'] for a in soup.find_all('a', href=True) if a.text]
</code>```

It's a web scraping thing, but just know it returns a list.

And in the form of a list, it returns results such as these:

'catalogue/category/books/travel_2/index.html',

'catalogue/category/books/mystery_3/index.html',

'catalogue/category/books/historical-fiction_4/index.html'

Before I print my list, I'd like to remove various useless information (such as "catalogue/", "category/" and "books/", so that only the important information is displayed (travel, mystery or historical fiction).

I was able to replace one thing successfully using:

```<code>mylist = [item.replace("catalogue/category/", "") for item in mylist]
</code>```

Which worked great. But I don't believe .replace will take more than 2 arguments, which leaves me unable to remove additional things from my results such as "index.html" . I'd rather not write that line for everything I want to replace. That's why I'm trying to use keys and values from a dictionary as the .replace() arguments:

```<code>replacedict = {"catalogue/category/": "" , "index.html": ""}

mylist = [a['href'] for a in soup.find_all('a', href=True) if a.text]

def replace_all(mylist, replacedict):
     for k, v in replacedict.items():
         mylist = [item.replace(k, v) for item in mylist]
     return mylist

replace_all(mylist, replacedict)

print(mylist)
</code>```

Right now, the program isn't throwing any errors when it's run. But it's also simply NOT doing what i'm asking of it. It's just returning a large list of results shown above with nothing removed or replaced out of them.

Very confused, although I'm sure the answer is right in front of me.

Appreciate any and all help, couldn't find a question quite like this one anywhere.

答:
Why not just get the part of each URL that you're interested in by splitting the string into a list of strings.  E.g.:

```<code>$ python
Python 3.7.2 (default, Dec 27 2018, 07:35:06)
[Clang 10.0.0 (clang-1000.11.45.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> string_list = ['catalogue/category/books/travel_2/index.html', 'catalogue/category/books/mystery_3/index.html', 'catalogue/category/books/historical-fiction_4/index.html']
>>> array_list = [s.split('/') for s in string_list]
>>> array_list
[['catalogue', 'category', 'books', 'travel_2', 'index.html'], ['catalogue', 'category', 'books', 'mystery_3', 'index.html'], ['catalogue', 'category', 'books', 'historical-fiction_4', 'index.html']]
>>> [a[3] for a in array_list]
['travel_2', 'mystery_3', 'historical-fiction_4']
</code>```

That should work if the URLs are always structured the way you've shown.

Body:
I have a long list that is generated using Beautiful Soup in Python 3.

Right now, the list is generated as so..

mylist = [a['href'] for a in soup.find_all('a', href=True) if a.text]

It's a web scraping thing, but just know it returns a list.

And in the form of a list, it returns results such as these:

'catalogue/category/books/travel_2/index.html',

'catalogue/category/books/mystery_3/index.html',

'catalogue/category/books/historical-fiction_4/index.html'

Before I print my list, I'd like to remove various useless information (such as "catalogue/", "category/" and "books/", so that only the important information is displayed (travel, mystery or historical fiction).

I was able to replace one thing successfully using:

mylist = [item.replace("catalogue/category/", "") for item in mylist]

Which worked great. But I don't believe .replace will take more than 2 arguments, which leaves me unable to remove additional things from my results such as "index.html" . I'd rather not write that line for everything I want to replace. That's why I'm trying to use keys and values from a dictionary as the .replace() arguments:

replacedict = {"catalogue/category/": "" , "index.html": ""}

mylist = [a['href'] for a in soup.find_all('a', href=True) if a.text]

def replace_all(mylist, replacedict):
     for k, v in replacedict.items():
         mylist = [item.replace(k, v) for item in mylist]
     return mylist

replace_all(mylist, replacedict)

print(mylist)

Right now, the program isn't throwing any errors when it's run. But it's also simply NOT doing what i'm asking of it. It's just returning a large list of results shown above with nothing removed or replaced out of them.

Very confused, although I'm sure the answer is right in front of me.

Appreciate any and all help, couldn't find a question quite like this one anywhere.

Body:
Why not just get the part of each URL that you're interested in by splitting the string into a list of strings.  E.g.:

$ python
Python 3.7.2 (default, Dec 27 2018, 07:35:06)
[Clang 10.0.0 (clang-1000.11.45.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> string_list = ['catalogue/category/books/travel_2/index.html', 'catalogue/category/books/mystery_3/index.html', 'catalogue/category/books/historical-fiction_4/index.html']
>>> array_list = [s.split('/') for s in string_list]
>>> array_list
[['catalogue', 'category', 'books', 'travel_2', 'index.html'], ['catalogue', 'category', 'books', 'mystery_3', 'index.html'], ['catalogue', 'category', 'books', 'historical-fiction_4', 'index.html']]
>>> [a[3] for a in array_list]
['travel_2', 'mystery_3', 'historical-fiction_4']

That should work if the URLs are always structured the way you've shown.


问:
Saving an array in Swift
I am trying to save an array to the documents directory but the save function fails. I can tell this because it returns a value of false and when I try to read the array back I get nil.

I have included some code below I have written this with some various debug statements just trying to get it to work so sorry if ic seems a little weired.

```<code>@IBAction func Save(_ sender: Any) {

    var values = [jpyTextField.text,eurTextField.text]
    var answer: Bool

    let manager = FileManager.default
    let documents = manager.urls(for: .documentDirectory, in: .userDomainMask)
    let docURL = documents.first

    answer = (values as NSArray).write(to:docURL! , atomically: true)

    let readData = NSMutableArray (contentsOf: docURL!)

    print (docURL)
    print (readData?[0])
    print (answer)
}
</code>```

Here is what I get back

```<code>file:///Users/kka/Library/Developer/CoreSimulator/Devices/82C6DEF6-20D0-4BB0-875A-C80D4AE9A4BD/data/Containers/Data/Application/F54CBC83-6756-466F-8701-E728F5C6DBF1/Documents/
nil
false
</code>```

答:
You need to specify a file name</p>

```<code>let dir = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask).first!
let fileURL = dir.appendingPathComponent("file.txt")
// write to fileURL
</code>```

Title: Saving an array in Swift

Body:
I am trying to save an array to the documents directory but the save function fails. I can tell this because it returns a value of false and when I try to read the array back I get nil.

I have included some code below I have written this with some various debug statements just trying to get it to work so sorry if ic seems a little weired.

@IBAction func Save(_ sender: Any) {

    var values = [jpyTextField.text,eurTextField.text]
    var answer: Bool

    let manager = FileManager.default
    let documents = manager.urls(for: .documentDirectory, in: .userDomainMask)
    let docURL = documents.first

    answer = (values as NSArray).write(to:docURL! , atomically: true)

    let readData = NSMutableArray (contentsOf: docURL!)

    print (docURL)
    print (readData?[0])
    print (answer)
}

Here is what I get back

file:///Users/kka/Library/Developer/CoreSimulator/Devices/82C6DEF6-20D0-4BB0-875A-C80D4AE9A4BD/data/Containers/Data/Application/F54CBC83-6756-466F-8701-E728F5C6DBF1/Documents/
nil
false

Body:
You need to specify a file name

let dir = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask).first!
let fileURL = dir.appendingPathComponent("file.txt")
// write to fileURL


问:
Pandas - Identify if a date in a pandas column is falls on Saturday or Sunday
I want to identify if a date value is on a Saturday or Sunday. If it is, then I want to change said date to the friday prior. (example df and result df below - these are real dates from May 2019, as you can see there is a Saturday(5/4/19) and Sunday(5/26/19) date pulled into the week prior in the results df. The (5/2/19) date stays the same as it is a weekday not a weekend.

```<code>import pandas as pd
data = [['5/3/19','5/4/19'],['5/3/19','5/2/19'],['5/24/19','5/26/19']]
df = pd.DataFrame(data,columns=['Date1','Date2'])
print(df)

data2 = [['5/3/19','5/3/19'],['5/3/19','5/2/19'],['5/24/19','5/24/19']]
resultsdf = pd.DataFrame(data2,columns=['Date1','Date2'])
print(resultsdf)
</code>```

I am running this code for a Month to Date range. I have to have the code to group a large data set into a MTD view - but I am having trouble creating variables for a list of sat/sunday ( as there will be 8-10 weekend dates per month) Here is my code:

```<code>import pandas as pd
import xlrd # added when using visual studio
import datetime
from datetime import datetime
today = pd.to_datetime(datetime.now().date())
day_of_week = today.dayofweek
last_monday = today - pd.to_timedelta(day_of_week, unit='d')
month_day = today.day
start_month_scope = today - pd.to_timedelta(month_day, unit = 'd')
first_day_of_month = start_month_scope + pd.to_timedelta(1, unit = 'd')
</code>```

The problem with the code below is that is only works if I need to find the saturday & sunday on 7 day (M-f) period.

```<code>last_sunday = last_monday + pd.to_timedelta(6, unit='d')
last_sat = last_sunday - pd.to_timedelta(1, unit='d')
last_fri = last_sunday - pd.to_timedelta(2, unit='d')
</code>```

I need some variable to input to the code below:

```<code>finaldf.loc[(finaldf['Due_Date'] == last_sat), 'Due_Date'] = last_fri
</code>```

So 'last_fri' would be just the most recent friday from the given saturday or sunday, so referencing my example df there is the 5/4/19 example which gets pulled back into a 5/3/19 (the last friday for that instance) or the 5/26/19 gets pulled back to 5/24/19. As you can see the code would have to understand first if the date is saturday and sunday then next what is the closest friday prior.

* if all the above got confusing please just reference my df -> resulftdf *

答:
I think something like this could also be possible:</p>

```<code>finaldf['Due_Date'] = [x + pd.to_timedelta(4 - x.dayofweek, unit='d') if x.dayofweek > 4 else x for x in findaldf.Due_Date]
</code>```

Body:
I want to identify if a date value is on a Saturday or Sunday. If it is, then I want to change said date to the friday prior. (example df and result df below - these are real dates from May 2019, as you can see there is a Saturday(5/4/19) and Sunday(5/26/19) date pulled into the week prior in the results df. The (5/2/19) date stays the same as it is a weekday not a weekend.

import pandas as pd
data = [['5/3/19','5/4/19'],['5/3/19','5/2/19'],['5/24/19','5/26/19']]
df = pd.DataFrame(data,columns=['Date1','Date2'])
print(df)

data2 = [['5/3/19','5/3/19'],['5/3/19','5/2/19'],['5/24/19','5/24/19']]
resultsdf = pd.DataFrame(data2,columns=['Date1','Date2'])
print(resultsdf)

I am running this code for a Month to Date range. I have to have the code to group a large data set into a MTD view - but I am having trouble creating variables for a list of sat/sunday ( as there will be 8-10 weekend dates per month) Here is my code:

import pandas as pd
import xlrd # added when using visual studio
import datetime
from datetime import datetime
today = pd.to_datetime(datetime.now().date())
day_of_week = today.dayofweek
last_monday = today - pd.to_timedelta(day_of_week, unit='d')
month_day = today.day
start_month_scope = today - pd.to_timedelta(month_day, unit = 'd')
first_day_of_month = start_month_scope + pd.to_timedelta(1, unit = 'd')

The problem with the code below is that is only works if I need to find the saturday & sunday on 7 day (M-f) period.

last_sunday = last_monday + pd.to_timedelta(6, unit='d')
last_sat = last_sunday - pd.to_timedelta(1, unit='d')
last_fri = last_sunday - pd.to_timedelta(2, unit='d')

I need some variable to input to the code below:

finaldf.loc[(finaldf['Due_Date'] == last_sat), 'Due_Date'] = last_fri

So 'last_fri' would be just the most recent friday from the given saturday or sunday, so referencing my example df there is the 5/4/19 example which gets pulled back into a 5/3/19 (the last friday for that instance) or the 5/26/19 gets pulled back to 5/24/19. As you can see the code would have to understand first if the date is saturday and sunday then next what is the closest friday prior.

* if all the above got confusing please just reference my df -> resulftdf *

Body:
I think something like this could also be possible:

finaldf['Due_Date'] = [x + pd.to_timedelta(4 - x.dayofweek, unit='d') if x.dayofweek > 4 else x for x in findaldf.Due_Date]


问:
Upgrade spring boot version from 1.5.13 to 1.5.19 occurred Error creating bean with name 'payloadRootAnnotationMethodEndpointMapping'
Error creating bean with name 'payloadRootAnnotationMethodEndpointMapping' defined in class path resource [org/springframework/ws/config/annotation/DelegatingWsConfiguration.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.ws.server.endpoint.mapping.PayloadRootAnnotationMethodEndpointMapping]: Factory method 'payloadRootAnnotationMethodEndpointMapping' threw exception; nested exception is java.lang.ExceptionInInitializerError

Failed to instantiate [org.springframework.ws.server.endpoint.mapping.PayloadRootAnnotationMethodEndpointMapping]: Factory method 'payloadRootAnnotationMethodEndpointMapping' threw exception; nested exception is java.lang.ExceptionInInitializerError

Not supported: http://javax.xml.XMLConstants/property/accessExternalDTD

答:
I have fixed this problem by adding the following to pom.xml:

```<code><exclusion>
   <groupId>xalan</groupId>
   <artifactId>xalan</artifactId>
</exclusion>
</code>```

Maybe it shouldn't have xalan in the Maven dependencies.

Body:
I have fixed this problem by adding the following to pom.xml:

<exclusion>
   <groupId>xalan</groupId>
   <artifactId>xalan</artifactId>
</exclusion>

Maybe it shouldn't have xalan in the Maven dependencies.


问:
AWS ELB Intermittent 502 gateway timeout error
I have an ELB in front of a node.js application deployed on 3 EC2 instances.

I have started observing intermittent HTTP 502 Bad Gateway error.

Following is an excerpt from my access logs. There is no pattern to these 502 errors so I am unable to narrow down the cause ?

Is it an ELB issue or an app issue ?

Can access logs help me figure this out ?

This occurs for 5 requests in every 100 requests

```<code>*type*                     https
*timestamp*                2019-05-08T14:50:11.438405Z
*elb*                      <my-elb>
*client:port*              clientIp:port
*target:port*              targetIp:port
*request_processing_time*  0
*target_processing_time*.  2.596
*response_processing_time* -1
*elb_status_code*          502
*target_status_code*       -
*received_bytes*           792
*sent_bytes*               293
*request*                  POST https://app/app-url/2.0/resourcepath/id/abc?queryParamA=abc&queryParamB=false&queryParamC=6b84c34 HTTP/1.1
*user_agent*               Apache-CXF/3.2.5
*ssl_cipher*               ssl-cipher
*ssl_protocol*             TLSv1.2
*target_group_arn*         arn
*trace_id*                 traceId
*domain_name*              cool-domain-name
*chosen_cert_arn*          session-reused
*matched_rule_priority*    0
*request_creation_time*    2019-05-08T14:50:08.841000Z
*actions_executed*         forward
*redirect_url*             -
*error_reason*             -
</code>```

答:
A 502 on an ELB generally points to an app/server issue. The ELB is having problems communicating with the app server.  Check the app logs for restarts or other errors.

Per the RFC:

10.5.3 502 Bad Gateway

  The server, while acting as a gateway or proxy, received an invalid response from the upstream server it accessed in attempting to fulfill the request.

Possible causes are empty or incomplete headers or response body, caused by broken connections. Look for 500 server errors in the app log also.

In your case,  an app server crash can cause 502 errors on the ELB.

See

https://en.wikipedia.org/wiki/List_of_HTTP_status_codes

Body:
I have an ELB in front of a node.js application deployed on 3 EC2 instances.

I have started observing intermittent HTTP 502 Bad Gateway error.

Following is an excerpt from my access logs. There is no pattern to these 502 errors so I am unable to narrow down the cause ?

Is it an ELB issue or an app issue ?

Can access logs help me figure this out ?

This occurs for 5 requests in every 100 requests

*type*                     https
*timestamp*                2019-05-08T14:50:11.438405Z
*elb*                      <my-elb>
*client:port*              clientIp:port
*target:port*              targetIp:port
*request_processing_time*  0
*target_processing_time*.  2.596
*response_processing_time* -1
*elb_status_code*          502
*target_status_code*       -
*received_bytes*           792
*sent_bytes*               293
*request*                  POST https://app/app-url/2.0/resourcepath/id/abc?queryParamA=abc&queryParamB=false&queryParamC=6b84c34 HTTP/1.1
*user_agent*               Apache-CXF/3.2.5
*ssl_cipher*               ssl-cipher
*ssl_protocol*             TLSv1.2
*target_group_arn*         arn
*trace_id*                 traceId
*domain_name*              cool-domain-name
*chosen_cert_arn*          session-reused
*matched_rule_priority*    0
*request_creation_time*    2019-05-08T14:50:08.841000Z
*actions_executed*         forward
*redirect_url*             -
*error_reason*             -