问:
How can I set the options for a MySQL Connector/Python X DevAPI connection?
How can I set the "--local-infile[={0|1}]" option for a mysqlx connection?

My python code looks the following:

```<code>import mysqlx

# Connect to server on localhost
session = mysqlx.get_session({
    'host': 'localhost',
    'port': 33060,
    'user': 'root',
    'password': '12345678',
})
</code>```

答:
Currently, there is no LOAD DATA support in the X Protocol, which means that X DevAPI clients don't provide such an option. You'll need a Classic Protocol connection for that.

Disclaimer: I'm the lead developer of the MySQL X DevAPI Connector for Node.js

Body:
How can I set the "--local-infile[={0|1}]" option for a mysqlx connection?

My python code looks the following:

import mysqlx

# Connect to server on localhost
session = mysqlx.get_session({
    'host': 'localhost',
    'port': 33060,
    'user': 'root',
    'password': '12345678',
})


来源: stackexchange

问:
rabbitmq error at startup
This is my setup.

OS: 2.6.35-28-generic-pae #50-Ubuntu SMP Fri Mar 18 20:43:15 UTC 2011 i686 GNU/Linux

Erlant OTP : R14B03

RabbitMQ : rabbitmq-server-2.4.1 (built from source using R14B03, used "make USE_SPECS=false" to fix a issue that was poppoing while build.

build goes fine.

when I start server
./rabbitmq-server start, I get following error.

ERROR: generation of boot script file

/export/workdir/data/rabbitmq/mnesia/rabbit@MyHostname-plugins-expand/rabbit.script failed:

{'EXIT',{{badmatch,{erl_prim_loader,error}},

         [{xref_utils,find_beam,1},

          {xref_base,'-make_libs/5-fun-0-',2},

          {lists,foldl,3},

          {xref_base,make_libs,5},

          {xref_base,do_set_up,1},

          {xref_base,do_set_up,2},

          {xref_base,q,3},

          {xref,handle_call,3}]}}

Any clues, I prefer to use latest version of rabbit and build it myself rather then going to older versions.

thanks

答:
That is a traceback. If you read and Erlang tutorial, that will explain how to read it.

That sounds to me like a typographical error in a config file. Go over any config file entries that you made, particularly related to plugins.

If you didn't set up any plugins at all, then I suggest downloading the collection of plugins used to make rabbitmq-management work, and configure that. These days it is unusual to have a RabbitMQ install with no plugins at all.

ContentLicense: CC BY-SA 3.0

Title: rabbitmq error at startup


问:
How to have a Python Script run constantly in the background connected to a Node.JS server
Currently I have a Node.JS server that needs to run a python script. Part of this script imports some data from a server that takes time which is detrimental to the experience as this Python script needs to be ran frequently.

What I want is for the Node.JS server to run this Python script when the server is ran, then in the background constantly have it keep running, with the ability to (from the Node.JS server) call a python function that returns data.

So far I have this on the Node.JS server that runs a python script and outputs the response. This is repeated every time data is needed to be retrieved:

const util = require('util');
    var options = {
    mode: 'text',
    args: [sentenceToUse]
    };

```<code>const readPy = util.promisify(PythonShell.run);
const test = await readPy("spacyTest.py", options);

    var response = test.toString();
    response = response.toString();
    response = response.replace(/'/g, '"');
    response = JSON.parse(response);
return(response);
console.log(test);
</code>```

'''

How can I keep this running in the background without restarting the python script every time data is needed?

答:
It seems you need to change the python script itself to keep running and respond to requests from its parent. If the python script now runs and exits, there's nothing you can do from node.js to stop that. You need to change the python script.

Here are some options:

The python script can regularly send data back to its parent on stdout and you can read that data from nodejs as it arrives.

You can put a little local http server or socket.io server in the python script on a known port and have your nodejs parent communicate with that to make requests and get responses.

The Python-shell module also shows you how to communicate between node.js parent and the python script here in the doc, so that is an option too.

Since options 1 and 3 are built-in already, I would probably start with one of those until you find some reason they aren't sufficient.

ContentLicense: CC BY-SA 4.0

Body:
Currently I have a Node.JS server that needs to run a python script. Part of this script imports some data from a server that takes time which is detrimental to the experience as this Python script needs to be ran frequently.

What I want is for the Node.JS server to run this Python script when the server is ran, then in the background constantly have it keep running, with the ability to (from the Node.JS server) call a python function that returns data.

So far I have this on the Node.JS server that runs a python script and outputs the response. This is repeated every time data is needed to be retrieved:

const util = require('util');
    var options = {
    mode: 'text',
    args: [sentenceToUse]
    };

const readPy = util.promisify(PythonShell.run);
const test = await readPy("spacyTest.py", options);

    var response = test.toString();
    response = response.toString();
    response = response.replace(/'/g, '"');
    response = JSON.parse(response);
return(response);
console.log(test);

'''

How can I keep this running in the background without restarting the python script every time data is needed?


问:
filter object of objects dynamic key value
how can i filter object of objects with dynamic key value in javaScript or lodash.

my data is :

```<code>var names = [
  {
    name: 'c1',
    tags: {
      key1: 'value1-1',
      key2: 'value1-2',
    },
  },
  {
    name: 'c2',
    tags: {
      key1: 'value2-1',
      key2: 'value1-2',
    },
  },
  {
    name: 'c3',
    tags: {
      key3: 'value3-1',
      key4: 'value3-2',
    },
  }
]
</code>```

and filter data is:

```<code>var filterData = {
  key3: { 0: 'value3-1' },
  key4: { 0: 'value3-2' }
}
</code>```

and i expect filter and after filter show like this:

```<code>afterFilter = [
      {
        name: 'c3',
        tags: {
          key3: 'value3-1',
          key4: 'value3-2',
        }
      }
    ]
</code>```

答:
You can do something on these lines.

```<code>var names = [
  {
    name: 'c1',
    tags: {
      key1: 'value1-1',
      key2: 'value1-2',
    },
  },
  {
    name: 'c2',
    tags: {
      key1: 'value2-1',
      key2: 'value1-2',
    },
  },
  {
    name: 'c3',
    tags: {
      key3: 'value3-1',
      key4: 'value3-2',
    },
  }
]

var path = "tags.key1";
var value = "value1-1"
var filtered = []
names.forEach(elem => {
  var arr = path.split('.')
  // make a copy so that you do not operate on original entry
  var item = JSON.parse(JSON.stringify(elem));
  while(arr.length) {

    item = item && item[arr[0]]
    arr.shift();
  }
  if (item === value) {
    filtered.push(elem)
  }
})

// Filtered is result
console.log(filtered)
</code>```

Codepen: https://codepen.io/kishin-karra/pen/dyYxEoZ

Body:
how can i filter object of objects with dynamic key value in javaScript or lodash.

my data is :

var names = [
  {
    name: 'c1',
    tags: {
      key1: 'value1-1',
      key2: 'value1-2',
    },
  },
  {
    name: 'c2',
    tags: {
      key1: 'value2-1',
      key2: 'value1-2',
    },
  },
  {
    name: 'c3',
    tags: {
      key3: 'value3-1',
      key4: 'value3-2',
    },
  }
]

and filter data is:

var filterData = {
  key3: { 0: 'value3-1' },
  key4: { 0: 'value3-2' }
}

and i expect filter and after filter show like this:

afterFilter = [
      {
        name: 'c3',
        tags: {
          key3: 'value3-1',
          key4: 'value3-2',
        }
      }
    ]

Body:
You can do something on these lines.

var names = [
  {
    name: 'c1',
    tags: {
      key1: 'value1-1',
      key2: 'value1-2',
    },
  },
  {
    name: 'c2',
    tags: {
      key1: 'value2-1',
      key2: 'value1-2',
    },
  },
  {
    name: 'c3',
    tags: {
      key3: 'value3-1',
      key4: 'value3-2',
    },
  }
]

var path = "tags.key1";
var value = "value1-1"
var filtered = []
names.forEach(elem => {
  var arr = path.split('.')
  // make a copy so that you do not operate on original entry
  var item = JSON.parse(JSON.stringify(elem));
  while(arr.length) {

    item = item && item[arr[0]]
    arr.shift();
  }
  if (item === value) {
    filtered.push(elem)
  }
})

// Filtered is result
console.log(filtered)

Codepen: https://codepen.io/kishin-karra/pen/dyYxEoZ


问:
Cannot access a disposed object in ASP.NET Core when injecting DbContext
On an ASP.NET Core project I have the following on Startup:

```<code>  services.AddDbContext<Context>(x => x.UseSqlServer(connectionString));

  services.AddTransient<IValidationService, ValidationService>();

  services.AddTransient<IValidator<Model>, ModelValidator>();
</code>```

The ValidationService is as follows:

```<code>public interface IValidationService {
    Task<List<Error>> ValidateAsync<T>(T model);
}

public class ValidationService : IValidationService {
    private readonly IServiceProvider _provider;

    public ValidationService(IServiceProvider provider) {
        _provider = provider;
    }

    public async Task<List<Error>> ValidateAsync<T>(T model) {
        IValidator<T> validator = _provider.GetRequiredService<IValidator<T>>();

        return await validator.ValidateAsync(model);
    }
}
</code>```

And the ModelValidator is as follows:

```<code>public class ModelValidator : AbstractValidator<Model> {
  public ModelValidator(Context context) {
    // Some code using context
  }
}
</code>```

When I inject a IValidationService in a controller and use it as:

```<code>List<Error> errors = await _validator.ValidateAsync(order);
</code>```

I get the error:

System.ObjectDisposedException: Cannot access a disposed object. A
  common cause of this error is disposing a context that was resolved
  from dependency injection and then later trying to use the same
  context instance elsewhere in your application.  This may occur is you
  are calling Dispose() on the context, or wrapping the context in a
  using statement.  If you are using dependency injection, you should
  let the dependency injection container take care of disposing context
  instances. Object name: 'Context'.

Any idea why I am having this error when using Context inside ModelValidator.

How to fix this?

UPDATE

So I changed the code to:

```<code>services.AddScoped<IValidationService, ValidationService>();

services.AddScoped<IValidator<Model>, ModelValidator>();
</code>```

But I get the same error ...

UPDATE - Seed Data Code inside Configure method on Startup

So on Configure method I have:

```<code>if (hostingEnvironment.IsDevelopment())
  applicationBuilder.SeedData();
</code>```

And the SeedData extension is:

```<code>public static class DataSeedExtensions {
    private static IServiceProvider _provider;

    public static void SeedData(this IApplicationBuilder builder) {
        _provider = builder.ApplicationServices;
        _type = type;

        using (Context context = (Context)_provider.GetService<Context>()) {
            await context.Database.MigrateAsync();
            // Insert data code
    }
}
</code>```

What am I missing?

UPDATE - A possible solution

Changing my Seed method to the following seems to work:

```<code>using (IServiceScope scope =
    _provider.GetRequiredService<IServiceScopeFactory>().CreateScope()) {
    Context context = _provider.GetService<Context>();
    // Insert data in database
}
</code>```

答:
I was facing a similar error and later was able to resolve it.

I was calling the async method without using await.

old code

```<code>var newUser = _repo.Register(newUserToCreate);
</code>```

with the fix made

```<code>var newUser = await _repo.Register(newUserToCreate);
</code>```

Body:
On an ASP.NET Core project I have the following on Startup:

  services.AddDbContext<Context>(x => x.UseSqlServer(connectionString));

  services.AddTransient<IValidationService, ValidationService>();

  services.AddTransient<IValidator<Model>, ModelValidator>();

The ValidationService is as follows:

public interface IValidationService {
    Task<List<Error>> ValidateAsync<T>(T model);
}

public class ValidationService : IValidationService {
    private readonly IServiceProvider _provider;

    public ValidationService(IServiceProvider provider) {
        _provider = provider;
    }

    public async Task<List<Error>> ValidateAsync<T>(T model) {
        IValidator<T> validator = _provider.GetRequiredService<IValidator<T>>();

        return await validator.ValidateAsync(model);
    }
}

And the ModelValidator is as follows:

public class ModelValidator : AbstractValidator<Model> {
  public ModelValidator(Context context) {
    // Some code using context
  }
}

When I inject a IValidationService in a controller and use it as:

List<Error> errors = await _validator.ValidateAsync(order);

I get the error:

System.ObjectDisposedException: Cannot access a disposed object. A
  common cause of this error is disposing a context that was resolved
  from dependency injection and then later trying to use the same
  context instance elsewhere in your application.  This may occur is you
  are calling Dispose() on the context, or wrapping the context in a
  using statement.  If you are using dependency injection, you should
  let the dependency injection container take care of disposing context
  instances. Object name: 'Context'.

Any idea why I am having this error when using Context inside ModelValidator.

How to fix this?

UPDATE

So I changed the code to:

services.AddScoped<IValidationService, ValidationService>();

services.AddScoped<IValidator<Model>, ModelValidator>();

But I get the same error ...

UPDATE - Seed Data Code inside Configure method on Startup

So on Configure method I have:

if (hostingEnvironment.IsDevelopment())
  applicationBuilder.SeedData();

And the SeedData extension is:

public static class DataSeedExtensions {
    private static IServiceProvider _provider;

    public static void SeedData(this IApplicationBuilder builder) {
        _provider = builder.ApplicationServices;
        _type = type;

        using (Context context = (Context)_provider.GetService<Context>()) {
            await context.Database.MigrateAsync();
            // Insert data code
    }
}

What am I missing?

UPDATE - A possible solution

Changing my Seed method to the following seems to work:

using (IServiceScope scope =
    _provider.GetRequiredService<IServiceScopeFactory>().CreateScope()) {
    Context context = _provider.GetService<Context>();
    // Insert data in database
}

Body:
I was facing a similar error and later was able to resolve it.

I was calling the async method without using await.

old code

var newUser = _repo.Register(newUserToCreate);

with the fix made

var newUser = await _repo.Register(newUserToCreate);


问:
Convert MDB to CSV
I have a MS Access mdb file. I need to convert it to a CSV file. How do i do it? Please do not point me to any freeware.

Thanks in advance

答:
Being no freeware, as you requested, I could recommend you Spectral Core's Full Convert Enterprise.

I've used it successfully in the past, too.

Update:

Since you clarified that you need a programmatic solution, I do recommend you do it manually:

Open a connection to the MDB file through ADO.NET.

Iterate all tables.

Create a text file (CSV) for each table.

For each table, iterate all rows.

For each row, write a new line in the text file.

For each row, iterate all columns.

For each column, write the value to the text file in the current row.

Title: Convert MDB to CSV


问:
When tkinter.PhotoImage works with png files, why use pillow?
I have seen many tkinter tutorials on images and almost all of them say that you have to use pillow to read png images, but tkinter.PhotoImage also does the job. Has there been a recent update to tkinter.PhotoUpdate or is there another reason for using pillow?</p>

答:
Has there been a recent update to tkinter.PhotoUpdate or is there another reason for using pillow?

Yes, there has been a fairly recent update to tkinter. PNG support is fairly recent, and still not available in every version of tkinter. You need a version of tkinter that is built on top of tk 8.6 or later.

You can check your version of tk by looking at the TkVersion attribute of the tkinter module:

```<code>import tkinter as tk
print(f"tk version: {tk.TkVersion}")
</code>```

Body:
Has there been a recent update to tkinter.PhotoUpdate or is there another reason for using pillow?

Yes, there has been a fairly recent update to tkinter. PNG support is fairly recent, and still not available in every version of tkinter. You need a version of tkinter that is built on top of tk 8.6 or later.

You can check your version of tk by looking at the TkVersion attribute of the tkinter module:

import tkinter as tk
print(f"tk version: {tk.TkVersion}")


问:
Unable to understand the function of key in sorting python lists
Sorry I can't understand what happens when key = key:</p>

```<code>def key(x):
    return x[1]

a = [(1, 2), (3, 1), (5, 10), (11, -3)]
a.sort(key=key)
</code>```

答:
You need to first understand what is a keyword (named) argument:
https://treyhunner.com/2018/04/keyword-arguments-in-python/

Thereafter, you can see that for the sort method that belongs to the "a" list, you can see that it has a named argument of key: https://www.programiz.com/python-programming/methods/list/sort

The named argument key takes in a function in which the return value of that function will be used for sorting, which in your case, the key function defined at line 1 will use the second element of each tuple in the list for sorting.

i.e.

first key on the LHS = named argument of sort method belonging to a list

second key on the RHS = key function defined that returns a value to be used for sorting the list

Body:
Sorry I can't understand what happens when key = key:

def key(x):
    return x[1]

a = [(1, 2), (3, 1), (5, 10), (11, -3)]
a.sort(key=key)


答:
sort takes a keyword argument named key. With a.sort(key=key), the function key is given for the keyword argument key. The sort function then applied the key function to every element in the list and then sorts the list by the result of this function call.

This may get less confusing when you rename the function, e.g. key_function.

```<code>def key_function(x):
    return x[1]

a = [(1, 2), (3, 1), (5, 10), (11, -3)]
a.sort(key=key_function)
</code>```

The function returns the second element of the input value, so the value for (1, 2) will be 2 and for (3, 1) it will be 1. These values will then determine how the tuples will be ordered other than how tuples will usually be sorted.

Title: Unable to understand the function of key in sorting python lists

Body:
sort takes a keyword argument named key. With a.sort(key=key), the function key is given for the keyword argument key. The sort function then applied the key function to every element in the list and then sorts the list by the result of this function call.

This may get less confusing when you rename the function, e.g. key_function.

def key_function(x):
    return x[1]

a = [(1, 2), (3, 1), (5, 10), (11, -3)]
a.sort(key=key_function)

The function returns the second element of the input value, so the value for (1, 2) will be 2 and for (3, 1) it will be 1. These values will then determine how the tuples will be ordered other than how tuples will usually be sorted.


问:
float() argument must be a string or a number, not 'Timestamp': python pandas
I have the following command:

```<code>from datetime import datetime
import pandas as pd
import matplotlib.pyplot as plt

data = pd.read_csv('data/C2A2_data/BinnedCsvs_d400/fb441e62df2d58994928907a91895ec62c2c42e6cd075c2700843b89.csv')
newdata = data[(data['Date'] >= '2005-01-01') & (data['Date'] <= '2014-12-12')]
datamax = newdata[newdata['Element']=='TMAX']
datamin = newdata[newdata['Element']=='TMIN']
datamax = datamax.groupby('Date').max()
datamin = datamin.groupby('Date').min()
datamax = datamax.reset_index()
datamin = datamin.reset_index()
datamax = datamax[~datamax['Date'].str.contains("02-29")]
datamin = datamin[~datamin['Date'].str.contains("02-29")]

breakoutdata = data[(data['Date']  > '2014-12-31')]
datamax2015 = breakoutdata[breakoutdata['Element']=='TMAX']
datamin2015 = breakoutdata[breakoutdata['Element']=='TMIN']
datamax2015 = datamax2015.groupby('Date').max()
datamin2015 = datamin2015.groupby('Date').min()
datamax2015 = datamax2015.reset_index()
datamin2015 = datamin2015.reset_index()
datamax2015 = datamax2015[~datamax2015['Date'].str.contains("02-29")]
datamin2015 = datamin2015[~datamin2015['Date'].str.contains("02-29")]

datamin['Date'] = pd.to_datetime(datamin['Date'])
datamin2015['Date'] = pd.to_datetime(datamin2015['Date'])
datamin["day_of_year"] = datamin["Date"].dt.dayofyear
datamin2015["day_of_year"] = datamin["Date"].dt.dayofyear
dataminappend = datamin2015.join(datamin,on="day_of_year",how="left",lsuffix="2015")
lower = dataminappend.loc[dataminappend["Data_Value2015"]<dataminappend["Data_Value"]]

datamax['Date'] = pd.to_datetime(datamax['Date'])
datamax2015['Date'] = pd.to_datetime(datamax2015['Date'])
datamax["day_of_year"] = datamax["Date"].dt.dayofyear
datamax2015["day_of_year"] = datamax["Date"].dt.dayofyear
datamaxappend = datamax2015.join(datamax,on="day_of_year",how="left",lsuffix="2015")

upper = datamaxappend.loc[datamaxappend["Data_Value2015"]>datamaxappend["Data_Value"]]
upper = upper.rename(columns={"Date2015":"Date","Data_Value2015":"Data_Value"})
lower = lower.rename(columns={"Date2015":"Date","Data_Value2015":"Data_Value"})
upper['Date'] = upper['Date'].map(lambda x: x.strftime("%d/%m/%Y, %H:%M:%S"))
lower['Date'] = lower['Date'].map(lambda x: x.strftime("%d/%m/%Y, %H:%M:%S"))

ax = plt.gca()
plt.plot(datamax['Date'],datamax['Data_Value'],color='red')
plt.plot(datamax['Date'],datamax['Data_Value'], color='blue')
plt.scatter(upper['Date'],upper['Data_Value'],color='red')
plt.scatter(lower['Date'],lower['Data_Value'], color='blue')

plt.ylabel("Temperature (degrees C)",color='navy')
plt.xlabel("Date",color='navy',labelpad=15)
plt.title('Record high and low temperatures by day (2005-2014)', alpha=1.0,color='brown',y=1.08)
ax.legend(loc='upper center', bbox_to_anchor=(0.5, -0.35),
          fancybox=False,labels=['Record high','Record low'])
plt.xticks(rotation=30)
plt.fill_between(range(len(datamax['Date'])), datamax['Data_Value'], datamin['Data_Value'],color='yellow',alpha=0.8)
plt.show()
</code>```

However, I keep getting the error: TypeError: float() argument must be a string or a number, not 'Timestamp'

For this line: plt.scatter(upper['Date'],upper['Data_Value'],color='red')

Would anybody be able to give me a helping hand?

I cannot see anything obviously wrong with my code...

Here is a snippet of the dataframe data:

```<code> ID        Date Element  Data_Value
0       USW00094889  2014-11-12    TMAX          22
1       USC00208972  2009-04-29    TMIN          56
2       USC00200032  2008-05-26    TMAX         278
3       USC00205563  2005-11-11    TMAX         139
4       USC00200230  2014-02-27    TMAX        -106
5       USW00014833  2010-10-01    TMAX         194
6       USC00207308  2010-06-29    TMIN         144
7       USC00203712  2005-10-04    TMAX         289
8       USW00004848  2007-12-14    TMIN         -16
9       USC00200220  2011-04-21    TMAX          72
10      USC00205822  2013-01-16    TMAX          11
11      USC00205822  2008-05-29    TMIN          28
12      USC00203712  2008-10-17    TMIN          17
13      USC00205563  2006-05-14    TMAX         183
14      USC00200842  2006-05-14    TMAX         122
15      USC00205563  2014-12-07    TMAX          67
16      USC00205822  2008-09-07    TMAX         250
17      USC00205450  2006-04-22    TMIN          67
18      USC00203712  2008-02-22    TMAX         -44
19      USC00205563  2015-01-03    TMIN         -39
20      USC00201250  2011-03-28    TMIN         -78
21      USC00200230  2008-02-10    TMAX          -6
22      USC00207320  2008-02-03    TMIN         -39

[165085 rows x 4 columns]
</code>```

In case anybody wants an easier way to manipulate the code above.

答:
Python code which returns a line graph of the record high and record low temperatures by day of the year over the period 2005-2014. The area between the record high and record low temperatures for each day should be shaded.

Then, overlay a scatter of the 2015 data for any points (highs and lows) for which the ten year record (2005-2014) record high or record low was broken in 2015.

Remove leap year dates (i.e. 29th February).

/

```<code>from datetime import datetime
import pandas as pd
import matplotlib.pyplot as plt

pd.set_option("display.max_rows",None,"display.max_columns",None)
data = pd.read_csv('data/C2A2_data/BinnedCsvs_d400/fb441e62df2d58994928907a91895ec62c2c42e6cd075c2700843b89.csv')
newdata = data[(data['Date'] >= '2005-01-01') & (data['Date'] <= '2014-12-12')]
datamax = newdata[newdata['Element']=='TMAX']
datamin = newdata[newdata['Element']=='TMIN']
datamax['Date'] = pd.to_datetime(datamax['Date'])
datamin['Date'] = pd.to_datetime(datamin['Date'])
datamax["day_of_year"] = datamax["Date"].dt.dayofyear
datamax = datamax.groupby('day_of_year').max()
datamin["day_of_year"] = datamin["Date"].dt.dayofyear
datamin = datamin.groupby('day_of_year').min()
datamax = datamax.reset_index()
datamin = datamin.reset_index()
datamin['Date'] = datamin['Date'].dt.strftime('%Y-%m-%d')
datamax['Date'] = datamax['Date'].dt.strftime('%Y-%m-%d')
datamax = datamax[~datamax['Date'].str.contains("02-29")]
datamin = datamin[~datamin['Date'].str.contains("02-29")]

breakoutdata = data[(data['Date']  > '2014-12-31')]
datamax2015 = breakoutdata[breakoutdata['Element']=='TMAX']
datamin2015 = breakoutdata[breakoutdata['Element']=='TMIN']
datamax2015['Date'] = pd.to_datetime(datamax2015['Date'])
datamin2015['Date'] = pd.to_datetime(datamin2015['Date'])
datamax2015["day_of_year"] = datamax2015["Date"].dt.dayofyear
datamax2015 = datamax2015.groupby('day_of_year').max()
datamin2015["day_of_year"] = datamin2015["Date"].dt.dayofyear
datamin2015 = datamin2015.groupby('day_of_year').min()
datamax2015 = datamax2015.reset_index()
datamin2015 = datamin2015.reset_index()
datamin2015['Date'] = datamin2015['Date'].dt.strftime('%Y-%m-%d')
datamax2015['Date'] = datamax2015['Date'].dt.strftime('%Y-%m-%d')
datamax2015 = datamax2015[~datamax2015['Date'].str.contains("02-29")]
datamin2015 = datamin2015[~datamin2015['Date'].str.contains("02-29")]

dataminappend = datamin2015.join(datamin,on="day_of_year",rsuffix="_new")
lower = dataminappend.loc[dataminappend["Data_Value_new"]>dataminappend["Data_Value"]]
datamaxappend = datamax2015.join(datamax,on="day_of_year",rsuffix="_new")
upper = datamaxappend.loc[datamaxappend["Data_Value_new"]<datamaxappend["Data_Value"]]

upper['Date'] = pd.to_datetime(upper['Date'])
lower['Date'] = pd.to_datetime(lower['Date'])
datamax['Date'] = pd.to_datetime(datamax['Date'])
datamin['Date'] = pd.to_datetime(datamin['Date'])

ax = plt.gca()
plt.plot(datamax['day_of_year'],datamax['Data_Value'],color='red')
plt.plot(datamin['day_of_year'],datamin['Data_Value'], color='blue')
plt.scatter(upper['day_of_year'],upper['Data_Value'],color='purple')
plt.scatter(lower['day_of_year'],lower['Data_Value'], color='cyan')

plt.ylabel("Temperature (degrees C)",color='navy')
plt.xlabel("Day of the year",color='navy',labelpad=15)
plt.title('Record high and low temperatures by day between 2005-2014)', alpha=1.0,color='brown',y=1.08)
ax.legend(loc='upper center', bbox_to_anchor=(0.5, -0.35),fancybox=False,labels=['Record high','Record low'])
plt.xticks(rotation=30)
plt.fill_between(range(len(datamax['Date'])), datamax['Data_Value'], datamin['Data_Value'],color='yellow',alpha=0.8)
plt.show()
</code>```

/

I have converted the 'Date' column to a string using Datamin['Date'] = datamin['Date'].dt.strftime('%Y-%m-%d').

I have then converted this back to 'datetime' format using upper['Date'] = pd.to_datetime(upper['Date'])

I then used 'date of year' as the x-value.

Body:
I have the following command:

from datetime import datetime
import pandas as pd
import matplotlib.pyplot as plt

data = pd.read_csv('data/C2A2_data/BinnedCsvs_d400/fb441e62df2d58994928907a91895ec62c2c42e6cd075c2700843b89.csv')
newdata = data[(data['Date'] >= '2005-01-01') & (data['Date'] <= '2014-12-12')]
datamax = newdata[newdata['Element']=='TMAX']
datamin = newdata[newdata['Element']=='TMIN']
datamax = datamax.groupby('Date').max()
datamin = datamin.groupby('Date').min()
datamax = datamax.reset_index()
datamin = datamin.reset_index()
datamax = datamax[~datamax['Date'].str.contains("02-29")]
datamin = datamin[~datamin['Date'].str.contains("02-29")]

breakoutdata = data[(data['Date']  > '2014-12-31')]
datamax2015 = breakoutdata[breakoutdata['Element']=='TMAX']
datamin2015 = breakoutdata[breakoutdata['Element']=='TMIN']
datamax2015 = datamax2015.groupby('Date').max()
datamin2015 = datamin2015.groupby('Date').min()
datamax2015 = datamax2015.reset_index()
datamin2015 = datamin2015.reset_index()
datamax2015 = datamax2015[~datamax2015['Date'].str.contains("02-29")]
datamin2015 = datamin2015[~datamin2015['Date'].str.contains("02-29")]

datamin['Date'] = pd.to_datetime(datamin['Date'])
datamin2015['Date'] = pd.to_datetime(datamin2015['Date'])
datamin["day_of_year"] = datamin["Date"].dt.dayofyear
datamin2015["day_of_year"] = datamin["Date"].dt.dayofyear
dataminappend = datamin2015.join(datamin,on="day_of_year",how="left",lsuffix="2015")
lower = dataminappend.loc[dataminappend["Data_Value2015"]<dataminappend["Data_Value"]]

datamax['Date'] = pd.to_datetime(datamax['Date'])
datamax2015['Date'] = pd.to_datetime(datamax2015['Date'])
datamax["day_of_year"] = datamax["Date"].dt.dayofyear
datamax2015["day_of_year"] = datamax["Date"].dt.dayofyear
datamaxappend = datamax2015.join(datamax,on="day_of_year",how="left",lsuffix="2015")

upper = datamaxappend.loc[datamaxappend["Data_Value2015"]>datamaxappend["Data_Value"]]
upper = upper.rename(columns={"Date2015":"Date","Data_Value2015":"Data_Value"})
lower = lower.rename(columns={"Date2015":"Date","Data_Value2015":"Data_Value"})
upper['Date'] = upper['Date'].map(lambda x: x.strftime("%d/%m/%Y, %H:%M:%S"))
lower['Date'] = lower['Date'].map(lambda x: x.strftime("%d/%m/%Y, %H:%M:%S"))

ax = plt.gca()
plt.plot(datamax['Date'],datamax['Data_Value'],color='red')
plt.plot(datamax['Date'],datamax['Data_Value'], color='blue')
plt.scatter(upper['Date'],upper['Data_Value'],color='red')
plt.scatter(lower['Date'],lower['Data_Value'], color='blue')

plt.ylabel("Temperature (degrees C)",color='navy')
plt.xlabel("Date",color='navy',labelpad=15)
plt.title('Record high and low temperatures by day (2005-2014)', alpha=1.0,color='brown',y=1.08)
ax.legend(loc='upper center', bbox_to_anchor=(0.5, -0.35),
          fancybox=False,labels=['Record high','Record low'])
plt.xticks(rotation=30)
plt.fill_between(range(len(datamax['Date'])), datamax['Data_Value'], datamin['Data_Value'],color='yellow',alpha=0.8)
plt.show()

However, I keep getting the error: TypeError: float() argument must be a string or a number, not 'Timestamp'

For this line: plt.scatter(upper['Date'],upper['Data_Value'],color='red')

Would anybody be able to give me a helping hand?

I cannot see anything obviously wrong with my code...

Here is a snippet of the dataframe data:

 ID        Date Element  Data_Value
0       USW00094889  2014-11-12    TMAX          22
1       USC00208972  2009-04-29    TMIN          56
2       USC00200032  2008-05-26    TMAX         278
3       USC00205563  2005-11-11    TMAX         139
4       USC00200230  2014-02-27    TMAX        -106
5       USW00014833  2010-10-01    TMAX         194
6       USC00207308  2010-06-29    TMIN         144
7       USC00203712  2005-10-04    TMAX         289
8       USW00004848  2007-12-14    TMIN         -16
9       USC00200220  2011-04-21    TMAX          72
10      USC00205822  2013-01-16    TMAX          11
11      USC00205822  2008-05-29    TMIN          28
12      USC00203712  2008-10-17    TMIN          17
13      USC00205563  2006-05-14    TMAX         183
14      USC00200842  2006-05-14    TMAX         122
15      USC00205563  2014-12-07    TMAX          67
16      USC00205822  2008-09-07    TMAX         250
17      USC00205450  2006-04-22    TMIN          67
18      USC00203712  2008-02-22    TMAX         -44
19      USC00205563  2015-01-03    TMIN         -39
20      USC00201250  2011-03-28    TMIN         -78
21      USC00200230  2008-02-10    TMAX          -6
22      USC00207320  2008-02-03    TMIN         -39

[165085 rows x 4 columns]

In case anybody wants an easier way to manipulate the code above.

Body:
Python code which returns a line graph of the record high and record low temperatures by day of the year over the period 2005-2014. The area between the record high and record low temperatures for each day should be shaded.

Then, overlay a scatter of the 2015 data for any points (highs and lows) for which the ten year record (2005-2014) record high or record low was broken in 2015.

Remove leap year dates (i.e. 29th February).

/

from datetime import datetime
import pandas as pd
import matplotlib.pyplot as plt

pd.set_option("display.max_rows",None,"display.max_columns",None)
data = pd.read_csv('data/C2A2_data/BinnedCsvs_d400/fb441e62df2d58994928907a91895ec62c2c42e6cd075c2700843b89.csv')
newdata = data[(data['Date'] >= '2005-01-01') & (data['Date'] <= '2014-12-12')]
datamax = newdata[newdata['Element']=='TMAX']
datamin = newdata[newdata['Element']=='TMIN']
datamax['Date'] = pd.to_datetime(datamax['Date'])
datamin['Date'] = pd.to_datetime(datamin['Date'])
datamax["day_of_year"] = datamax["Date"].dt.dayofyear
datamax = datamax.groupby('day_of_year').max()
datamin["day_of_year"] = datamin["Date"].dt.dayofyear
datamin = datamin.groupby('day_of_year').min()
datamax = datamax.reset_index()
datamin = datamin.reset_index()
datamin['Date'] = datamin['Date'].dt.strftime('%Y-%m-%d')
datamax['Date'] = datamax['Date'].dt.strftime('%Y-%m-%d')
datamax = datamax[~datamax['Date'].str.contains("02-29")]
datamin = datamin[~datamin['Date'].str.contains("02-29")]

breakoutdata = data[(data['Date']  > '2014-12-31')]
datamax2015 = breakoutdata[breakoutdata['Element']=='TMAX']
datamin2015 = breakoutdata[breakoutdata['Element']=='TMIN']
datamax2015['Date'] = pd.to_datetime(datamax2015['Date'])
datamin2015['Date'] = pd.to_datetime(datamin2015['Date'])
datamax2015["day_of_year"] = datamax2015["Date"].dt.dayofyear
datamax2015 = datamax2015.groupby('day_of_year').max()
datamin2015["day_of_year"] = datamin2015["Date"].dt.dayofyear
datamin2015 = datamin2015.groupby('day_of_year').min()
datamax2015 = datamax2015.reset_index()
datamin2015 = datamin2015.reset_index()
datamin2015['Date'] = datamin2015['Date'].dt.strftime('%Y-%m-%d')
datamax2015['Date'] = datamax2015['Date'].dt.strftime('%Y-%m-%d')
datamax2015 = datamax2015[~datamax2015['Date'].str.contains("02-29")]
datamin2015 = datamin2015[~datamin2015['Date'].str.contains("02-29")]

dataminappend = datamin2015.join(datamin,on="day_of_year",rsuffix="_new")
lower = dataminappend.loc[dataminappend["Data_Value_new"]>dataminappend["Data_Value"]]
datamaxappend = datamax2015.join(datamax,on="day_of_year",rsuffix="_new")
upper = datamaxappend.loc[datamaxappend["Data_Value_new"]<datamaxappend["Data_Value"]]

upper['Date'] = pd.to_datetime(upper['Date'])
lower['Date'] = pd.to_datetime(lower['Date'])
datamax['Date'] = pd.to_datetime(datamax['Date'])
datamin['Date'] = pd.to_datetime(datamin['Date'])

ax = plt.gca()
plt.plot(datamax['day_of_year'],datamax['Data_Value'],color='red')
plt.plot(datamin['day_of_year'],datamin['Data_Value'], color='blue')
plt.scatter(upper['day_of_year'],upper['Data_Value'],color='purple')
plt.scatter(lower['day_of_year'],lower['Data_Value'], color='cyan')

plt.ylabel("Temperature (degrees C)",color='navy')
plt.xlabel("Day of the year",color='navy',labelpad=15)
plt.title('Record high and low temperatures by day between 2005-2014)', alpha=1.0,color='brown',y=1.08)
ax.legend(loc='upper center', bbox_to_anchor=(0.5, -0.35),fancybox=False,labels=['Record high','Record low'])
plt.xticks(rotation=30)
plt.fill_between(range(len(datamax['Date'])), datamax['Data_Value'], datamin['Data_Value'],color='yellow',alpha=0.8)
plt.show()

/

I have converted the 'Date' column to a string using Datamin['Date'] = datamin['Date'].dt.strftime('%Y-%m-%d').

I have then converted this back to 'datetime' format using upper['Date'] = pd.to_datetime(upper['Date'])

I then used 'date of year' as the x-value.


问:
Reading a json webpage using python
I am trying to read a json webpage and insert it into a dataframe.

import urllib3
import json
import pandas as pd

url = 'https://covid.ourworldindata.org/data/owid-covid-data.json'
http = urllib3.PoolManager()
r = http.request('GET', url, headers={'Accept': 'application/json'})

data = json.loads(r.data.decode('utf-8'))
data_copy = data

df = pd.DataFrame.from_dict(data_copy, orient='index')
df.transpose()
```

I need to use the same code I previewed but the output has to be in the following format:

答:
Instead of using

```<code>pd.DataFrame.from_dictionary()
</code>```

use

```<code>pd.DataFrame.from_records()
</code>```

This method converts structured or record ndarray to DataFrame. If i had some code i would have tried to code the solution.

Title: Reading a json webpage using python

Body:
I am trying to read a json webpage and insert it into a dataframe.

import urllib3
import json
import pandas as pd

url = 'https://covid.ourworldindata.org/data/owid-covid-data.json'
http = urllib3.PoolManager()
r = http.request('GET', url, headers={'Accept': 'application/json'})

data = json.loads(r.data.decode('utf-8'))
data_copy = data

df = pd.DataFrame.from_dict(data_copy, orient='index')
df.transpose()

I need to use the same code I previewed but the output has to be in the following format:

Body:
Instead of using

pd.DataFrame.from_dictionary()

use

pd.DataFrame.from_records()

This method converts structured or record ndarray to DataFrame. If i had some code i would have tried to code the solution.


问:
Connecting a Javascript client with Node.js/express server
My friend finished developing a functioning client website via Javascript. I have my server code running in which it grabs certain data via MySQL. Could you guys give us some tips on how we could connect the Node.js server code to the client side? (in which we're hoping for the client side and server side to be separate, and have a medium connecting both together). From what I've gathered from my research, socket could be a viable option? Any help is greatly appreciated!! </p>

答: You will probebly need some sort of api on the server side. In the client side you can then fetch the data drom the server side. You can use the fetch api of the brower or like TArvela mentioned using ajax or something like that to fetch the (json) data. Creating a api in express is fairly easy and you should be able to find out how you can do that. </p>


问:
grails and AJAX dynamic search
I want to organize search by users usernames.
When I write some letters, I want to see list of users,
which usernames start  with that letters.
After clicking one of values I have to update users data.

Is there any easy plugin or other way to do it?

I use grail-1.3.2.

答: You may use JQuery Autocomplete to implement such feature... Have a look at the demo..</p>

Title: grails and AJAX dynamic search


问:
Do signals contain data pyqt5?
I have trouble understanding whether signals contain some data or not.
for example the signal windowTitleChanged contains a data of type str which needs to be passed to it's slot:

```<code>self.windowTitleChanged.connect(lambda x  : self.onTitle(x))

## or self.windowTitleChanged.connect(self.onTitle) which also automatically sends a data

def onTitle(self,k):
        print(k)
</code>```

While some signals like clicked do not send data unless they are set as checkable (which contain a bool type data). clicking on the pushBotton in the code below does nothing (which surprisingly to me does not raise an error despite the fact that def buttonClicked(self,pressed) requires a pressed argument.

```<code>btn2.clicked.connect(self.buttonClicked)

def buttonClicked(self,pressed):
        if pressed:

            self.statusBar().showMessage("l")
</code>```

It seems to me that some signals like windowTitleChanged contain a data and can be used within the slot, while some contain a None type data (like clicked signal).
Is what i've understood correct?

答:
It seems to me that some signals likewindowTitleChanged contain a data and can be used within the slot, while some contain a None type data (like clicked signal). Is what i've understood correct?

Yes, you are right. You can even see this in action by creating a custom signal.

```<code>from PyQt5.QtCore import pyqtSignal, Qobject
class Analyzer(QObject):
    analyze_completed = pyqtSignal(bool)
</code>```

In the above snippet if you emit the analyze_completed signal with a bool the slots that are connected to this signal will receive that bool as a parameter.

Title: Do signals contain data pyqt5?

Body:
I have trouble understanding whether signals contain some data or not.
for example the signal windowTitleChanged contains a data of type str which needs to be passed to it's slot:

self.windowTitleChanged.connect(lambda x  : self.onTitle(x))

## or self.windowTitleChanged.connect(self.onTitle) which also automatically sends a data

def onTitle(self,k):
        print(k)

While some signals like clicked do not send data unless they are set as checkable (which contain a bool type data). clicking on the pushBotton in the code below does nothing (which surprisingly to me does not raise an error despite the fact that def buttonClicked(self,pressed) requires a pressed argument.

btn2.clicked.connect(self.buttonClicked)

def buttonClicked(self,pressed):
        if pressed:

            self.statusBar().showMessage("l")

It seems to me that some signals like windowTitleChanged contain a data and can be used within the slot, while some contain a None type data (like clicked signal).
Is what i've understood correct?

Body:
It seems to me that some signals likewindowTitleChanged contain a data and can be used within the slot, while some contain a None type data (like clicked signal). Is what i've understood correct?

Yes, you are right. You can even see this in action by creating a custom signal.

from PyQt5.QtCore import pyqtSignal, Qobject
class Analyzer(QObject):
    analyze_completed = pyqtSignal(bool)

In the above snippet if you emit the analyze_completed signal with a bool the slots that are connected to this signal will receive that bool as a parameter.


问:
How to watch for a value which depends on another value vue?
First off, sorry for the vague title; I couldn't come up with a really succinct one. Here's my situation:

Let's say I have users, topics and comments in a vuex store. Users have names, a list of topics as well as the topic they have currently selected:

```<code>users: {
  1: {
      name: 'Tom',
      topics: [1, 2],
      selectedTopic: null // set to 1 and then 2 when user clicks "next"
    },
  2: {... }
},
topics:
  1: {
      title: 'Post!',
      comments: [1, 2],
      selectedComment: null
    },
  2: { ... }
},
comments:
  1: {
      title: 'Cool!'
    },
  2: { ... }
}
</code>```

If a user is selected, I want to show their name. If a user also selects a topic, this topic should be displayed as well; they can also cycle through the topics. For every topic, they can cycle through the comments in the same way. Thus:

```<code>computed: {
  // some getters
  // ...
  user () {
    return this.getUser(this.userId)
  },
  topic () {
    return this.getTopic(this.user.topics[this.user.selectedTopic])
  },
  comment () {
    return this.getComment(this.topic.comments(this.topic.selectedComment])
  }
</code>```

In this case, the cycling works, but the console shows a TypeError: "this.user.topics is undefined" or "this.user.selectedTopicis undefined" at the start (only sometimes), which is true, of course, since we haven't selected a user yet. On the other hand, if I replace this with

```<code>  topic () {
    if (this.user) return this.getTopic(this.user.topics[this.user.selectedTopic])
  }
</code>```

I don't get any errors, but neither does the value update when the selected topic is changed. Is there a good way to handle the issue?

答:
Based on a comment by Ivo Gelov, I arrived at

return this.getTopic((this.user.topics || {})[this.user.selectedTopic])

This seems to do the trick.

Body:
First off, sorry for the vague title; I couldn't come up with a really succinct one. Here's my situation:

Let's say I have users, topics and comments in a vuex store. Users have names, a list of topics as well as the topic they have currently selected:

users: {
  1: {
      name: 'Tom',
      topics: [1, 2],
      selectedTopic: null // set to 1 and then 2 when user clicks "next"
    },
  2: {... }
},
topics:
  1: {
      title: 'Post!',
      comments: [1, 2],
      selectedComment: null
    },
  2: { ... }
},
comments:
  1: {
      title: 'Cool!'
    },
  2: { ... }
}

If a user is selected, I want to show their name. If a user also selects a topic, this topic should be displayed as well; they can also cycle through the topics. For every topic, they can cycle through the comments in the same way. Thus:

computed: {
  // some getters
  // ...
  user () {
    return this.getUser(this.userId)
  },
  topic () {
    return this.getTopic(this.user.topics[this.user.selectedTopic])
  },
  comment () {
    return this.getComment(this.topic.comments(this.topic.selectedComment])
  }

In this case, the cycling works, but the console shows a TypeError: "this.user.topics is undefined" or "this.user.selectedTopicis undefined" at the start (only sometimes), which is true, of course, since we haven't selected a user yet. On the other hand, if I replace this with

  topic () {
    if (this.user) return this.getTopic(this.user.topics[this.user.selectedTopic])
  }

I don't get any errors, but neither does the value update when the selected topic is changed. Is there a good way to handle the issue?


问:
How can I create a column with incremented values from a column with cumulated values in MySQL?
Table1 contains a column with cumulated values (all positive integers):

```<code>id ValuesCum
1  5
2  8
3  20
</code>```

I would like to write a statement that returns an extra column with the incremented values for each row. The output should read something like:

```<code>id ValuesCum ValuesInc
1  5         (5)
2  8         3
3  20        12
</code>```

Does anyone have a solution for this?

答:
You can use a correlated subquery to get the value of ValuesCum of the previous id:

```<code>select t.*,
  t.ValuesCum -
  coalesce((select ValuesCum from tablename where id < t.id order by id desc limit 1), 0) ValuesInc
from tablename t
</code>```

See the demo.

Results:

```<code>| id  | ValuesCum | ValuesInc |
| --- | --------- | --------- |
| 1   | 5         | 5         |
| 2   | 8         | 3         |
| 3   | 20        | 12        |
</code>```

Body:
Table1 contains a column with cumulated values (all positive integers):

id ValuesCum
1  5
2  8
3  20

I would like to write a statement that returns an extra column with the incremented values for each row. The output should read something like:

id ValuesCum ValuesInc
1  5         (5)
2  8         3
3  20        12

Does anyone have a solution for this?

Body:
You can use a correlated subquery to get the value of ValuesCum of the previous id:

select t.*,
  t.ValuesCum -
  coalesce((select ValuesCum from tablename where id < t.id order by id desc limit 1), 0) ValuesInc
from tablename t

See the demo.

Results:

| id  | ValuesCum | ValuesInc |
| --- | --------- | --------- |
| 1   | 5         | 5         |
| 2   | 8         | 3         |
| 3   | 20        | 12        |


问:
Reading S3 files from a manifest and processing them in parallel using Pandas
I have about 50k to read from S3 using a manifest file. I have to read contents of every single (JSON) file into a dataframe and process the files (normalize them as database tables). Right now I have a working code that take about 15hours to process the 50k files. I have to run this as a daily job. Is there any way to parallel process larger amount to files or any better way to speed up process?

Updating the question with the code

```<code>import json
import pandas as pd
import os
import gzip
import boto3
from datetime import datetime,timezone,timedelta

session = boto3.session.Session()
s3 = session.resource('s3')
client = session.client('s3')

#read the S3 inventory report, get the keys of files that are modified on sysdate-1
dt=(datetime.now(timezone.utc) + timedelta(days=-1)).strftime('%Y-%m-%d')
dtz=dt+'T00-00Z'
print('reading inventory report for', dtz)
inventory_bucket = 'xxx'
manifest_key='s3-bucket'+dtz+'/manifest.json'
manifest = json.load(s3.Object(inventory_bucket, manifest_key).get()['Body'])
df=pd.DataFrame()
for obj in manifest['files']:
        gzip_obj = s3.Object(bucket_name=inventory_bucket, key=obj['key'])
        print('csv obj:', gzip_obj)
        buffer = gzip.open(gzip_obj.get()["Body"], mode='rt')
        reader = pd.read_csv(buffer)
        reader.columns=['id','key','modified_date']
        print('converting csv obj to dataframe')
        df=df.append(reader[(reader.modified_date>dt)])
source_keys=list(df['key'])
s3_bucket_source='yyy'

#download the files to a tmp folder
local='/tmp/'
print("downloading from S3")
for k in source_keys:
  k_path=k.replace('/', '-')
  dest_pathname = os.path.join(local, k_path)
  if not os.path.exists(os.path.dirname(dest_pathname)):
      os.makedirs(os.path.dirname(dest_pathname))
      client.download_file(s3_bucket_source, k, dest_pathname)

#read the latest jsons from tmp folder
path_to_json =  os.path.expanduser('/tmp/')
json_files = [pos_json for pos_json in os.listdir(path_to_json) if pos_json.endswith('.json')]

#loop through the jsons and normalize each file contents into df_table1, df_table2, df_table3
for index, js in enumerate(json_files):
    with open(os.path.join(path_to_json, js)) as json_file:
        print('processing file:', js)
        d=json.loads(json_file.read())
        v=d['key1'][0]['key2']
        if isinstance(v, list):
                for i, v2 in enumerate(v):
                    df_table1, df_table2, df_table3 = normalizeJSON(d,i,v2['id'])
                    #normalize is the custom function to split the nested json into relational tables
        else:
            print('invalid json')
</code>```

I use the S3 inventory report to get the list of most recent modified files from the manifest, download the files to a tmp location and read them one by one to do what I need to do

答:
You use the multiprocessing module to download JSON files in parallel.
You code contains 3 for blocks. You can do each one of them in parallel. An example of how to do this for the first for follows:

The first for:

```<code>df=pd.DataFrame()
for obj in manifest['files']:
        gzip_obj = s3.Object(bucket_name=inventory_bucket, key=obj['key'])
        print('csv obj:', gzip_obj)
        buffer = gzip.open(gzip_obj.get()["Body"], mode='rt')
        reader = pd.read_csv(buffer)
        reader.columns=['id','key','modified_date']
        print('converting csv obj to dataframe')
        df=df.append(reader[(reader.modified_date>dt)])
</code>```

becomes:

```<code>def get_reader(obj):
    gzip_obj = s3.Object(bucket_name=inventory_bucket, key=obj['key'])
    print('csv obj:', gzip_obj)
    buffer = gzip.open(gzip_obj.get()["Body"], mode='rt')
    reader = pd.read_csv(buffer)
    reader.columns=['id','key','modified_date']
    print('converting csv obj to dataframe')
    return reader[(reader.modified_date>dt)]

num_of_workers = 4
df = pd.DataFrame()
with multiprocessing.Pool(num_of_workers) as p:
    results = p.map(get_reader, manifest['files'])

for result in results:
    df = df.append(result)
</code>```

You can do the same for the other for blocks

Body:
I have about 50k to read from S3 using a manifest file. I have to read contents of every single (JSON) file into a dataframe and process the files (normalize them as database tables). Right now I have a working code that take about 15hours to process the 50k files. I have to run this as a daily job. Is there any way to parallel process larger amount to files or any better way to speed up process?

Updating the question with the code

import json
import pandas as pd
import os
import gzip
import boto3
from datetime import datetime,timezone,timedelta

session = boto3.session.Session()
s3 = session.resource('s3')
client = session.client('s3')

#read the S3 inventory report, get the keys of files that are modified on sysdate-1
dt=(datetime.now(timezone.utc) + timedelta(days=-1)).strftime('%Y-%m-%d')
dtz=dt+'T00-00Z'
print('reading inventory report for', dtz)
inventory_bucket = 'xxx'
manifest_key='s3-bucket'+dtz+'/manifest.json'
manifest = json.load(s3.Object(inventory_bucket, manifest_key).get()['Body'])
df=pd.DataFrame()
for obj in manifest['files']:
        gzip_obj = s3.Object(bucket_name=inventory_bucket, key=obj['key'])
        print('csv obj:', gzip_obj)
        buffer = gzip.open(gzip_obj.get()["Body"], mode='rt')
        reader = pd.read_csv(buffer)
        reader.columns=['id','key','modified_date']
        print('converting csv obj to dataframe')
        df=df.append(reader[(reader.modified_date>dt)])
source_keys=list(df['key'])
s3_bucket_source='yyy'

#download the files to a tmp folder
local='/tmp/'
print("downloading from S3")
for k in source_keys:
  k_path=k.replace('/', '-')
  dest_pathname = os.path.join(local, k_path)
  if not os.path.exists(os.path.dirname(dest_pathname)):
      os.makedirs(os.path.dirname(dest_pathname))
      client.download_file(s3_bucket_source, k, dest_pathname)

#read the latest jsons from tmp folder
path_to_json =  os.path.expanduser('/tmp/')
json_files = [pos_json for pos_json in os.listdir(path_to_json) if pos_json.endswith('.json')]

#loop through the jsons and normalize each file contents into df_table1, df_table2, df_table3
for index, js in enumerate(json_files):
    with open(os.path.join(path_to_json, js)) as json_file:
        print('processing file:', js)
        d=json.loads(json_file.read())
        v=d['key1'][0]['key2']
        if isinstance(v, list):
                for i, v2 in enumerate(v):
                    df_table1, df_table2, df_table3 = normalizeJSON(d,i,v2['id'])
                    #normalize is the custom function to split the nested json into relational tables
        else:
            print('invalid json')

I use the S3 inventory report to get the list of most recent modified files from the manifest, download the files to a tmp location and read them one by one to do what I need to do

Body:
You use the multiprocessing module to download JSON files in parallel.
You code contains 3 for blocks. You can do each one of them in parallel. An example of how to do this for the first for follows:

The first for:

df=pd.DataFrame()
for obj in manifest['files']:
        gzip_obj = s3.Object(bucket_name=inventory_bucket, key=obj['key'])
        print('csv obj:', gzip_obj)
        buffer = gzip.open(gzip_obj.get()["Body"], mode='rt')
        reader = pd.read_csv(buffer)
        reader.columns=['id','key','modified_date']
        print('converting csv obj to dataframe')
        df=df.append(reader[(reader.modified_date>dt)])

becomes:

def get_reader(obj):
    gzip_obj = s3.Object(bucket_name=inventory_bucket, key=obj['key'])
    print('csv obj:', gzip_obj)
    buffer = gzip.open(gzip_obj.get()["Body"], mode='rt')
    reader = pd.read_csv(buffer)
    reader.columns=['id','key','modified_date']
    print('converting csv obj to dataframe')
    return reader[(reader.modified_date>dt)]

num_of_workers = 4
df = pd.DataFrame()
with multiprocessing.Pool(num_of_workers) as p:
    results = p.map(get_reader, manifest['files'])

for result in results:
    df = df.append(result)

You can do the same for the other for blocks


问:
Flutter "argument type not assignable" Error with two identical types
Flutter shows me this error, but the two types are identical.

[CartItem it's just a simple model; there was a conflict since another widget had the same name but I resolved it using "as" in the import statement]

The argument type List<CartItems> (where CartItem is defined in /Users/marco/Documenti Locali/shop_app/lib/providers/cart.dart)' can't be assigned to the parameter type  List<CartItems> (where CartItem is defined in /Users/marco/Documenti Locali/shop_app/lib/providers/cart.dart)'.dartargument_type_not_assignable

list.dart(56, 16): List is defined in /Users/marco/flutter/bin/cache/pkg/sky_engine/lib/core/list.dart

cart.dart(3, 7): CartItem is defined in /Users/marco/Documenti Locali/shop_app/lib/providers/cart.dart

list.dart(56, 16): List is defined in /Users/marco/flutter/bin/cache/pkg/sky_engine/lib/core/list.dart

cart.dart(3, 7): CartItem is defined in /Users/marco/Documenti Locali/shop_app/lib/providers/cart.dart

Peek Problem (⌥F8)
No quick fixes available

答:
you should use the library prefix in the usage too so if your namespace is

import 'package:app/xxxx/shop_app/lib/providers/cart.dart as shop_cart;

you should specify it in the generic List as well such as;

List shopCarts = etc...;


问:
count of subset with given sum
I was trying to find a way to give back the count of subsets that are equal to the sum. the array would include 5 numbers. I wrote a code that at the end counts all the branches instead of the correct subsets that are equal to the sum and I can't find out how to change it.</p>

```<code>import java.util.*;

public class Assignment4 {

    public static int countSubsetSums(int[] arr, int sum) {
        return countSubsetSums(arr,sum, 0);
    }

    public static int countSubsetSums(int[] arr, int sum, int i) {
        int count=0;
        if(sum==0) {
            count++;
        }
        else if(i >= arr.length) {
            return count;
        }
        else {
            if(sum>=0) {
                count= countSubsetSums(arr, sum- arr[i], i++) + countSubsetSums(arr, sum, i++);
            }
        }
        return count;
    }

    public static void main(String[] args) {
        Scanner myScanner= new Scanner(System.in);
        System.out.println("Enter 5 numbers into array");
        int [] myArr= new int [5];
        for (int i = 0; i < myArr.length; i++) {
            myArr[i]= myScanner.nextInt();
        }
        System.out.println("Enter a number into sum");
        int sum= myScanner.nextInt();
        System.out.println(countSubsetSums(myArr,sum));

    }

}
</code>```

答:
I think your logic was mostly right, but the base cases were a little off (fixed code below):

If i is out of bounds or sum is negative, then there is no subsets that sum to the given sum starting past i, so we return 0.

If the sum is equal to 0, then there is only one subset of arr that equals sum which is the empty subset (assuming arr elements are strictly positive. If arr can include 0s, then this is a totally different problem).

Lastly, if a[i] is less than sum, then we can add the count of subsets that use a[i] else we just counts subsets without a[i]

```<code>public static int countSubsetSums(int[] arr, int sum) {
    return countSubsetSums(arr,sum, 0);
}

public static int countSubsetSums(int[] arr, int sum, int i) {
    if (i >= arr.length || sum < 0) {
      return 0;
    }
    if (sum == 0) {
        return 1;
    }

    int count=0;
    if (arr[i] <= sum) {
      count = countSubsetSums(arr, sum - arr[i], i+1);
    }
    count += countSubsetSums(arr, sum, i+1);
    return count;
}
</code>```

Title: count of subset with given sum

Body:
I was trying to find a way to give back the count of subsets that are equal to the sum. the array would include 5 numbers. I wrote a code that at the end counts all the branches instead of the correct subsets that are equal to the sum and I can't find out how to change it.

import java.util.*;

public class Assignment4 {

    public static int countSubsetSums(int[] arr, int sum) {
        return countSubsetSums(arr,sum, 0);
    }

    public static int countSubsetSums(int[] arr, int sum, int i) {
        int count=0;
        if(sum==0) {
            count++;
        }
        else if(i >= arr.length) {
            return count;
        }
        else {
            if(sum>=0) {
                count= countSubsetSums(arr, sum- arr[i], i++) + countSubsetSums(arr, sum, i++);
            }
        }
        return count;
    }

    public static void main(String[] args) {
        Scanner myScanner= new Scanner(System.in);
        System.out.println("Enter 5 numbers into array");
        int [] myArr= new int [5];
        for (int i = 0; i < myArr.length; i++) {
            myArr[i]= myScanner.nextInt();
        }
        System.out.println("Enter a number into sum");
        int sum= myScanner.nextInt();
        System.out.println(countSubsetSums(myArr,sum));

    }

}

Body:
I think your logic was mostly right, but the base cases were a little off (fixed code below):

If i is out of bounds or sum is negative, then there is no subsets that sum to the given sum starting past i, so we return 0.

If the sum is equal to 0, then there is only one subset of arr that equals sum which is the empty subset (assuming arr elements are strictly positive. If arr can include 0s, then this is a totally different problem).

Lastly, if a[i] is less than sum, then we can add the count of subsets that use a[i] else we just counts subsets without a[i]

public static int countSubsetSums(int[] arr, int sum) {
    return countSubsetSums(arr,sum, 0);
}

public static int countSubsetSums(int[] arr, int sum, int i) {
    if (i >= arr.length || sum < 0) {
      return 0;
    }
    if (sum == 0) {
        return 1;
    }

    int count=0;
    if (arr[i] <= sum) {
      count = countSubsetSums(arr, sum - arr[i], i+1);
    }
    count += countSubsetSums(arr, sum, i+1);
    return count;
}


问:
Clean up the tweets for sentiment analysis
I used the following code to clean the corpus of the Username in the Tweet. But some names are in the word cloud. Why are they wrong?

removeUsername <- function(x) gsub('@[^[:space:]]*', '', x)

removeURL <- function(x) gsub("http[:alnum:]*","",x)
removeNumPunct<- function(x) gsub("[^[:alpha:][:space:]]*","",x)
removeUsername <- function(x) gsub('@[^[:space:]]*', '', x)
mycorpus <- tm_map(mycorpus,PlainTextDocument)
mycorpus <- tm_map(mycorpus,content_transformer(removeURL))
mycorpus <- tm_map(mycorpus,content_transformer(tolower))
mycorpus <- tm_map(mycorpus,stripWhitespace)
mycorpus <- tm_map(mycorpus,content_transformer(removeNumPunct))
mycorpus <- tm_map(mycorpus,content_transformer(removeNumbers))
mycorpus <- tm_map(mycorpus, content_transformer(removeUsername))

Stopwords <- readLines("/Users/Stopwords.txt")
mycorpus <- tm_map(mycorpus, function(x) removeWords(x, Stopwords ))

```

答:
Your problem is the order in which you execute your code. You first remove punctuation, and then you want to remove twitter handles. Which will not be found as removing the punctuation already removed the @. Same goes from stripping the whitespace, this is a step you should do at the end.

The following order should take care of the twitter handles and whitespace.

```<code>mycorpus <- tm_map(mycorpus, PlainTextDocument)
mycorpus <- tm_map(mycorpus, content_transformer(removeUsername))
mycorpus <- tm_map(mycorpus, content_transformer(removeURL))
mycorpus <- tm_map(mycorpus, content_transformer(tolower))
mycorpus <- tm_map(mycorpus, content_transformer(removeNumPunct))
mycorpus <- tm_map(mycorpus, content_transformer(removeNumbers))
mycorpus <- tm_map(mycorpus, stripWhitespace)
</code>```

Also tm already contains functions fro removing puntuation and numbers, which can be used without content_transformer, like in the example below.

```<code>mycorpus <- tm_map(mycorpus, removePunctuation)
mycorpus <- tm_map(mycorpus, removeNumbers)
</code>```

You could check the package quanteda. It has a lot of these twitter functions already included as flags in the tokens function.

Body:
Your problem is the order in which you execute your code. You first remove punctuation, and then you want to remove twitter handles. Which will not be found as removing the punctuation already removed the @. Same goes from stripping the whitespace, this is a step you should do at the end.

The following order should take care of the twitter handles and whitespace.

mycorpus <- tm_map(mycorpus, PlainTextDocument)
mycorpus <- tm_map(mycorpus, content_transformer(removeUsername))
mycorpus <- tm_map(mycorpus, content_transformer(removeURL))
mycorpus <- tm_map(mycorpus, content_transformer(tolower))
mycorpus <- tm_map(mycorpus, content_transformer(removeNumPunct))
mycorpus <- tm_map(mycorpus, content_transformer(removeNumbers))
mycorpus <- tm_map(mycorpus, stripWhitespace)

Also tm already contains functions fro removing puntuation and numbers, which can be used without content_transformer, like in the example below.

mycorpus <- tm_map(mycorpus, removePunctuation)
mycorpus <- tm_map(mycorpus, removeNumbers)

You could check the package quanteda. It has a lot of these twitter functions already included as flags in the tokens function.


问:
Getting Error while fetching data location wise with $near query in mongoose
```<code>query = {
  "location" : {
       "$near" : {
                    "$geometry": {
                        "type": "Point" ,
                        "coordinates": [18.55,73.78]
                    }
                }
            }
       }
  }

planner returned error: unable to find index for $geoNear query
    at Connection.<anonymous> (D:\Projects\api\node_modules\mongodb-core\lib\connection\pool.js:443:61)
    at Connection.emit (events.js:198:13)
    at Connection.EventEmitter.emit (domain.js:466:23)
    at processMessage (D:\Projects\api\node_modules\mongodb-core\lib\connection\connection.js:364:10)
    at Socket.<anonymous> (D:\Projects\api\node_modules\mongodb-core\lib\connection\connection.js:533:15)
    at Socket.emit (events.js:198:13)
    at Socket.EventEmitter.emit (domain.js:466:23)
    at addChunk (_stream_readable.js:288:12)
    at readableAddChunk (_stream_readable.js:269:11)
    at Socket.Readable.push (_stream_readable.js:224:10)
    at TCP.onStreamRead [as onread] (internal/stream_base_commons.js:94:17)
  ok: 0,
</code>```

For this I have created 2dsphere on location field as specified in the below link

https://docs.mongodb.com/manual/core/2dsphere/

```<code>db.collectionName.createIndex( { location : "2dsphere" } )
</code>```

Help me to understand If I am doing anything wrong.

答:
try changing $near to $nearSphere and also add $minDistance: 0 and $maxDistance: yourmaxdistance next to $geometry if it still doesnt work...so it will look like

```<code>query = {
  location: {
     $nearSphere: {
        $geometry: {
            type: "Point" ,
            coordinates: [18.55,73.78],
        },
        $minDistance: 0,
        $maxDistance: //your max will go here
     }
  }
}
</code>```

If the above does not work try creating the index again and keep a look out for the output...

Here is an example of an index creation that did not work...

```<code>{
        "ok" : 0,
        "errmsg" : "Unknown index plugin '2dSphere'",
        "code" : 67,
        "codeName" : "CannotCreateIndex",
        "operationTime" : Timestamp(1590940517, 1),
        "$clusterTime" : {
                "clusterTime" : Timestamp(1590940517, 1),
                "signature" : {
                        "hash" : BinData(0,""),
                        "keyId" : NumberLong("")
                }
        }
}
</code>```

Body:
query = {
  "location" : {
       "$near" : {
                    "$geometry": {
                        "type": "Point" ,
                        "coordinates": [18.55,73.78]
                    }
                }
            }
       }
  }

planner returned error: unable to find index for $geoNear query
    at Connection.<anonymous> (D:\Projects\api\node_modules\mongodb-core\lib\connection\pool.js:443:61)
    at Connection.emit (events.js:198:13)
    at Connection.EventEmitter.emit (domain.js:466:23)
    at processMessage (D:\Projects\api\node_modules\mongodb-core\lib\connection\connection.js:364:10)
    at Socket.<anonymous> (D:\Projects\api\node_modules\mongodb-core\lib\connection\connection.js:533:15)
    at Socket.emit (events.js:198:13)
    at Socket.EventEmitter.emit (domain.js:466:23)
    at addChunk (_stream_readable.js:288:12)
    at readableAddChunk (_stream_readable.js:269:11)
    at Socket.Readable.push (_stream_readable.js:224:10)
    at TCP.onStreamRead [as onread] (internal/stream_base_commons.js:94:17)
  ok: 0,

For this I have created 2dsphere on location field as specified in the below link

https://docs.mongodb.com/manual/core/2dsphere/

db.collectionName.createIndex( { location : "2dsphere" } )

Help me to understand If I am doing anything wrong.

Body:
try changing $near to $nearSphere and also add $minDistance: 0 and $maxDistance: yourmaxdistance next to $geometry if it still doesnt work...so it will look like

query = {
  location: {
     $nearSphere: {
        $geometry: {
            type: "Point" ,
            coordinates: [18.55,73.78],
        },
        $minDistance: 0,
        $maxDistance: //your max will go here
     }
  }
}

If the above does not work try creating the index again and keep a look out for the output...

Here is an example of an index creation that did not work...

{
        "ok" : 0,
        "errmsg" : "Unknown index plugin '2dSphere'",
        "code" : 67,
        "codeName" : "CannotCreateIndex",
        "operationTime" : Timestamp(1590940517, 1),
        "$clusterTime" : {
                "clusterTime" : Timestamp(1590940517, 1),
                "signature" : {
                        "hash" : BinData(0,""),
                        "keyId" : NumberLong("")
                }
        }
}


问:
Problem with rails + json
my controller admin

```<code>  def index_all_for_question_id

      @answers      = Question.find(params[:id]).answers

    respond_to do |format|
      format.ext_json { render :json => @answers.to_ext_json(:class => Answer, :include => [:respondent]) }
    end
  end
</code>```

my dataStore in extJS

```<code> var answers_datastore = new Ext.data.Store({
    autoLoad: true,
    proxy: new Ext.data.HttpProxy({
       url: '/answers/index_all_for_question_id/<%= @questions.first.id %>?format=ext_json',
        method: 'GET'}),
        reader: answers_reader
        });
</code>```

My question is: When i trying to enter
localhost:3000/answers/index_all_for_question_id/551?format=ext_json for example,

I get :

{"results":2,"answers":[{"answer":{"scale":1,"inquiry_id":277,"created_at":"2011-05-30T07:10:22Z","updated_at":"2011-05-30T07:10:22Z","text":"dfgfdghfdhfdh","id":275,"respondent":{"created_at":"2011-05-16T06:47:08Z","updated_at":"2011-05-16T06:47:08Z","id":109,"user_id":6,"email":"xxx@xxx"}}},{"answer":{"scale":1,"inquiry_id":278,"created_at":"2011-05-31T12:33:36Z","updated_at":"2011-05-31T12:33:36Z","text":"lolololol","id":290,"respondent":{"created_at":"2011-05-25T11:22:55Z","updated_at":"2011-05-25T11:22:55Z","id":110,"user_id":6,"email":"xxx@xxx"}}}]}

SO i dont what it! Every user can enter this line i get answers (if he not an admin). How i can solve this problem? Only admins can see this, for others should be redirect on some page or appears text "Sorry!".

Thank you

答: Hi I think that the simplest solution is to write custom before_filter where you can check if user has admin role and redirect to some page if no also you can use CanCan for authorization purposes

Title: Problem with rails + json

Body:
my controller admin

  def index_all_for_question_id

      @answers      = Question.find(params[:id]).answers

    respond_to do |format|
      format.ext_json { render :json => @answers.to_ext_json(:class => Answer, :include => [:respondent]) }
    end
  end

my dataStore in extJS

 var answers_datastore = new Ext.data.Store({
    autoLoad: true,
    proxy: new Ext.data.HttpProxy({
       url: '/answers/index_all_for_question_id/<%= @questions.first.id %>?format=ext_json',
        method: 'GET'}),
        reader: answers_reader
        });

My question is: When i trying to enter
localhost:3000/answers/index_all_for_question_id/551?format=ext_json for example,

I get :

{"results":2,"answers":[{"answer":{"scale":1,"inquiry_id":277,"created_at":"2011-05-30T07:10:22Z","updated_at":"2011-05-30T07:10:22Z","text":"dfgfdghfdhfdh","id":275,"respondent":{"created_at":"2011-05-16T06:47:08Z","updated_at":"2011-05-16T06:47:08Z","id":109,"user_id":6,"email":"xxx@xxx"}}},{"answer":{"scale":1,"inquiry_id":278,"created_at":"2011-05-31T12:33:36Z","updated_at":"2011-05-31T12:33:36Z","text":"lolololol","id":290,"respondent":{"created_at":"2011-05-25T11:22:55Z","updated_at":"2011-05-25T11:22:55Z","id":110,"user_id":6,"email":"xxx@xxx"}}}]}

SO i dont what it! Every user can enter this line i get answers (if he not an admin). How i can solve this problem? Only admins can see this, for others should be redirect on some page or appears text "Sorry!".

Thank you


问:
What is the difference between assertIs and assertTrue in unittest module, when these methods are applied on True values?
My working environment:

```<code>OS: Ubuntu 18.04 (64 bits)
Python version: 3.8.0 (64 bits)
</code>```

I've a question about the unittest documentation, more precisely, the difference between assertTrue and assertIs methods. Here is what the online documentation says:

https://docs.python.org/3/library/unittest.html

assertTrue(expr, msg=None)

assertFalse(expr, msg=None)

Test that expr is true (or false).

Note that this is equivalent to bool(expr) is True and  not to expr is
  True (use assertIs(expr, True) for the latter).

I don't really understand the difference, that is, in the case where I have a boolean type with a True value, then why should I use assertIs instead of assertTrue? Given that bool(True) is True also returns the True boolean value I don't see the difference and impact on tests. I'd appreciate if you could kindly make some clarification.

答:
To answer the question you have to understand the difference between is and == comparison operators, imagine you have two different boys, but they are both Peters, they have the same name so == will give you true, but is will give you False, because they are not one person.

import unittest

class MyClass:
    def __init__(self, value):
        self.value = value

    def __eq__(self, other):
        return self.value == other.value

class TestStringMethods(unittest.TestCase):

    def test_your_func(self):
        x = MyClass(5)
        y = MyClass(5)
        z = x
        print(x is y)  # False x is not y, they are separate object
        print(x == y)  # True 5 == 5
        print(x is z)  # True, because z is the reference to the same object
        print(z == y)  # True 5 == 5
        # self.assertIs(x, y)  # if you uncomment the line you'll get error
        self.assertIs(x, z)

if __name__ == '__main__':
    unittest.main()

```

As per AssertTruem it has absolutely different purpose to use, e.g. if []: is False and if [1] is True.

Body:
My working environment:

OS: Ubuntu 18.04 (64 bits)
Python version: 3.8.0 (64 bits)

I've a question about the unittest documentation, more precisely, the difference between assertTrue and assertIs methods. Here is what the online documentation says:

https://docs.python.org/3/library/unittest.html

assertTrue(expr, msg=None)

assertFalse(expr, msg=None)

Test that expr is true (or false).

Note that this is equivalent to bool(expr) is True and  not to expr is
  True (use assertIs(expr, True) for the latter).

I don't really understand the difference, that is, in the case where I have a boolean type with a True value, then why should I use assertIs instead of assertTrue? Given that bool(True) is True also returns the True boolean value I don't see the difference and impact on tests. I'd appreciate if you could kindly make some clarification.

Body:
To answer the question you have to understand the difference between is and == comparison operators, imagine you have two different boys, but they are both Peters, they have the same name so == will give you true, but is will give you False, because they are not one person.

import unittest

class MyClass:
    def __init__(self, value):
        self.value = value

    def __eq__(self, other):
        return self.value == other.value

class TestStringMethods(unittest.TestCase):

    def test_your_func(self):
        x = MyClass(5)
        y = MyClass(5)
        z = x
        print(x is y)  # False x is not y, they are separate object
        print(x == y)  # True 5 == 5
        print(x is z)  # True, because z is the reference to the same object
        print(z == y)  # True 5 == 5
        # self.assertIs(x, y)  # if you uncomment the line you'll get error
        self.assertIs(x, z)

if __name__ == '__main__':
    unittest.main()

As per AssertTruem it has absolutely different purpose to use, e.g. if []: is False and if [1] is True.


问:
How to calculate the number of the week from the start and end date period?
the main idea is to get the number of the week between two dates (from a period of start date and end date)!
Something like that: if the period is 01-05-2020 to 31-05-2020 and in the data picker I chouse 08-05-2020 result will be 2, the second week.

can someone help with that can't figure out by my self,
thank you!

here is JS date picker code line with setting up period 01-05-2020 to 31-05-2020, how to echo out from this js code number of week for future php usage or input value?

js

```<code><!-- js -->
<script src='https://cdnjs.cloudflare.com/ajax/libs/jquery/3.5.1/jquery.min.js' type='text/javascript'></script>
<!-- Datepicker -->
<link href='https://cdnjs.cloudflare.com/ajax/libs/bootstrap-datepicker/1.9.0/css/bootstrap-datepicker.min.css' rel='stylesheet' type='text/css'>
<script src='https://cdnjs.cloudflare.com/ajax/libs/bootstrap-datepicker/1.9.0/js/bootstrap-datepicker.min.js' type='text/javascript'></script>

<script type="text/javascript">
$(document).ready(function(){

  $('#datepicker').datepicker({
    format: "dd/mm/yyyy",
    startDate: new Date('2020-5-1'),
    endDate: new Date('2020-5-31')
  });

});
</script>
</code>```

html

```<code><div class="form-group">
<label for="formGroupExampleInput">Date of sale</label>
<input type="text" name="dos" value="<?php echo $doc; ?>" class="form-control" id='datepicker' id="formGroupExampleInput">
</div>
</code>```

答:
Thanks, I found a solution to my problem.
Here it is, maybe someone needed..

```<code><script>
function diff_weeks(dt2, dt1)
 {

  var diff =(dt2.getTime() - dt1.getTime()) / 1000;
  diff /= (60 * 60 * 24 * 7);
  return Math.abs(Math.round(diff));

 }

dt1 = new Date(2014,10,2);
dt2 = new Date(2014,10,11);
console.log(diff_weeks(dt1, dt2));

dt1 = new Date("June 13, 2014 08:11:00");
dt2 = new Date("October 19, 2014 11:13:00");
console.log(diff_weeks(dt1, dt2));
<script>
</code>```

lnk.
https://www.w3resource.com/javascript-exercises/javascript-date-exercise-47.php

Body:
the main idea is to get the number of the week between two dates (from a period of start date and end date)!
Something like that: if the period is 01-05-2020 to 31-05-2020 and in the data picker I chouse 08-05-2020 result will be 2, the second week.

can someone help with that can't figure out by my self,
thank you!

here is JS date picker code line with setting up period 01-05-2020 to 31-05-2020, how to echo out from this js code number of week for future php usage or input value?

js

<link href='https://cdnjs.cloudflare.com/ajax/libs/bootstrap-datepicker/1.9.0/css/bootstrap-datepicker.min.css' rel='stylesheet' type='text/css'>

html

Date of sale
<input type="text" name="dos" value="<?php echo $doc; ?>" class="form-control" id='datepicker' id="formGroupExampleInput">

Body:
Thanks, I found a solution to my problem.
Here it is, maybe someone needed..

<script>
function diff_weeks(dt2, dt1)
 {

  var diff =(dt2.getTime() - dt1.getTime()) / 1000;
  diff /= (60 * 60 * 24 * 7);
  return Math.abs(Math.round(diff));

 }

dt1 = new Date(2014,10,2);
dt2 = new Date(2014,10,11);
console.log(diff_weeks(dt1, dt2));

dt1 = new Date("June 13, 2014 08:11:00");
dt2 = new Date("October 19, 2014 11:13:00");
console.log(diff_weeks(dt1, dt2));
<script>

lnk.
https://www.w3resource.com/javascript-exercises/javascript-date-exercise-47.php


问:
How does DatagramSocket receive packet everytime and does not stuck
I create a DatagramSocket.
When I am waiting for the packet, I cannot do anything until the packet is received.
If I want to do something, such as print something when I am waiting for the packet.
How do I do it?

This is my code.

```<code>private DatagramSocket socket = new DatagramSocket(8080);
private DatagramPacket packet = new DatagramPacket();
bool b = true;
int x = 0;

do {
  socket.receive(packet);
  //if I receive a packet, print the payload of the packet
  System.out.println(x);
  x++;
} while (b);
</code>```

I want to keep print the "x". If I receive a packet, I will print the payload of the packet.

or

Can I put the receive function on the background? When I receive a packet, I print it immediately.

答: For this case, you can use threads. You are giving your main thread to infinite loop so that you can't do anything outside of the loop. If you create one more thread and use it for receiving packets, then your main thread can do other works.</p>

Body:
I create a DatagramSocket.
When I am waiting for the packet, I cannot do anything until the packet is received.
If I want to do something, such as print something when I am waiting for the packet.
How do I do it?

This is my code.

private DatagramSocket socket = new DatagramSocket(8080);
private DatagramPacket packet = new DatagramPacket();
bool b = true;
int x = 0;

do {
  socket.receive(packet);
  //if I receive a packet, print the payload of the packet
  System.out.println(x);
  x++;
} while (b);

I want to keep print the "x". If I receive a packet, I will print the payload of the packet.

or

Can I put the receive function on the background? When I receive a packet, I print it immediately.


问:
Platform IO :undefined reference to `init'/[.pio\build\uno\firmware.elf] Error 1
I use visual studio code for programing arduino uno using platform io. But when I build my code there is no mistake I seen on problem. there is no error script on there. but on terminal says like this:

Executing task: C:\Users\ridho.platformio\penv\Scripts\platformio.exe run <

Processing uno (platform: atmelavr; board: uno; framework: arduino)

Verbose mode can be enabled via -v, --verbose option

CONFIGURATION: https://docs.platformio.org/page/boards/atmelavr/uno.html

PLATFORM: Atmel AVR 2.1.0 > Arduino Uno

HARDWARE: ATMEGA328P 16MHz, 2KB RAM, 31.50KB Flash

DEBUG: Current (simavr) On-board (simavr)

PACKAGES:

framework-arduino-avr 5.0.0

toolchain-atmelavr 1.50400.190710 (5.4.0)

LDF: Library Dependency Finder ->

LDF Modes: Finder ~ chain, Compatibility ~ soft

Found 6 compatible libraries

Scanning dependencies...

No dependencies

Building in release mode

Linking .pio\build\uno\firmware.elf

C:\Users\ridho\AppData\Local\Temp\ccrKx04Q.ltrans0.ltrans.o: In function main':
***<artificial>:(.text.startup+0x0): undefined reference toinit'*
collect2.exe: error: ld returned 1 exit status
****** [.pio\build\uno\firmware.elf] Error 1*
==================================================== [FAILED] Took 1.43 seconds ====================================================
The terminal process terminated with exit code: 1

Terminal will be reused by tasks, press any key to close it.

I assume that the main problem is : undefined reference to `init' or*** [.pio\build\uno\firmware.elf] Error 1

I have read all same problem like this ask in platform io community ,and professional said that the solution is disable any antivirus. I just have kaspersky antivirus on my pc. I also disable win defender,win firewall. but when I build again. still same. anyone can help me?

this my antivirus that have been disabled by me:
this image by clicking

答:
just reinstall the VScode and Platform IO
Do a Clean install.
Delete all the files of the Platform IO and also Arduino files </p>


问:
Passing get paramter to controller
I realise this is probably a simple question. I'm trying to pass a get parameter to Scala Play controller :

I'm using Play2 and reading the doc:
https://www.playframework.com/documentation/2.8.x/ScalaActions it does not seem to be specified.

Here is my function in a controller:

```<code>  def sj(name: String) = Action { implicit request =>

      println("Name: "+name)
      println("Request is:"+request)
      Ok(Json.toJson(List(1,2,3)).toString());

  }
</code>```

Route points to this controller:

```<code>GET     /chart/data                    controllers.ChartController.sj
</code>```

Invoking http://localhost:9000/chart/data?name=test

invokes the function sj defined above but the println statements are not being printed to the console. How to pass the get parameter "name" to a controller function ?

答:
Modify your route:</p>

```<code>GET     /chart/data/:name                    controllers.ChartController.sj(name: string)
</code>```

Title: Passing get paramter to controller

Body:
I realise this is probably a simple question. I'm trying to pass a get parameter to Scala Play controller :

I'm using Play2 and reading the doc:
https://www.playframework.com/documentation/2.8.x/ScalaActions it does not seem to be specified.

Here is my function in a controller:

  def sj(name: String) = Action { implicit request =>

      println("Name: "+name)
      println("Request is:"+request)
      Ok(Json.toJson(List(1,2,3)).toString());

  }

Route points to this controller:

GET     /chart/data                    controllers.ChartController.sj

Invoking http://localhost:9000/chart/data?name=test

invokes the function sj defined above but the println statements are not being printed to the console. How to pass the get parameter "name" to a controller function ?

Body:
Modify your route:

GET     /chart/data/:name                    controllers.ChartController.sj(name: string)


问:
What is the difference between a normal file download and a PHP-driven download?
Is there a difference between directly downloading a file from a web server, and downloading the same file through a PHP script that uses these headers?  (Assume that all of the variables here contain correct values for the file being downloaded.)

```<code>header('Content-Description: File Transfer');
header('Content-Type: ' . $mimeType);
header('Content-Disposition: attachment; filename='.basename($file));
header('Content-Transfer-Encoding: binary');
header('Expires: 0');
header('Cache-Control: must-revalidate, post-check=0, pre-check=0');
header('Pragma: public');
header('Content-Length: ' . filesize($file));
</code>```

After reading the discussion here, i want to change the question :

With a file on server for user who can download.

What & when do we need to process this task by PHP script?

Same question with direct link of file

答: I would think this would also let you create a download for a resource that is normally forbidden for a user to directly access. It also seems like you could use this for a file that only exists in memory, without having to write it out to disk first.</p>

Body:
Is there a difference between directly downloading a file from a web server, and downloading the same file through a PHP script that uses these headers?  (Assume that all of the variables here contain correct values for the file being downloaded.)

header('Content-Description: File Transfer');
header('Content-Type: ' . $mimeType);
header('Content-Disposition: attachment; filename='.basename($file));
header('Content-Transfer-Encoding: binary');
header('Expires: 0');
header('Cache-Control: must-revalidate, post-check=0, pre-check=0');
header('Pragma: public');
header('Content-Length: ' . filesize($file));

After reading the discussion here, i want to change the question :

With a file on server for user who can download.

What & when do we need to process this task by PHP script?

Same question with direct link of file


问:
the column in csv that comes from the index of DataFrame does not have a header name
here is a pandas DataFrame

```<code>>>> print(df)
   A  B  C
0  0  1  2
1  3  4  5
2  6  7  8
</code>```

with df.to_csv('df.csv') I got this file

the column in csv that comes from the index of DataFrame does not have a header name. Is it possible to specify a column name with pandas?

答:
Try with rename_axis

```<code>df.rename_axis('index').to_csv('df.csv')
</code>```

Body:
here is a pandas DataFrame

>>> print(df)
   A  B  C
0  0  1  2
1  3  4  5
2  6  7  8

with df.to_csv('df.csv') I got this file

the column in csv that comes from the index of DataFrame does not have a header name. Is it possible to specify a column name with pandas?

Body:
Try with rename_axis

df.rename_axis('index').to_csv('df.csv')


问:
Adding images to Readme of plugin in pub.dev
I have a plugin called UPI India in pub.dev whose readme contains two images. Both images are visible in Readme of Github but not in the Readme of pub.dev

Images in Pub

Images in Github

I am using absolute path in Readme.

```

Does anyone know why this is happening?

答:
Problem solved!

I used the cdn link instead of repository link.

```

Hope it helps someone in future.

Body:
I have a plugin called UPI India in pub.dev whose readme contains two images. Both images are visible in Readme of Github but not in the Readme of pub.dev

Images in Pub

Images in Github

I am using absolute path in Readme.

Does anyone know why this is happening?

Body:
Problem solved!

I used the cdn link instead of repository link.

<img src="https://user-images.githubusercontent.com/42082172/82893565-d3643480-9f6e-11ea-96a7-493181df6214.gif" alt="How example looks" width="300" height="540">

<img src="https://user-images.githubusercontent.com/42082172/82893023-1245ba80-9f6e-11ea-868d-a55062d34086.jpg" alt="Success Status" width="300" height="540">

Hope it helps someone in future.


问:
Adding additional NICs to a Virtual Machine in Azure?
Is it true, that even for Virtual Machine's created in the latest platform (ARM), that if you initially created the machine with 1 NIC, that there is no way to add additional NICs to the VM?

I found a few random (non-Microsoft) articles that seem to indicate this is the case, which if so... is kind of retarded.

So I wanted to make sure that I'm understanding this correctly.

If I have to start all over and build a new machine just to add a NIC, I might consider just using Amazon AWS as I can't imagine this being a limitation over there.

答: There is an update to this. You can now add nics to existing Azure VMs. See the documentation here for tutorial - https://learn.microsoft.com/en-us/azure/virtual-network/virtual-network-network-interface-vm</p>


问:
How can I access variables in vector<struct> *obj?
How would I get my variables out of a vector?

I can't use the binary insertion operators or the equal operators.

Earlier, I declared a vector<someStruct> *vObj and allocated it, then returned the vObj
and called it in this function:

```<code>vector<sameStruct> firstFunc();

for (unsigned int x = 0; x < v->size(); x++)
{
   v[x];
}
</code>```

when I debug it, v[x] now has the full contents of the original vector, as it did before without the subscript/index.

But I don't think I've done anything to progress beyond that.

I just have about 4 variables inside my vector; when I debug it, it has the information that I need, but I can't get to it.

答:
As it is written v is a pointer to a vector of structs.

When you index directly into v all you are doing is pointer arithmatic. v[x] is the vector of structs at position x (assuming that v is an array if it is just a single object at the end of the pointer then v[x] for x>0 is just garbage). This is because it is applying the [x] not to the vector pointed to by v but to the pointer v itself.

You need to dereference the pointer and then index into the vector using something like:

```<code>(*v)[x];
</code>```

At this point you have a reference to the object at the xth position of the vector to get at its member functions / variables use:

```<code>(*v)[x].variable;
</code>```

or

```<code>(*v)[x].memberfunction(parameters);
</code>```

If you do not want to dereference the vector then access the element within it you might try something like:

```<code>v->at(x);
v->at(x).variable;
v->at(x).memberfunction;
</code>```

This way you are accessing a member function of an object in exactly the same manner as when you called:

```<code>v->size();
</code>```

I hope that this helps.

ContentLicense: CC BY-SA 2.5

Body:
How would I get my variables out of a vector?

I can't use the binary insertion operators or the equal operators.

Earlier, I declared a vector<someStruct> *vObj and allocated it, then returned the vObj
and called it in this function:

vector<sameStruct> firstFunc();

for (unsigned int x = 0; x < v->size(); x++)
{
   v[x];
}

when I debug it, v[x] now has the full contents of the original vector, as it did before without the subscript/index.

But I don't think I've done anything to progress beyond that.

I just have about 4 variables inside my vector; when I debug it, it has the information that I need, but I can't get to it.

LastEditorDisplayName: Daniel L

Body:
As it is written v is a pointer to a vector of structs.

When you index directly into v all you are doing is pointer arithmatic. v[x] is the vector of structs at position x (assuming that v is an array if it is just a single object at the end of the pointer then v[x] for x>0 is just garbage). This is because it is applying the [x] not to the vector pointed to by v but to the pointer v itself.

You need to dereference the pointer and then index into the vector using something like:

(*v)[x];

At this point you have a reference to the object at the xth position of the vector to get at its member functions / variables use:

(*v)[x].variable;

or

(*v)[x].memberfunction(parameters);

If you do not want to dereference the vector then access the element within it you might try something like:

v->at(x);
v->at(x).variable;
v->at(x).memberfunction;

This way you are accessing a member function of an object in exactly the same manner as when you called:

v->size();

I hope that this helps.

OwnerDisplayName: James Matta

LastEditorDisplayName: James Matta


问:
Debug Error! HEAP CORRUPTION DETECTED in C
I tried to use dynamic allocation and I didnt sucsses to write it without warnings. I get debbug error with the 'free' function:

This is my program:

```<code>#include <stdio.h>
#include <stdlib.h>
#include <stdbool.h>
#define size 1

int* symetri(int set1[size], int set2[size]);

int main() {
    int* difference;
    int set1[size], set2[size];
    for (int i = 0; i < size; i++)
        scanf("%d", &set1[i]);
    for (int i = 0; i < size; i++)
        scanf("%d", &set2[i]);
    difference = symetri(set1, set2);
    free(difference);
    return 0;
}

int* symetri(int set1[size], int set2[size])
{
    int* new_arr = (int*)malloc(size * sizeof(int));
    while (new_arr == NULL)
        new_arr = (int*)malloc(size * sizeof(int));
    int counter = 0;
    bool sym;
    for (int i = 0; i < size; i++) {
        new_arr[counter] = set1[i];
        counter++;
    }
    new_arr[counter] = -1;
    return new_arr;
}
</code>```

答:
If free crashes with an error message about heap corruption in a program this simple, it's almost certainly because you wrote to memory that doesn't belong to you. And you did, with this line at the end of symetri:

```<code>new_arr[counter] = -1;
</code>```

You need to make your allocation of new_arr another sizeof(int) longer if you want to write to the end after looping over it.

Body:
I tried to use dynamic allocation and I didnt sucsses to write it without warnings. I get debbug error with the 'free' function:

This is my program:

#include <stdio.h>
#include <stdlib.h>
#include <stdbool.h>
#define size 1

int* symetri(int set1[size], int set2[size]);

int main() {
    int* difference;
    int set1[size], set2[size];
    for (int i = 0; i < size; i++)
        scanf("%d", &set1[i]);
    for (int i = 0; i < size; i++)
        scanf("%d", &set2[i]);
    difference = symetri(set1, set2);
    free(difference);
    return 0;
}

int* symetri(int set1[size], int set2[size])
{
    int* new_arr = (int*)malloc(size * sizeof(int));
    while (new_arr == NULL)
        new_arr = (int*)malloc(size * sizeof(int));
    int counter = 0;
    bool sym;
    for (int i = 0; i < size; i++) {
        new_arr[counter] = set1[i];
        counter++;
    }
    new_arr[counter] = -1;
    return new_arr;
}

Body:
If free crashes with an error message about heap corruption in a program this simple, it's almost certainly because you wrote to memory that doesn't belong to you. And you did, with this line at the end of symetri:

new_arr[counter] = -1;

You need to make your allocation of new_arr another sizeof(int) longer if you want to write to the end after looping over it.


问:
WebSocket connection: can it follow a previous HTTP/1.1 request/response?
Can a WebSocket connection be initiated using a TCP (or TCP+TLS) connection already used by one or more regular HTTP request/responses, or does the client have to initiate a new TCP connection for each WebSocket connection?

[Asking in the context of extending an HTTP client to support also making WebSockets connections. The client maintains a pool of keep-alive connections, and wondering if when making a WebSocket connection, if it can use a TCP connection from the pool.]

答:
From https://www.rfc-editor.org/rfc/rfc6455#section-4.1

Once a connection to the server has been established (including a connection via a proxy or over a TLS-encrypted tunnel), the client MUST send an opening handshake to the server.

The language suggests to me that there shouldn't be anything [e.g. an HTTP request/response] between opening the connection and sending the WebSocket opening handshake would not adhere to the spec.

So no, connections alive from previous HTTP request/responses cannot be used for WebSockets.