问:
Custom AuthenticationFilter Setting `LoginProcessingUrl` is Invalid
I'm building an OAuth2 Authorization server that supports Restful API with Spring Authorization Server and Spring Security.
I want a SPA application built by React to provide a login interface at /login and submit the login information to the /api/login path with a Post request.
I extend UsernamePasswordauthenticationFilter to support Restful-style Post requests to parse Json data from body:

public class RestfulUsernamePasswordAuthenticationFilter extends UsernamePasswordAuthenticationFilter {

    public RestfulUsernamePasswordAuthenticationFilter(AuthenticationManager authenticationManager) {
        super(authenticationManager);
    }

    private String jsonUsername;
    private String jsonPassword;

    @Override
    protected String obtainPassword(HttpServletRequest request) {
        if (request.getContentType().equals(MediaType.APPLICATION_JSON_VALUE)) {
            return this.jsonPassword;
        } else {
            return super.obtainPassword(request);
        }

    }

    @Override
    protected String obtainUsername(HttpServletRequest request) {

        if (request.getContentType().equals(MediaType.APPLICATION_JSON_VALUE)) {
            return this.jsonUsername;
        } else {
            return super.obtainUsername(request);
        }

    }

    @Override
    public Authentication attemptAuthentication(HttpServletRequest request, HttpServletResponse response)
            throws AuthenticationException {
        if ("application/json".equals(request.getHeader("Content-Type"))) {
            try {
                /*
                 * HttpServletRequest can be read only once
                 */

                //json transformation
                Map<String, String> requestMap = new ObjectMapper().readValue(request.getInputStream(), Map.class);

                this.jsonUsername = requestMap.get("username");
                this.jsonPassword = requestMap.get("password");
            } catch (Exception e) {
                throw new AuthenticationServiceException(e.getMessage(), e);
            }
        }

        return super.attemptAuthentication(request, response);
    }
}
```
<p>In the configuration, I replaced the custom <code>RestfulUsernamePasswordauthenticationFilter</code> with <code>UsernamePasswordauthenticationFilter</code>, and used <code>.loginProcessUrl</code> to set the path for processing Post requests to <code>/api/login</code>:</p>
<pre class="lang-java prettyprint-override"><code>    @Override
    protected void configure(HttpSecurity http)
            throws Exception {
        http.addFilterAt(new RestfulUsernamePasswordAuthenticationFilter(authenticationManagerBean()),
                UsernamePasswordAuthenticationFilter.class);
        http.authorizeRequests()
                .antMatchers("/api/all")
                .permitAll()
                .antMatchers("/api/auth")
                .hasRole("USER")
                .antMatchers("/api/admin")
                .hasRole("ADMIN")
                .antMatchers("/login", "/register", "/api/login")
                .permitAll();

        http.formLogin()
                .loginPage("/login")
                .loginProcessingUrl("/api/login");
        http.csrf().disable();
    }
</code>```

The problem is that although I set the path to process Post requests through .loginProcessingUrl, it doesn't seem to work.
When a Post request is submitted to /api/login, it will be redirected to/login like all unauthenticated requests, and the request submitted to /login will take effect normally.

In the process of debugging, I found that .loginProcessingUrl will register this path in a UsernamePasswordconfirationFilter, but will not be processed by my custom RestfulUsernamePasswordShareationFilter. In the process of debugging, I found that .loginProcessingUrl will register this path in a UsernamePasswordFilter.
I want to know if there is any way to make .loginProcessingUrl work on my custom AuthenticationFilter.
At the same time, can I easily customize the path to which they accept requests when I add more custom Filter?
Maybe I will add more AuthenticationProvider that need to read Restful information in the future. how should I design the architecture of these Filter and Provider to make it easier to expand?

答:
I think I solved the problem after read this blog. As it said:

After you provided your custom filter, you can no longer use the Spring HttpSecurity builder. If you still use it, you’ll configure the default Filter, not yours!

We tell Spring to use our implementatin by the “addFIlterBefore” function.

After this little modification, the test APIs work the same way, the only difference is that you should provide ‘login’ and ‘password’ params in the POST request body (and not the query string).

So I have to manually set the AntPathRequestMatcher  withsetRequiresAuthenticationRequestMatcher of the filter class like this:

RestfulUsernamePasswordAuthenticationFilter restfulFilter = new RestfulUsernamePasswordAuthenticationFilter(
                this.authenticationManagerBean());
restfulFilter.setRequiresAuthenticationRequestMatcher(
                new AntPathRequestMatcher("/api/login", "POST"));
http.addFilterBefore(restfulFilter,
                UsernamePasswordAuthenticationFilter.class);
```

By this way, you can configure the path to handle the POST request just like you use loginProcessingUrl in the formal way.

You can also use same method to add more custom filter and authenticationProvider in the future, just configure them manually.

Body:
I'm building an OAuth2 Authorization server that supports Restful API with Spring Authorization Server and Spring Security.
I want a SPA application built by React to provide a login interface at /login and submit the login information to the /api/login path with a Post request.
I extend UsernamePasswordauthenticationFilter to support Restful-style Post requests to parse Json data from body:

public class RestfulUsernamePasswordAuthenticationFilter extends UsernamePasswordAuthenticationFilter {

    public RestfulUsernamePasswordAuthenticationFilter(AuthenticationManager authenticationManager) {
        super(authenticationManager);
    }

    private String jsonUsername;
    private String jsonPassword;

    @Override
    protected String obtainPassword(HttpServletRequest request) {
        if (request.getContentType().equals(MediaType.APPLICATION_JSON_VALUE)) {
            return this.jsonPassword;
        } else {
            return super.obtainPassword(request);
        }

    }

    @Override
    protected String obtainUsername(HttpServletRequest request) {

        if (request.getContentType().equals(MediaType.APPLICATION_JSON_VALUE)) {
            return this.jsonUsername;
        } else {
            return super.obtainUsername(request);
        }

    }

    @Override
    public Authentication attemptAuthentication(HttpServletRequest request, HttpServletResponse response)
            throws AuthenticationException {
        if ("application/json".equals(request.getHeader("Content-Type"))) {
            try {
                /*
                 * HttpServletRequest can be read only once
                 */

                //json transformation
                Map<String, String> requestMap = new ObjectMapper().readValue(request.getInputStream(), Map.class);

                this.jsonUsername = requestMap.get("username");
                this.jsonPassword = requestMap.get("password");
            } catch (Exception e) {
                throw new AuthenticationServiceException(e.getMessage(), e);
            }
        }

        return super.attemptAuthentication(request, response);
    }
}

In the configuration, I replaced the custom RestfulUsernamePasswordauthenticationFilter with UsernamePasswordauthenticationFilter, and used .loginProcessUrl to set the path for processing Post requests to /api/login:

    @Override
    protected void configure(HttpSecurity http)
            throws Exception {
        http.addFilterAt(new RestfulUsernamePasswordAuthenticationFilter(authenticationManagerBean()),
                UsernamePasswordAuthenticationFilter.class);
        http.authorizeRequests()
                .antMatchers("/api/all")
                .permitAll()
                .antMatchers("/api/auth")
                .hasRole("USER")
                .antMatchers("/api/admin")
                .hasRole("ADMIN")
                .antMatchers("/login", "/register", "/api/login")
                .permitAll();

        http.formLogin()
                .loginPage("/login")
                .loginProcessingUrl("/api/login");
        http.csrf().disable();
    }

The problem is that although I set the path to process Post requests through .loginProcessingUrl, it doesn't seem to work.
When a Post request is submitted to /api/login, it will be redirected to/login like all unauthenticated requests, and the request submitted to /login will take effect normally.

In the process of debugging, I found that .loginProcessingUrl will register this path in a UsernamePasswordconfirationFilter, but will not be processed by my custom RestfulUsernamePasswordShareationFilter. In the process of debugging, I found that .loginProcessingUrl will register this path in a UsernamePasswordFilter.
I want to know if there is any way to make .loginProcessingUrl work on my custom AuthenticationFilter.
At the same time, can I easily customize the path to which they accept requests when I add more custom Filter?
Maybe I will add more AuthenticationProvider that need to read Restful information in the future. how should I design the architecture of these Filter and Provider to make it easier to expand?

Body:
I think I solved the problem after read this blog. As it said:

After you provided your custom filter, you can no longer use the Spring HttpSecurity builder. If you still use it, you’ll configure the default Filter, not yours!

We tell Spring to use our implementatin by the “addFIlterBefore” function.

After this little modification, the test APIs work the same way, the only difference is that you should provide ‘login’ and ‘password’ params in the POST request body (and not the query string).

So I have to manually set the AntPathRequestMatcher  withsetRequiresAuthenticationRequestMatcher of the filter class like this:

RestfulUsernamePasswordAuthenticationFilter restfulFilter = new RestfulUsernamePasswordAuthenticationFilter(
                this.authenticationManagerBean());
restfulFilter.setRequiresAuthenticationRequestMatcher(
                new AntPathRequestMatcher("/api/login", "POST"));
http.addFilterBefore(restfulFilter,
                UsernamePasswordAuthenticationFilter.class);

By this way, you can configure the path to handle the POST request just like you use loginProcessingUrl in the formal way.

You can also use same method to add more custom filter and authenticationProvider in the future, just configure them manually.


来源: stackexchange

问:
C# - WPF - Bind Style.Setters
I have spent all day looking for a way to to bind property in Row Style Setters from DataGrid and the bind fails

```<code><DataGrid.RowStyle>
  <Style.Setters>
    <Setter Property="IsEnabled" Value="{Binding MyProperty}"/>
  </Style.Setters>
</DataGrid.RowStyle>
</code>```

I tried custom converters
And it did not work, does anyone have an idea?

答:
I am not sure really what to do here. Since you haven't given me a lot to work with. So I will assume that everything else is setup correctly and you are missing the style.

I remember that you might have to specify what to target in your style. In this case "DataGridRow".
So you need a
        </Style>
    </DataGrid.RowStyle>
```

ContentLicense: CC BY-SA 4.0

Title: C# - WPF - Bind Style.Setters

Body:
I have spent all day looking for a way to to bind property in Row Style Setters from DataGrid and the bind fails

<DataGrid.RowStyle>

</DataGrid.RowStyle>

I tried custom converters
And it did not work, does anyone have an idea?

Body:
I am not sure really what to do here. Since you haven't given me a lot to work with. So I will assume that everything else is setup correctly and you are missing the style.

I remember that you might have to specify what to target in your style. In this case "DataGridRow".
So you need a <Style TargetType="DataGridRow"> tag before the Style Setter

   <DataGrid.RowStyle>
        <Style TargetType="DataGridRow">
            <Style.Setters>
                <Setter Property="IsEnabled" Value="{Binding MyProperty}"/>
            </Style.Setters>
        </Style>
    </DataGrid.RowStyle>


问:
lua debug hook causing race condition?
I wanted to report some debug information for a parser I am writing in lua. I am using the debug hook facility for tracking but it seems like there is some form of race condition happening.

Here is my test code:

enters = 0
enters2 = 0
calls = 0
tailcalls = 0
returns = 0
lines = 0
other = 0
exits = 0

local function analyze(arg)
    enters = enters + 1
    enters2 = enters2 + 1

    if arg == "call" then
        calls = calls + 1
    elseif arg == "tail call" then
        tailcalls = tailcalls + 1
    elseif arg == "return" then
        returns = returns + 1
    elseif arg == "line" then
        lines = lines + 1
    else
        other = other + 1
    end

    exits = exits + 1
end

debug.sethook(analyze, "crl")

-- main code

print("enters = ", enters)
print("enters2 = ", enters2)
print("calls = ", calls)
print("tailcalls = ", tailcalls)
print("returns = ", returns)
print("lines = ", lines)
print("other = ", other)
print("exits = ", exits)

print("sum = ", calls + tailcalls + returns + lines + other)
```
<p>and here is the result:</p>
```enters =        429988
enters2 =       429991
calls =         97433
tailcalls =     7199
returns =       97436
lines =         227931
other =         0
exits =         430009
sum =   430012
```

Why does none of this add up? I am running lua 5.4.2 on Ubuntu 20.04, no custom c libraries, no further messing with the debug library.

答:
I found the problem...

The calls to the print function when printing the result also trigger the hook, which only affects the results that have not been printed yet.

Title: lua debug hook causing race condition?

Body:
I wanted to report some debug information for a parser I am writing in lua. I am using the debug hook facility for tracking but it seems like there is some form of race condition happening.

Here is my test code:

enters = 0
enters2 = 0
calls = 0
tailcalls = 0
returns = 0
lines = 0
other = 0
exits = 0

local function analyze(arg)
    enters = enters + 1
    enters2 = enters2 + 1

    if arg == "call" then
        calls = calls + 1
    elseif arg == "tail call" then
        tailcalls = tailcalls + 1
    elseif arg == "return" then
        returns = returns + 1
    elseif arg == "line" then
        lines = lines + 1
    else
        other = other + 1
    end

    exits = exits + 1
end

debug.sethook(analyze, "crl")

-- main code

print("enters = ", enters)
print("enters2 = ", enters2)
print("calls = ", calls)
print("tailcalls = ", tailcalls)
print("returns = ", returns)
print("lines = ", lines)
print("other = ", other)
print("exits = ", exits)

print("sum = ", calls + tailcalls + returns + lines + other)

and here is the result:

enters =        429988
enters2 =       429991
calls =         97433
tailcalls =     7199
returns =       97436
lines =         227931
other =         0
exits =         430009
sum =   430012

Why does none of this add up? I am running lua 5.4.2 on Ubuntu 20.04, no custom c libraries, no further messing with the debug library.


问:
Using Moq to mock HttpClient in the context of Thread
I'm working on creating unit tests using moq and I'm having trouble figuring out how to apply this framework to a portion of my program that deals with using an HttpClient. There's various resources I found that demonstrate how to mock an HttpClient response directly but the way my application makes use of HttpClient is slightly different with the utilization of Threads.

The test's skeleton:

```<code>public class MyTestClass
{
    public void myTest()
    {
        ClassA classObj = new ClassA();
        classObj.Start();
        // I'd like to use moq somewhere here to mock the response that occurs in DoThreadStuff() below
    }
}
</code>```

The class under testing:

```<code>public class ClassA
{
    private readonly Thread _myThread;
    private HttpClient _client;

    public ClassA()
    {
        // initialize some values
        _myThread = new Thread(DoThreadStuff);
    }

    public void Start()
    {
        _myThread.Start(); // starts DoThreadStuff()
    }

    private void DoThreadStuff()
    {
        var newClient = getNewHttpClient(); // utility function returns a HttpClient
        var response = newClient.GetAsync("/my/api/status/endpoint");
    }
}
</code>```

As you can see, when ClassA.Start() gets called, a new HttpClient gets created and used via GetAsync. What would the correct way to structure a test for this look like? Will I have to change the implementation of my existing classes to accommodate for Moq? Does anyone have experience with something very similar which I could take a look at?

答: You should consider to use a Factory/Repository-Pattern. They make it easier to Test your methods because you can just inject mocks of objects like HttpClient. The answer Peter Csala suggested is pretty dirty because you create a class that has the single purpose to be used in a test.</p>

Body:
I'm working on creating unit tests using moq and I'm having trouble figuring out how to apply this framework to a portion of my program that deals with using an HttpClient. There's various resources I found that demonstrate how to mock an HttpClient response directly but the way my application makes use of HttpClient is slightly different with the utilization of Threads.

The test's skeleton:

public class MyTestClass
{
    public void myTest()
    {
        ClassA classObj = new ClassA();
        classObj.Start();
        // I'd like to use moq somewhere here to mock the response that occurs in DoThreadStuff() below
    }
}

The class under testing:

public class ClassA
{
    private readonly Thread _myThread;
    private HttpClient _client;

    public ClassA()
    {
        // initialize some values
        _myThread = new Thread(DoThreadStuff);
    }

    public void Start()
    {
        _myThread.Start(); // starts DoThreadStuff()
    }

    private void DoThreadStuff()
    {
        var newClient = getNewHttpClient(); // utility function returns a HttpClient
        var response = newClient.GetAsync("/my/api/status/endpoint");
    }
}

As you can see, when ClassA.Start() gets called, a new HttpClient gets created and used via GetAsync. What would the correct way to structure a test for this look like? Will I have to change the implementation of my existing classes to accommodate for Moq? Does anyone have experience with something very similar which I could take a look at?


问:
Turning vectors of strings in a dataframe into categorical variables in R
I'm fairly new to R and am sure there's a way to do the following without using loops, which I'm more familiar with.

Take the following example where you have a bunch of names and fruits each person likes:

```<code>name <- c("Alice", "Bob")
preference <- list(c("apple", "pear"), c("banana", "apple"))
df <- as.data.frame(cbind(name, preference))
</code>```

How to I convert it to the following?

```<code>apple <- c(1, 1)
pear <- c(1, 0)
banana <- c(0, 1)
df2 <- data.frame(name, apple, pear, banana)
</code>```

My basic instinct is to first extract all the fruits then do a loop to check if each fruit is in each row's preference:

```<code>fruits <- unique(unlist(df$preference))
for (fruit in fruits) {
    df <- df %>% rowwise %>% mutate("{fruit}" := fruit %in% preference)
}
</code>```

This seems to work, but I'm pretty sure there's a better way to do this.

答:
```<code>df %>%
  unnest(everything()) %>%
  xtabs(~., .) %>%
  as.data.frame.matrix() %>%
  rownames_to_column('name')

   name apple banana pear
1 Alice     1      0    1
2   Bob     1      1    0
</code>```

Body:
I'm fairly new to R and am sure there's a way to do the following without using loops, which I'm more familiar with.

Take the following example where you have a bunch of names and fruits each person likes:

name <- c("Alice", "Bob")
preference <- list(c("apple", "pear"), c("banana", "apple"))
df <- as.data.frame(cbind(name, preference))

How to I convert it to the following?

apple <- c(1, 1)
pear <- c(1, 0)
banana <- c(0, 1)
df2 <- data.frame(name, apple, pear, banana)

My basic instinct is to first extract all the fruits then do a loop to check if each fruit is in each row's preference:

fruits <- unique(unlist(df$preference))
for (fruit in fruits) {
    df <- df %>% rowwise %>% mutate("{fruit}" := fruit %in% preference)
}

This seems to work, but I'm pretty sure there's a better way to do this.


答:
In tidyverse (assuming the 'preference' is a list column), unnest the 'preference' and then use pivot_wider to reshape back to 'wide' format with values_fn as length

```<code>library(dplyr)
library(tidyr)
df %>%
   unnest_longer(preference) %>%
    pivot_wider(names_from = preference, values_from = preference,
      values_fn = length, values_fill = 0)
</code>```

-output

```<code># A tibble: 2 × 4
  name  apple  pear banana
  <chr> <int> <int>  <int>
1 Alice     1     1      0
2 Bob       1     0      1
</code>```

data

```<code>df <- data.frame(name, preference = I(preference))
</code>```

Title: Turning vectors of strings in a dataframe into categorical variables in R

Body:
In tidyverse (assuming the 'preference' is a list column), unnest the 'preference' and then use pivot_wider to reshape back to 'wide' format with values_fn as length

library(dplyr)
library(tidyr)
df %>%
   unnest_longer(preference) %>%
    pivot_wider(names_from = preference, values_from = preference,
      values_fn = length, values_fill = 0)

-output

# A tibble: 2 × 4
  name  apple  pear banana
  <chr> <int> <int>  <int>
1 Alice     1     1      0
2 Bob       1     0      1

data

df <- data.frame(name, preference = I(preference))


问:
Why am I getting an error stated that k is not defined, even though i have returned it?
Why am I getting an error stated that k is not defined, even though i have returned it?</p>
```<code>import pyautogui
def askpass():
    password = pyautogui.password(text='Enter password for the database:', title='', default='', mask='*')
    if password=='root':
        print('password correct')
    else:
        askpass()
    k=password
    return k

askpass()
print(k)
</code>```

答:
EDIT:
I've missed the recursive askpass() when reading the code, if you entered the wrong password you would go into another call of the same function but that call's return doesn't get assigned to password. Now I don't recommend using recursion either in here because you just keep on adding up functions call, but I recommend doing this with a while loop like this:

def askpass():
    password = ""
    while password != "root":
        password = pyautogui.password(text='Enter password for the database:', title='', default='', mask='*')
    k=password
    return k
```
<p>OLD:</p>
<p>The correct syntax for this is</p>
<pre class="lang-py prettyprint-override"><code>import pyautogui
def askpass():
    password = pyautogui.password(text='Enter password for the database:', title='', default='', mask='*')
    if password=='root':
        print('password correct')
    else:
        password=askpass()
    k=password
    return k

k = askpass()
print(k)
</code>```

or

print(askpass())
```

When you return a variable, it returns it's value that you can then get like written above.

(That's how you can imagine it, but in python what actually happens the function returns a reference to a object that can then be assigned to a variable)

Body:
Why am I getting an error stated that k is not defined, even though i have returned it?

import pyautogui
def askpass():
    password = pyautogui.password(text='Enter password for the database:', title='', default='', mask='*')
    if password=='root':
        print('password correct')
    else:
        askpass()
    k=password
    return k

askpass()
print(k)

Body:
EDIT:
I've missed the recursive askpass() when reading the code, if you entered the wrong password you would go into another call of the same function but that call's return doesn't get assigned to password. Now I don't recommend using recursion either in here because you just keep on adding up functions call, but I recommend doing this with a while loop like this:

def askpass():
    password = ""
    while password != "root":
        password = pyautogui.password(text='Enter password for the database:', title='', default='', mask='*')
    k=password
    return k

OLD:

The correct syntax for this is

import pyautogui
def askpass():
    password = pyautogui.password(text='Enter password for the database:', title='', default='', mask='*')
    if password=='root':
        print('password correct')
    else:
        password=askpass()
    k=password
    return k

k = askpass()
print(k)

or

print(askpass())

When you return a variable, it returns it's value that you can then get like written above.

(That's how you can imagine it, but in python what actually happens the function returns a reference to a object that can then be assigned to a variable)


问:
Total ground time
I am working with a dataset from Kaggle that contains information about flights. A glimpse of the dataset is shown below

I want to create a new dataset that should contain the aggregate view on the airplane,
for each flight day, each airline, total availability and total night_availability, where total availability is the total time that each aircraft was not flying in each flight day (ground time) and night availability is the ground time available between 23:00 and 07:00.

For example a plane can flight many times during each day

, I can find the total flight time for a plane for a day by summarizing the times (in minutes)  of the column ELAPSED_TIME. Then the ground time is the difference of the total minutes of a day and the sum of the ELAPSED_TIME column.

How can I create this dataset using pandas?

Edit:

To create the dataset I followed these steps:

```<code>import pandas as pd

# read dataset
flights = pd.read_csv('flights.csv',
                      dtype = {'DEPARTURE_TIME': object, 'ARRIVAL_TIME': object},
                      low_memory=False)
# keep only columns of interest
F = flights[['YEAR', 'MONTH', 'DAY',
                           'DAY_OF_WEEK', 'AIRLINE',
                           'FLIGHT_NUMBER', 'TAIL_NUMBER',
                           'DEPARTURE_TIME', 'ARRIVAL_TIME',
                           'ELAPSED_TIME', 'CANCELLED']]
# drop rows contain na
f = F.dropna()
# format columns to datetime
f['DEPARTURE_TIME'] = pd.to_datetime(f['DEPARTURE_TIME'],format='%H%M', errors='coerce').dt.time

f['ARRIVAL_TIME'] = pd.to_datetime(f['ARRIVAL_TIME'], format='%H%M', errors = 'coerce').dt.time
</code>```

答:
```<code>import pandas as pd

# read dataset
flights = pd.read_csv('flights.csv',
                      dtype = {'DEPARTURE_TIME': object, 'ARRIVAL_TIME': object},
                      low_memory=False)
# keep only columns of interest
F = flights[['YEAR', 'MONTH', 'DAY',
                           'DAY_OF_WEEK', 'AIRLINE',
                           'FLIGHT_NUMBER', 'TAIL_NUMBER',
                           'DEPARTURE_TIME', 'ARRIVAL_TIME',
                           'ELAPSED_TIME', 'CANCELLED']]
# drop rows contain na
f = F.dropna()

# according to your method of elapsed time sum

df2=f.groupby(['TAIL_NUMBER','YEAR','MONTH','DAY'])['ELAPSED_TIME'].sum().reset_index()
df2['ground_time']=24*60-df2['ELAPSED_TIME']
print(df2)

</code>```

Now for the second part where you needed night total ground time  assume a is departure time and b is arrival time .

for the part which requires you to calculate the night  total ground time

```<code>#logic is here
#if b>a
    # if b<=7
    #    return b-a
    # if b>=7 and a<=7
    #    if b>=23
    #       return (b-23 )+(7-a)
    #     else
    #    return 7-a
    # if b>=23 and a>=23
    #    return b-a
    # if b>=23
    #   return b-23

# if b<a
    # if b<7 and a>23 we need to do b-a
    # if b>7 and a>23 we need to do 7-a
    # if b<7 and a<23 we need to do b-23
    # if b>7 and a<23 we need to do 7-23

def change(x):

    a=pd.to_datetime(x['DEPARTURE_TIME'],format='%H%M',errors='coerce')
    b=pd.to_datetime(x['ARRIVAL_TIME'],format='%H%M',errors='coerce')
    c=pd.to_datetime("2300",format='%H%M')
    d=pd.to_datetime("0700",format='%H%M')
    if b>a :
        if b<=d :
            return pd.Timedelta(b-a).seconds/60.0;
        if b>=d and a<=d:
            if b>=c:
                return (pd.Timedelta(b-c).seconds/60.0)+pd.Timedelta(d-a).seconds/60.0;
            else :
                return pd.Timedelta(d-a).seconds/60.0;
        if b>=c and a>=c:
            return pd.Timedelta(b-a).seconds/60.0;
        if b>=c :
            return pd.Timedelta(b-c).seconds/60.0;
        else :
            return 0

    else:
        if b<=d and a>=c :
            return pd.Timedelta(b-a).seconds/60.0 ;
        if b>=d and a>=c :
            return pd.Timedelta(d-a).seconds/60.0 ;
        if b<=d and a<=c :
            return pd.Timedelta(b-c).seconds/60.0;
        if b>=d and a<=c :
            return pd.Timedelta(d-c).seconds/60.0;
        return 0

#night_flight_time is the time spent between departure and arrival which lies in between (23:00 and 07:00)
f['night_flight_time']=f.apply(change,axis=1)
#print(f)
df3=f.groupby(['TAIL_NUMBER','YEAR','MONTH','DAY'])['night_flight_time'].sum().reset_index()
# here i have subtracted the night_flight_time  from total night duration i.e. 8 hours
df3['night_ground_time']=8*60-df3['night_flight_time']
print(df3)

</code>```

Title: Total ground time

Body:
I am working with a dataset from Kaggle that contains information about flights. A glimpse of the dataset is shown below

I want to create a new dataset that should contain the aggregate view on the airplane,
for each flight day, each airline, total availability and total night_availability, where total availability is the total time that each aircraft was not flying in each flight day (ground time) and night availability is the ground time available between 23:00 and 07:00.

For example a plane can flight many times during each day

, I can find the total flight time for a plane for a day by summarizing the times (in minutes)  of the column ELAPSED_TIME. Then the ground time is the difference of the total minutes of a day and the sum of the ELAPSED_TIME column.

How can I create this dataset using pandas?

Edit:

To create the dataset I followed these steps:

import pandas as pd

# read dataset
flights = pd.read_csv('flights.csv',
                      dtype = {'DEPARTURE_TIME': object, 'ARRIVAL_TIME': object},
                      low_memory=False)
# keep only columns of interest
F = flights[['YEAR', 'MONTH', 'DAY',
                           'DAY_OF_WEEK', 'AIRLINE',
                           'FLIGHT_NUMBER', 'TAIL_NUMBER',
                           'DEPARTURE_TIME', 'ARRIVAL_TIME',
                           'ELAPSED_TIME', 'CANCELLED']]
# drop rows contain na
f = F.dropna()
# format columns to datetime
f['DEPARTURE_TIME'] = pd.to_datetime(f['DEPARTURE_TIME'],format='%H%M', errors='coerce').dt.time

f['ARRIVAL_TIME'] = pd.to_datetime(f['ARRIVAL_TIME'], format='%H%M', errors = 'coerce').dt.time

Body:
import pandas as pd

# read dataset
flights = pd.read_csv('flights.csv',
                      dtype = {'DEPARTURE_TIME': object, 'ARRIVAL_TIME': object},
                      low_memory=False)
# keep only columns of interest
F = flights[['YEAR', 'MONTH', 'DAY',
                           'DAY_OF_WEEK', 'AIRLINE',
                           'FLIGHT_NUMBER', 'TAIL_NUMBER',
                           'DEPARTURE_TIME', 'ARRIVAL_TIME',
                           'ELAPSED_TIME', 'CANCELLED']]
# drop rows contain na
f = F.dropna()

# according to your method of elapsed time sum

df2=f.groupby(['TAIL_NUMBER','YEAR','MONTH','DAY'])['ELAPSED_TIME'].sum().reset_index()
df2['ground_time']=24*60-df2['ELAPSED_TIME']
print(df2)

Now for the second part where you needed night total ground time  assume a is departure time and b is arrival time .

for the part which requires you to calculate the night  total ground time

#logic is here
#if b>a
    # if b<=7
    #    return b-a
    # if b>=7 and a<=7
    #    if b>=23
    #       return (b-23 )+(7-a)
    #     else
    #    return 7-a
    # if b>=23 and a>=23
    #    return b-a
    # if b>=23
    #   return b-23

# if b<a
    # if b<7 and a>23 we need to do b-a
    # if b>7 and a>23 we need to do 7-a
    # if b<7 and a<23 we need to do b-23
    # if b>7 and a<23 we need to do 7-23

def change(x):

    a=pd.to_datetime(x['DEPARTURE_TIME'],format='%H%M',errors='coerce')
    b=pd.to_datetime(x['ARRIVAL_TIME'],format='%H%M',errors='coerce')
    c=pd.to_datetime("2300",format='%H%M')
    d=pd.to_datetime("0700",format='%H%M')
    if b>a :
        if b<=d :
            return pd.Timedelta(b-a).seconds/60.0;
        if b>=d and a<=d:
            if b>=c:
                return (pd.Timedelta(b-c).seconds/60.0)+pd.Timedelta(d-a).seconds/60.0;
            else :
                return pd.Timedelta(d-a).seconds/60.0;
        if b>=c and a>=c:
            return pd.Timedelta(b-a).seconds/60.0;
        if b>=c :
            return pd.Timedelta(b-c).seconds/60.0;
        else :
            return 0

    else:
        if b<=d and a>=c :
            return pd.Timedelta(b-a).seconds/60.0 ;
        if b>=d and a>=c :
            return pd.Timedelta(d-a).seconds/60.0 ;
        if b<=d and a<=c :
            return pd.Timedelta(b-c).seconds/60.0;
        if b>=d and a<=c :
            return pd.Timedelta(d-c).seconds/60.0;
        return 0

#night_flight_time is the time spent between departure and arrival which lies in between (23:00 and 07:00)
f['night_flight_time']=f.apply(change,axis=1)
#print(f)
df3=f.groupby(['TAIL_NUMBER','YEAR','MONTH','DAY'])['night_flight_time'].sum().reset_index()
# here i have subtracted the night_flight_time  from total night duration i.e. 8 hours
df3['night_ground_time']=8*60-df3['night_flight_time']
print(df3)


问:
how to open and edit py file in command prompt windows 10
I'm finding for a syntax equivalent to mac os terminal open filename.py in windows command prompt.
I've search internet to find answer like type in filename (or location) filename.py into command prompt, but that just run a script or even start filename.py doesn't seems to work too. how do i edit get to edit my code via command prompt?

答:
to edit a file you can simply use notepad

```<code>notepad file.txt
</code>```

you can install vim or nano and use them in windows

Body:
to edit a file you can simply use notepad

notepad file.txt

you can install vim or nano and use them in windows


问:
Can you use the output of a function with user input to "call" a dataframe?
I have some code in R that invites a user to put in a year between 2010 and 2021.

```<code>chosen.year <- readline(promt = "choose year between 2010 and 2021:")
y.chosen.year <- paste("y", chosen.year)
year.input <- gsub(" ", "", y.chosen.year, fixed = TRUE)
</code>```

The output that is stored in year.input is for e.g. 2015: y2015.

I have a dataframe for each year between 2010 and 2021 that is called y2010, y2011 etc.

Is it possible to later use year.input in another function that would otherwhise require me to write y2015 (so that the user can choose a year that will be used later  on)?

Example:

```<code>myspdf2 <- merge(myspdf1, year.input, by.x "abc", by .y "def")
</code>```

Instead of:

```<code>myspdf2 <- merge(myspdf1, y2015, by.x "abc", by .y "def")
</code>```

I tried the method above but it did not work.

答:
Assuming promt= is not in your real code, two options:

Combine all years into one frame, including the year in the data (if not there already).

years <- ls(pattern = "^y\\d{4}$")
allyears <- Map(
  function(x, yr) transform(x, year = yr),
  mget(years), years)
subset(allyears, year == chosen.year)
```
</li>
<li><p>Combine all years into a list of frames, and subset from there:</p>
<pre class="lang-r prettyprint-override"><code>allyears <- mget(ls(pattern = "^y\\d{4}$"))
allyears[[ chosen.year ]]
</code>```

(This assumes that a chosen.year will only reference one of the multiple frames.)

Ultimately I suspect that this is not about merge so much about subset (one-frame) or [[-extraction (list of frames).

A third option that I'm not fond of, but offered to round out the answer:

Just get the data. BTW, you should use either paste0(.) or paste(., sep=""), otherwise you'll get y 2015 instead of y2015. This is much more direct than paste(.) and gsub(" ", "", .).

year.input <- paste0("y", chosen.year)
get(year.input)
```

Body:
I have some code in R that invites a user to put in a year between 2010 and 2021.

chosen.year <- readline(promt = "choose year between 2010 and 2021:")
y.chosen.year <- paste("y", chosen.year)
year.input <- gsub(" ", "", y.chosen.year, fixed = TRUE)

The output that is stored in year.input is for e.g. 2015: y2015.

I have a dataframe for each year between 2010 and 2021 that is called y2010, y2011 etc.

Is it possible to later use year.input in another function that would otherwhise require me to write y2015 (so that the user can choose a year that will be used later  on)?

Example:

myspdf2 <- merge(myspdf1, year.input, by.x "abc", by .y "def")

Instead of:

myspdf2 <- merge(myspdf1, y2015, by.x "abc", by .y "def")

I tried the method above but it did not work.

Body:
Assuming promt= is not in your real code, two options:

Combine all years into one frame, including the year in the data (if not there already).

years <- ls(pattern = "^y\\d{4}$")
allyears <- Map(
  function(x, yr) transform(x, year = yr),
  mget(years), years)
subset(allyears, year == chosen.year)

Combine all years into a list of frames, and subset from there:

allyears <- mget(ls(pattern = "^y\\d{4}$"))
allyears[[ chosen.year ]]

(This assumes that a chosen.year will only reference one of the multiple frames.)

Ultimately I suspect that this is not about merge so much about subset (one-frame) or [[-extraction (list of frames).

A third option that I'm not fond of, but offered to round out the answer:

Just get the data. BTW, you should use either paste0(.) or paste(., sep=""), otherwise you'll get y 2015 instead of y2015. This is much more direct than paste(.) and gsub(" ", "", .).

year.input <- paste0("y", chosen.year)
get(year.input)


问:
How do I create an instance of a Class passed as parameter to a function?
I'm building a library in Kotlin and here's my usecase

I have a base class

```<code>abstract class Component(...) {
  // ... class body
}
</code>```

I want the users of my library to define their own sub-classes like say:

```<code>class MyComponent() : Component() {
  // .. class body
}
</code>```

How can I write a helper function that takes in this derived class as a param and create an instance out of it. Something like:

```<code>fun helper(component: Class, props: HashMap<String, String>) : Component {
  // somehow create a new instance of Class and return it?
}
</code>```

Thanks!

答:
You can have users pass a constructor reference:

```<code>
fun helper(componentConstructor: ()->Component, props: Map<String, String>) : Component {
  val component = componentConstructor()
    // set it up and return it.
}

// usage:
val component = helper(::MyComponent, emptyMap())
</code>```

Better for props not to require a specific type of map since it doesn’t matter here. Needless burden for users of your library.

Body:
I'm building a library in Kotlin and here's my usecase

I have a base class

abstract class Component(...) {
  // ... class body
}

I want the users of my library to define their own sub-classes like say:

class MyComponent() : Component() {
  // .. class body
}

How can I write a helper function that takes in this derived class as a param and create an instance out of it. Something like:

fun helper(component: Class, props: HashMap<String, String>) : Component {
  // somehow create a new instance of Class and return it?
}

Thanks!

Body:
You can have users pass a constructor reference:

fun helper(componentConstructor: ()->Component, props: Map<String, String>) : Component {
  val component = componentConstructor()
    // set it up and return it.
}

// usage:
val component = helper(::MyComponent, emptyMap())

Better for props not to require a specific type of map since it doesn’t matter here. Needless burden for users of your library.


问:
Finding the maximum product of a path in binary tree
I need some help to come up with an algorithm to calculate maximum product of the values along any path from leaf to root of a binary tree. So far I have:

```<code>int max_path_prod_rec(Tree* t, int max_so_far, int path_prod) {
    if (is_leaf(t))
        return t->value;

    int left = max_path_prod_rec(t->left, max_so_far, path_prod);
    int right = max_path_prod_rec(t->right, max_so_far, path_prod);

    if (left > right)
        return left * t->value;
    else
        return right * t->value;
}

int max_path_prod(Tree* t) {
    require("path product only defined for non-empty trees", t != NULL);
    return max_path_prod_rec(t, INT_MIN, 1);
}
</code>```

The problem is this function doesn't work when there are 2 negative numbers in a row, like 10 * (-3) * (-6). How can I improve upon this?

Edit: examples:

```<code>    //   10
    //  2   5
    // 1 -3 4 -6
    t = node(node(leaf(1), 2, leaf(-3)), 10, node(leaf(4), 5, leaf(-6)));
    printf("%i\n", max_path_prod(t));
</code>```

Will return 10 * 5 * 4 == 200. But

```<code>    //   10
    //  2   -5
    // 1 -3 4 -6
    t = node(node(leaf(1), 2, leaf(-3)), 10, node(leaf(4), -5, leaf(-6)));
    printf("%i\n", max_path_prod(t));
</code>```

Returns 10 * 2 * 1 == 20 instead of 10 * (-5) * (-6) == 300

答:
The basic issue is that when processing a subtree, you need to account for sign changes that will be applied when the subtree result is incorporated into the result for the overall tree.  If the overall result is going to be negative then you want to minimize the absolute value; otherwise, you want to maximize it.  But you can know that, and in fact, you can do even better.

I am not going to write a solution to this exercise for you, but you have a reasonable foundation, and I will try to set you on the right path.  Consider your function signature:

```<code>int max_path_prod_rec(Tree* t, int max_so_far, int path_prod)
</code>```

Now look at your implementation of that function.  Are you really using max_so_far and path_prod in the ways that their names imply? What would happen if you updated your function to pass correct values of those parameters when it recurses?  How could it use one or both of them to help it achieve the objective?

Body:
I need some help to come up with an algorithm to calculate maximum product of the values along any path from leaf to root of a binary tree. So far I have:

int max_path_prod_rec(Tree* t, int max_so_far, int path_prod) {
    if (is_leaf(t))
        return t->value;

    int left = max_path_prod_rec(t->left, max_so_far, path_prod);
    int right = max_path_prod_rec(t->right, max_so_far, path_prod);

    if (left > right)
        return left * t->value;
    else
        return right * t->value;
}

int max_path_prod(Tree* t) {
    require("path product only defined for non-empty trees", t != NULL);
    return max_path_prod_rec(t, INT_MIN, 1);
}

The problem is this function doesn't work when there are 2 negative numbers in a row, like 10 * (-3) * (-6). How can I improve upon this?

Edit: examples:

    //   10
    //  2   5
    // 1 -3 4 -6
    t = node(node(leaf(1), 2, leaf(-3)), 10, node(leaf(4), 5, leaf(-6)));
    printf("%i\n", max_path_prod(t));

Will return 10 * 5 * 4 == 200. But

    //   10
    //  2   -5
    // 1 -3 4 -6
    t = node(node(leaf(1), 2, leaf(-3)), 10, node(leaf(4), -5, leaf(-6)));
    printf("%i\n", max_path_prod(t));

Returns 10 * 2 * 1 == 20 instead of 10 * (-5) * (-6) == 300

Body:
The basic issue is that when processing a subtree, you need to account for sign changes that will be applied when the subtree result is incorporated into the result for the overall tree.  If the overall result is going to be negative then you want to minimize the absolute value; otherwise, you want to maximize it.  But you can know that, and in fact, you can do even better.

I am not going to write a solution to this exercise for you, but you have a reasonable foundation, and I will try to set you on the right path.  Consider your function signature:

int max_path_prod_rec(Tree* t, int max_so_far, int path_prod)

Now look at your implementation of that function.  Are you really using max_so_far and path_prod in the ways that their names imply? What would happen if you updated your function to pass correct values of those parameters when it recurses?  How could it use one or both of them to help it achieve the objective?


问:
Why totalDocsExamined is non zero even when query uses index in MongoDB?
From the official MongoDB docs its written, that a query is covered if it uses an index:

When an index covers a query, the explain result has an IXSCAN stage
that is not a descendant of a FETCH stage, and in the executionStats,
the totalDocsExamined is 0.

Then why when I find by _id, which is an index, and I configure the .explain() method, I get this result: totalDocsExamined = 1?

This is the query:

```<code>db.getCollection('orders').find({_id: ObjectId("70c457ef3f969259164df3e1")}).explain("executionStats")
</code>```

答:
That query isn't covered.

From the doc:

A covered query is a query that can be satisfied entirely using an index.

"satisfied entirely" means that all fields that are either queried or might be returned are contained in the index.

In order for that query to be covered by the default index on the _id field, it would need to have a projection that excluded all other fields, such as:

```<code>db.getCollection('orders').find({_id: ObjectId("70c457ef3f969259164df3e1")}, {_id:1})
</code>```

Body:
From the official MongoDB docs its written, that a query is covered if it uses an index:

When an index covers a query, the explain result has an IXSCAN stage
that is not a descendant of a FETCH stage, and in the executionStats,
the totalDocsExamined is 0.

Then why when I find by _id, which is an index, and I configure the .explain() method, I get this result: totalDocsExamined = 1?

This is the query:

db.getCollection('orders').find({_id: ObjectId("70c457ef3f969259164df3e1")}).explain("executionStats")

Body:
That query isn't covered.

From the doc:

A covered query is a query that can be satisfied entirely using an index.

"satisfied entirely" means that all fields that are either queried or might be returned are contained in the index.

In order for that query to be covered by the default index on the _id field, it would need to have a projection that excluded all other fields, such as:

db.getCollection('orders').find({_id: ObjectId("70c457ef3f969259164df3e1")}, {_id:1})


问:
Caused by: java.lang.RuntimeException: Duplicate class com.google.common.base.FinalizablePhantomReference
I'm working on Android Studio Bumblebee | 2021.1.1

I'm having this error when I try to compile the project:

```<code>Caused by: java.lang.RuntimeException: Duplicate class com.google.common.base.FinalizablePhantomReference found in modules jetified-google-collect-snapshot-20080530 (com.google.code.google-collections:google-collect:snapshot-20080530) and jetified-guava-28.1-android (com.google.guava:guava:28.1-android)
</code>```

Is simple annoying, I not able to recognize which exactly generate the error (The duplicate class).

Please let me know if you have any clue or if you have had the same problem before and know how to solve it, thanks!

答:
Add the following implementation in the app level build.gradle

```<code>implementation 'com.google.firebase:firebase-inappmessaging-display:17.2.0'
implementation 'com.google.guava:guava:27.0.1-android'
</code>```

Body:
I'm working on Android Studio Bumblebee | 2021.1.1

I'm having this error when I try to compile the project:

Caused by: java.lang.RuntimeException: Duplicate class com.google.common.base.FinalizablePhantomReference found in modules jetified-google-collect-snapshot-20080530 (com.google.code.google-collections:google-collect:snapshot-20080530) and jetified-guava-28.1-android (com.google.guava:guava:28.1-android)

Is simple annoying, I not able to recognize which exactly generate the error (The duplicate class).

Please let me know if you have any clue or if you have had the same problem before and know how to solve it, thanks!

Body:
Add the following implementation in the app level build.gradle

implementation 'com.google.firebase:firebase-inappmessaging-display:17.2.0'
implementation 'com.google.guava:guava:27.0.1-android'


问:
Jetson AGX Xavier python3 matplotlib 3.3.4 installation error
I want to use yolov5 at my Jetson AGX Xavier developer kit and I have to upgrade matplotlib to version 3.3.4 highest version that python3.6 support. I'm using python version 3.6.9(default of Jetson AGX Xavier) and python3.6 support matplotlib version 3.3.4. But I CANNOT upgrade that over 2.1.1(and this version is default also).

I've upgrade setuptools and I tried all command I can.

```<code>$ sudo apt-get install python3-matplotlib
$ python3 -m pip install --upgrade matplotlib
$ python3 -m pip install matplotlib==3.3.4
</code>```

How can I solve this problem?

Error code and image is here.

```<code>Defaulting to user installation because normal site-packages is not writeable
Collecting matplotlib==3.3.4
  Using cached matplotlib-3.3.4.tar.gz (37.9 MB)
  Preparing metadata (setup.py) ... error
  ERROR: Command errored out with exit status -4:
   command: /usr/bin/python3 -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-tk3o5hkn/matplotlib_53b7f655efb14a6a9d86b117497e1927/setup.py'"'"'; __file__='"'"'/tmp/pip-install-tk3o5hkn/matplotlib_53b7f655efb14a6a9d86b117497e1927/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-sz1ykkr8
       cwd: /tmp/pip-install-tk3o5hkn/matplotlib_53b7f655efb14a6a9d86b117497e1927/
  Complete output (19 lines):

  Edit setup.cfg to change the build options; suppress output with --quiet.

  BUILDING MATPLOTLIB
    matplotlib: yes [3.3.4]
        python: yes [3.6.9 (default, Dec  8 2021, 21:08:43)  [GCC 8.4.0]]
      platform: yes [linux]
   sample_data: yes [installing]
         tests: no  [skipping due to configuration]
        macosx: no  [Mac OS-X only]

  running egg_info
  creating /tmp/pip-pip-egg-info-sz1ykkr8/matplotlib.egg-info
  writing /tmp/pip-pip-egg-info-sz1ykkr8/matplotlib.egg-info/PKG-INFO
  writing dependency_links to /tmp/pip-pip-egg-info-sz1ykkr8/matplotlib.egg-info/dependency_links.txt
  writing namespace_packages to /tmp/pip-pip-egg-info-sz1ykkr8/matplotlib.egg-info/namespace_packages.txt
  writing requirements to /tmp/pip-pip-egg-info-sz1ykkr8/matplotlib.egg-info/requires.txt
  writing top-level names to /tmp/pip-pip-egg-info-sz1ykkr8/matplotlib.egg-info/top_level.txt
  writing manifest file '/tmp/pip-pip-egg-info-sz1ykkr8/matplotlib.egg-info/SOURCES.txt'
  ----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/22/d4/e7ca532e68a9357742604e1e4ae35d9c09a4a810de39a9d80402bd12f50f/matplotlib-3.3.4.tar.gz#sha256=3e477db76c22929e4c6876c44f88d790aacdf3c3f8f3a90cb1975c0bf37825b0 (from https://pypi.org/simple/matplotlib/) (requires-python:>=3.6). Command errored out with exit status -4: python setup.py egg_info Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement matplotlib==3.3.4 (from versions: 0.86, 0.86.1, 0.86.2, 0.91.0, 0.91.1, 1.0.1, 1.1.0, 1.1.1, 1.2.0, 1.2.1, 1.3.0, 1.3.1, 1.4.0, 1.4.1rc1, 1.4.1, 1.4.2, 1.4.3, 1.5.0, 1.5.1, 1.5.2, 1.5.3, 2.0.0b1, 2.0.0b2, 2.0.0b3, 2.0.0b4, 2.0.0rc1, 2.0.0rc2, 2.0.0, 2.0.1, 2.0.2, 2.1.0rc1, 2.1.0, 2.1.1, 2.1.2, 2.2.0rc1, 2.2.0, 2.2.2, 2.2.3, 2.2.4, 2.2.5, 3.0.0rc2, 3.0.0, 3.0.1, 3.0.2, 3.0.3, 3.1.0rc1, 3.1.0rc2, 3.1.0, 3.1.1, 3.1.2, 3.1.3, 3.2.0rc1, 3.2.0rc3, 3.2.0, 3.2.1, 3.2.2, 3.3.0rc1, 3.3.0, 3.3.1, 3.3.2, 3.3.3, 3.3.4)
ERROR: No matching distribution found for matplotlib==3.3.4
</code>```

error

答:
It seems that your python packages have broken, There can be two ways to install matplotlib

Way-1: Try to install matplotlib in venv

Way-2: Uninstall python and then reinstall python again.

Note: In Second way, don't restart jetson until new python installed completely, because python3 is already comes with jetson, so if you will uninstall it and then restart jetson, it will not work fine.

Body:
I want to use yolov5 at my Jetson AGX Xavier developer kit and I have to upgrade matplotlib to version 3.3.4 highest version that python3.6 support. I'm using python version 3.6.9(default of Jetson AGX Xavier) and python3.6 support matplotlib version 3.3.4. But I CANNOT upgrade that over 2.1.1(and this version is default also).

I've upgrade setuptools and I tried all command I can.

$ sudo apt-get install python3-matplotlib
$ python3 -m pip install --upgrade matplotlib
$ python3 -m pip install matplotlib==3.3.4

How can I solve this problem?

Error code and image is here.

Defaulting to user installation because normal site-packages is not writeable
Collecting matplotlib==3.3.4
  Using cached matplotlib-3.3.4.tar.gz (37.9 MB)
  Preparing metadata (setup.py) ... error
  ERROR: Command errored out with exit status -4:
   command: /usr/bin/python3 -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-tk3o5hkn/matplotlib_53b7f655efb14a6a9d86b117497e1927/setup.py'"'"'; __file__='"'"'/tmp/pip-install-tk3o5hkn/matplotlib_53b7f655efb14a6a9d86b117497e1927/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-sz1ykkr8
       cwd: /tmp/pip-install-tk3o5hkn/matplotlib_53b7f655efb14a6a9d86b117497e1927/
  Complete output (19 lines):

  Edit setup.cfg to change the build options; suppress output with --quiet.

  BUILDING MATPLOTLIB
    matplotlib: yes [3.3.4]
        python: yes [3.6.9 (default, Dec  8 2021, 21:08:43)  [GCC 8.4.0]]
      platform: yes [linux]
   sample_data: yes [installing]
         tests: no  [skipping due to configuration]
        macosx: no  [Mac OS-X only]

  running egg_info
  creating /tmp/pip-pip-egg-info-sz1ykkr8/matplotlib.egg-info
  writing /tmp/pip-pip-egg-info-sz1ykkr8/matplotlib.egg-info/PKG-INFO
  writing dependency_links to /tmp/pip-pip-egg-info-sz1ykkr8/matplotlib.egg-info/dependency_links.txt
  writing namespace_packages to /tmp/pip-pip-egg-info-sz1ykkr8/matplotlib.egg-info/namespace_packages.txt
  writing requirements to /tmp/pip-pip-egg-info-sz1ykkr8/matplotlib.egg-info/requires.txt
  writing top-level names to /tmp/pip-pip-egg-info-sz1ykkr8/matplotlib.egg-info/top_level.txt
  writing manifest file '/tmp/pip-pip-egg-info-sz1ykkr8/matplotlib.egg-info/SOURCES.txt'
  ----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/22/d4/e7ca532e68a9357742604e1e4ae35d9c09a4a810de39a9d80402bd12f50f/matplotlib-3.3.4.tar.gz#sha256=3e477db76c22929e4c6876c44f88d790aacdf3c3f8f3a90cb1975c0bf37825b0 (from https://pypi.org/simple/matplotlib/) (requires-python:>=3.6). Command errored out with exit status -4: python setup.py egg_info Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement matplotlib==3.3.4 (from versions: 0.86, 0.86.1, 0.86.2, 0.91.0, 0.91.1, 1.0.1, 1.1.0, 1.1.1, 1.2.0, 1.2.1, 1.3.0, 1.3.1, 1.4.0, 1.4.1rc1, 1.4.1, 1.4.2, 1.4.3, 1.5.0, 1.5.1, 1.5.2, 1.5.3, 2.0.0b1, 2.0.0b2, 2.0.0b3, 2.0.0b4, 2.0.0rc1, 2.0.0rc2, 2.0.0, 2.0.1, 2.0.2, 2.1.0rc1, 2.1.0, 2.1.1, 2.1.2, 2.2.0rc1, 2.2.0, 2.2.2, 2.2.3, 2.2.4, 2.2.5, 3.0.0rc2, 3.0.0, 3.0.1, 3.0.2, 3.0.3, 3.1.0rc1, 3.1.0rc2, 3.1.0, 3.1.1, 3.1.2, 3.1.3, 3.2.0rc1, 3.2.0rc3, 3.2.0, 3.2.1, 3.2.2, 3.3.0rc1, 3.3.0, 3.3.1, 3.3.2, 3.3.3, 3.3.4)
ERROR: No matching distribution found for matplotlib==3.3.4

error


问:
RDP or VNC in browser
Is anyone aware of good, usable and FREE (for commercial use) in-browser solutions to connect to a Windows server? I can't seem to find any inbrowser RDP. There seems to be some VNC based solutions (Guacamole?, terms unclear) and I can't tell if they're worth a shot.</p>

答: You can try ThinRDP, which is an HTML5 solution to connect to Windows, Terminal Server, or Linux running xRDP. It's not free though. </p>

ContentLicense: CC BY-SA 2.5

Title: RDP or VNC in browser

ContentLicense: CC BY-SA 3.0


问:
IPFS X-Ipfs-Path on static images referenced on a dynamic non-IPFS https page forces localhost gateway to load over https
I'm trying to utilize IPFS to load static content, such as images and javascript libraries, on a dynamic site loaded on the http protocol.

For example https://www.example.com/ is a normal web 2.0 page, with an image reference here https://www.example.com/images/myimage.jpg

When the request is made on myimage.jpg, the following header is served

```<code>x-ipfs-path: /ipfs/QmXXXXXXXXXXXXXXXXX/images/myimage.jpg
</code>```

Which then gets translated by the IPFS Companion browser plugin as:

```<code>https://127.0.0.1:8081/ipfs/QmXXXXXXXXXXXXXXXXX/images/myimage.jpg
</code>```

The problem being, is that it has directed to an SSL page on the local IP, which won't load due to a protocol error. (changing the above from https to http works)

Now, if I were to request https://www.example.com/images/myimage.jpg directly from the address bar, it loads the following:

```<code>http://localhost:8081/ipfs/QmYcJvDhjQJrMRFLsuWAJRDJigP38fiz2GiHoFrUQ53eNi/images/myimage.jpg
</code>```

And then a 301 to:

```<code>http://(some other hash).ipfs.localhost:8081/images/myimage.jpg
</code>```

Resulting in the image loading successfully.

I'm assuming because the initial page is served over SSL, it wants to serve the static content over SSL as well. I also assume that's why it then uses the local IP over https, rather than localhost in the other route.

My question is, how do I get this to work?

Is there a header which lets IPFS companion to force it to load over http? If so, I'm assuming this would cause browser security warnings due to mixed content. I have tried adding this header without luck: X-Forwarded-Proto: http

Do I need to do something to enable SSL over 127.0.0.1, connecting this up with my local node? If so, this doesn't seem to be the default setup for clients, and worry that all the content will show broken images if they do not follow some extra steps.

Is it even possible to serve static content over IPFS from non-IPFS pages?

Any hints appreciated!

Edit: This appears to effect the Chrome engine and and Firefox.

答:
Looks like a configuration error on your end.

Using IPFS Companion with default settings on a clean browser profile works as expected.

Opening: https://example.com/ipfs/bafybeigdyrzt5sfp7udm7hu76uh7y26nf3efuylqabf3oclgtqy55fbzdi redirects fine to http://localhost:8080/ipfs/bafybeigdyrzt5sfp7udm7hu76uh7y26nf3efuylqabf3oclgtqy55fbzdi which then redirects to unique Origin based on the root CID: http://bafybeigdyrzt5sfp7udm7hu76uh7y26nf3efuylqabf3oclgtqy55fbzdi.ipfs.localhost:8080/

You use custom port (8081), which means you changed the Gateway address in ipfs-companion Preferences at some point.

Potential fix: go there and make sure your "Local gateway" is set to http://localhost:8081 (instead of https://).

If you have http:// there, then see if you have some other extension or browser setting forcing https:// (check if this behavior occurs on a browser profile, and then add extensions/settings one by one, to identify the source of the problem).

Body:
I'm trying to utilize IPFS to load static content, such as images and javascript libraries, on a dynamic site loaded on the http protocol.

For example https://www.example.com/ is a normal web 2.0 page, with an image reference here https://www.example.com/images/myimage.jpg

When the request is made on myimage.jpg, the following header is served

x-ipfs-path: /ipfs/QmXXXXXXXXXXXXXXXXX/images/myimage.jpg

Which then gets translated by the IPFS Companion browser plugin as:

https://127.0.0.1:8081/ipfs/QmXXXXXXXXXXXXXXXXX/images/myimage.jpg

The problem being, is that it has directed to an SSL page on the local IP, which won't load due to a protocol error. (changing the above from https to http works)

Now, if I were to request https://www.example.com/images/myimage.jpg directly from the address bar, it loads the following:

http://localhost:8081/ipfs/QmYcJvDhjQJrMRFLsuWAJRDJigP38fiz2GiHoFrUQ53eNi/images/myimage.jpg

And then a 301 to:

http://(some other hash).ipfs.localhost:8081/images/myimage.jpg

Resulting in the image loading successfully.

I'm assuming because the initial page is served over SSL, it wants to serve the static content over SSL as well. I also assume that's why it then uses the local IP over https, rather than localhost in the other route.

My question is, how do I get this to work?

Is there a header which lets IPFS companion to force it to load over http? If so, I'm assuming this would cause browser security warnings due to mixed content. I have tried adding this header without luck: X-Forwarded-Proto: http

Do I need to do something to enable SSL over 127.0.0.1, connecting this up with my local node? If so, this doesn't seem to be the default setup for clients, and worry that all the content will show broken images if they do not follow some extra steps.

Is it even possible to serve static content over IPFS from non-IPFS pages?

Any hints appreciated!

Edit: This appears to effect the Chrome engine and and Firefox.


问:
Autohotkey ELSE statements not working. Any replacement?
I‘m making an autohotkey script.

What the problem is that the ELSE commands are sometimes not working.

How to fix this problem?

答:
When you code in AHK v1, one of the most common sources of confusion is the distinction between the deprecated, traditional if statements and more reasonable, modern if statements using expressions.

The official documentation says:

An If statement that contains an expression is usually differentiated from a traditional If statement such as if FoundColor != Blue by enclosing the expression in parentheses, ...

What happened to you was that you meant to use modern, universal if statements when AHK interpreted them as deprecated ones, which was why your else didn't get recognized.

The parentheses aren't really required every single time, but until you get the hang of it, make sure you use them as in:

```<code>if (expression)
  DoThis
else
  OrThat
</code>```

Body:
When you code in AHK v1, one of the most common sources of confusion is the distinction between the deprecated, traditional if statements and more reasonable, modern if statements using expressions.

The official documentation says:

An If statement that contains an expression is usually differentiated from a traditional If statement such as if FoundColor != Blue by enclosing the expression in parentheses, ...

What happened to you was that you meant to use modern, universal if statements when AHK interpreted them as deprecated ones, which was why your else didn't get recognized.

The parentheses aren't really required every single time, but until you get the hang of it, make sure you use them as in:

if (expression)
  DoThis
else
  OrThat


问:
How can I fetch data in my content script? (chrome extension)
I'm trying to fetch a simple .json file from my we accessible ressources into my content script. Here is what I tried so far:

manifest.json

{
    "manifest_version": 3,
    "name": "MyExtension",
    "version": "0.0",
    "content_scripts": [{
        "matches": ["*://*/*"],
        "js": ["content.js"]
    }],
    "icons":{
            "128": "images/logo128x128.png"
    },
    "web_accessible_resources": [{
      "resources": ["data/data.json"],
      "matches": ["https://web-accessible-resources-1.glitch.me/*"]
    }]
}```
</div>
</div>
</p>
<p>content.js</p>
<p><div class="snippet" data-lang="js" data-hide="false" data-console="true" data-babel="false">
<div class="snippet-code">
<pre class="snippet-code-js lang-js prettyprint-override"><code>const jsondict_url = chrome.runtime.getURL("data.json")
console.log(jsondict_url)

var jsondict = fetch(jsondict_url)

console.log(jsondict)</code>```

This is the error message I get:

chrome-extension://hbjmkaohgphjcegliejkcehbmjeggnnb/data.json
physique-generale-mecanique-PHYS-101-A#:1

       Denying load of chrome-extension://hbjmkaohgphjcegliejkcehbmjeggnnb/data.json. Resources must be listed in the web_accessible_resources manifest key in order to be loaded by pages outside the extension.

content.js:6 Promise {<pending>}[[Prototype]]: Promise
[[PromiseState]]: "rejected"
[[PromiseResult]]: TypeError: Failed to fetch
    at chrome-extension://hbjmkaohgphjcegliejkcehbmjeggnnb/content.js:4:16
content.js:4

       GET chrome-extension://invalid/ net::ERR_FAILED
(anonymous) @ content.js:4
content.js:4

      Uncaught (in promise) TypeError: Failed to fetch
    at content.js:4:16
(anonymous) @ content.js:4```

Edit: my files are organized this way

images/
logo128x128.png

data/data.json

content.js

manifest.js

答:
Correcting the path in 'content.js' to 'data/data.js' helped.

I found one solution to my issue, I think I was misusing the function fetch. This introduction https://developers.google.com/web/updates/2015/03/introduction-to-fetch to it uses the following code:

fetch('./api/some.json') // 'data/data.json' in my case
  .then(
    function(response) {
      if (response.status !== 200) {
        console.log('Looks like there was a problem. Status Code: ' +
          response.status);
        return;
      }

      // Examine the text in the response
      response.json().then(function(data) {
        console.log(data);
      });
    }
  )
  .catch(function(err) {
    console.log('Fetch Error :-S', err);
  });```

My extension works fine with this.

Body:
I'm trying to fetch a simple .json file from my we accessible ressources into my content script. Here is what I tried so far:

manifest.json

{
    "manifest_version": 3,
    "name": "MyExtension",
    "version": "0.0",
    "content_scripts": [{
        "matches": ["*://*/*"],
        "js": ["content.js"]
    }],
    "icons":{
            "128": "images/logo128x128.png"
    },
    "web_accessible_resources": [{
      "resources": ["data/data.json"],
      "matches": ["https://web-accessible-resources-1.glitch.me/*"]
    }]
}

content.js

const jsondict_url = chrome.runtime.getURL("data.json")
console.log(jsondict_url)

var jsondict = fetch(jsondict_url)

console.log(jsondict)

This is the error message I get:

chrome-extension://hbjmkaohgphjcegliejkcehbmjeggnnb/data.json
physique-generale-mecanique-PHYS-101-A#:1

       Denying load of chrome-extension://hbjmkaohgphjcegliejkcehbmjeggnnb/data.json. Resources must be listed in the web_accessible_resources manifest key in order to be loaded by pages outside the extension.

content.js:6 Promise {<pending>}[[Prototype]]: Promise
[[PromiseState]]: "rejected"
[[PromiseResult]]: TypeError: Failed to fetch
    at chrome-extension://hbjmkaohgphjcegliejkcehbmjeggnnb/content.js:4:16
content.js:4

       GET chrome-extension://invalid/ net::ERR_FAILED
(anonymous) @ content.js:4
content.js:4

      Uncaught (in promise) TypeError: Failed to fetch
    at content.js:4:16
(anonymous) @ content.js:4

Edit: my files are organized this way

images/
logo128x128.png

data/data.json

content.js

manifest.js

Body:
Correcting the path in 'content.js' to 'data/data.js' helped.

I found one solution to my issue, I think I was misusing the function fetch. This introduction https://developers.google.com/web/updates/2015/03/introduction-to-fetch to it uses the following code:

fetch('./api/some.json') // 'data/data.json' in my case
  .then(
    function(response) {
      if (response.status !== 200) {
        console.log('Looks like there was a problem. Status Code: ' +
          response.status);
        return;
      }

      // Examine the text in the response
      response.json().then(function(data) {
        console.log(data);
      });
    }
  )
  .catch(function(err) {
    console.log('Fetch Error :-S', err);
  });

My extension works fine with this.


问:
Unity animation not completing
i'm beginner on unity and game development. i'm trying to make a platformer game with unity, and i want to play a animation when the player is moving right or left, but the problem is that the isMoving variable for some reason is true animation is playing for a few moments (less than a second) and then is going to false again and the idle animation is playing.

My Animator (Exit time is disabled for all transitions):

My Movement script:

```<code>public class PlayerMovement : MonoBehaviour
{

    private bool canPlayerJump;
    private float horizontalInput;
    private Animator animator;

    // Start is called before the first frame update
    void Start()
    {
        animator = GetComponent<Animator>();
    }

    // Update is called once per frame
    void Update()
    {

        if (Input.GetKeyDown(KeyCode.Space) && canPlayerJump)
        {
            GetComponent<Rigidbody>().AddForce(Vector3.up * 5, ForceMode.VelocityChange);
        }

        if (Input.GetKeyDown(KeyCode.D) || Input.GetKeyDown(KeyCode.A))
        {
            animator.SetBool("isMoving", true);
        } else
        {
            animator.SetBool("isMoving", false);
        }

        horizontalInput = Input.GetAxis("Horizontal");

    }

    private void FixedUpdate()
    {
        GetComponent<Rigidbody>().velocity = new Vector3(horizontalInput * 5, GetComponent<Rigidbody>().velocity.y, 0);
    }

    private void OnCollisionEnter(Collision collision)
    {
       canPlayerJump = true;
       Debug.Log(collision.gameObject.name);
    }

    private void OnCollisionExit(Collision collision)
    {
        canPlayerJump = false;
    }

}
</code>```

答:
Input.GetKeyDown

is only true in the one frame the button went down.

Returns true during the frame the user starts pressing down the key identified by the key KeyCode enum parameter.

You rather want to use Input.GetKey which stay true as long as the key stays pressed.

Returns true while the user holds down the key identified by the key KeyCode enum parameter

alternatively in order to not have redundant accesses you could also simply use

```<code>horizontalInput = Input.GetAxis("Horizontal");

animator.SetBool("isMoving", !Mathf.Approximately(horizontalInput, 0));
</code>```

And as a general note: Cache the component references and don't use GetComponent over and over again!

```<code>[SerializeField] private Rigidbody _rigidbody;

private void Awake ()
{
    if(!_rigodbody) _rigidbody = GetComponent<Rigidbody>();
}
</code>```

then later everywhere use _rigidbdoy instead of GetComponent<Rigidbody>()

Title: Unity animation not completing

Body:
i'm beginner on unity and game development. i'm trying to make a platformer game with unity, and i want to play a animation when the player is moving right or left, but the problem is that the isMoving variable for some reason is true animation is playing for a few moments (less than a second) and then is going to false again and the idle animation is playing.

My Animator (Exit time is disabled for all transitions):

My Movement script:

public class PlayerMovement : MonoBehaviour
{

    private bool canPlayerJump;
    private float horizontalInput;
    private Animator animator;

    // Start is called before the first frame update
    void Start()
    {
        animator = GetComponent<Animator>();
    }

    // Update is called once per frame
    void Update()
    {

        if (Input.GetKeyDown(KeyCode.Space) && canPlayerJump)
        {
            GetComponent<Rigidbody>().AddForce(Vector3.up * 5, ForceMode.VelocityChange);
        }

        if (Input.GetKeyDown(KeyCode.D) || Input.GetKeyDown(KeyCode.A))
        {
            animator.SetBool("isMoving", true);
        } else
        {
            animator.SetBool("isMoving", false);
        }

        horizontalInput = Input.GetAxis("Horizontal");

    }

    private void FixedUpdate()
    {
        GetComponent<Rigidbody>().velocity = new Vector3(horizontalInput * 5, GetComponent<Rigidbody>().velocity.y, 0);
    }

    private void OnCollisionEnter(Collision collision)
    {
       canPlayerJump = true;
       Debug.Log(collision.gameObject.name);
    }

    private void OnCollisionExit(Collision collision)
    {
        canPlayerJump = false;
    }

}

Body:
Input.GetKeyDown

is only true in the one frame the button went down.

Returns true during the frame the user starts pressing down the key identified by the key KeyCode enum parameter.

You rather want to use Input.GetKey which stay true as long as the key stays pressed.

Returns true while the user holds down the key identified by the key KeyCode enum parameter

alternatively in order to not have redundant accesses you could also simply use

horizontalInput = Input.GetAxis("Horizontal");

animator.SetBool("isMoving", !Mathf.Approximately(horizontalInput, 0));

And as a general note: Cache the component references and don't use GetComponent over and over again!

[SerializeField] private Rigidbody _rigidbody;

private void Awake ()
{
    if(!_rigodbody) _rigidbody = GetComponent<Rigidbody>();
}

then later everywhere use _rigidbdoy instead of GetComponent<Rigidbody>()


问:
Accessing Delta Lake Table in Databricks via Spark in MLflow project
I am currently accessing deltalake table from databricks notebook using spark. However now I need to access delta tables from MLflow project. MLflow spark api only allows logging and loading of SparkML models. Any idea on how can I accomplish this?

Currently I am trying to access spark via this code in MLflow project:

spark = pyspark.sql.SparkSession._instantiatedSession
if spark is None:
  # NB: If there is no existing Spark context, create a new local one.
  # NB: We're disabling caching on the new context since we do not need it and we want to
  # avoid overwriting cache of underlying Spark cluster when executed on a Spark Worker
  # (e.g. as part of spark_udf).
  spark = ( pyspark.sql.SparkSession.builder \
   .config("spark.python.worker.reuse", True)
   .config("spark.databricks.io.cache.enabled", False)
   # In Spark 3.1 and above, we need to set this conf explicitly to enable creating
   # a SparkSession on the workers
   .config("spark.executor.allowSparkContext", "true")
   .master("local[*]")
   .appName("MLflow Project")
   .getOrCreate()
  )
```
<p>But I am getting this error:</p>
```py4j.protocol.Py4JJavaError: An error occurred while calling None.org.apache.spark.api.java.JavaSparkContext.
```

答:
It should be done the same way as for normal Spark projects that doesn't run in Notebooks:

Add dependencies to spark-submit or pyspark, you need to install delta-spark package to use code completion, etc. (--conf could be set from code itself, see next step):

pyspark --packages io.delta:delta-core_2.12:1.1.0 \
  --conf "spark.sql.extensions=io.delta.sql.DeltaSparkSessionExtension" \
  --conf "spark.sql.catalog.spark_catalog=org.apache.spark.sql.delta.catalog.DeltaCatalog"
```
<ul>
<li>create a <code>SparkSession</code> object and use it:</li>
</ul>
<pre class="lang-py prettyprint-override"><code>import pyspark
from delta import *

builder = pyspark.sql.SparkSession.builder.appName("MyApp") \
  .config("spark.sql.extensions", "io.delta.sql.DeltaSparkSessionExtension") \
  .config("spark.sql.catalog.spark_catalog", "org.apache.spark.sql.delta.catalog.DeltaCatalog")

spark = configure_spark_with_delta_pip(builder).getOrCreate()
</code>```

These steps are covered in the Quickstart guide included into Delta documentation.

If data resides on Azure Data Lake Storage, or AWS S3, or GCP, you may need to add additional packages & configurations, but that's also covered in documentation.

Body:
I am currently accessing deltalake table from databricks notebook using spark. However now I need to access delta tables from MLflow project. MLflow spark api only allows logging and loading of SparkML models. Any idea on how can I accomplish this?

Currently I am trying to access spark via this code in MLflow project:

spark = pyspark.sql.SparkSession._instantiatedSession
if spark is None:
  # NB: If there is no existing Spark context, create a new local one.
  # NB: We're disabling caching on the new context since we do not need it and we want to
  # avoid overwriting cache of underlying Spark cluster when executed on a Spark Worker
  # (e.g. as part of spark_udf).
  spark = ( pyspark.sql.SparkSession.builder \
   .config("spark.python.worker.reuse", True)
   .config("spark.databricks.io.cache.enabled", False)
   # In Spark 3.1 and above, we need to set this conf explicitly to enable creating
   # a SparkSession on the workers
   .config("spark.executor.allowSparkContext", "true")
   .master("local[*]")
   .appName("MLflow Project")
   .getOrCreate()
  )

But I am getting this error:

py4j.protocol.Py4JJavaError: An error occurred while calling None.org.apache.spark.api.java.JavaSparkContext.

Body:
It should be done the same way as for normal Spark projects that doesn't run in Notebooks:

Add dependencies to spark-submit or pyspark, you need to install delta-spark package to use code completion, etc. (--conf could be set from code itself, see next step):

pyspark --packages io.delta:delta-core_2.12:1.1.0 \
  --conf "spark.sql.extensions=io.delta.sql.DeltaSparkSessionExtension" \
  --conf "spark.sql.catalog.spark_catalog=org.apache.spark.sql.delta.catalog.DeltaCatalog"

create a SparkSession object and use it:

import pyspark
from delta import *

builder = pyspark.sql.SparkSession.builder.appName("MyApp") \
  .config("spark.sql.extensions", "io.delta.sql.DeltaSparkSessionExtension") \
  .config("spark.sql.catalog.spark_catalog", "org.apache.spark.sql.delta.catalog.DeltaCatalog")

spark = configure_spark_with_delta_pip(builder).getOrCreate()

These steps are covered in the Quickstart guide included into Delta documentation.

If data resides on Azure Data Lake Storage, or AWS S3, or GCP, you may need to add additional packages & configurations, but that's also covered in documentation.


问:
UISearchBar displays in the root view but not in any other view on the tab bar
I have implemented the search bar for the table in the first tab and this works fine and filters fine, however, with the same code and the xib linked up all the same etc on the second tab it no longer displays the search bar. Is there a simple reason to this?</p>

答: Yes, there is a simple reason—the search bar is a subview of your main view, so when that view gets swiped off screen, it disappears with that view. To fix this, you'll have to add the search bar to the view that holds your tab bar controller—you'll probably have to put it in MainWindow.xib. You can still connect it to your tab bar controller if you need to—just add a reference to in the XIB.</p>


问:
Clustering script fails with German, but works like expected with English
I have a script to cluster keywords, utilizing pandas and polyfuzz. With English, it works like expected. Trying to use the script with keywords in German, the script recognizes multiple keywords wrongly.

What means "wrongly recognized": clustering recognizes the first and second word in the keyword. And as you can see on the screenshot, columns G and H (First Word and Second Word) contain other words, then corresponding keywords in column B (Keyword):

The script fails not always with German - multiple keywords are clustered correctly. But the part of wrongly recognized keywords is very high, up to 20%.

Could somebody explain to me why the script failed with German keywords and, in the best case, improve the script enabling it to work with German?

Here is the part of the script, which does clustering:

```<code># find keywords from one column in another in any order and count the frequency
df_matched['Cluster Name'] = df_matched['Cluster Name'].str.strip()
df_matched['Keyword'] = df_matched['Keyword'].str.strip()

df_matched['First Word'] = df_matched['Cluster Name'].str.split(" ").str[0]
df_matched['Second Word'] = df_matched['Cluster Name'].str.split(" ").str[1]
df_matched['Total Keywords'] = df_matched['First Word'].str.count(' ') + 1

def ismatch(s):
    A = set(s["First Word"].split())
    B = set(s['Keyword'].split())
    return A.intersection(B) == A

df_matched['Found'] = df_matched.apply(ismatch, axis=1)

df_matched = df_matched. fillna('')

def ismatch(s):
    A = set(s["Second Word"].split())
    B = set(s['Keyword'].split())
    return A.intersection(B) == A
df_matched['Found 2'] = df_matched.apply(ismatch, axis=1)

# todo - document this algo. Essentially if it matches on the second word only, it renames the cluster to the second word
# clean up code nd variable names
df_matched.loc[(df_matched["Found"] == False) & (df_matched["Found 2"] == True), "Cluster Name"] = df_matched["Second Word"]
df_matched.loc[(df_matched["Found"] == False) & (df_matched["Found 2"] == False), "Cluster Name"] = "zzz_no_cluster_available"

# count cluster_size
df_matched['Cluster Size'] = df_matched['Cluster Name'].map(df_matched.groupby('Cluster Name')['Cluster Name'].count())
df_matched.loc[df_matched["Cluster Size"] == 1, "Cluster Name"] = "zzz_no_cluster_available"

df_matched = df_matched.sort_values(by="Cluster Name", ascending=True)
</code>```

Here are two datasets:

Working dataset in English: http://dl.dropboxusercontent.com/s/zrobh2x4bs3ztlf/working-dataset-english.txt

Badly working dataset in German: http://dl.dropboxusercontent.com/s/i1p3j3zi1t0cev3/badly-working-dataset-german.txt

And here, the working Colab with the whole script.

答:
I opened the full code to understand where df_matched came from.
I'm not 100% sure of what you are trying to do, but I think that the problem comes from before the snippet you shared here.

It comes from the way that df_matched is created. It uses fuzzy matching to create clusters. So the words of "Cluster Name" are not all guaranteed to be present in "Keyword".

If you run the code for the English data, and check the words in position -1 and -2 (last two words of the Cluster Name) instead of 0 and 1...

df_matched['First Word'] = df_matched['Cluster Name'].str.split(" ").str[-1]
df_matched['Second Word'] = df_matched['Cluster Name'].str.split(" ").str[-2]
```
<p>...then calculate how many of them are not found...</p>
<pre class="lang-py prettyprint-override"><code>print((~df_matched["Found"]).sum())
print((~df_matched["Found 2"]).sum())
# 140
# 10
</code>```

...you can see that for 104 out of 158 rows, the last word is not part of the keywords.

(I don't know if you care about the first two words more than the last two... but this looks worse than the 20% you noticed in the German data.)

For the German one the problem is more visible because this language uses a lot of compound words and many frequent suffixes (e.g., "ung")... So they will fuzzy-match a lot.

Example of df_matched for German: the "From" words are not present in "To"... but there are large overlaps.

This is df_matched for English: some words of "From" are not even close to the words in "To"... and similarity score can be worse than in the German dataset.

Possible improvements

I think that the part where you could improve the clustering is this (from the colab notebook)

df_1_list = df_1.Keyword.tolist()  # create list from df
model = PolyFuzz("TF-IDF")

cluster_tags = df_1_list[::]
cluster_tags = set(cluster_tags)
cluster_tags = list(cluster_tags)

print("Cleaning up the cluster tags.. Please be patient!")
substrings = {w1 for w1 in tqdm(cluster_tags) for w2 in cluster_tags if w1 in w2 and w1 != w2}
longest_word = set(cluster_tags) - substrings
longest_word = list(longest_word)
shortest_word_list = list(set(cluster_tags) - set(longest_word))

try:
    model.match(df_1_list, shortest_word_list)
except ValueError:
    print("Empty Dataframe, Can't Match - Check the URL Filter!")
    sys.exit()

model.group(link_min_similarity=sim_match_percent)
df_matched = model.get_matches()
```
<p>Here you compute the similarity between <code>df_1_list</code> and <code>shortest_word_list</code>.
<code>shortest_word_list</code> is created by looking for substrings, which might lead to weird clusters is German because of compound words.</p>
<p>You could try and normalize the text with (language-specific) stemming or lemmatization before / instead of checking for substrings and creating clusters. This should help and transform each word in their "root form" and retain their meaning.</p>
<p>Yoy can use the spaCy library, which provide language-specific
pretrained models for stemming, embedding and other language operations.
You can select the correct model for each language and use the lemmatization function to replace each word of <code>df_1_list</code> with their "base form" before trying to cluster.</p>
<p>Lemmatization example</p>
<pre class="lang-py prettyprint-override"><code>import spacy

nlp = spacy.load("en_core_web_sm") # load English or German model
lemmatizer = nlp.get_pipe("lemmatizer")
print(lemmatizer.mode)  # 'rule'

doc = nlp("I was reading the paper.")
print([token.lemma_ for token in doc])
# ['I', 'be', 'read', 'the', 'paper', '.']
</code>```

Link to spaCy German model: https://spacy.io/models/de

Body:
I have a script to cluster keywords, utilizing pandas and polyfuzz. With English, it works like expected. Trying to use the script with keywords in German, the script recognizes multiple keywords wrongly.

What means "wrongly recognized": clustering recognizes the first and second word in the keyword. And as you can see on the screenshot, columns G and H (First Word and Second Word) contain other words, then corresponding keywords in column B (Keyword):

The script fails not always with German - multiple keywords are clustered correctly. But the part of wrongly recognized keywords is very high, up to 20%.

Could somebody explain to me why the script failed with German keywords and, in the best case, improve the script enabling it to work with German?

Here is the part of the script, which does clustering:

# find keywords from one column in another in any order and count the frequency
df_matched['Cluster Name'] = df_matched['Cluster Name'].str.strip()
df_matched['Keyword'] = df_matched['Keyword'].str.strip()

df_matched['First Word'] = df_matched['Cluster Name'].str.split(" ").str[0]
df_matched['Second Word'] = df_matched['Cluster Name'].str.split(" ").str[1]
df_matched['Total Keywords'] = df_matched['First Word'].str.count(' ') + 1

def ismatch(s):
    A = set(s["First Word"].split())
    B = set(s['Keyword'].split())
    return A.intersection(B) == A

df_matched['Found'] = df_matched.apply(ismatch, axis=1)

df_matched = df_matched. fillna('')

def ismatch(s):
    A = set(s["Second Word"].split())
    B = set(s['Keyword'].split())
    return A.intersection(B) == A
df_matched['Found 2'] = df_matched.apply(ismatch, axis=1)

# todo - document this algo. Essentially if it matches on the second word only, it renames the cluster to the second word
# clean up code nd variable names
df_matched.loc[(df_matched["Found"] == False) & (df_matched["Found 2"] == True), "Cluster Name"] = df_matched["Second Word"]
df_matched.loc[(df_matched["Found"] == False) & (df_matched["Found 2"] == False), "Cluster Name"] = "zzz_no_cluster_available"

# count cluster_size
df_matched['Cluster Size'] = df_matched['Cluster Name'].map(df_matched.groupby('Cluster Name')['Cluster Name'].count())
df_matched.loc[df_matched["Cluster Size"] == 1, "Cluster Name"] = "zzz_no_cluster_available"

df_matched = df_matched.sort_values(by="Cluster Name", ascending=True)

Here are two datasets:

Working dataset in English: http://dl.dropboxusercontent.com/s/zrobh2x4bs3ztlf/working-dataset-english.txt

Badly working dataset in German: http://dl.dropboxusercontent.com/s/i1p3j3zi1t0cev3/badly-working-dataset-german.txt

And here, the working Colab with the whole script.

Body:
I opened the full code to understand where df_matched came from.
I'm not 100% sure of what you are trying to do, but I think that the problem comes from before the snippet you shared here.

It comes from the way that df_matched is created. It uses fuzzy matching to create clusters. So the words of "Cluster Name" are not all guaranteed to be present in "Keyword".

If you run the code for the English data, and check the words in position -1 and -2 (last two words of the Cluster Name) instead of 0 and 1...

df_matched['First Word'] = df_matched['Cluster Name'].str.split(" ").str[-1]
df_matched['Second Word'] = df_matched['Cluster Name'].str.split(" ").str[-2]

...then calculate how many of them are not found...

print((~df_matched["Found"]).sum())
print((~df_matched["Found 2"]).sum())
# 140
# 10

...you can see that for 104 out of 158 rows, the last word is not part of the keywords.

(I don't know if you care about the first two words more than the last two... but this looks worse than the 20% you noticed in the German data.)

For the German one the problem is more visible because this language uses a lot of compound words and many frequent suffixes (e.g., "ung")... So they will fuzzy-match a lot.

Example of df_matched for German: the "From" words are not present in "To"... but there are large overlaps.

This is df_matched for English: some words of "From" are not even close to the words in "To"... and similarity score can be worse than in the German dataset.

Possible improvements

I think that the part where you could improve the clustering is this (from the colab notebook)

df_1_list = df_1.Keyword.tolist()  # create list from df
model = PolyFuzz("TF-IDF")

cluster_tags = df_1_list[::]
cluster_tags = set(cluster_tags)
cluster_tags = list(cluster_tags)

print("Cleaning up the cluster tags.. Please be patient!")
substrings = {w1 for w1 in tqdm(cluster_tags) for w2 in cluster_tags if w1 in w2 and w1 != w2}
longest_word = set(cluster_tags) - substrings
longest_word = list(longest_word)
shortest_word_list = list(set(cluster_tags) - set(longest_word))

try:
    model.match(df_1_list, shortest_word_list)
except ValueError:
    print("Empty Dataframe, Can't Match - Check the URL Filter!")
    sys.exit()

model.group(link_min_similarity=sim_match_percent)
df_matched = model.get_matches()

Here you compute the similarity between df_1_list and shortest_word_list.
shortest_word_list is created by looking for substrings, which might lead to weird clusters is German because of compound words.

You could try and normalize the text with (language-specific) stemming or lemmatization before / instead of checking for substrings and creating clusters. This should help and transform each word in their "root form" and retain their meaning.

Yoy can use the spaCy library, which provide language-specific
pretrained models for stemming, embedding and other language operations.
You can select the correct model for each language and use the lemmatization function to replace each word of df_1_list with their "base form" before trying to cluster.

Lemmatization example

import spacy

nlp = spacy.load("en_core_web_sm") # load English or German model
lemmatizer = nlp.get_pipe("lemmatizer")
print(lemmatizer.mode)  # 'rule'

doc = nlp("I was reading the paper.")
print([token.lemma_ for token in doc])
# ['I', 'be', 'read', 'the', 'paper', '.']

Link to spaCy German model: https://spacy.io/models/de


问:
Moving a MDToolbar in kivymd
i don't know how to move the toolbar to the top of the screen, before when i was creating a toolbar it was placed at the top of the screen by default. How can i fix this problem?
output image

KV = ('''
Screen:

    MDToolbar:
        title: "WeatherApp"

    MDLabel:
        text: "Hello World"
''')

class WeatherApp(MDApp):

    Window.size = (350, 630)

    def build(self):

        self.title = "WeatherApp"
        self.theme_cls.theme_style = 'Dark'
        return Builder.load_string(KV)

WeatherApp().run()
```

答:
As you are using Screen as your root widget (also a parent of MDToolbar), which is a subclass of FloatLayout, you have to explicitly set any widget's position, otherwise all will be stacked on each other at (0, 0), the default position.

So if you want it to be placed on the top, use attr. pos_hint as,

Screen:

    MDToolbar:
        title: "WeatherApp"
        pos_hint: {"top" : 1}
```

Title: Moving a MDToolbar in kivymd