问:
Python3 AttributeError I don't understand
The code at the bottom generates the following traceback in my program.

```<code>Traceback (most recent call last):                                                                                │
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1832, in full_dispatch_request                                          │
    rv = self.dispatch_request()                                                                                │
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1818, in dispatch_request                                               │
    return self.view_functions[rule.endpoint](**req.view_args)                                                                             │
  File "/usr/local/lib/python3.6/site-packages/flask_restplus/api.py", line 325, in wrapper                                                │
    resp = resource(*args, **kwargs)                                                                                │
  File "/usr/local/lib/python3.6/site-packages/flask/views.py", line 87, in view                                                           │
    self = view.view_class(*class_args, **class_kwargs)                                                                                │
  File "/etc/changes/api/__init__.py", line 22, in __init__                                                                                │
    self.request_event = RequestEvent(request.environ)                                                                                │
  File "/etc/changes/event/request_event.py", line 11, in __init__                                                                         │
    super(RequestEvent, self).__init__(event_data)                                                                                │
  File "/etc/changes/event/__init__.py", line 14, in __init__                                                                              │
    self._load_json_object(event_data)                                                                                │
  File "/etc/changes/event/request_event.py", line 19, in _load_json_object                                                                │
    self.client_cert = self._get_certification_details().get('client_serial', None)                                                        │
  File "/etc/changes/event/request_event.py", line 23, in _get_certification_details                                                       │
    CLIENT_VERIFY = self.request_environment.get('CLIENT_VERIFY', None)                                                                    │
AttributeError: 'RequestEvent' object has no attribute 'request_environment'
</code>```

I don't understand why, since the attribute is clearly made before the method. Any clue with which you can help me out? Some of the code has been removed for clarity.

```<code>from event import Event
from manager.manager_factory import ManagerFactory

class RequestEvent(Event):

    def __init__(self, request_environment, event_data={}):
        super(RequestEvent, self).__init__(event_data)
        self.request_environment = request_environment
        self.certification_details = self._get_certification_details()

    def _load_json_object(self,json_obj):
        super()._load_json_object(json_obj)
        self.type = 'request'
        self.trigger_invocation = json_obj.get('trigger_invocation', False)
        self.client_cert = self._get_certification_details().get('client_serial', None)
        self.tenant = ManagerFactory().authentication_manager

    def _get_certification_details(self):
        CLIENT_VERIFY = self.request_environment.get('CLIENT_VERIFY', None)
                   client_certificate_details = {}
        client_certificate_details['client_verify'] = CLIENT_VERIFY
        return client_certificate_details
</code>```

答:
Check the trace-back carefully.

You have a recursive definition there. The super __init__ is calling the _load_json_object method of the subclass. Which, in turn, is calling the _get_vertification_details method. All this is happening before you defined the request_environment attribute.

Body:
The code at the bottom generates the following traceback in my program.

Traceback (most recent call last):                                                                                │
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1832, in full_dispatch_request                                          │
    rv = self.dispatch_request()                                                                                │
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1818, in dispatch_request                                               │
    return self.view_functions[rule.endpoint](**req.view_args)                                                                             │
  File "/usr/local/lib/python3.6/site-packages/flask_restplus/api.py", line 325, in wrapper                                                │
    resp = resource(*args, **kwargs)                                                                                │
  File "/usr/local/lib/python3.6/site-packages/flask/views.py", line 87, in view                                                           │
    self = view.view_class(*class_args, **class_kwargs)                                                                                │
  File "/etc/changes/api/__init__.py", line 22, in __init__                                                                                │
    self.request_event = RequestEvent(request.environ)                                                                                │
  File "/etc/changes/event/request_event.py", line 11, in __init__                                                                         │
    super(RequestEvent, self).__init__(event_data)                                                                                │
  File "/etc/changes/event/__init__.py", line 14, in __init__                                                                              │
    self._load_json_object(event_data)                                                                                │
  File "/etc/changes/event/request_event.py", line 19, in _load_json_object                                                                │
    self.client_cert = self._get_certification_details().get('client_serial', None)                                                        │
  File "/etc/changes/event/request_event.py", line 23, in _get_certification_details                                                       │
    CLIENT_VERIFY = self.request_environment.get('CLIENT_VERIFY', None)                                                                    │
AttributeError: 'RequestEvent' object has no attribute 'request_environment'

I don't understand why, since the attribute is clearly made before the method. Any clue with which you can help me out? Some of the code has been removed for clarity.

from event import Event
from manager.manager_factory import ManagerFactory

class RequestEvent(Event):

    def __init__(self, request_environment, event_data={}):
        super(RequestEvent, self).__init__(event_data)
        self.request_environment = request_environment
        self.certification_details = self._get_certification_details()

    def _load_json_object(self,json_obj):
        super()._load_json_object(json_obj)
        self.type = 'request'
        self.trigger_invocation = json_obj.get('trigger_invocation', False)
        self.client_cert = self._get_certification_details().get('client_serial', None)
        self.tenant = ManagerFactory().authentication_manager

    def _get_certification_details(self):
        CLIENT_VERIFY = self.request_environment.get('CLIENT_VERIFY', None)
                   client_certificate_details = {}
        client_certificate_details['client_verify'] = CLIENT_VERIFY
        return client_certificate_details


来源: stackexchange

问:
Julia | DataFrame | Replacing missing Values
How can we replace missing values with 0.0 for a column in a DataFrame?

答:
There are a few different approaches to this problem (valid for Julia 1.x):

Base.replace!

Probably the easiest approach is to use replace! or replace from base Julia. Here is an example with replace!:

```<code>julia> using DataFrames

julia> df = DataFrame(x = [1, missing, 3])
3×1 DataFrame
│ Row │ x       │
│     │ Int64⍰  │
├─────┼─────────┤
│ 1   │ 1       │
│ 2   │ missing │
│ 3   │ 3       │

julia> replace!(df.x, missing => 0);

julia> df
3×1 DataFrame
│ Row │ x      │
│     │ Int64⍰ │
├─────┼────────┤
│ 1   │ 1      │
│ 2   │ 0      │
│ 3   │ 3      │
</code>```

However, note that at this point the type of column x still allows missing values:

```<code>julia> typeof(df.x)
Array{Union{Missing, Int64},1}
</code>```

This is also indicated by the question mark following Int64 in column x when the data frame is printed out. You can change this by using disallowmissing! (from the DataFrames.jl package):

```<code>julia> disallowmissing!(df, :x)
3×1 DataFrame
│ Row │ x     │
│     │ Int64 │
├─────┼───────┤
│ 1   │ 1     │
│ 2   │ 0     │
│ 3   │ 3     │
</code>```

Alternatively, if you use replace (without the exclamation mark) as follows, then the output will already disallow missing values:

```<code>julia> df = DataFrame(x = [1, missing, 3]);

julia> df.x = replace(df.x, missing => 0);

julia> df
3×1 DataFrame
│ Row │ x     │
│     │ Int64 │
├─────┼───────┤
│ 1   │ 1     │
│ 2   │ 0     │
│ 3   │ 3     │
</code>```

Finally, you can replace missing in all columns at once by using mapcols:

```<code>julia> df = DataFrame(a=[1, missing, 3], b=[4, 5, missing]);

julia> mapcols(col -> replace(col, missing => 0), df)
3×2 DataFrame
 Row │ a      b
     │ Int64  Int64
─────┼──────────────
   1 │     1      4
   2 │     0      5
   3 │     3      0
</code>```

Base.ismissing with logical indexing

You can use ismissing with logical indexing to assign a new value to all missing entries of an array:

```<code>julia> df = DataFrame(x = [1, missing, 3]);

julia> df.x[ismissing.(df.x)] .= 0;

julia> df
3×1 DataFrame
│ Row │ x      │
│     │ Int64⍰ │
├─────┼────────┤
│ 1   │ 1      │
│ 2   │ 0      │
│ 3   │ 3      │
</code>```

Base.coalesce

Another approach is to use coalesce:

```<code>julia> df = DataFrame(x = [1, missing, 3]);

julia> df.x = coalesce.(df.x, 0);

julia> df
3×1 DataFrame
│ Row │ x     │
│     │ Int64 │
├─────┼───────┤
│ 1   │ 1     │
│ 2   │ 0     │
│ 3   │ 3     │
</code>```

DataFramesMeta

Both replace and coalesce can be used with the @transform macro from the DataFramesMeta.jl package:

```<code>julia> using DataFramesMeta

julia> df = DataFrame(x = [1, missing, 3]);

julia> @transform(df, x = replace(:x, missing => 0))
3×1 DataFrame
│ Row │ x     │
│     │ Int64 │
├─────┼───────┤
│ 1   │ 1     │
│ 2   │ 0     │
│ 3   │ 3     │
</code>```
```<code>julia> df = DataFrame(x = [1, missing, 3]);

julia> @transform(df, x = coalesce.(:x, 0))
3×1 DataFrame
│ Row │ x     │
│     │ Int64 │
├─────┼───────┤
│ 1   │ 1     │
│ 2   │ 0     │
│ 3   │ 3     │
</code>```

Additional documentation

Julia manual

Julia manual - function reference

DataFrames.jl manual

ContentLicense: CC BY-SA 4.0

Body:
There are a few different approaches to this problem (valid for Julia 1.x):

Base.replace!

Probably the easiest approach is to use replace! or replace from base Julia. Here is an example with replace!:

julia> using DataFrames

julia> df = DataFrame(x = [1, missing, 3])
3×1 DataFrame
│ Row │ x       │
│     │ Int64⍰  │
├─────┼─────────┤
│ 1   │ 1       │
│ 2   │ missing │
│ 3   │ 3       │

julia> replace!(df.x, missing => 0);

julia> df
3×1 DataFrame
│ Row │ x      │
│     │ Int64⍰ │
├─────┼────────┤
│ 1   │ 1      │
│ 2   │ 0      │
│ 3   │ 3      │

However, note that at this point the type of column x still allows missing values:

julia> typeof(df.x)
Array{Union{Missing, Int64},1}

This is also indicated by the question mark following Int64 in column x when the data frame is printed out. You can change this by using disallowmissing! (from the DataFrames.jl package):

julia> disallowmissing!(df, :x)
3×1 DataFrame
│ Row │ x     │
│     │ Int64 │
├─────┼───────┤
│ 1   │ 1     │
│ 2   │ 0     │
│ 3   │ 3     │

Alternatively, if you use replace (without the exclamation mark) as follows, then the output will already disallow missing values:

julia> df = DataFrame(x = [1, missing, 3]);

julia> df.x = replace(df.x, missing => 0);

julia> df
3×1 DataFrame
│ Row │ x     │
│     │ Int64 │
├─────┼───────┤
│ 1   │ 1     │
│ 2   │ 0     │
│ 3   │ 3     │

Finally, you can replace missing in all columns at once by using mapcols:

julia> df = DataFrame(a=[1, missing, 3], b=[4, 5, missing]);

julia> mapcols(col -> replace(col, missing => 0), df)
3×2 DataFrame
 Row │ a      b
     │ Int64  Int64
─────┼──────────────
   1 │     1      4
   2 │     0      5
   3 │     3      0

Base.ismissing with logical indexing

You can use ismissing with logical indexing to assign a new value to all missing entries of an array:

julia> df = DataFrame(x = [1, missing, 3]);

julia> df.x[ismissing.(df.x)] .= 0;

julia> df
3×1 DataFrame
│ Row │ x      │
│     │ Int64⍰ │
├─────┼────────┤
│ 1   │ 1      │
│ 2   │ 0      │
│ 3   │ 3      │

Base.coalesce

Another approach is to use coalesce:

julia> df = DataFrame(x = [1, missing, 3]);

julia> df.x = coalesce.(df.x, 0);

julia> df
3×1 DataFrame
│ Row │ x     │
│     │ Int64 │
├─────┼───────┤
│ 1   │ 1     │
│ 2   │ 0     │
│ 3   │ 3     │

DataFramesMeta

Both replace and coalesce can be used with the @transform macro from the DataFramesMeta.jl package:

julia> using DataFramesMeta

julia> df = DataFrame(x = [1, missing, 3]);

julia> @transform(df, x = replace(:x, missing => 0))
3×1 DataFrame
│ Row │ x     │
│     │ Int64 │
├─────┼───────┤
│ 1   │ 1     │
│ 2   │ 0     │
│ 3   │ 3     │

julia> df = DataFrame(x = [1, missing, 3]);

julia> @transform(df, x = coalesce.(:x, 0))
3×1 DataFrame
│ Row │ x     │
│     │ Int64 │
├─────┼───────┤
│ 1   │ 1     │
│ 2   │ 0     │
│ 3   │ 3     │

Additional documentation

Julia manual

Julia manual - function reference

DataFrames.jl manual


问:
Android Layout Inspector fails with timeout error
When I try to inspect my application's layout in a simulator, by going Tools => Layout Inspector, it always fails with one of the errors:

1.

Error obtaining view hierarchy: There was a timeout error capturing the layout data from the device.
      The device may be too slow, the captured view may be too complex, or the view may contain animations.

Please retry with a simplified view and ensure the device is responsive.

2.

No active windows displayed by com.instabridge.android.dev

The layout is not particularly complex, and it does not contain animations, just map with some views. Android Studio is 3.4.1. How do I fix it? Or how to find the cause of the errors?

答:
I had the same issue on Android Studio 3.4, but I was able to use the Layout Inspector on Android Studio 3.5 Beta 1 without any problems.

I am working with a very complex view, so I suspect that is what led to the trouble on Android Studio 3.4. Both the failure I saw and the success were with a physical device.


问:
Edit headers of fancybox ajax request
Hello I am trying to alter ajax request headers while opening fancybox.
So far I can not seem to make it work even according to documentation. After stating type to be ajax i should be able to edit ajax according to jquery documentation.

So far it does work without headers like this (everything is inside jQ on click event):

```<code>$.fancybox.open( $(this), {
        type: 'ajax',
        helpers: {
            overlay: {
                locked: false
            }
        }
    });
</code>```

The problem comes when i try to alter ajax request. This is what i tried and does not work:

```<code>$.fancybox.open({
            type: 'ajax',
            ajax: {
                url: siteUrl + href,
                type: 'GET',
                headers: {
                    'fooheader' : 'bar'
                }
            },
            helpers: {
                overlay: {
                    locked: false
                }
            }
        });
</code>```

What happens when I execute previous code is that it redirects me back to the same page i clicked from.

Does anyone have experience with adding headers inside fancybox ajax request?

答: The script merges ajax option and passes to jQuery ajax method - http://api.jquery.com/jquery.ajax/ - as settings and everything works fine.

Title: Edit headers of fancybox ajax request

Body:
Hello I am trying to alter ajax request headers while opening fancybox.
So far I can not seem to make it work even according to documentation. After stating type to be ajax i should be able to edit ajax according to jquery documentation.

So far it does work without headers like this (everything is inside jQ on click event):

$.fancybox.open( $(this), {
        type: 'ajax',
        helpers: {
            overlay: {
                locked: false
            }
        }
    });

The problem comes when i try to alter ajax request. This is what i tried and does not work:

$.fancybox.open({
            type: 'ajax',
            ajax: {
                url: siteUrl + href,
                type: 'GET',
                headers: {
                    'fooheader' : 'bar'
                }
            },
            helpers: {
                overlay: {
                    locked: false
                }
            }
        });

What happens when I execute previous code is that it redirects me back to the same page i clicked from.

Does anyone have experience with adding headers inside fancybox ajax request?


问:
fullcalendar - display time in dayGridWeek
I use fullcalendar to display events but I have problem. How to display time in left in dayGridWeek?

code js :

```<code>var displayDisponibiltyBabySitter = function(){
            var calendarEl = document.getElementById('disponiblity');
            if(calendarEl){
                var calendar = new FullCalendar.Calendar(calendarEl, {
                    defaultView: 'dayGridWeek' ,
                    validRange: function(nowDate){
                        return {start: nowDate}
                    },
                    header: {
                        left: 'prev,next today',
                        center: 'title',
                        right: 'dayGridWeek,timeGridDay'
                    },
                    locale: _mpop.current_lang,
                    timeZone: 'Europe/Paris',
                    eventBackgroundColor : "#444",
                    eventTextColor: '#EEE',
                    slotDuration: '00:60:00',
                    slotLabelFormat: [
                        {
                            hour: '2-digit',
                            minute: '2-digit',
                            hour12: false
                        }
                    ],
                    views: {
                        dayGridWeek: {
                            eventTimeFormat: {
                                hour: 'numeric',
                                minute: '2-digit',
                                omitZeroMinute: true,
                                meridiem: 'narrow'
                            }
                        }
                    },
                    events: Routing.generate('front.babysitter.disponibilities',{'id': $("#dispo-babysitter").data("id")}),
                    plugins: [ 'dayGrid', 'timeGrid', 'list' ]
                });
                calendar.render();
            }

        };
</code>```

thanks in advance

I have this result:

but how to get this result:

答:
You've got your defaultView set as 'dayGridWeek'

set it to:

```<code>defaultView: 'timeGridWeek'
</code>```

and you should be fine...

Body:
I use fullcalendar to display events but I have problem. How to display time in left in dayGridWeek?

code js :

var displayDisponibiltyBabySitter = function(){
            var calendarEl = document.getElementById('disponiblity');
            if(calendarEl){
                var calendar = new FullCalendar.Calendar(calendarEl, {
                    defaultView: 'dayGridWeek' ,
                    validRange: function(nowDate){
                        return {start: nowDate}
                    },
                    header: {
                        left: 'prev,next today',
                        center: 'title',
                        right: 'dayGridWeek,timeGridDay'
                    },
                    locale: _mpop.current_lang,
                    timeZone: 'Europe/Paris',
                    eventBackgroundColor : "#444",
                    eventTextColor: '#EEE',
                    slotDuration: '00:60:00',
                    slotLabelFormat: [
                        {
                            hour: '2-digit',
                            minute: '2-digit',
                            hour12: false
                        }
                    ],
                    views: {
                        dayGridWeek: {
                            eventTimeFormat: {
                                hour: 'numeric',
                                minute: '2-digit',
                                omitZeroMinute: true,
                                meridiem: 'narrow'
                            }
                        }
                    },
                    events: Routing.generate('front.babysitter.disponibilities',{'id': $("#dispo-babysitter").data("id")}),
                    plugins: [ 'dayGrid', 'timeGrid', 'list' ]
                });
                calendar.render();
            }

        };

thanks in advance

I have this result:

but how to get this result:

Body:
You've got your defaultView set as 'dayGridWeek'

set it to:

defaultView: 'timeGridWeek'

and you should be fine...


问:
Attach the DotTrace Performance profiler with Visual Studio Load Testing
Is there, Any way I can attach the DotTrace performance profiler with Visual studio Load testing project. I am checking the load testing of the ASP.NET Web Application. I am trying to create stress test and trying to do more analysis. I am just curious that is there any way there we can attach DotTrace performance profiler with the Visual Studio Load test and check the details of the Load test.</p>

答: This answer just incase if you dont have production or TEST environment access to attach the process and all. You can connect to the same TEST DB or Dev DB when load testing is happening from the local environment then attach the process of the IIS running app and check the dotTrace analysis.</p>


问:
Getting all installed applications with Category.LAUNCHER
My Problem:

I want to get all installed applications with the Intent Category Category.LAUNCHER.
So I want to get all apps that would be displayed in the Launcher

and then resolve the package and the name.

Such as package: "com.android.phone" -- name: "Phone"

My code basically works, but i think, no i know this can be made easier:

```<code>final PackageManager pm = getPackageManager();
Intent main = new Intent(Intent.ACTION_MAIN, null);
main.addCategory(Intent.CATEGORY_LAUNCHER);
List<ResolveInfo> packages = pm.queryIntentActivities(main, 0);

for (ResolveInfo resolve_info : packages)
{
    try
    {
        //THIS TWO
        String package_name = resolve_info.activityInfo.packageName;
        String app_name = (String)pm.getApplicationLabel(pm.getApplicationInfo(package_name, PackageManager.GET_META_DATA));

        Log.i("TEST", "package = <" + package_name + "> name = <" + app_name + ">");
    }
    catch(Exception e)
    {
        //package not found -- should never happen
    }
}
</code>```

So is there a better / faster or easier way?

答:
```<code>    Intent startupIntent = new Intent(Intent.ACTION_MAIN);
    startupIntent.addCategory(Intent.CATEGORY_LAUNCHER);
    PackageManager pm =  this.getPackageManager();
    List<ResolveInfo> activities = pm.queryIntentActivities(startupIntent, 0);
    Log.i(TAG, "Found " + activities.size() + " activities.");
</code>```

<p>in MainActivity which extends AppCompatActivity or Activity </p>

ContentLicense: CC BY-SA 3.0

Body:
My Problem:

I want to get all installed applications with the Intent Category Category.LAUNCHER.
So I want to get all apps that would be displayed in the Launcher

and then resolve the package and the name.

Such as package: "com.android.phone" -- name: "Phone"

My code basically works, but i think, no i know this can be made easier:

final PackageManager pm = getPackageManager();
Intent main = new Intent(Intent.ACTION_MAIN, null);
main.addCategory(Intent.CATEGORY_LAUNCHER);
List<ResolveInfo> packages = pm.queryIntentActivities(main, 0);

for (ResolveInfo resolve_info : packages)
{
    try
    {
        //THIS TWO
        String package_name = resolve_info.activityInfo.packageName;
        String app_name = (String)pm.getApplicationLabel(pm.getApplicationInfo(package_name, PackageManager.GET_META_DATA));

        Log.i("TEST", "package = <" + package_name + "> name = <" + app_name + ">");
    }
    catch(Exception e)
    {
        //package not found -- should never happen
    }
}

So is there a better / faster or easier way?

Body:
    Intent startupIntent = new Intent(Intent.ACTION_MAIN);
    startupIntent.addCategory(Intent.CATEGORY_LAUNCHER);
    PackageManager pm =  this.getPackageManager();
    List<ResolveInfo> activities = pm.queryIntentActivities(startupIntent, 0);
    Log.i(TAG, "Found " + activities.size() + " activities.");

in MainActivity which extends AppCompatActivity or Activity


问:
NSTableViewDataSource dealloc in objective-c
I'm currently learning objective-c and I'm currently training with NSTableView.

Here is my problem :

I have linked my tableview to my controller through Interface Builder so that it has a datasource, I have implemented NSTableViewDataSource protocol in my controller and I have implemented both -(NSInteger) numberOfRowsInTableView: and -(id) tableView:objectValueForTableColumn:row: methods.

I have created a raw business class ("person") and I succeeded to display its content into my NSTableView.

But then, I put some NSLog in my dealloc methods to see whether the memory was freed or not and it seems that my array as well as my "person" instances are never released.

here is my dealloc code in the controller:

```<code>-(void)dealloc
{
    NSLog(@"the array is about to be deleted. current retain : %d",[personnes retainCount]);

    [personnes release];
    [super dealloc];
}
</code>```

and in my "person" class

```<code>-(void) dealloc
{
    NSLog(@"%@ is about to be deleted. current retain : %d",[self prenom],[self retainCount]);

    [self->nom release];
    [self->prenom release];

    [super dealloc];
}
</code>```

When these deallocs are supposed to be called in the application lifecycle? Because I expected them to be called at the window closure, but it didn't.

In the hope of beeing clear enough,

Thanks :)

KiTe.

答: Don't ever use retainCount. The results are misleading at best. If you practice proper memory management practices, you'll be fine. Have you had any memory issues/crashes?

Body:
I'm currently learning objective-c and I'm currently training with NSTableView.

Here is my problem :

I have linked my tableview to my controller through Interface Builder so that it has a datasource, I have implemented NSTableViewDataSource protocol in my controller and I have implemented both -(NSInteger) numberOfRowsInTableView: and -(id) tableView:objectValueForTableColumn:row: methods.

I have created a raw business class ("person") and I succeeded to display its content into my NSTableView.

But then, I put some NSLog in my dealloc methods to see whether the memory was freed or not and it seems that my array as well as my "person" instances are never released.

here is my dealloc code in the controller:

-(void)dealloc
{
    NSLog(@"the array is about to be deleted. current retain : %d",[personnes retainCount]);

    [personnes release];
    [super dealloc];
}

and in my "person" class

-(void) dealloc
{
    NSLog(@"%@ is about to be deleted. current retain : %d",[self prenom],[self retainCount]);

    [self->nom release];
    [self->prenom release];

    [super dealloc];
}

When these deallocs are supposed to be called in the application lifecycle? Because I expected them to be called at the window closure, but it didn't.

In the hope of beeing clear enough,

Thanks :)

KiTe.


问:
Deadlock when using fork, exec and pipes in a parallel environment
I am spawning a child process using fork and exec. Using two pipes to provide input and receive output from that process.

It works just fine most of the time but when I use something like openmp to test how it performs in concurrent environments it hangs in the read syscall, or waitpid sometimes.

When I straceed the child process I found that it is also blocked on the read syscall. Which is weird because I only wait on reading in the parent process after I've provided all of my input and closed the write end of the pipe.

I tried to create an MVCE but it is sort of long. I don't know how to make it any shorter. I removed most of the error checking code for the sake of simplicity.

Note that there are no globals in my code. And I'm not trying to read/write from the same file descriptors in multiple threads.

I can't think of what could go wrong. So hopefully you guys can spot what I'm doing wrong.

There goes:

```<code>#include <string.h>
#include <assert.h>
#include <unistd.h>
#include <limits.h>
#include <sys/wait.h>
#include <stdio.h>
#include <stdlib.h>

size_t
min(size_t first, size_t second)
{
    if(first < second)
    {
        return first;
    }

    return second;
}

struct RDI_Buffer
{
    char* data;
    size_t size;
};

typedef struct RDI_Buffer RDI_Buffer;

RDI_Buffer
rdi_buffer_init()
{
    RDI_Buffer b = {0};
    return b;
}

RDI_Buffer
rdi_buffer_new(size_t size)
{
    RDI_Buffer b;

    b.data = malloc(size);
    b.size = size;
    return b;
}

void
rdi_buffer_free(RDI_Buffer b)
{
    if(!b.data)
    {
        return;
    }

    free(b.data);
}

RDI_Buffer
rdi_buffer_resize(RDI_Buffer b, size_t new_size)
{
    if(!b.data)
    {
        return rdi_buffer_new(new_size);
    }

    char* new_data = realloc(b.data, new_size);

    if(new_data)
    {
        b.size = new_size;
        b.data = new_data;
        return b;
    }

    RDI_Buffer output = rdi_buffer_new(new_size);
    memcpy(output.data, b.data, output.size);
    rdi_buffer_free(b);
    return output;
}

RDI_Buffer
rdi_buffer_null_terminate(RDI_Buffer b)
{
    b = rdi_buffer_resize(b, b.size + 1);
    b.data[b.size - 1] = '\0';
    return b;
}

static RDI_Buffer
rw_from_fd(int w_fd, int r_fd, RDI_Buffer input)
{
    const size_t CHUNK_SIZE = 4096;

    assert(input.size <= CHUNK_SIZE);

    write(w_fd, input.data, input.size);
    close(w_fd);

    RDI_Buffer output = rdi_buffer_new(CHUNK_SIZE);

    read(r_fd, output.data, CHUNK_SIZE);

    close(r_fd);
    return output;
}

int main()
{
#pragma omp parallel for
    for(size_t i = 0; i < 100; i++)
    {
        char* thing =
                "Hello this is a sort of long text so that we can test how "
                "well this works. It should go with cat and be printed.";

        RDI_Buffer input_buffer;
        input_buffer.data = thing;
        input_buffer.size = strlen(thing);

        int main_to_sub[2];
        int sub_to_main[2];

        pipe(main_to_sub);
        pipe(sub_to_main);

        int pid = fork();

        if(pid == 0)
        {
            dup2(main_to_sub[0], STDIN_FILENO);
            dup2(sub_to_main[1], STDOUT_FILENO);

            close(main_to_sub[1]);
            close(main_to_sub[0]);
            close(sub_to_main[1]);
            close(sub_to_main[0]);

            char* argv[] = {"cat", NULL};

            execvp("cat", argv);
            exit(1);
        }

        close(main_to_sub[0]);
        close(sub_to_main[1]);

        RDI_Buffer output =
                rw_from_fd(main_to_sub[1], sub_to_main[0], input_buffer);

        int *status = NULL;
        waitpid(pid, status, 0);

        if(status)
        {
            printf("%d\n", *status);
        }

        output = rdi_buffer_null_terminate(output);

        if(strcmp(output.data, thing) == 0)
        {
            printf("good\n");
        }
        else
        {
            printf("bad\n");
        }

        rdi_buffer_free(output);
    }
}
</code>```

Make sure you compile and link with -fopenmp. Like so: gcc main.c -fopenmp

答:
The reason for the issue turns out to be the open files that are inherited to the child processes as Jonathan Leffler and Mevet explained in their answers. Please read their answers if you have that problem then refer back to my answer if you still do not understand or don't know what to do.

I will share my explanation in a way I would have understood right away. Also share my code solution to the issue.

Consider the following scenario:
Process A opens a pipe (which is two files).

Process A spawns Process B to communicate to it with the pipe. However it also creates Process C which inherits the pipe (the two files).

Now process B will continuously call read(2) on the pipe which is a blocking system call. (It will wait until someone writes to the pipe)

Process A finishes writing and closes their end of the pipe. Normally that would cause the read(2) syscall in Process B to fail and the program would exit (That's what we want).

However in our case since Process C does have an open write end of the pipe, the read(2) syscall in Process B will not fail and will block waiting for a write from the open write-end in Process C.

Things will be okay when Process C just ends.

The real deadlock would arrise in a different scenario where both B and C hold pipes for one another (As explained in Mevet's answer). Each one of them will be waiting for the other to close their ends of the pipes. Which will never happen causing the deadlock.

My solution was to close all open files I don't need right after the fork(2)

```<code>int pid = fork();

if(pid == 0)
{
    int exceptions[2] = {main_to_sub[0], sub_to_main[1]};
    close_all_descriptors(exceptions);
    dup2(main_to_sub[0], STDIN_FILENO);
    dup2(sub_to_main[1], STDOUT_FILENO);

    close(main_to_sub[0]);
    close(sub_to_main[1]);

    char* argv[] = {"cat", NULL};

    execvp("cat", argv);
    exit(1);
}
</code>```

Here's the implementation of close_all_descriptors

```<code>#include <fcntl.h>
#include <errno.h>

static int
is_within(int fd, int arr[2])
{
    for(int i = 0; i < 2; i++)
    {
        if(fd == arr[i])
        {
            return 1;
        }
    }

    return 0;
}

static int
fd_is_valid(int fd)
{
    return fcntl(fd, F_GETFD) != -1 || errno != EBADF;
}

static void
close_all_descriptors(int exceptions[2])
{
    // getdtablesize returns the max number of files that can be open. It's 1024 on my system
    const int max_fd = getdtablesize();

    // starting at 3 because I don't want to close stdin/out/err
    // let dup2(2) do that
    for (int fd = 3; fd <= max_fd; fd++)
    {
        if(fd_is_valid(fd) && !is_within(fd, exceptions))
        {
            close(fd);
        }
    }
}
</code>```

Body:
I am spawning a child process using fork and exec. Using two pipes to provide input and receive output from that process.

It works just fine most of the time but when I use something like openmp to test how it performs in concurrent environments it hangs in the read syscall, or waitpid sometimes.

When I straceed the child process I found that it is also blocked on the read syscall. Which is weird because I only wait on reading in the parent process after I've provided all of my input and closed the write end of the pipe.

I tried to create an MVCE but it is sort of long. I don't know how to make it any shorter. I removed most of the error checking code for the sake of simplicity.

Note that there are no globals in my code. And I'm not trying to read/write from the same file descriptors in multiple threads.

I can't think of what could go wrong. So hopefully you guys can spot what I'm doing wrong.

There goes:

#include <string.h>
#include <assert.h>
#include <unistd.h>
#include <limits.h>
#include <sys/wait.h>
#include <stdio.h>
#include <stdlib.h>

size_t
min(size_t first, size_t second)
{
    if(first < second)
    {
        return first;
    }

    return second;
}

struct RDI_Buffer
{
    char* data;
    size_t size;
};

typedef struct RDI_Buffer RDI_Buffer;

RDI_Buffer
rdi_buffer_init()
{
    RDI_Buffer b = {0};
    return b;
}

RDI_Buffer
rdi_buffer_new(size_t size)
{
    RDI_Buffer b;

    b.data = malloc(size);
    b.size = size;
    return b;
}

void
rdi_buffer_free(RDI_Buffer b)
{
    if(!b.data)
    {
        return;
    }

    free(b.data);
}

RDI_Buffer
rdi_buffer_resize(RDI_Buffer b, size_t new_size)
{
    if(!b.data)
    {
        return rdi_buffer_new(new_size);
    }

    char* new_data = realloc(b.data, new_size);

    if(new_data)
    {
        b.size = new_size;
        b.data = new_data;
        return b;
    }

    RDI_Buffer output = rdi_buffer_new(new_size);
    memcpy(output.data, b.data, output.size);
    rdi_buffer_free(b);
    return output;
}

RDI_Buffer
rdi_buffer_null_terminate(RDI_Buffer b)
{
    b = rdi_buffer_resize(b, b.size + 1);
    b.data[b.size - 1] = '\0';
    return b;
}

static RDI_Buffer
rw_from_fd(int w_fd, int r_fd, RDI_Buffer input)
{
    const size_t CHUNK_SIZE = 4096;

    assert(input.size <= CHUNK_SIZE);

    write(w_fd, input.data, input.size);
    close(w_fd);

    RDI_Buffer output = rdi_buffer_new(CHUNK_SIZE);

    read(r_fd, output.data, CHUNK_SIZE);

    close(r_fd);
    return output;
}

int main()
{
#pragma omp parallel for
    for(size_t i = 0; i < 100; i++)
    {
        char* thing =
                "Hello this is a sort of long text so that we can test how "
                "well this works. It should go with cat and be printed.";

        RDI_Buffer input_buffer;
        input_buffer.data = thing;
        input_buffer.size = strlen(thing);

        int main_to_sub[2];
        int sub_to_main[2];

        pipe(main_to_sub);
        pipe(sub_to_main);

        int pid = fork();

        if(pid == 0)
        {
            dup2(main_to_sub[0], STDIN_FILENO);
            dup2(sub_to_main[1], STDOUT_FILENO);

            close(main_to_sub[1]);
            close(main_to_sub[0]);
            close(sub_to_main[1]);
            close(sub_to_main[0]);

            char* argv[] = {"cat", NULL};

            execvp("cat", argv);
            exit(1);
        }

        close(main_to_sub[0]);
        close(sub_to_main[1]);

        RDI_Buffer output =
                rw_from_fd(main_to_sub[1], sub_to_main[0], input_buffer);

        int *status = NULL;
        waitpid(pid, status, 0);

        if(status)
        {
            printf("%d\n", *status);
        }

        output = rdi_buffer_null_terminate(output);

        if(strcmp(output.data, thing) == 0)
        {
            printf("good\n");
        }
        else
        {
            printf("bad\n");
        }

        rdi_buffer_free(output);
    }
}

Make sure you compile and link with -fopenmp. Like so: gcc main.c -fopenmp

Body:
The reason for the issue turns out to be the open files that are inherited to the child processes as Jonathan Leffler and Mevet explained in their answers. Please read their answers if you have that problem then refer back to my answer if you still do not understand or don't know what to do.

I will share my explanation in a way I would have understood right away. Also share my code solution to the issue.

Consider the following scenario:
Process A opens a pipe (which is two files).

Process A spawns Process B to communicate to it with the pipe. However it also creates Process C which inherits the pipe (the two files).

Now process B will continuously call read(2) on the pipe which is a blocking system call. (It will wait until someone writes to the pipe)

Process A finishes writing and closes their end of the pipe. Normally that would cause the read(2) syscall in Process B to fail and the program would exit (That's what we want).

However in our case since Process C does have an open write end of the pipe, the read(2) syscall in Process B will not fail and will block waiting for a write from the open write-end in Process C.

Things will be okay when Process C just ends.

The real deadlock would arrise in a different scenario where both B and C hold pipes for one another (As explained in Mevet's answer). Each one of them will be waiting for the other to close their ends of the pipes. Which will never happen causing the deadlock.

My solution was to close all open files I don't need right after the fork(2)

int pid = fork();

if(pid == 0)
{
    int exceptions[2] = {main_to_sub[0], sub_to_main[1]};
    close_all_descriptors(exceptions);
    dup2(main_to_sub[0], STDIN_FILENO);
    dup2(sub_to_main[1], STDOUT_FILENO);

    close(main_to_sub[0]);
    close(sub_to_main[1]);

    char* argv[] = {"cat", NULL};

    execvp("cat", argv);
    exit(1);
}

Here's the implementation of close_all_descriptors

#include <fcntl.h>
#include <errno.h>

static int
is_within(int fd, int arr[2])
{
    for(int i = 0; i < 2; i++)
    {
        if(fd == arr[i])
        {
            return 1;
        }
    }

    return 0;
}

static int
fd_is_valid(int fd)
{
    return fcntl(fd, F_GETFD) != -1 || errno != EBADF;
}

static void
close_all_descriptors(int exceptions[2])
{
    // getdtablesize returns the max number of files that can be open. It's 1024 on my system
    const int max_fd = getdtablesize();

    // starting at 3 because I don't want to close stdin/out/err
    // let dup2(2) do that
    for (int fd = 3; fd <= max_fd; fd++)
    {
        if(fd_is_valid(fd) && !is_within(fd, exceptions))
        {
            close(fd);
        }
    }
}


问:
In Corda, am I able to include a party as an optional signer?
Is optional transaction signing is possible? I know that the command explicitly outlines the required signers (parties that must sign a transaction for it to be notarised), but can a party have the option to sign? How could we implement this?

The use-case is an anti-money laundering entity that will sign depending on the level of trust of the transacting party.

```<code>val requiredSigners = listOf<PublicKey>()
val optionalSigners = listOf<PublicKey>()

tb.addCommand(TestContract.Commands.TestCommand(), requiredSigners, optionalSigners)
</code>```

The code above isn't possible using the existing SDK.

答:
If the level of trust is known when creating a transaction, we can explicitly indicate the required signers when adding the TestCommand() to the transaction builder.

Once a transaction is signed, it becomes immutable. Which means we are unable to add additional required signers.

To enable optional signers, we would adjust the contents of the required signers based on the inputs to and outputs from the transaction builder (or any other business logic). Then in the Corda contract - we could verify the number of required signatures based on that same business logic.

In short: Commands define who needs to sign, but the check logic in your verify method in your Corda contract is what looks at the states included and decides which public keys should be in the commands. It is therefore perfectly possible to have conditional logic surrounding required signers.

Body:
Is optional transaction signing is possible? I know that the command explicitly outlines the required signers (parties that must sign a transaction for it to be notarised), but can a party have the option to sign? How could we implement this?

The use-case is an anti-money laundering entity that will sign depending on the level of trust of the transacting party.

val requiredSigners = listOf<PublicKey>()
val optionalSigners = listOf<PublicKey>()

tb.addCommand(TestContract.Commands.TestCommand(), requiredSigners, optionalSigners)

The code above isn't possible using the existing SDK.


问:
select distinct window function in PostgreSQL
I have a data table like the following:

How do you select the distinct continuities for each Longitudinal distance in a 10-step rolling window?

Ideally, I'd want something in the Ideal output column in which the results are arrays.

Thanks.

答:
One method uses arrays:

```<code>select t.*,
       (select count(distinct c) from unnest(ar) c) as num_distinct
from (select t.*,
             array_agg(continuity) over (order by distance rows between 9 preceding and current row) ar
      from t
     ) t;
</code>```

EDIT:

Or, if you want the values, aggregate them:

```<code>select t.*,
       (select array_agg(distinct c) from unnest(ar) c) as num_distinct
from (select t.*,
             array_agg(continuity) over (order by distance rows between 9 preceding and current row) ar
      from t
     ) t;
</code>```

Body:
One method uses arrays:

select t.*,
       (select count(distinct c) from unnest(ar) c) as num_distinct
from (select t.*,
             array_agg(continuity) over (order by distance rows between 9 preceding and current row) ar
      from t
     ) t;

EDIT:

Or, if you want the values, aggregate them:

select t.*,
       (select array_agg(distinct c) from unnest(ar) c) as num_distinct
from (select t.*,
             array_agg(continuity) over (order by distance rows between 9 preceding and current row) ar
      from t
     ) t;


问:
Correct use of Angular TypeScript constructor - Can't resolve all parameters for
```<code>@Injectable({
  providedIn: 'root',
  /* useFactory: () => new MyService(MyAnotherService.myInteger) */
})
export class MyService{

  constructor(private someInteger?: number) {
    // doThings with someInteger
  }
</code>```

--aot build will complain that Warning: Can't resolve all parameters for...

In fact, due to the useFactory: () part, we are still able to run our application correctly. But I do not know if this is the right way or not

We can, of course, change the constructor to

```<code>constructor(private anotherService?: MyAnotherService) {
        // doThings with anotherService
      }
</code>```

To shut the build warning,
However I do not think that MyService should be dependent on MyAnotherService when It can be just dependent on a primitive, It's also bad for code reusability

What is the best thing to do in this situation?

My idea is that. Angular use Decorator to instruct that it is an Angular component ( am I right? )
So the main logic body should not always be coupled to Angular DI logic, especially if It is a service. We can pull the service code and plug it elsewhere ( Using DI or not )

答:
You can use an optional injection token

token.ts

```<code>import {InjectionToken} from '@angular/core';

export const INTEGER_TOKEN= new InjectionToken<number>('IntegerToken');
</code>```

app.module.ts

```<code>import {INTEGER_TOKEN} from './token';

@NgModule({
//...
  ],
  providers: [
    {
      provide: INTEGER_TOKEN,
      useValue: 1 //Or factory if you need one
    },
</code>```

service.ts

```<code>import {INTEGER_TOKEN} from './token';

import {Injectable, Inject, Optional} from '@angular/core';

@Injectable({
  providedIn: 'root',

})
export class MyService{

  constructor(@Optional() @Inject(INTEGER_TOKEN) private someInteger?: number) {
    // doThings with someInteger
  }
</code>```

Body:
@Injectable({
  providedIn: 'root',
  /* useFactory: () => new MyService(MyAnotherService.myInteger) */
})
export class MyService{

  constructor(private someInteger?: number) {
    // doThings with someInteger
  }

--aot build will complain that Warning: Can't resolve all parameters for...

In fact, due to the useFactory: () part, we are still able to run our application correctly. But I do not know if this is the right way or not

We can, of course, change the constructor to

constructor(private anotherService?: MyAnotherService) {
        // doThings with anotherService
      }

To shut the build warning,
However I do not think that MyService should be dependent on MyAnotherService when It can be just dependent on a primitive, It's also bad for code reusability

What is the best thing to do in this situation?

My idea is that. Angular use Decorator to instruct that it is an Angular component ( am I right? )
So the main logic body should not always be coupled to Angular DI logic, especially if It is a service. We can pull the service code and plug it elsewhere ( Using DI or not )

Body:
You can use an optional injection token

token.ts

import {InjectionToken} from '@angular/core';

export const INTEGER_TOKEN= new InjectionToken<number>('IntegerToken');

app.module.ts

import {INTEGER_TOKEN} from './token';

@NgModule({
//...
  ],
  providers: [
    {
      provide: INTEGER_TOKEN,
      useValue: 1 //Or factory if you need one
    },

service.ts

import {INTEGER_TOKEN} from './token';

import {Injectable, Inject, Optional} from '@angular/core';

@Injectable({
  providedIn: 'root',

})
export class MyService{

  constructor(@Optional() @Inject(INTEGER_TOKEN) private someInteger?: number) {
    // doThings with someInteger
  }


问:
How to stop service account credential printing on Jenkins console?
I have Jenkins server which builds project, build docker image and push it to Google Container Registry.

I am using service account to login to GCR using command docker login -u _json_key -p "$(cat ${service-account.json})" https://gcr.io as stated here

When this line is executed, content of service-account.json file is printed on console in Jenkins pipeline.

How would I stop credential being printed on console?

答:
If you store the JSON Key File and the username associated with it as a credential in Jenkins, you can then reference the key file inside of a withCredentials step (https://jenkins.io/doc/pipeline/steps/credentials-binding/)

Any credentials used inside of a withCredentials step will be masked automatically


问:
Date before the current date should not be selected
I am trying to show a popup message when any date before current day is clicked in the calendar? This is the function that has the date the user has clicked. setCalendarControlDate(year, month, day) has the date what user has selected. How can i compare the dates and show the error. </p>

```<code>function setCalendarControlDate(year, month, day) {

calendarControl.setDate(year, month, day);
var arguments = "changeDataGrid;day=" + day;
arguments += ";month=" + month;
arguments += ";year=" + year;
//  arguments += ";rowId=" + rowId;
arguments += ";" + rowId;
arguments += ";";

document.getElementById(targetControlId.id).value = arguments;
document.forms[0].submit();
</code>```

<p>}</p>

答:
```<code>var selectedDate = new Date(year, month-1, day);

var valid = selectedDate >= new Date();

// If today was 12th April 2011
year = 2011;
month = 4;
day = 13;

var selectedDate = new Date(year, month-1, day);

var valid = selectedDate >= new Date();

valid ; //# => true

year = 2011;
month = 4;
day = 11;

selectedDate = new Date(year, month-1, day);

var valid = selectedDate >= new Date();

valid ; //# => false
</code>```

Body:
I am trying to show a popup message when any date before current day is clicked in the calendar? This is the function that has the date the user has clicked. setCalendarControlDate(year, month, day) has the date what user has selected. How can i compare the dates and show the error.

function setCalendarControlDate(year, month, day) {

calendarControl.setDate(year, month, day);
var arguments = "changeDataGrid;day=" + day;
arguments += ";month=" + month;
arguments += ";year=" + year;
//  arguments += ";rowId=" + rowId;
arguments += ";" + rowId;
arguments += ";";

document.getElementById(targetControlId.id).value = arguments;
document.forms[0].submit();

}


问:
php - The best solution for storing access_token
I'm creating a little application where users must authenticate with Facebook (I'm using OAuth2). At this moment all of this procedure looks like this:

The user will click on "FB Login" button.

The user will be asked to login to Facebook and allow permission. If user allows it will return an authorization code.

Now we will use the authorization code to get an access token.

We can store the access token in session to start a user session.

Now we can use the access token to access to different user resources.

And here I have question - Is it best (ie the safest way) to store the access_token for later use? "Hide" it simply to the database and take it when it's needed or can it develop some original hash algorithm to secure it?

答:
Access tokens have a short lifetime - if you have a server side session, it's easier to keep an access token there (not in a database), so that all requests can use it. It's better to reuse a single access token than getting a new one for each request by using a refresh token. One reason is performance. Getting a new access token may revoke older access tokens created by the same refresh token, but it depends on your OAuth2 server implementation.

The backend session should be safe enough, as long as your server doesn't keep its session data in a place that anyone can read (some temporary disk storage).

When keeping access tokens in a session, you will probably have to find a way how to refresh them - you should prevent multiple requests (threads) from refreshing a single access token.


问:
Find separated maxima in numpy
Given a numpy ndarray, how to find all peaks, but each peak should be at least some fixed distance away from the others. The reason is that often there is one peak, and then the next many peaks are just the neighboring points of the first peak, so they are not any "new" peaks.

I am sure there must be better implementations, and that is what I am asking for.
To illustrate what I would like, here is an example of something I just threw together (I have edited to improved the example, and added an image at the end):

import numpy as np
from matplotlib import pyplot as plt
import scipy as sp
import scipy.ndimage
def findpeaks(objective, maxpeaks=10, gap=0):
    obj = objective.copy()
    peaks = np.zeros([maxpeaks, 2], dtype=int)
    for n in range(maxpeaks):
        peak = np.unravel_index(np.argmax(obj, axis=None), obj.shape)
        peaks[n,:] = peak
        obj[np.maximum(0,peak[0]-gap):np.minimum(40,peak[0]+gap),
            np.maximum(0,peak[1]-gap):np.minimum(40,peak[1]+gap)] = 0
    return peaks

np.random.seed(12345)
x = np.random.uniform(0, 1, [40,40])
x = sp.ndimage.filters.gaussian_filter(x, [3,3], mode='constant')

peaks = findpeaks(x, 10, 5)
print(peaks.T)
print(x[peaks[:,0],peaks[:,1]])
print(np.array(np.unravel_index(x.flatten().argsort()[-10:][::-1],x.shape)))
for p in peaks:
    x[p[0]-1:p[0]+2,p[1]]=0
    x[p[0],p[1]-1:p[1]+2]=0
plt.imshow(x, cmap='gray')
```

<p>The output is</p>

<pre class="lang-py prettyprint-override"><code>[[27 11 26 24  5 16 25 16 13 31]
 [11 14 16  5 13 34 21 14  8 16]]
[0.55472915 0.54961331 0.53829221 0.5353206  0.53512158 0.52064241
 0.51729225 0.51557288 0.51025817 0.50846277]
[[27 27 26 28 27 28 26 27 26 11]
 [11 12 11 11 10 12 12 13 10 14]]
</code>```

It takes a random x array, and applies a gaussian filter just to make it smooth. This smoothness is why once you find one maximum, the others will usually be its neighbors.

The findpeaks method finds separated peaks. The locations of those peaks is the first two lines in the output.

The middle two lines are the values of the 10 maxima.

The last two lines are the locations of the largest points using a cutoff. As is visible, you just get a number of points which are contiguous.

答:
scipy.signal.find_peaks is exactly what you're looking for :-)

from the docs, the distance keyword does exactly that - requires the peak to be higher than all points up to a distance of distance from it.

Title: Find separated maxima in numpy

Body:
Given a numpy ndarray, how to find all peaks, but each peak should be at least some fixed distance away from the others. The reason is that often there is one peak, and then the next many peaks are just the neighboring points of the first peak, so they are not any "new" peaks.

I am sure there must be better implementations, and that is what I am asking for.
To illustrate what I would like, here is an example of something I just threw together (I have edited to improved the example, and added an image at the end):

import numpy as np
from matplotlib import pyplot as plt
import scipy as sp
import scipy.ndimage
def findpeaks(objective, maxpeaks=10, gap=0):
    obj = objective.copy()
    peaks = np.zeros([maxpeaks, 2], dtype=int)
    for n in range(maxpeaks):
        peak = np.unravel_index(np.argmax(obj, axis=None), obj.shape)
        peaks[n,:] = peak
        obj[np.maximum(0,peak[0]-gap):np.minimum(40,peak[0]+gap),
            np.maximum(0,peak[1]-gap):np.minimum(40,peak[1]+gap)] = 0
    return peaks

np.random.seed(12345)
x = np.random.uniform(0, 1, [40,40])
x = sp.ndimage.filters.gaussian_filter(x, [3,3], mode='constant')

peaks = findpeaks(x, 10, 5)
print(peaks.T)
print(x[peaks[:,0],peaks[:,1]])
print(np.array(np.unravel_index(x.flatten().argsort()[-10:][::-1],x.shape)))
for p in peaks:
    x[p[0]-1:p[0]+2,p[1]]=0
    x[p[0],p[1]-1:p[1]+2]=0
plt.imshow(x, cmap='gray')

The output is

[[27 11 26 24  5 16 25 16 13 31]
 [11 14 16  5 13 34 21 14  8 16]]
[0.55472915 0.54961331 0.53829221 0.5353206  0.53512158 0.52064241
 0.51729225 0.51557288 0.51025817 0.50846277]
[[27 27 26 28 27 28 26 27 26 11]
 [11 12 11 11 10 12 12 13 10 14]]

It takes a random x array, and applies a gaussian filter just to make it smooth. This smoothness is why once you find one maximum, the others will usually be its neighbors.

The findpeaks method finds separated peaks. The locations of those peaks is the first two lines in the output.

The middle two lines are the values of the 10 maxima.

The last two lines are the locations of the largest points using a cutoff. As is visible, you just get a number of points which are contiguous.


问:
Find min and max of every column for every label in pandas.DataFrame
I have a DataFrame called df and it has 4 columns, like the one presented below:

```<code>A      B      C      Class
12     13     22     1
8      15     20     1
9      14     25     1
18     9      35     2
5      14     30     2
4      12     28     2
35     87     67     3
35     82     66     3
20     7      32     4
10     8      32     4
22     7      31     4
...    ...    ...    ...
</code>```

What I want is to find the min and max for every column with respect to the class.
In other words, I would like to get a result similar to the one below:

```<code>Class: 1
A: [8, 12]
B: [13, 15]
C: [20, 25]

Class: 2
A: [4, 18]
B: [9, 14]
C: [28, 35]

Class: 3
A: [35, 35]
B: [82, 87]
C: [66, 67]
...
</code>```

Body:
I have a DataFrame called df and it has 4 columns, like the one presented below:

A      B      C      Class
12     13     22     1
8      15     20     1
9      14     25     1
18     9      35     2
5      14     30     2
4      12     28     2
35     87     67     3
35     82     66     3
20     7      32     4
10     8      32     4
22     7      31     4
...    ...    ...    ...

What I want is to find the min and max for every column with respect to the class.
In other words, I would like to get a result similar to the one below:

Class: 1
A: [8, 12]
B: [13, 15]
C: [20, 25]

Class: 2
A: [4, 18]
B: [9, 14]
C: [28, 35]

Class: 3
A: [35, 35]
B: [82, 87]
C: [66, 67]
...

Body: df.groupby(df['Class']).agg(['min', 'max'])


问:
How do you select an element that is the child of another element in Appium using wd and javascript?
I have a React-Native application on Android with multiple Pickers (dropdowns) on a screen. I have an accessibilityLabel assigned to the first picker component called "Pick Description" and I have an accessibilityLabel assigned to the second picker component called "Pick Price". I am able to find the second picker component on the page using driver.waitForElementByAccessibilityId( "Pick Price" ) but I need to be able to find out what the text value is in the picker, i.e. the value that has been selected. I am using Jest, JavaScript, and wd.

I have opened the app with Appium Desktop and selected the picker and I can see that it has a child component that is an android.widget.TextView with a resource-id of "android:id/text1".

If I use driver.waitForElementByAccessibilityId( "Pick Price" ).waitForElementById( "android:id/text1" ).text(); it gets the text from the first picker, i.e. "Pick Description". It looks like it is just returning the first element with the ID of "text1" on the page.

How do I go about getting the text value from a React-Native Picker component when I have an accessibilityLabel assigned to the Picker component but no accessibilityLabel assigned to the TextView?

答:
You can consider moving to XPath selector and amend your code to look like:

```<code>driver.waitForElementByAccessibilityId( "Pick Price" ).waitForElementByXPath( ".//[@android:id/text1]" ).text();
</code>```

this way the second web element search scope will be relative to the first element.

References:

XPath Syntax -> Selecting Nodes

XPath Operators & Functions

Body:
You can consider moving to XPath selector and amend your code to look like:

driver.waitForElementByAccessibilityId( "Pick Price" ).waitForElementByXPath( ".//[@android:id/text1]" ).text();

this way the second web element search scope will be relative to the first element.

References:

XPath Syntax -> Selecting Nodes

XPath Operators & Functions


问:
Python to delete a row in excel spreadsheet
I have a really large excel file and i need to delete about 20,000 rows, contingent on meeting a simple condition and excel won't let me delete such a complex range when using a filter. The condition is:

If the first column contains the value, X, then I need to be able to delete the entire row.

I'm trying to automate this using python and xlwt, but am not quite sure where to start. Seeking some code snippits to get me started...
Grateful for any help that's out there!

答:
I like using COM objects for this kind of fun:

```<code>import win32com.client
from win32com.client import constants

f = r"h:\Python\Examples\test.xls"
DELETE_THIS = "X"

exc = win32com.client.gencache.EnsureDispatch("Excel.Application")
exc.Visible = 1
exc.Workbooks.Open(Filename=f)

row = 1
while True:
    exc.Range("B%d" % row).Select()
    data = exc.ActiveCell.FormulaR1C1
    exc.Range("A%d" % row).Select()
    condition = exc.ActiveCell.FormulaR1C1

    if data == '':
        break
    elif condition == DELETE_THIS:
        exc.Rows("%d:%d" % (row, row)).Select()
        exc.Selection.Delete(Shift=constants.xlUp)
    else:
        row += 1

# Before
#
#      a
#      b
# X    c
#      d
#      e
# X    d
#      g
#

# After
#
#      a
#      b
#      d
#      e
#      g
</code>```

I usually record snippets of Excel macros and glue them together with Python as I dislike Visual Basic :-D.

Body:
I like using COM objects for this kind of fun:

import win32com.client
from win32com.client import constants

f = r"h:\Python\Examples\test.xls"
DELETE_THIS = "X"

exc = win32com.client.gencache.EnsureDispatch("Excel.Application")
exc.Visible = 1
exc.Workbooks.Open(Filename=f)

row = 1
while True:
    exc.Range("B%d" % row).Select()
    data = exc.ActiveCell.FormulaR1C1
    exc.Range("A%d" % row).Select()
    condition = exc.ActiveCell.FormulaR1C1

    if data == '':
        break
    elif condition == DELETE_THIS:
        exc.Rows("%d:%d" % (row, row)).Select()
        exc.Selection.Delete(Shift=constants.xlUp)
    else:
        row += 1

# Before
#
#      a
#      b
# X    c
#      d
#      e
# X    d
#      g
#

# After
#
#      a
#      b
#      d
#      e
#      g

I usually record snippets of Excel macros and glue them together with Python as I dislike Visual Basic :-D.


问:
System.Web.Mvc.HtmlHelper' does not contain a definition for 'ActionLink'
I would like to use custom @Html.ActionLink

I am trying to use the following code:-

```<code>public static class LinkExtensions
{
    public static MvcHtmlString MyActionLink(
        this HtmlHelper htmlHelper,
        string linkText,
        string action,
        string controller)
    {
        var currentAction = htmlHelper.ViewContext.RouteData.GetRequiredString("action");
        var currentController = mlHelper.ViewContext.RouteData.GetRequiredString("controller");

        if (action == currentAction && controller == currentController)
        {
          var anchor = new TagBuilder("a");
          anchor.Attributes["href"] = "#";
          anchor.AddCssClass("currentPageCSS");
          anchor.SetInnerText(linkText);
          return MvcHtmlString.Create(anchor.ToString());
         }

         return htmlHelper.ActionLink(linkText, action, controller);
    }
}
</code>```

From Custom ActionLink helper that knows what page you're on

But I am getting

System.Web.Mvc.HtmlHelper' does not contain a definition for
  'ActionLink' and no extension method 'ActionLink' accepting a first
  argument of type 'System.Web.Mvc.HtmlHelper' could be found (are you
  missing a using directive or an assembly reference?

答: My issue was, I had incomplete syntax, "@Html.actionLink", in a view. Looks like I had started to add an action link and went a different direction but forgot to remove the partial action link, this caused the same error as above.... So check your syntax as that will throw the same runtime error. Good luck!</p>

Body:
I would like to use custom @Html.ActionLink

I am trying to use the following code:-

public static class LinkExtensions
{
    public static MvcHtmlString MyActionLink(
        this HtmlHelper htmlHelper,
        string linkText,
        string action,
        string controller)
    {
        var currentAction = htmlHelper.ViewContext.RouteData.GetRequiredString("action");
        var currentController = mlHelper.ViewContext.RouteData.GetRequiredString("controller");

        if (action == currentAction && controller == currentController)
        {
          var anchor = new TagBuilder("a");
          anchor.Attributes["href"] = "#";
          anchor.AddCssClass("currentPageCSS");
          anchor.SetInnerText(linkText);
          return MvcHtmlString.Create(anchor.ToString());
         }

         return htmlHelper.ActionLink(linkText, action, controller);
    }
}

From Custom ActionLink helper that knows what page you're on

But I am getting

System.Web.Mvc.HtmlHelper' does not contain a definition for
  'ActionLink' and no extension method 'ActionLink' accepting a first
  argument of type 'System.Web.Mvc.HtmlHelper' could be found (are you
  missing a using directive or an assembly reference?


问:
Is there a better/cleaner way to format this script
I'm pretty new to python (made 4 scripts to cleanup folders and move some stuff around) and now i have a new challenge.

I have this csv file with data about some pdf's (name of the pdf, times it needs to be printed, etc.)

I have the following code atm and everything works fine, but i find it "unclean" and i hope there was a better way to format some things.
This is my code:

```<code>import csv, shutil, os

with open('\\\\fatboy.leleu.be\\Psm\\Mailing Templates\\OhGreen\\Verwerking\\OhGreen POS week 19.csv') as csvfile:
    csvReader = csv.reader(csvfile, delimiter=';')
    rows = csv.reader(csvfile)
    headers = next(rows)

    for row in csvReader:
        NAAM = row[0]
        Formaat = row[1]
        Tekstkleur = row[2]
        Gouvy = row[3]
        LouvainLaNeuve = row[4]
        Nannine = row[5]
        Tournai = row[6]
        SaintGeorges = row[7]
        Aarschot = row[8]
        Balen = row[9]
        Ekeren = row[10]
        Ninove = row[11]
        SintKatelijneWaver = row[12]
        Olen = row[13]
        Zwijndrecht = row[14]

        if Gouvy > "0":
            file = NAAM + ".pdf"
            path = ("\\\\fatboy.leleu.be\\Psm\\Mailing Templates\\OhGreen\\Verwerking\\" + Formaat + "\\" + file)
            dest = ("\\\\fatboy.leleu.be\\Psm\\Mailing Templates\\OhGreen\\Verwerking\\2print\\" + Formaat + "\\" + NAAM + "_GOUVY" + ".pdf")
            shutil.copy(path, dest)
            print(dest , " changed name and moved to ", dest)

        if LouvainLaNeuve > "0":
            file = NAAM + ".pdf"
            path = ("\\\\fatboy.leleu.be\\Psm\\Mailing Templates\\OhGreen\\Verwerking\\" + Formaat + "\\" + file)
            dest = ("\\\\fatboy.leleu.be\\Psm\\Mailing Templates\\OhGreen\\Verwerking\\2print\\" + Formaat + "\\" + NAAM + "_LOUVAIN_LA_NEUVE" + ".pdf")
            shutil.copy(path, dest)
            print(dest , " changed name and moved to ", dest)

        if Nannine > "0":
            file = NAAM + ".pdf"
            path = ("\\\\fatboy.leleu.be\\Psm\\Mailing Templates\\OhGreen\\Verwerking\\" + Formaat + "\\" + file)
            dest = ("\\\\fatboy.leleu.be\\Psm\\Mailing Templates\\OhGreen\\Verwerking\\2print\\" + Formaat + "\\" + NAAM + "_NANNINE" + ".pdf")
            shutil.copy(path, dest)
            print(dest , " changed name and moved to ", dest)
</code>```

This code goes on for every city name in the csv (12 in total)
I want to look for a way to make this cleaner.
Like i don't need to make an if loop for every city and a loop for all the header names etc.

Any tips are welcome.
PS: I won't use Pandas because i can't get it installed properly and it works fine already in basic python

答:
First of all, store the strings as variables. You seem to be reusing the same string a lot, so if you store it as the variable BASE_PATH, you can do something like this:

```<code>BASE_PATH = "\\\\fatboy.leleu.be\\Psm\\Mailing\\Templates\\OhGreen\\Verwerking\\"
path = BASE_PATH + Formaat + "\\" + file
</code>```

or even

```<code>BASE_PATH = "\\\\fatboy.leleu.be\\Psm\\Mailing\\Templates\\OhGreen\\Verwerking\\{}\\{}"
path = BASE_PATH.format(Formaat, file)
</code>```

Secondly, you can store the variables you want to check for as a dictionary and loop over its keys. For example:

BASE_PATH = "\\\\fatboy.leleu.be\\Psm\\Mailing\\Templates\\OhGreen\\Verwerking\\{}\\{}"
DEST_PATH = "\\\\fatboy.leleu.be\\Psm\\Mailing\\Templates\\OhGreen\\Verwerking\\2print\\{}\\{}.pdf"
name_index_mapping = {
  "GOUVY": 3,
  "LOUVAIN_LA_NEUVE": 4,
  "NANNINE": 5
}

for (name, index) in name_index_mapping.items():
  if row[index] > "0":
    file = NAAM + ".pdf"
    path = BASE_PATH.format(Formaat, file)
    dest = DEST_PATH.format(Formaat, NAAM, name)
    shutil.copy(path, dest)
    print(path, "changed name and moved to", dest)
```

Finally, are you sure you need to compare the value to the literal string "0" and not the number 0? Comparing strings using greater/smaller than logic can give unexpected results.

Body:
I'm pretty new to python (made 4 scripts to cleanup folders and move some stuff around) and now i have a new challenge.

I have this csv file with data about some pdf's (name of the pdf, times it needs to be printed, etc.)

I have the following code atm and everything works fine, but i find it "unclean" and i hope there was a better way to format some things.
This is my code:

import csv, shutil, os

with open('\\\\fatboy.leleu.be\\Psm\\Mailing Templates\\OhGreen\\Verwerking\\OhGreen POS week 19.csv') as csvfile:
    csvReader = csv.reader(csvfile, delimiter=';')
    rows = csv.reader(csvfile)
    headers = next(rows)

    for row in csvReader:
        NAAM = row[0]
        Formaat = row[1]
        Tekstkleur = row[2]
        Gouvy = row[3]
        LouvainLaNeuve = row[4]
        Nannine = row[5]
        Tournai = row[6]
        SaintGeorges = row[7]
        Aarschot = row[8]
        Balen = row[9]
        Ekeren = row[10]
        Ninove = row[11]
        SintKatelijneWaver = row[12]
        Olen = row[13]
        Zwijndrecht = row[14]

        if Gouvy > "0":
            file = NAAM + ".pdf"
            path = ("\\\\fatboy.leleu.be\\Psm\\Mailing Templates\\OhGreen\\Verwerking\\" + Formaat + "\\" + file)
            dest = ("\\\\fatboy.leleu.be\\Psm\\Mailing Templates\\OhGreen\\Verwerking\\2print\\" + Formaat + "\\" + NAAM + "_GOUVY" + ".pdf")
            shutil.copy(path, dest)
            print(dest , " changed name and moved to ", dest)

        if LouvainLaNeuve > "0":
            file = NAAM + ".pdf"
            path = ("\\\\fatboy.leleu.be\\Psm\\Mailing Templates\\OhGreen\\Verwerking\\" + Formaat + "\\" + file)
            dest = ("\\\\fatboy.leleu.be\\Psm\\Mailing Templates\\OhGreen\\Verwerking\\2print\\" + Formaat + "\\" + NAAM + "_LOUVAIN_LA_NEUVE" + ".pdf")
            shutil.copy(path, dest)
            print(dest , " changed name and moved to ", dest)

        if Nannine > "0":
            file = NAAM + ".pdf"
            path = ("\\\\fatboy.leleu.be\\Psm\\Mailing Templates\\OhGreen\\Verwerking\\" + Formaat + "\\" + file)
            dest = ("\\\\fatboy.leleu.be\\Psm\\Mailing Templates\\OhGreen\\Verwerking\\2print\\" + Formaat + "\\" + NAAM + "_NANNINE" + ".pdf")
            shutil.copy(path, dest)
            print(dest , " changed name and moved to ", dest)

This code goes on for every city name in the csv (12 in total)
I want to look for a way to make this cleaner.
Like i don't need to make an if loop for every city and a loop for all the header names etc.

Any tips are welcome.
PS: I won't use Pandas because i can't get it installed properly and it works fine already in basic python

Body:
First of all, store the strings as variables. You seem to be reusing the same string a lot, so if you store it as the variable BASE_PATH, you can do something like this:

BASE_PATH = "\\\\fatboy.leleu.be\\Psm\\Mailing\\Templates\\OhGreen\\Verwerking\\"
path = BASE_PATH + Formaat + "\\" + file

or even

BASE_PATH = "\\\\fatboy.leleu.be\\Psm\\Mailing\\Templates\\OhGreen\\Verwerking\\{}\\{}"
path = BASE_PATH.format(Formaat, file)

Secondly, you can store the variables you want to check for as a dictionary and loop over its keys. For example:

BASE_PATH = "\\\\fatboy.leleu.be\\Psm\\Mailing\\Templates\\OhGreen\\Verwerking\\{}\\{}"
DEST_PATH = "\\\\fatboy.leleu.be\\Psm\\Mailing\\Templates\\OhGreen\\Verwerking\\2print\\{}\\{}.pdf"
name_index_mapping = {
  "GOUVY": 3,
  "LOUVAIN_LA_NEUVE": 4,
  "NANNINE": 5
}

for (name, index) in name_index_mapping.items():
  if row[index] > "0":
    file = NAAM + ".pdf"
    path = BASE_PATH.format(Formaat, file)
    dest = DEST_PATH.format(Formaat, NAAM, name)
    shutil.copy(path, dest)
    print(path, "changed name and moved to", dest)

Finally, are you sure you need to compare the value to the literal string "0" and not the number 0? Comparing strings using greater/smaller than logic can give unexpected results.


问:
reset validations after form reset(ReactiveForms)
HTML:

```<code><form [formGroup]="locationForm" #myForm="ngForm" class="formContainer">

<div class="buttonContainer">
      <section class="btn1">
          <button
          (click)="addAnotherMethod()"
          mat-raised-button
          color="primary"> Add another method </button>
      </section>

      <section>
          <button
          (click)="addAnotherLocation()"
          mat-raised-button
          color="primary"> Add another location </button>
      </section>
    </div>
</code>```

.TS File

```<code>@ViewChild('myForm') currentLocationForm;

addAnotherLocation(): void{
    if(this.locationForm.valid){
      //send data to redux
      }))
    }
    this.currentLocationForm.resetForm()
  }

addAnotherMethod: void {
    if(this.locationForm.valid){
     //send data to redux
      }))
    }
    this.currentLocationForm.resetForm()
  }
</code>```

With the above code I'm able to reset the form after the data is sent to redux,but after reset, the validation error messages in the new form comes up saying to fill the required fields. I need help in using the correct method in resetting this form and clearing validations when new form is shown. I may be also using wrong method to call two button functions.
Thank you.

答:
You have 2 things in your form: FormGroup and FormGroupDirective. To get the errors to go away, you need to reset both of them:

```<code>this.currentLocationForm.resetForm();
this.locationForm.reset();
</code>```

See this post for more details about why.

Body:
HTML:

<form [formGroup]="locationForm" #myForm="ngForm" class="formContainer">

          <button
          (click)="addAnotherMethod()"
          mat-raised-button
          color="primary"> Add another method </button>

          <button
          (click)="addAnotherLocation()"
          mat-raised-button
          color="primary"> Add another location </button>

.TS File

@ViewChild('myForm') currentLocationForm;

addAnotherLocation(): void{
    if(this.locationForm.valid){
      //send data to redux
      }))
    }
    this.currentLocationForm.resetForm()
  }

addAnotherMethod: void {
    if(this.locationForm.valid){
     //send data to redux
      }))
    }
    this.currentLocationForm.resetForm()
  }

With the above code I'm able to reset the form after the data is sent to redux,but after reset, the validation error messages in the new form comes up saying to fill the required fields. I need help in using the correct method in resetting this form and clearing validations when new form is shown. I may be also using wrong method to call two button functions.
Thank you.

Body:
You have 2 things in your form: FormGroup and FormGroupDirective. To get the errors to go away, you need to reset both of them:

this.currentLocationForm.resetForm();
this.locationForm.reset();

See this post for more details about why.


问:
How to fix "TypeError: a bytes-like object is required, not 'str'" error in python?
I'm upgrading a python 2.7 code to python 3.6, but every time I'm trying to write something on console using logging I get this error

TypeError: a bytes-like object is required, not 'str'

I've read most of the similiar questions with this but none of them has worked.

# mainTest.py module
from config import logger
log = logger.getLogger(__file__)

def function():
    message = "testing"
    log.info(message)
    # code
```

<pre class="lang-py prettyprint-override"><code># logger.py
import logging
import os
import classpathDir as cf

def getLogger(loggerForFile):
    logger = logging.getLogger(os.path.basename(loggerForFile))
    if not logger.handlers:
        logging.basicConfig(format='%(asctime)s %(name)-12s %(levelname)-8s %(message)s',
                            datefmt='%d/%m/%Y %I:%M:%S %p', filename=cf.ROOT_DIR + "/output/processing.log",
                            filemode='wb', level=logging.DEBUG)
        console = logging.StreamHandler()
        console.setLevel(logging.DEBUG)
        # set a format which is simpler for console use
        formatter = logging.Formatter('%(asctime)s %(name)-12s: %(levelname)-8s %(message)s',
                                      datefmt='%d/%m/%Y %I:%M:%S')
        # tell the handler to use this format
        console.setFormatter(formatter)
        # add the handler to the root logger
        logger.addHandler(console)
    return logger

if __name__ == '__main__':
    print("Logging config module")
</code>```

When I was using this very same code on python2.7 i got this output:

22/05/2019 01:38:11 mainTest.py   : INFO     testing
```

<p>On python 3.6 with the same code I got this error:</p>

<pre class="lang-py prettyprint-override"><code>22/05/2019 03:17:59 mainRF.py   : INFO     testing
--- Logging error ---
Traceback (most recent call last):
  File "/usr/lib/python3.6/logging/__init__.py", line 996, in emit
    stream.write(msg)
TypeError: a bytes-like object is required, not 'str'
Call stack:
  File "mainTest.py", line 126, in <module>
    run_combinations()
  File "mainTest.py", line 20, in run_combinations
    log.info(message)
Message: 'testing'
Arguments: ()
</code>```

答:
When setting logging basic config I changed
python filemode='wb' to
python filemode='w' and it worked properly.

Body:
I'm upgrading a python 2.7 code to python 3.6, but every time I'm trying to write something on console using logging I get this error

TypeError: a bytes-like object is required, not 'str'

I've read most of the similiar questions with this but none of them has worked.

# mainTest.py module
from config import logger
log = logger.getLogger(__file__)

def function():
    message = "testing"
    log.info(message)
    # code

# logger.py
import logging
import os
import classpathDir as cf

def getLogger(loggerForFile):
    logger = logging.getLogger(os.path.basename(loggerForFile))
    if not logger.handlers:
        logging.basicConfig(format='%(asctime)s %(name)-12s %(levelname)-8s %(message)s',
                            datefmt='%d/%m/%Y %I:%M:%S %p', filename=cf.ROOT_DIR + "/output/processing.log",
                            filemode='wb', level=logging.DEBUG)
        console = logging.StreamHandler()
        console.setLevel(logging.DEBUG)
        # set a format which is simpler for console use
        formatter = logging.Formatter('%(asctime)s %(name)-12s: %(levelname)-8s %(message)s',
                                      datefmt='%d/%m/%Y %I:%M:%S')
        # tell the handler to use this format
        console.setFormatter(formatter)
        # add the handler to the root logger
        logger.addHandler(console)
    return logger

if __name__ == '__main__':
    print("Logging config module")

When I was using this very same code on python2.7 i got this output:

22/05/2019 01:38:11 mainTest.py   : INFO     testing

On python 3.6 with the same code I got this error:

22/05/2019 03:17:59 mainRF.py   : INFO     testing
--- Logging error ---
Traceback (most recent call last):
  File "/usr/lib/python3.6/logging/__init__.py", line 996, in emit
    stream.write(msg)
TypeError: a bytes-like object is required, not 'str'
Call stack:
  File "mainTest.py", line 126, in <module>
    run_combinations()
  File "mainTest.py", line 20, in run_combinations
    log.info(message)
Message: 'testing'
Arguments: ()


问:
Creating space in looping
How to create empty space in a for loop to save the output if I don't know the number of looping that will be iterated and save the result in a certain file format in a folder?</p>

```<code>data<-
for(i in 1:9){
for(j in 1:5){
   for (b in 1:5){
  for (f in 1:(max(data[,'colum']))) {
model.fit<-lm(y~x)
#again I want to pull out the coefficients and all necessessary values and put them somewhere as matrix or anything else..
I also want to add another model and put the results together
      }
   }
 }
}
write.table ()
</code>```

答:
As mentioned in the comments, you can initialize a list and an auxiliary position variable aux. For your case, something similar to this:

```<code>mylist = list()
aux = 1

for(i in 1:9){
    for(j in 1:5){
       for (b in 1:5){
          for (f in 1:(max(data[,'colum']))) {
              mylist[[aux]]<-lm(y~x)
              aux = aux + 1
          }
       }
    }
}
write.table ()
</code>```

In each iteration, the model is added to your list and the auxiliary variable is updated by one.

Edit

In case you want to add two models per iteration you can do:

```<code>mylist = list()
aux = 1

for(i in 1:9){
    for(j in 1:5){
       for (b in 1:5){
          for (f in 1:(max(data[,'colum']))) {
              mylist[[aux]]<-lm(y~x)
              aux = aux + 1
              mylist[[aux]] <- lm2(y~x)
              aux = aux + 1
          }
       }
    }
}
write.table ()
</code>```

This way, all odd numbers will have model.fit and the even numbers model.fit1.

Another alternative is to use nested lists: for each step, a sublist containing model.fit and model.fit1 can be created as follows:

```<code>mylist = list()
aux = 1

for(i in 1:9){
    for(j in 1:5){
       for (b in 1:5){
          for (f in 1:(max(data[,'colum']))) {
              mylist[[aux]]<-list(lm(y~x),lm2(y~x))
              aux = aux + 1
          }
       }
    }
}
write.table ()
</code>```

Then, for each step you can access model.fit by doing mylist[[aux]][[1]] and model.fit1 by doing mylist[[aux]][[2]].

Title: Creating space in looping

Body:
How to create empty space in a for loop to save the output if I don't know the number of looping that will be iterated and save the result in a certain file format in a folder?

data<-
for(i in 1:9){
for(j in 1:5){
   for (b in 1:5){
  for (f in 1:(max(data[,'colum']))) {
model.fit<-lm(y~x)
#again I want to pull out the coefficients and all necessessary values and put them somewhere as matrix or anything else..
I also want to add another model and put the results together
      }
   }
 }
}
write.table ()

Body:
As mentioned in the comments, you can initialize a list and an auxiliary position variable aux. For your case, something similar to this:

mylist = list()
aux = 1

for(i in 1:9){
    for(j in 1:5){
       for (b in 1:5){
          for (f in 1:(max(data[,'colum']))) {
              mylist[[aux]]<-lm(y~x)
              aux = aux + 1
          }
       }
    }
}
write.table ()

In each iteration, the model is added to your list and the auxiliary variable is updated by one.

Edit

In case you want to add two models per iteration you can do:

mylist = list()
aux = 1

for(i in 1:9){
    for(j in 1:5){
       for (b in 1:5){
          for (f in 1:(max(data[,'colum']))) {
              mylist[[aux]]<-lm(y~x)
              aux = aux + 1
              mylist[[aux]] <- lm2(y~x)
              aux = aux + 1
          }
       }
    }
}
write.table ()

This way, all odd numbers will have model.fit and the even numbers model.fit1.

Another alternative is to use nested lists: for each step, a sublist containing model.fit and model.fit1 can be created as follows:

mylist = list()
aux = 1

for(i in 1:9){
    for(j in 1:5){
       for (b in 1:5){
          for (f in 1:(max(data[,'colum']))) {
              mylist[[aux]]<-list(lm(y~x),lm2(y~x))
              aux = aux + 1
          }
       }
    }
}
write.table ()

Then, for each step you can access model.fit by doing mylist[[aux]][[1]] and model.fit1 by doing mylist[[aux]][[2]].


问:
Subtracting value from previous row sql by group
I have the following dataset:

```<code>DATA_FIM    CENTRO  C   M   ESTOQUE
2018-02-01  HD01    CD  70  539.000
2018-03-01  HD01    CD  70  511.000
2018-04-01  HD01    CD  70  468.000
2018-05-01  HD01    CD  70  447.000
2018-06-01  HD01    CD  70  382.000
2018-07-01  HD01    CD  70  348.000
2018-08-01  HD01    CD  70  285.000
2018-09-01  HD01    CD  70  245.000
2018-10-01  HD01    CD  70  221.000
2018-11-01  HD01    CD  70  207.000
2018-12-01  HD01    CD  70  122.000
2018-12-21  HD01    CD  70  101.000
2018-02-01  HD01    CD  71  164.000
2018-03-01  HD01    CD  71  147.000
2018-04-01  HD01    CD  71  124.000
2018-05-01  HD01    CD  71  107.000
2018-06-01  HD01    CD  71  78.000
</code>```

1- Considering the 2nd, 3rd, 4th columns as a group, I want to subtract the 5th column values from previous row of same column.

In order to create a new row.

[![Data][2]][2]

It's very similar with this problem: [Issue:13196190][3], however, in my case I have multiple index, and I am not sure how to solve.

UPDATED

I am using the suggested codes

```<code>select
ZBI_FAT_PRODUCT.DATA_FIM,
        ZBI_DIM_EMPRESA.CENTRO,
        ZBI_DIM_EMPRESA.CANAL,
        ZBI_DIM_PRODUCT.MATERIAL,
        ZBI_DIM_PRODUCT.TITULO,
        ZBI_FAT_PRODUCT.ESTOQUE_VENDA,
        LAG(ZBI_FAT_PRODUCT.ESTOQUE_VENDA, 1, ZBI_FAT_PRODUCT.ESTOQUE_VENDA) OVER (PARTITION BY ZBI_DIM_EMPRESA.CENTRO, ZBI_DIM_PRODUCT.MATERIAL, ZBI_DIM_PRODUCT.TITULO ORDER BY ZBI_FAT_PRODUCT.DATA_FIM) - ZBI_FAT_PRODUCT.ESTOQUE_VENDA AS NEW
FROM ZBI_DIM_EMPRESA INNER JOIN (ZBI_DIM_PRODUCT INNER JOIN ZBI_FAT_PRODUCT ON ZBI_DIM_PRODUCT.BK = ZBI_FAT_PRODUCT.BK_MATERIAL) ON ZBI_DIM_EMPRESA.BK = ZBI_FAT_PRODUCT.BK_EMPRESA;
</code>```

答:
You seem to just want lag():

```<code>select el.*,
       (lag(estoque, 0, estoque) over (partition by centro, c, m order by date_mif) -
        estoque
       ) as diff
from energylog el;
</code>```

EDIT:

For the edited question, only two columns seem to comprise a group

```<code>select el.*,
       (lag(estoque, 0, value) over (partition by centro, canal, order by date_mif) -
        estoque
       ) as diff
from energylog el;
</code>```

Body:
I have the following dataset:

DATA_FIM    CENTRO  C   M   ESTOQUE
2018-02-01  HD01    CD  70  539.000
2018-03-01  HD01    CD  70  511.000
2018-04-01  HD01    CD  70  468.000
2018-05-01  HD01    CD  70  447.000
2018-06-01  HD01    CD  70  382.000
2018-07-01  HD01    CD  70  348.000
2018-08-01  HD01    CD  70  285.000
2018-09-01  HD01    CD  70  245.000
2018-10-01  HD01    CD  70  221.000
2018-11-01  HD01    CD  70  207.000
2018-12-01  HD01    CD  70  122.000
2018-12-21  HD01    CD  70  101.000
2018-02-01  HD01    CD  71  164.000
2018-03-01  HD01    CD  71  147.000
2018-04-01  HD01    CD  71  124.000
2018-05-01  HD01    CD  71  107.000
2018-06-01  HD01    CD  71  78.000

1- Considering the 2nd, 3rd, 4th columns as a group, I want to subtract the 5th column values from previous row of same column.

In order to create a new row.

[![Data][2]][2]

It's very similar with this problem: [Issue:13196190][3], however, in my case I have multiple index, and I am not sure how to solve.

UPDATED

I am using the suggested codes

select
ZBI_FAT_PRODUCT.DATA_FIM,
        ZBI_DIM_EMPRESA.CENTRO,
        ZBI_DIM_EMPRESA.CANAL,
        ZBI_DIM_PRODUCT.MATERIAL,
        ZBI_DIM_PRODUCT.TITULO,
        ZBI_FAT_PRODUCT.ESTOQUE_VENDA,
        LAG(ZBI_FAT_PRODUCT.ESTOQUE_VENDA, 1, ZBI_FAT_PRODUCT.ESTOQUE_VENDA) OVER (PARTITION BY ZBI_DIM_EMPRESA.CENTRO, ZBI_DIM_PRODUCT.MATERIAL, ZBI_DIM_PRODUCT.TITULO ORDER BY ZBI_FAT_PRODUCT.DATA_FIM) - ZBI_FAT_PRODUCT.ESTOQUE_VENDA AS NEW
FROM ZBI_DIM_EMPRESA INNER JOIN (ZBI_DIM_PRODUCT INNER JOIN ZBI_FAT_PRODUCT ON ZBI_DIM_PRODUCT.BK = ZBI_FAT_PRODUCT.BK_MATERIAL) ON ZBI_DIM_EMPRESA.BK = ZBI_FAT_PRODUCT.BK_EMPRESA;

Body:
You seem to just want lag():

select el.*,
       (lag(estoque, 0, estoque) over (partition by centro, c, m order by date_mif) -
        estoque
       ) as diff
from energylog el;

EDIT:

For the edited question, only two columns seem to comprise a group

select el.*,
       (lag(estoque, 0, value) over (partition by centro, canal, order by date_mif) -
        estoque
       ) as diff
from energylog el;


问:
How to enforce utf8mb4 on table creation with to_sql?
I'm importing some data from an API in Python, formatting it and saving it to a MySQL database with to_sql.

```<code>results, types, valid = self.process_data(data, [])
        if valid:
            results.to_sql(
                con=self.db.connection,
                name="degreed_" + method,
                if_exists="replace",
                index=False,
                dtype=types,
            )
</code>```

In my connection I have specified utf8mb4 as the charset:

```<code>self.connection = create_engine(
            'mysql+mysqlconnector://{0}:{1}@{2}/{3}?charset=utf8mb4'.
            format(database_username, database_password, database_ip, database_name))
</code>```

and in my types I have text columns as:

```<code>NVARCHAR(length=500, collation='utf8mb4_bin').
</code>```

However, I still get the error message:

```<code>COLLATION 'utf8mb4_bin' is not valid for CHARACTER SET 'utf8'
</code>```

In MySQL my character_set_client is utf8mb4 and the default table charset is utf8mb4. Why is the character set utf8?

Apologies if I'm doing anything stupid here, I'm quite new to sqlalchemy and mysql in general.

答: Turns out the issue was that I was using NVARCHAR instead of VARCHAR, so the result was being cast as UTF8.</p>

Body:
I'm importing some data from an API in Python, formatting it and saving it to a MySQL database with to_sql.

results, types, valid = self.process_data(data, [])
        if valid:
            results.to_sql(
                con=self.db.connection,
                name="degreed_" + method,
                if_exists="replace",
                index=False,
                dtype=types,
            )

In my connection I have specified utf8mb4 as the charset:

self.connection = create_engine(
            'mysql+mysqlconnector://{0}:{1}@{2}/{3}?charset=utf8mb4'.
            format(database_username, database_password, database_ip, database_name))

and in my types I have text columns as:

NVARCHAR(length=500, collation='utf8mb4_bin').

However, I still get the error message:

COLLATION 'utf8mb4_bin' is not valid for CHARACTER SET 'utf8'

In MySQL my character_set_client is utf8mb4 and the default table charset is utf8mb4. Why is the character set utf8?

Apologies if I'm doing anything stupid here, I'm quite new to sqlalchemy and mysql in general.


问:
Adding external JavaScript file to Magento
How to add external JavaScript file to Magento, so it's code would be included on every frontend page?</p>

答:
None of the methods above worked for me because the script was not hosted on the same domain as the website and had to be controlled using a config variable.

This was my solution:

```<code>    /** @var Mage_Core_Model_Layout $layout */
    $layout = Mage::getSingleton('core/layout');

    /** @var Mage_Core_Block_Text $block */
    $block = $layout->createBlock('Mage_Core_Block_Text', $name);

    $block->setText('<script type="text/javascript" src="'.$url.'"></script>');

    /** @var Mage_Page_Block_Html_Head $head */
    $head = $layout->getBlock('head');

    $head->append($block);
</code>```

I did this in an observer observing controller_action_layout_generate_blocks_after

Body:
None of the methods above worked for me because the script was not hosted on the same domain as the website and had to be controlled using a config variable.

This was my solution:

    /** @var Mage_Core_Model_Layout $layout */
    $layout = Mage::getSingleton('core/layout');

    /** @var Mage_Core_Block_Text $block */
    $block = $layout->createBlock('Mage_Core_Block_Text', $name);

    $block->setText('<script type="text/javascript" src="'.$url.'"></script>');

    /** @var Mage_Page_Block_Html_Head $head */
    $head = $layout->getBlock('head');

    $head->append($block);

I did this in an observer observing controller_action_layout_generate_blocks_after