问:
Multi-threading and Single-threading performance issues in CPU-bound task
The two following single-threading and multi-threading scripts are taking the same time when I give as input a big number like 555550000

single thread

```<code>import threading, time
a=[]
def print_factors(x):
   for i in range(1, x + 1):
       if x % i == 0:
           a.append(i)

n=int(input("Please enter a large number"))
print ("Starting time is %s" % ( time.ctime(time.time()) ))
print("The factors of",n,"are:")
thread = threading.Thread(target=print_factors,args=(n,))
thread.start()
thread.join()
print("Finishing time is %s" % (time.ctime(time.time())))
print(a)
</code>```

multi thread

```<code>import threading, time
a=[]
def print_factors1(x):
   for i in range(1, int(x/2)):
       if x % i == 0:
           a.append(i)

def print_factors2(x):
    for i in range(int(x/2), x+1):
        if x % i == 0:
            a.append(i)

n=int(input("Please enter a large number"))
print ("Starting time is %s" % ( time.ctime(time.time()) ))
thread1 = threading.Thread(target=print_factors1,args=(n,))
thread2 = threading.Thread(target=print_factors2,args=(n,))
print("The factors of",n,"are:")
thread1.start()
thread2.start()
thread2.join()
print("Finishing time is %s" % (time.ctime(time.time())))
print(a)
</code>```

I am trying to understand the difference between single-threading and multi-threading in terms of time taken to got the results.

I'm measuring similar timings for both types and I cannot figuring out the reasons.

答:
Your problem is GIL, the Global Interpreter Lock.

The Python Global Interpreter Lock or GIL, in simple words, is a mutex
  (or a lock) that allows only one thread to hold the control of the
  Python interpreter.

You can found detailed informations about GIL here (just a fast search on Google and you can find a lot more sources):

https://wiki.python.org/moin/GlobalInterpreterLock

What is the global interpreter lock (GIL) in CPython?

https://medium.com/python-features/pythons-gil-a-hurdle-to-multithreaded-program-d04ad9c1a63

You need to change your implementation to use processes instead of threads.

I changed your script as follows:

```<code>from multiprocessing import Pool
import time
def print_factors1(x):
    a=[]
    for i in range(1, int(x/2)):
        if x % i == 0:
            a.append(i)
    return a

def print_factors2(x):
    a=[]
    for i in range(int(x/2), x+1):
        if x % i == 0:
            a.append(i)
    return a

if __name__ == '__main__':
    n=int(input("Please enter a large number"))
    pool = Pool(processes=2)
    print ("Starting time is %s" % ( time.ctime(time.time()) ))

    process1 = pool.apply_async(print_factors1,[n])
    process2 = pool.apply_async(print_factors2,[n])

    pool.close()
    pool.join()

    print("Finishing time is %s" % (time.ctime(time.time())))
    print("The factors of",n,"are:")
    print(process1.get())
    print(process2.get())
</code>```

Take into account that threads share the memory, processes don't.

Body:
The two following single-threading and multi-threading scripts are taking the same time when I give as input a big number like 555550000

single thread

import threading, time
a=[]
def print_factors(x):
   for i in range(1, x + 1):
       if x % i == 0:
           a.append(i)

n=int(input("Please enter a large number"))
print ("Starting time is %s" % ( time.ctime(time.time()) ))
print("The factors of",n,"are:")
thread = threading.Thread(target=print_factors,args=(n,))
thread.start()
thread.join()
print("Finishing time is %s" % (time.ctime(time.time())))
print(a)

multi thread

import threading, time
a=[]
def print_factors1(x):
   for i in range(1, int(x/2)):
       if x % i == 0:
           a.append(i)

def print_factors2(x):
    for i in range(int(x/2), x+1):
        if x % i == 0:
            a.append(i)

n=int(input("Please enter a large number"))
print ("Starting time is %s" % ( time.ctime(time.time()) ))
thread1 = threading.Thread(target=print_factors1,args=(n,))
thread2 = threading.Thread(target=print_factors2,args=(n,))
print("The factors of",n,"are:")
thread1.start()
thread2.start()
thread2.join()
print("Finishing time is %s" % (time.ctime(time.time())))
print(a)

I am trying to understand the difference between single-threading and multi-threading in terms of time taken to got the results.

I'm measuring similar timings for both types and I cannot figuring out the reasons.

Body:
Your problem is GIL, the Global Interpreter Lock.

The Python Global Interpreter Lock or GIL, in simple words, is a mutex
  (or a lock) that allows only one thread to hold the control of the
  Python interpreter.

You can found detailed informations about GIL here (just a fast search on Google and you can find a lot more sources):

https://wiki.python.org/moin/GlobalInterpreterLock

What is the global interpreter lock (GIL) in CPython?

https://medium.com/python-features/pythons-gil-a-hurdle-to-multithreaded-program-d04ad9c1a63

You need to change your implementation to use processes instead of threads.

I changed your script as follows:

from multiprocessing import Pool
import time
def print_factors1(x):
    a=[]
    for i in range(1, int(x/2)):
        if x % i == 0:
            a.append(i)
    return a

def print_factors2(x):
    a=[]
    for i in range(int(x/2), x+1):
        if x % i == 0:
            a.append(i)
    return a

if __name__ == '__main__':
    n=int(input("Please enter a large number"))
    pool = Pool(processes=2)
    print ("Starting time is %s" % ( time.ctime(time.time()) ))

    process1 = pool.apply_async(print_factors1,[n])
    process2 = pool.apply_async(print_factors2,[n])

    pool.close()
    pool.join()

    print("Finishing time is %s" % (time.ctime(time.time())))
    print("The factors of",n,"are:")
    print(process1.get())
    print(process2.get())

Take into account that threads share the memory, processes don't.


来源: stackexchange

问:
Use group by with find_in_batches in Rails
I have a topic_followers table in MySQL with schema like this

```<code>id, user_id, topic_id, creation_date, modified_date
</code>```

I tried to use expression below to get list of topics each user is following:

```<code>TopicFollower.select("user_id, group_concat(topic_id) as topic_ids").group("user_id").find_in_batches { |group| ... }
</code>```

However, find_in_batches automatically adds "ORDER BY topic_followers.id ASC" to the query and forces me to add "id" to select clause which conflicts agains group clause. Any suggestion how I can do batch requests with group?

Another related question, is there a rule of thumb on when I should start batching the requests? What's the decision process like?

答:
```<code>TopicFollower.find_in_batches(batch_size: 300) do |objects|
 objects.group_by { |object| object&.user_id}
end
</code>```

ContentLicense: CC BY-SA 4.0

Body:
I have a topic_followers table in MySQL with schema like this

id, user_id, topic_id, creation_date, modified_date

I tried to use expression below to get list of topics each user is following:

TopicFollower.select("user_id, group_concat(topic_id) as topic_ids").group("user_id").find_in_batches { |group| ... }

However, find_in_batches automatically adds "ORDER BY topic_followers.id ASC" to the query and forces me to add "id" to select clause which conflicts agains group clause. Any suggestion how I can do batch requests with group?

Another related question, is there a rule of thumb on when I should start batching the requests? What's the decision process like?


问:
Python: How to label this dataset
Hi I have df like this:

```<code>df = pd.DataFrame({'CaseNo':[1,1,1,1,2,2,3,3,3,4,4],
                   'Category':['A','A','IOU','A','B','B','IOU','IOU','IOU','C','IOU']})
</code>```

I'd like to label CaseNo col based on this condition:

For each CaseNo, if there is an occurrence of value 'IOU' in Category column with any values (except 'IOU') preceding it - label 'YES', else label 'NO'.

How can I do this in Python? Expected outcome:

答:
The following code does what you need. It uses groupby and index. The advantage is you can always modify the function func to make an update in what you want. The other answers are no doubt specific but can't be modified much.

def func(x):
    try:
        return 'YES' if x.index('IOU')!=0 else 'NO'
    except:
        return "NO"
mapper = df.groupby("CaseNo")['Category'].agg(list).apply(func)
df['Label'] = df['CaseNo'].apply(lambda x: mapper[x])
print(df)
```

<p>Output:</p>

```    CaseNo Category Label
0        1        A   YES
1        1        A   YES
2        1      IOU   YES
3        1        A   YES
4        2        B    NO
5        2        B    NO
6        3      IOU    NO
7        3      IOU    NO
8        3      IOU    NO
9        4        C   YES
10       4      IOU   YES
```

Title: Python: How to label this dataset

Body:
Hi I have df like this:

df = pd.DataFrame({'CaseNo':[1,1,1,1,2,2,3,3,3,4,4],
                   'Category':['A','A','IOU','A','B','B','IOU','IOU','IOU','C','IOU']})

I'd like to label CaseNo col based on this condition:

For each CaseNo, if there is an occurrence of value 'IOU' in Category column with any values (except 'IOU') preceding it - label 'YES', else label 'NO'.

How can I do this in Python? Expected outcome:

Body:
The following code does what you need. It uses groupby and index. The advantage is you can always modify the function func to make an update in what you want. The other answers are no doubt specific but can't be modified much.

def func(x):
    try:
        return 'YES' if x.index('IOU')!=0 else 'NO'
    except:
        return "NO"
mapper = df.groupby("CaseNo")['Category'].agg(list).apply(func)
df['Label'] = df['CaseNo'].apply(lambda x: mapper[x])
print(df)

Output:

    CaseNo Category Label
0        1        A   YES
1        1        A   YES
2        1      IOU   YES
3        1        A   YES
4        2        B    NO
5        2        B    NO
6        3      IOU    NO
7        3      IOU    NO
8        3      IOU    NO
9        4        C   YES
10       4      IOU   YES


问:
Laravel Eloquent adding brackets to Eloquent SQL Statement
Got a bit of an issue that I am banging my head against a brick wall about and done a lot of digging but cant seem to find an answer.

So I have a search query which is like this ( Excuse the roughness of the code ) more a concept.

```<code> $slots = DB::table('slots')->select(DB::raw('DISTINCT user_id'));

 $slots->when( $date_start, function( $query, $date_start )  {
     return $query->where('start_date', '>=', $date_start->format('Y-m-d') );
 } );

 $slots->when( $date_end, function( $query, $date_end )  {
     return $query->where('start_date', '<=', $date_end->format('Y-m-d') );
 } );

 if( isset($dow)  ) {
   foreach( $days_array as $k => $v ) {
       if(isset($dow[$k])) {
            foreach($dow[$k] as $_k => $_v ) {
               if( !$dow_initial_where_added ) {
                    $slots->where('dow', '=', $v )
                       ->where('start_time', ">=", $thresholds[$_k]['start'] )
                       ->where('start_time', "<=", $thresholds[$_k]['end'] );
                       $dow_initial_where_added = true;
                } else {
                     $slots->orWhere('dow', '=', $v )
                         ->where('start_time', ">=", $thresholds[$_k]['start'] )
                         ->where('start_time', "<=", $thresholds[$_k]['end'] );
                }
            }
        }
}

$slots->where('status', '<', 3);

$slots->when( $instuctors_ids, function( $query, $instuctors_ids )  {
   return $query->whereIn('user_id', $instuctors_ids);
} );

$slots->get();
</code>```

Now this works how I would of kind of expected and gives me something like this:

```<code>Select DISTINCT user_id from `slots` where `start_date` >= 2020-06-01 and `start_date` <= 2020-06-30 and `dow` = thu and `start_time` >= 05:00 and `start_time` <= 11:59 or `dow` = thu and `start_time` >= 12:00 and `start_time` <= 17:59 or `dow` = thu and `start_time` >= 18:00 and `start_time` <= 23:59 and `status` < 3 and `user_id` in (15) order by RAND()
</code>```

What I really need is for it to do this:

```<code>Select DISTINCT user_id from `slots` where (`start_date` >= 2020-06-01 and `start_date` <= 2020-06-30 and `dow` = thu and `start_time` >= 05:00 and `start_time` <= 11:59 or `dow` = thu and `start_time` >= 12:00 and `start_time` <= 17:59 or `dow` = thu and `start_time` >= 18:00 and `start_time` <= 23:59 and `status` < 3) and `user_id` in (15) order by RAND()
</code>```

So brackets around everything apart from the And 'User_id' IN Statment.

I dont even know if its possible or not but would rather use Laravel's inbuilt query functions than write a raw query.

答:
Seems wrapping it all in a closure does the trick. </p>

```<code>$slots = DB::table('slots')->select(DB::raw('DISTINCT user_id'))->where( function($query) use ($date_start, $date_end, $thresholds, $dow, $time, $days_array) {

    $query->when( $date_start, function( $query, $date_start )  {
                        return $query->where('start_date', '>=', $date_start->format('Y-m-d') );
    } );

    $query->when( $date_end, function( $query, $date_end )  {
        return $query->where('start_date', '<=', $date_end->format('Y-m-d') );
    } );

    $dow_initial_where_added = false;

    if( isset($dow)  ) {

        foreach( $days_array as $k => $v ) {
            if(isset($dow[$k])) {
                foreach($dow[$k] as $_k => $_v ) {
                    if( !$dow_initial_where_added ) {
                            $query->where('dow', '=', $v )
                                ->where('start_time', ">=", $thresholds[$_k]['start'] )
                                ->where('start_time', "<=", $thresholds[$_k]['end'] );
                                $dow_initial_where_added = true;
                    } else {
                            $query->orWhere('dow', '=', $v )
                                ->where('start_time', ">=", $thresholds[$_k]['start'] )
                                ->where('start_time', "<=", $thresholds[$_k]['end'] );

                     }
                }
            }
        }
    }

});

$slots->where('status', '<', 3);

$slots->when( $instuctors_ids, function( $query, $instuctors_ids )  {
   return $query->whereIn('user_id', $instuctors_ids);
} );

$slots->get();
</code>```

Body:
Got a bit of an issue that I am banging my head against a brick wall about and done a lot of digging but cant seem to find an answer.

So I have a search query which is like this ( Excuse the roughness of the code ) more a concept.

 $slots = DB::table('slots')->select(DB::raw('DISTINCT user_id'));

 $slots->when( $date_start, function( $query, $date_start )  {
     return $query->where('start_date', '>=', $date_start->format('Y-m-d') );
 } );

 $slots->when( $date_end, function( $query, $date_end )  {
     return $query->where('start_date', '<=', $date_end->format('Y-m-d') );
 } );

 if( isset($dow)  ) {
   foreach( $days_array as $k => $v ) {
       if(isset($dow[$k])) {
            foreach($dow[$k] as $_k => $_v ) {
               if( !$dow_initial_where_added ) {
                    $slots->where('dow', '=', $v )
                       ->where('start_time', ">=", $thresholds[$_k]['start'] )
                       ->where('start_time', "<=", $thresholds[$_k]['end'] );
                       $dow_initial_where_added = true;
                } else {
                     $slots->orWhere('dow', '=', $v )
                         ->where('start_time', ">=", $thresholds[$_k]['start'] )
                         ->where('start_time', "<=", $thresholds[$_k]['end'] );
                }
            }
        }
}

$slots->where('status', '<', 3);

$slots->when( $instuctors_ids, function( $query, $instuctors_ids )  {
   return $query->whereIn('user_id', $instuctors_ids);
} );

$slots->get();

Now this works how I would of kind of expected and gives me something like this:

Select DISTINCT user_id from `slots` where `start_date` >= 2020-06-01 and `start_date` <= 2020-06-30 and `dow` = thu and `start_time` >= 05:00 and `start_time` <= 11:59 or `dow` = thu and `start_time` >= 12:00 and `start_time` <= 17:59 or `dow` = thu and `start_time` >= 18:00 and `start_time` <= 23:59 and `status` < 3 and `user_id` in (15) order by RAND()

What I really need is for it to do this:

Select DISTINCT user_id from `slots` where (`start_date` >= 2020-06-01 and `start_date` <= 2020-06-30 and `dow` = thu and `start_time` >= 05:00 and `start_time` <= 11:59 or `dow` = thu and `start_time` >= 12:00 and `start_time` <= 17:59 or `dow` = thu and `start_time` >= 18:00 and `start_time` <= 23:59 and `status` < 3) and `user_id` in (15) order by RAND()

So brackets around everything apart from the And 'User_id' IN Statment.

I dont even know if its possible or not but would rather use Laravel's inbuilt query functions than write a raw query.

Body:
Seems wrapping it all in a closure does the trick.

$slots = DB::table('slots')->select(DB::raw('DISTINCT user_id'))->where( function($query) use ($date_start, $date_end, $thresholds, $dow, $time, $days_array) {

    $query->when( $date_start, function( $query, $date_start )  {
                        return $query->where('start_date', '>=', $date_start->format('Y-m-d') );
    } );

    $query->when( $date_end, function( $query, $date_end )  {
        return $query->where('start_date', '<=', $date_end->format('Y-m-d') );
    } );

    $dow_initial_where_added = false;

    if( isset($dow)  ) {

        foreach( $days_array as $k => $v ) {
            if(isset($dow[$k])) {
                foreach($dow[$k] as $_k => $_v ) {
                    if( !$dow_initial_where_added ) {
                            $query->where('dow', '=', $v )
                                ->where('start_time', ">=", $thresholds[$_k]['start'] )
                                ->where('start_time', "<=", $thresholds[$_k]['end'] );
                                $dow_initial_where_added = true;
                    } else {
                            $query->orWhere('dow', '=', $v )
                                ->where('start_time', ">=", $thresholds[$_k]['start'] )
                                ->where('start_time', "<=", $thresholds[$_k]['end'] );

                     }
                }
            }
        }
    }

});

$slots->where('status', '<', 3);

$slots->when( $instuctors_ids, function( $query, $instuctors_ids )  {
   return $query->whereIn('user_id', $instuctors_ids);
} );

$slots->get();


问:
Blue Prism - Hot to use Invoke Javascript function
I have a issue with understanding how in BP Invoke Javascript works?
On online SAP system i need to enter a login and password in to 2 textboxes on page.
When I run my code in the Firefox console

```<code>document.getElementsByName('sap-user')[0].value = 'test';
</code>```

the textbox on page is filled with test and is working.

When I try to do the same in Blue Prism nothing happens.
I have created a variable which is holding the script:

```<code><script>
function test(){
document.getElementsByName('sap-user')[0].value = 'test';
return;}
</script>
</code>```

and running the code in the Navigate stage.

In the arguments I m passing the variable [jScript] name which is holding the script.

Element3 is the textbox spy element on page.

Dont know how to run the script in Blue Prism?

I'm to new in the javascript maybe my code is wrong?

Thank you for any help,
Mirek

答:
The details in this answer should help; summary is that you call Insert to add the JS to the page and then Invoke to fire the method.  The Arguments you send to the Invoke are in JSON format, e.g. "[{}]" even if the method requires no arguments.

Unable to Invoke Inserted Javascript Fragment

Body:
I have a issue with understanding how in BP Invoke Javascript works?
On online SAP system i need to enter a login and password in to 2 textboxes on page.
When I run my code in the Firefox console

document.getElementsByName('sap-user')[0].value = 'test';

the textbox on page is filled with test and is working.

When I try to do the same in Blue Prism nothing happens.
I have created a variable which is holding the script:

and running the code in the Navigate stage.

In the arguments I m passing the variable [jScript] name which is holding the script.

Element3 is the textbox spy element on page.

Dont know how to run the script in Blue Prism?

I'm to new in the javascript maybe my code is wrong?

Thank you for any help,
Mirek


问:
pytest cannot import module while python can
I am working on a package in Python. I use virtualenv. I set the path to the root of the module in a .pth path in my virtualenv, so that I can import modules of the package while developing the code and do testing (Question 1: is it a good way to do?). This works fine  (here is an example, this is the behavior I want):

```<code>(VEnvTestRc) zz@zz:~/Desktop/GitFolders/rc$ python
Python 2.7.12 (default, Jul  1 2016, 15:12:24)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from rc import ns
>>> exit()
(VEnvTestRc) zz@zz:~/Desktop/GitFolders/rc$ python tests/test_ns.py
issued command: echo hello
command output: hello
</code>```

However, if I try to use PyTest, I get some import error messages:

```<code>(VEnvTestRc) zz@zz:~/Desktop/GitFolders/rc$ pytest
=========================================== test session starts ============================================
platform linux2 -- Python 2.7.12, pytest-3.0.5, py-1.4.31, pluggy-0.4.0
rootdir: /home/zz/Desktop/GitFolders/rc, inifile:
collected 0 items / 1 errors

================================================== ERRORS ==================================================
________________________________ ERROR collecting tests/test_ns.py ________________________________
ImportError while importing test module '/home/zz/Desktop/GitFolders/rc/tests/test_ns.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/test_ns.py:2: in <module>
    from rc import ns
E   ImportError: cannot import name ns
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
========================================= 1 error in 0.09 seconds ==========================================
(VEnvTestRc) zz@zz:~/Desktop/GitFolders/rc$ which pytest
/home/zz/Desktop/VirtualEnvs/VEnvTestRc/bin/pytest
</code>```

I am a bit puzzled, it looks like this indicates an import error, but Python does it fine so why is there a problem specifically with PyTest? Any suggestion to the reason / remedy (Question 2)? I googled and stack-overflowed the 'ImportError: cannot import' error for PyTest, but the hits I got were related to missing python path and remedy to this, which does not seem to be the problem here. Any suggestions?

答:
Kept everything same and just added a blank test file at the root folder .. Solved it

Here are the findings, this problem really bugged me for a while.
My folder structure was

```<code>mathapp/
    - server.py
    - configuration.py
    - __init__.py
    - static/
       - home.html
tests/
    - functional
       - test_errors.py
    - unit
       - test_add.py
</code>```

and pytest would complain with the ModuleNotFoundError and gives the hint:

make sure your test modules/packages have valid Python names.

I introduced a mock test file at the same level as mathsapp and tests directory. The file contained nothing. Now pytest does not complain.

Result without the file

```<code>$ pytest
============================= test session starts =============================
platform win32 -- Python 3.8.2, pytest-5.4.2, py-1.8.1, pluggy-0.13.1
rootdir: C:\mak2006\workspace\0github\python-rest-app-cont
collected 1 item / 1 error

=================================== ERRORS ====================================
_______________ ERROR collecting tests/functional/test_func.py ________________
ImportError while importing test module 'C:\mainak\workspace\0github\python-rest-app-cont\tests\functional\test_func.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests\functional\test_func.py:4: in <module>
    from mathapp.service import sum
E   ModuleNotFoundError: No module named 'mathapp'
=========================== short test summary info ===========================
ERROR tests/functional/test_func.py
!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
============================== 1 error in 0.24s ===============================
</code>```

Results with the file

```<code>$ pytest
============================= test session starts =============================
platform win32 -- Python 3.8.2, pytest-5.4.2, py-1.8.1, pluggy-0.13.1
rootdir: C:\mak2006\workspace\0github\python-rest-app-cont
collected 2 items

tests\functional\test_func.py .                                          [ 50%]
tests\unit\test_unit.py .                                                [100%]

============================== 2 passed in 0.11s ==============================
</code>```

ContentLicense: CC BY-SA 3.0

Body:
I am working on a package in Python. I use virtualenv. I set the path to the root of the module in a .pth path in my virtualenv, so that I can import modules of the package while developing the code and do testing (Question 1: is it a good way to do?). This works fine  (here is an example, this is the behavior I want):

(VEnvTestRc) zz@zz:~/Desktop/GitFolders/rc$ python
Python 2.7.12 (default, Jul  1 2016, 15:12:24)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from rc import ns
>>> exit()
(VEnvTestRc) zz@zz:~/Desktop/GitFolders/rc$ python tests/test_ns.py
issued command: echo hello
command output: hello

However, if I try to use PyTest, I get some import error messages:

(VEnvTestRc) zz@zz:~/Desktop/GitFolders/rc$ pytest
=========================================== test session starts ============================================
platform linux2 -- Python 2.7.12, pytest-3.0.5, py-1.4.31, pluggy-0.4.0
rootdir: /home/zz/Desktop/GitFolders/rc, inifile:
collected 0 items / 1 errors

================================================== ERRORS ==================================================
________________________________ ERROR collecting tests/test_ns.py ________________________________
ImportError while importing test module '/home/zz/Desktop/GitFolders/rc/tests/test_ns.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/test_ns.py:2: in <module>
    from rc import ns
E   ImportError: cannot import name ns
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
========================================= 1 error in 0.09 seconds ==========================================
(VEnvTestRc) zz@zz:~/Desktop/GitFolders/rc$ which pytest
/home/zz/Desktop/VirtualEnvs/VEnvTestRc/bin/pytest

I am a bit puzzled, it looks like this indicates an import error, but Python does it fine so why is there a problem specifically with PyTest? Any suggestion to the reason / remedy (Question 2)? I googled and stack-overflowed the 'ImportError: cannot import' error for PyTest, but the hits I got were related to missing python path and remedy to this, which does not seem to be the problem here. Any suggestions?

Body:
Kept everything same and just added a blank test file at the root folder .. Solved it

Here are the findings, this problem really bugged me for a while.
My folder structure was

mathapp/
    - server.py
    - configuration.py
    - __init__.py
    - static/
       - home.html
tests/
    - functional
       - test_errors.py
    - unit
       - test_add.py

and pytest would complain with the ModuleNotFoundError and gives the hint:

make sure your test modules/packages have valid Python names.

I introduced a mock test file at the same level as mathsapp and tests directory. The file contained nothing. Now pytest does not complain.

Result without the file

$ pytest
============================= test session starts =============================
platform win32 -- Python 3.8.2, pytest-5.4.2, py-1.8.1, pluggy-0.13.1
rootdir: C:\mak2006\workspace\0github\python-rest-app-cont
collected 1 item / 1 error

=================================== ERRORS ====================================
_______________ ERROR collecting tests/functional/test_func.py ________________
ImportError while importing test module 'C:\mainak\workspace\0github\python-rest-app-cont\tests\functional\test_func.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests\functional\test_func.py:4: in <module>
    from mathapp.service import sum
E   ModuleNotFoundError: No module named 'mathapp'
=========================== short test summary info ===========================
ERROR tests/functional/test_func.py
!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
============================== 1 error in 0.24s ===============================

Results with the file

$ pytest
============================= test session starts =============================
platform win32 -- Python 3.8.2, pytest-5.4.2, py-1.8.1, pluggy-0.13.1
rootdir: C:\mak2006\workspace\0github\python-rest-app-cont
collected 2 items

tests\functional\test_func.py .                                          [ 50%]
tests\unit\test_unit.py .                                                [100%]

============================== 2 passed in 0.11s ==============================


问:
How to collect elements of a collection basing on a result of some method?
Suppose we have a list of values sorted according to some ordering. We also have a map of elements mapped to these values. We want to obtain a collection of elements from the map in the same order as their keys are in the list. A straightforward method to do this is:

```<code>val order = Seq("a", "b", "c")
val map = Map("a" -> "aaa", "c" -> "ccc")

val elems = order.map(map.get(_)).filter(_.isDefined).map(_.get)
</code>```

However the program needs to iterate over the collection three times. Is it possible to implement this functionality more efficiently? In particular, is it possible to do this with collect method?

答:
You can use flatMap for that. Here is an example:

```<code>List(1,2,3,4,5).flatMap(x => if (x%2 == 1) Some(2*x) else None)
</code>```

This is equivalent to

```<code>List(1,2,3,4,5).filter(_%2==1).map(2*)
</code>```

Body:
Suppose we have a list of values sorted according to some ordering. We also have a map of elements mapped to these values. We want to obtain a collection of elements from the map in the same order as their keys are in the list. A straightforward method to do this is:

val order = Seq("a", "b", "c")
val map = Map("a" -> "aaa", "c" -> "ccc")

val elems = order.map(map.get(_)).filter(_.isDefined).map(_.get)

However the program needs to iterate over the collection three times. Is it possible to implement this functionality more efficiently? In particular, is it possible to do this with collect method?

Body:
You can use flatMap for that. Here is an example:

List(1,2,3,4,5).flatMap(x => if (x%2 == 1) Some(2*x) else None)

This is equivalent to

List(1,2,3,4,5).filter(_%2==1).map(2*)


答:
IIUC and assuming that YES should only be set when there's at least one IOU and some value other than IOU preceding it:

```<code>m = (df.Category.eq('IOU').view('i1')
       .groupby(df.CaseNo)
       .transform(lambda x: x.diff().eq(1).any()))
df['Label'] = m.map({True:'Yes',False:'No'})
</code>```

```<code>print(df)

    CaseNo Category Label
0        1        A   Yes
1        1        A   Yes
2        1      IOU   Yes
3        1        A   Yes
4        2        B    No
5        2        B    No
6        3      IOU    No
7        3      IOU    No
8        3      IOU    No
9        4        C   Yes
10       4      IOU   Yes
</code>```

The idea here is that Series.diff() will be 1, when there is a 1 (IOU) preceded by a 0 (no IOU)

Title: Python: How to label this dataset

Body:
IIUC and assuming that YES should only be set when there's at least one IOU and some value other than IOU preceding it:

m = (df.Category.eq('IOU').view('i1')
       .groupby(df.CaseNo)
       .transform(lambda x: x.diff().eq(1).any()))
df['Label'] = m.map({True:'Yes',False:'No'})

print(df)

    CaseNo Category Label
0        1        A   Yes
1        1        A   Yes
2        1      IOU   Yes
3        1        A   Yes
4        2        B    No
5        2        B    No
6        3      IOU    No
7        3      IOU    No
8        3      IOU    No
9        4        C   Yes
10       4      IOU   Yes

The idea here is that Series.diff() will be 1, when there is a 1 (IOU) preceded by a 0 (no IOU)


问:
how to get a date from a date input using javascript
hello so i have a javascript function that takes two dates from two date inputs, compares them and returns a boolean value except it doesn't work

function checkdate() {

 var datefrom = new Date();
 var dateto = new Date();

 datefrom = getElementById("date_from").innerHTML;
 dateto = getElementById("date_to").innerHTML;

 if (datefrom > dateto) {
  alert("Dateto must be bigger than datefrom  ");

  return false;
 } else return true;

}```
<pre class="snippet-code-html lang-html prettyprint-override"><code><form onsubmit="return checkdate()" class="forms-sample" method="post"
   action="addstage.php">
   <div class="form-group row">
      <label for="exampleInputUsername2" class="col-sm-3 col-form-label">Date
      Début</label>
      <div class="col-sm-9">
         <input type="date" class="form-control"
            id="date_from" placeholder="Debut" name="datedebut"  required="required">
      </div>
   </div>
   <div class="form-group row">
      <label for="exampleInputEmail2" class="col-sm-3 col-form-label">Date
      Fin</label>
      <div class="col-sm-9">
         <input type="date" class="form-control"
            id="date_to" placeholder="Fin" name="datefin" required="required">
      </div>
   </div>
   <div class="form-check form-check-flat form-check-primary"></div>
   <button type="submit" class="btn btn-primary mr-2" value="addStage"
      name="action">Add</button>
</form></code>```

答:
Two problems:

You're calling getElementById globally, but should be calling it on the document object.

You're using .innerHtml instead of .value

As an aside you also don't need to create new Date() objects just to throw them away.  Simply set the values you want right away:

```<code>var datefrom = document.getElementById("date_from").value;
var dateto = document.getElementById("date_to").value;
</code>```

Note: These values are strings, not dates.  According to MDN:

The displayed date format will differ from the actual value — the displayed date is formatted based on the locale of the user's browser, but the parsed value is always formatted yyyy-mm-dd.

So for comparison purposes this should still work, since yyyy-mm-dd is an incrementing value.  But if you need a Date object then MDN also says there's a .valueAsDate property (though has no example on that page), as well as a .valueAsNumber which returns a UNIX timestamp that should be reliable for creating a Date.

Otherwise dates can be tricky in JavaScript, and libraries like Moment.js exist to make that easier.

Body:
hello so i have a javascript function that takes two dates from two date inputs, compares them and returns a boolean value except it doesn't work

function checkdate() {

 var datefrom = new Date();
 var dateto = new Date();

 datefrom = getElementById("date_from").innerHTML;
 dateto = getElementById("date_to").innerHTML;

 if (datefrom > dateto) {
  alert("Dateto must be bigger than datefrom  ");

  return false;
 } else return true;

}

<form onsubmit="return checkdate()" class="forms-sample" method="post"
   action="addstage.php">

      Date
      Début

         <input type="date" class="form-control"
            id="date_from" placeholder="Debut" name="datedebut"  required="required">

      Date
      Fin

         <input type="date" class="form-control"
            id="date_to" placeholder="Fin" name="datefin" required="required">

   <button type="submit" class="btn btn-primary mr-2" value="addStage"
      name="action">Add</button>
</form>

Body:
Two problems:

You're calling getElementById globally, but should be calling it on the document object.

You're using .innerHtml instead of .value

As an aside you also don't need to create new Date() objects just to throw them away.  Simply set the values you want right away:

var datefrom = document.getElementById("date_from").value;
var dateto = document.getElementById("date_to").value;

Note: These values are strings, not dates.  According to MDN:

The displayed date format will differ from the actual value — the displayed date is formatted based on the locale of the user's browser, but the parsed value is always formatted yyyy-mm-dd.

So for comparison purposes this should still work, since yyyy-mm-dd is an incrementing value.  But if you need a Date object then MDN also says there's a .valueAsDate property (though has no example on that page), as well as a .valueAsNumber which returns a UNIX timestamp that should be reliable for creating a Date.

Otherwise dates can be tricky in JavaScript, and libraries like Moment.js exist to make that easier.


问:
Bug in Numpy ndarray min/max method
I'm assuming I'm doing something wrong here, but I'm working on a project in Pycharm, which notified me when using the ndarray.max() function that initial was undefined (parameter 'initial' unfilled). Looking at the documentation, it does show that there is no default value for initial argument.

When ctrl-clicking the ndarray.max() function in Pycharm, opens the following function:

```<code>    def max(self, axis=None, out=None, keepdims=False, initial, *args, **kwargs): # real signature unknown; NOTE: unreliably restored from __doc__
    """
    a.max(axis=None, out=None, keepdims=False, initial=<no value>, where=True)

        Return the maximum along a given axis.

        Refer to `numpy.amax` for full documentation.

        See Also
        --------
        numpy.amax : equivalent function
    """
    pass
</code>```

Which appears to not even do anything. Either way, the code works, only an IDE error is given.

Am I using the wrong function? I know there's amax and max, as well as the package level numpy.max, but the above seems to be unwanted behaviour.

If this is a bug, I wouldn't know how to report it / start an issue or whatever, haha.

答:
In an Ipython session, numpy 1.18,  np.max?? (like the charm click?) shows:

```<code>def amax(a, axis=None, out=None, keepdims=np._NoValue,
     initial=np._NoValue, where=np._NoValue):
     .... docs

     return _wrapreduction(a, np.maximum, 'max', axis, None, out,
                      keepdims=keepdims, initial=initial, where=where)

File:      /usr/local/lib/python3.6/dist-packages/numpy/core/fromnumeric.py
</code>```

So initial is given this unique numpy constant ._NoValue.  The actual action is equivalent to

```<code> np.maximum.reduce(a, ....)
</code>```

initial was added in version 1.15.  np.maximum.reduce?? docs gives more information on initial (which is generally applicable to binary ufunc).

Title: Bug in Numpy ndarray min/max method

Body:
I'm assuming I'm doing something wrong here, but I'm working on a project in Pycharm, which notified me when using the ndarray.max() function that initial was undefined (parameter 'initial' unfilled). Looking at the documentation, it does show that there is no default value for initial argument.

When ctrl-clicking the ndarray.max() function in Pycharm, opens the following function:

    def max(self, axis=None, out=None, keepdims=False, initial, *args, **kwargs): # real signature unknown; NOTE: unreliably restored from __doc__
    """
    a.max(axis=None, out=None, keepdims=False, initial=<no value>, where=True)

        Return the maximum along a given axis.

        Refer to `numpy.amax` for full documentation.

        See Also
        --------
        numpy.amax : equivalent function
    """
    pass

Which appears to not even do anything. Either way, the code works, only an IDE error is given.

Am I using the wrong function? I know there's amax and max, as well as the package level numpy.max, but the above seems to be unwanted behaviour.

If this is a bug, I wouldn't know how to report it / start an issue or whatever, haha.

Body:
In an Ipython session, numpy 1.18,  np.max?? (like the charm click?) shows:

def amax(a, axis=None, out=None, keepdims=np._NoValue,
     initial=np._NoValue, where=np._NoValue):
     .... docs

     return _wrapreduction(a, np.maximum, 'max', axis, None, out,
                      keepdims=keepdims, initial=initial, where=where)

File:      /usr/local/lib/python3.6/dist-packages/numpy/core/fromnumeric.py

So initial is given this unique numpy constant ._NoValue.  The actual action is equivalent to

 np.maximum.reduce(a, ....)

initial was added in version 1.15.  np.maximum.reduce?? docs gives more information on initial (which is generally applicable to binary ufunc).


问:
proper phpdoc @return for factory method
I am curious how to document the following scenario. Imagine a group of classes:

```<code>abstract class Personality {}
class Me extends Personality {}
class Myself extends Personality {}
class Irene extends Personality {}
</code>```

If I am writing a factory-esque method to instantiate and return a subclass of class Personality, how would I properly document the @return?

```<code>@return mixed  A "Personality" subclass object
</code>```

or

```<code>@return Personality  A "Personality" subclass object
</code>```

答:
```<code>@return Personality A "Personality" subclass object
</code>```

is the best. And IDE will provide you with autocomplete properties for the Personality class.

Body:
I am curious how to document the following scenario. Imagine a group of classes:

abstract class Personality {}
class Me extends Personality {}
class Myself extends Personality {}
class Irene extends Personality {}

If I am writing a factory-esque method to instantiate and return a subclass of class Personality, how would I properly document the @return?

@return mixed  A "Personality" subclass object

or

@return Personality  A "Personality" subclass object

Body:
@return Personality A "Personality" subclass object

is the best. And IDE will provide you with autocomplete properties for the Personality class.


问:
SAS-Append data to a table and increment a flag on the resultant table
I have two tables on SAS "Table_A" and "Table_A_Archive", as part my ETL process "Table_A" is created on a daily basis and the data should be archived on "Table_A_Archive". When the data is archived on "Table_A_Archive" a a flag will be created/updated "DT_FLAG".

On the first day this is how the table would look like

```<code> "Table_A"
 | ID  | Load_Date
 ------ -------------
 | 100 | 01JUN2020:12:13:56

 "Table_A_Archive"
  | ID  | Load_Date           | DT_FLAG
  ------ --------------------- ---------
  | 100 | 01JUN2020:12:13:56  | 1
</code>```

On Day 2

```<code> "Table_A"
 | ID  | Load_Date
 ------ ------------
 | 101 | 02JUN2020:12:13:56

 "Table_A_Archive"
 | ID  | Load_Date           | DT_FLAG
 ------ --------------------- ---------
 | 100 | 01JUN2020:12:13:56  | 2
 | 101 | 02JUN2020:12:13:56  | 1
</code>```

The new data should be loaded with the DT_FLAG of 1 and old records DT_FLAG should be incremented by 1. Load_Date is the pivotal point. I have written a SAS code but it seems a bit messy, can someone please help me with a SAS Datastep

```<code> %macro Cntl_archive(table_name=,arch_table_name=);
 %GLOBAL WRK;

 %if %sysfunc(exist(&arch_table_name.))  %then %do;

 proc append base=&arch_table_name. data=&table_name. force;
 run;

 proc sql;
 Create table TEMP as
 Select distinct Load_Date,Load_Date as WRK from &arch_table_name.
 order by Load_Date desc
 ;quit;

 proc rank data=TEMP descending out=TEMP;
 var WRK;
 ranks count;
 run;

 data &arch_table_name. (drop=DT_FLAG);
 set    &arch_table_name.;
 run;

 proc sql;
 Create table &arch_table_name. as
 Select T0.*,T1.count as DT_FLAG from &arch_table_name. T0
 inner join TEMP T1 on T0.Load_Date=T1.Load_Date
 ;quit
 %end;
 %else %do;

data &arch_table_name.;
set &table_name.;
DT_FLAG= 1;
IS_ACTIVE='';
run;
%end;
%mend Cntl_archive;
</code>```

答:
Here is a method using the MODIFY statement to update the values of DT_FLAG in the existing observations in place and append the new values.

First lets make the initial A and use it to create an empty A_ARCHIVE with the extra variable. (Note I renamed your timestamp variable to avoid the confusion caused by having a variable named "date" that has datetime values instead of date values.)

```<code>data a ;
  input id load_dt :datetime.;
  format load_dt datetime19.;
cards;
100 01JUN2020:12:13:56
;

data a_archive;
  stop;
  set a ;
  dt_flag=0;
run;
</code>```

Now let's append A to A_ARCHIVE.

```<code>data a_archive;
  do while(not eof1);
    modify a_archive end=eof1;
    dt_flag=sum(dt_flag,1);
    replace;
  end;
  do until(eof2);
    set a end=eof2;
    dt_flag=1;
    output;
  end;
run;
</code>```

Now you can make a new version of A and re-run the same data step to append it.

```<code>data a ;
  input id load_dt :datetime.;
  format load_dt datetime19.;
cards;
101 02JUN2020:12:13:56
;

data a_archive;
  do while(not eof1);
    modify a_archive end=eof1;
    dt_flag=sum(dt_flag,1);
    replace;
  end;
  do until(eof2);
    set a end=eof2;
    dt_flag=1;
    output;
  end;
run;
</code>```

Results:

```<code>Obs     id                load_dt  dt_flag

 1     100     01JUN2020:12:13:56       2
 2     101     02JUN2020:12:13:56       1
</code>```

Body:
I have two tables on SAS "Table_A" and "Table_A_Archive", as part my ETL process "Table_A" is created on a daily basis and the data should be archived on "Table_A_Archive". When the data is archived on "Table_A_Archive" a a flag will be created/updated "DT_FLAG".

On the first day this is how the table would look like

 "Table_A"
 | ID  | Load_Date
 ------ -------------
 | 100 | 01JUN2020:12:13:56

 "Table_A_Archive"
  | ID  | Load_Date           | DT_FLAG
  ------ --------------------- ---------
  | 100 | 01JUN2020:12:13:56  | 1

On Day 2

 "Table_A"
 | ID  | Load_Date
 ------ ------------
 | 101 | 02JUN2020:12:13:56

 "Table_A_Archive"
 | ID  | Load_Date           | DT_FLAG
 ------ --------------------- ---------
 | 100 | 01JUN2020:12:13:56  | 2
 | 101 | 02JUN2020:12:13:56  | 1

The new data should be loaded with the DT_FLAG of 1 and old records DT_FLAG should be incremented by 1. Load_Date is the pivotal point. I have written a SAS code but it seems a bit messy, can someone please help me with a SAS Datastep

 %macro Cntl_archive(table_name=,arch_table_name=);
 %GLOBAL WRK;

 %if %sysfunc(exist(&arch_table_name.))  %then %do;

 proc append base=&arch_table_name. data=&table_name. force;
 run;

 proc sql;
 Create table TEMP as
 Select distinct Load_Date,Load_Date as WRK from &arch_table_name.
 order by Load_Date desc
 ;quit;

 proc rank data=TEMP descending out=TEMP;
 var WRK;
 ranks count;
 run;

 data &arch_table_name. (drop=DT_FLAG);
 set    &arch_table_name.;
 run;

 proc sql;
 Create table &arch_table_name. as
 Select T0.*,T1.count as DT_FLAG from &arch_table_name. T0
 inner join TEMP T1 on T0.Load_Date=T1.Load_Date
 ;quit
 %end;
 %else %do;

data &arch_table_name.;
set &table_name.;
DT_FLAG= 1;
IS_ACTIVE='';
run;
%end;
%mend Cntl_archive;

Body:
Here is a method using the MODIFY statement to update the values of DT_FLAG in the existing observations in place and append the new values.

First lets make the initial A and use it to create an empty A_ARCHIVE with the extra variable. (Note I renamed your timestamp variable to avoid the confusion caused by having a variable named "date" that has datetime values instead of date values.)

data a ;
  input id load_dt :datetime.;
  format load_dt datetime19.;
cards;
100 01JUN2020:12:13:56
;

data a_archive;
  stop;
  set a ;
  dt_flag=0;
run;

Now let's append A to A_ARCHIVE.

data a_archive;
  do while(not eof1);
    modify a_archive end=eof1;
    dt_flag=sum(dt_flag,1);
    replace;
  end;
  do until(eof2);
    set a end=eof2;
    dt_flag=1;
    output;
  end;
run;

Now you can make a new version of A and re-run the same data step to append it.

data a ;
  input id load_dt :datetime.;
  format load_dt datetime19.;
cards;
101 02JUN2020:12:13:56
;

data a_archive;
  do while(not eof1);
    modify a_archive end=eof1;
    dt_flag=sum(dt_flag,1);
    replace;
  end;
  do until(eof2);
    set a end=eof2;
    dt_flag=1;
    output;
  end;
run;

Results:

Obs     id                load_dt  dt_flag

 1     100     01JUN2020:12:13:56       2
 2     101     02JUN2020:12:13:56       1


问:
T-SQL query to show table definition?
What is a query that will show me the full definition, including indexes and keys for a SQL Server table?  I want a pure query - and know that SQL Studio can give this to me, but I am often on "wild" computers that have only the most bare-bones apps and I have no rights to install studio.  But SQLCMD is always an option.

UPDATE:  I have tried sp_help, but is just yields one record which shows Name, Owner, Type and Created_Datetime.  Is there something else I am missing with sp_help?

Here is what I call:

sp_help airports

Note that I really do want the DDL that defines the table.

答:
This will return columns, datatypes, and indexes defined on the table:

```<code>--List all tables in DB
select * from sysobjects where xtype = 'U'

--Table Definition
sp_help TableName
</code>```

This will return triggers defined on the table:

```<code>--Triggers in SQL Table
select * from sys.triggers where parent_id = object_id(N'SQLTableName')
</code>```

Title: T-SQL query to show table definition?

Body:
This will return columns, datatypes, and indexes defined on the table:

--List all tables in DB
select * from sysobjects where xtype = 'U'

--Table Definition
sp_help TableName

This will return triggers defined on the table:

--Triggers in SQL Table
select * from sys.triggers where parent_id = object_id(N'SQLTableName')


问:
DeeplabV3 MobileNetV2 Quantization for USB Coral TPU with custom input size ((100, 100) rather than (513, 513))
I have been following the steps in https://github.com/tensorflow/models/blob/394baa9f21424d3522ccfbdcee8acd3840891ff6/research/deeplab/g3doc/quantize.md to quantize a DeeplabV3 MobileNetV2 model for the USB Coral TPU but I cannot get it to work.

Even when I start from the models provided at the bottom (not the 8_bits, the raw models), I cannot get it to run. And I even get different problem for both models.

For the deeplabv3_mnv2_pascal_train_aug:

- The deeplab/train.py script for quantizing runs successfully.

- The deeplab/export_model.py scripts step works.

- The tflite_convert works too.

- EdgeTPU compiler gets cryptic error message: Internal compiler error. Aborting!

For the deeplabv3_mnv2_dm05_pascal_trainaug:
- The deeplab/train.py script for quantizing runs gets:
ValueError: Total size of new array must be unchanged for MobilenetV2/Conv/weights lh_shape: [(3, 3, 3, 16)], rh_shape: [(3, 3, 3, 32)]
I tried this solution unsuccessfully.

Using the .pb file directly, the deeplab/train.py with quantization aware training fails with python crashing.

Doing a post-quantization in python directly works.

edgetpu_compiler works.

But once ran on the TPU, we get 0.87 TPS, most likely because the edgetpu-converter cannot convert all to edgtpu.

Any guidance to get it working would be great. I am also open to solutions not following the guide from tensorflow.

答:
I believe you've reached out to us and my colleague already shared the procedure, but I'll sahre it here for other to reference:</p>

```<code>python deeplab/export_model.py \
    --checkpoint_path=../../deeplabv3_mnv2_dm05_pascal_trainaug_8bit/model.ckpt \
    --quantize_delay_step=0 \
    --crop_size=55 \
    --crop_size=55 \
    --model_variant="mobilenet_v2" \
    --decoder_output_stride="" \
    --export_path=../../deeplabv3_mnv2_dm05_pascal_trainaug_8bit/frozen_inference_graph.pb \
    --depth_multiplier=0.5
</code>```

```<code>tflite_convert \
  --graph_def_file=../../frozen_inference_graph.pb \
  --output_file=../../frozen_inference_graph.tflite \
  --output_format=TFLITE \
  --input_shape=1,55,55,3 \
  --input_arrays="MobilenetV2/MobilenetV2/input" \
  --inference_type=QUANTIZED_UINT8 \
  --inference_input_type=QUANTIZED_UINT8 \
  --std_dev_values=128 \
  --mean_values=128 \
  --change_concat_input_ranges=true \
  --output_arrays="ArgMax"
</code>```

Body:
I believe you've reached out to us and my colleague already shared the procedure, but I'll sahre it here for other to reference:

python deeplab/export_model.py \
    --checkpoint_path=../../deeplabv3_mnv2_dm05_pascal_trainaug_8bit/model.ckpt \
    --quantize_delay_step=0 \
    --crop_size=55 \
    --crop_size=55 \
    --model_variant="mobilenet_v2" \
    --decoder_output_stride="" \
    --export_path=../../deeplabv3_mnv2_dm05_pascal_trainaug_8bit/frozen_inference_graph.pb \
    --depth_multiplier=0.5

tflite_convert \
  --graph_def_file=../../frozen_inference_graph.pb \
  --output_file=../../frozen_inference_graph.tflite \
  --output_format=TFLITE \
  --input_shape=1,55,55,3 \
  --input_arrays="MobilenetV2/MobilenetV2/input" \
  --inference_type=QUANTIZED_UINT8 \
  --inference_input_type=QUANTIZED_UINT8 \
  --std_dev_values=128 \
  --mean_values=128 \
  --change_concat_input_ranges=true \
  --output_arrays="ArgMax"


问:
How to apply a css class in a element that hasn't been rendered yet?
I need to add a css class to an element that when the page is loaded is still not created. It gets created only when the dropdown is open. This is my code below where I loop through a list and add a class to the elements but, as I said, the class '.dropdown-popup-row' is not found because the drop starts closed.

What I'm doing at the moment is verify if '.dropdown-list' exists in the body (if so, it means that the dropdown is open)

```<code>const str = "[0,1,2]";
const arr = JSON.parse(str);

var dropdownList = document.querySelector('.dropdown-list');

if (document.body.contains(dropdownList)) {
    var dropdownElements = document.querySelectorAll('.dropdown-popup-row');

    if(dropdownElements) {

    for(var i = 0; i < arr.length; i++) {
        var disabledIndex = arr[i];
        dropdownElements[disabledIndex].classList.add("-disabled");
    }
}
}
</code>```

this works fine in the browser console with the dropdown open (therefore .dropdown-list is rendered and available in the DOM). I can't figure out a way to add this class to this element. The error I've been getting is: "Uncaught TypeError: Cannot read property 'classList' of undefined".

Thank you guys!

答:
Why don't you add that class when that element is being created?
Is it created by an external library ?</p>

Body:
I need to add a css class to an element that when the page is loaded is still not created. It gets created only when the dropdown is open. This is my code below where I loop through a list and add a class to the elements but, as I said, the class '.dropdown-popup-row' is not found because the drop starts closed.

What I'm doing at the moment is verify if '.dropdown-list' exists in the body (if so, it means that the dropdown is open)

const str = "[0,1,2]";
const arr = JSON.parse(str);

var dropdownList = document.querySelector('.dropdown-list');

if (document.body.contains(dropdownList)) {
    var dropdownElements = document.querySelectorAll('.dropdown-popup-row');

    if(dropdownElements) {

    for(var i = 0; i < arr.length; i++) {
        var disabledIndex = arr[i];
        dropdownElements[disabledIndex].classList.add("-disabled");
    }
}
}

this works fine in the browser console with the dropdown open (therefore .dropdown-list is rendered and available in the DOM). I can't figure out a way to add this class to this element. The error I've been getting is: "Uncaught TypeError: Cannot read property 'classList' of undefined".

Thank you guys!


问:
Task timed out when trying to use SES SendEmail operation
I'm trying to send an email message using AWS SES with Python 3.7 lambda function.

When I'm trying to test the function and see if it sends the email message, I get a message that the task timed out.

It reaches to the code part where it sends the message, but the message isn't being sent at any time, and the task just got timeout.

This is the code that I'm using to send the message:

```<code>from __future__ import print_function
import boto3
import json
import decimal
from datetime import datetime
from boto3.dynamodb.conditions import Key, Attr
from botocore.exceptions import ClientError

ses = boto3.client(
    'ses',
    region_name = 'us-east-1',
    endpoint_url = 'https://email.us-east-1.amazonaws.com'
)

try:
    response = ses.send_email(
        Destination = {
            'ToAddresses': [
                email
            ],
        },
        Message = {
            'Body': {
                'Html': {
                    'Charset': CHARSET,
                    'Data': BODY_HTML
                },
                'Text': {
                    'Charset': CHARSET,
                    'Data': BODY_TEXT
                },
            },
            'Subject': {
                'Charset': CHARSET,
                'Data': SUBJECT
            }
        },
        Source = SENDER
    )
except ClientError as e:
    print(e)
else:
    print('Email sent! Message ID:'),
    print(response['MessageId'])
</code>```

All of the above variables are hardcoded in the code, and definitely has value (I have printed it to the console to ensure it).

答:
I tried unsuccessfully for a long time to get this to work with a boto3 SES client and just couldn't do it, even after creating an SES endpoint (my lambda is in a VPC). It kept timing out without any other error. In case this helps anyone else, I switched to using smtplib as explained here: https://www.authsmtp.com/python/index.html. I don't know if there is a downside to this approach; I only know that it worked for me.

```<code>import boto3
import json
from email.mime.multipart import MIMEMultipart
from email.mime.text import MIMEText
import smtplib
</code>```

and then (I have the SES SMTP credentials stored in AWS Secrets Manager):

```<code># get SES smtp email credentials from Secrets Manager
client2 = boto3.client('secretsmanager')
response = client2.get_secret_value(
SecretId='secretIDhere'
)
secretDict = json.loads(response['SecretString'])
smtpusername = secretDict['username']
smtppassword = secretDict['password']
smtphost = 'email-smtp.us-east-1.amazonaws.com:25'
</code>```

and finally

```<code># send email
msg = MIMEMultipart()
message = "Message text here"
msg['From'] = "email address here"
msg['To'] = "email address here"
msg['Subject'] = "Subject line here"
msg.attach(MIMEText(message, 'plain'))
server = smtplib.SMTP(smtphost)
server.starttls()
server.login(smtpusername, smtppassword)
server.sendmail(msg['From'], msg['To'], msg.as_string())
server.quit()
</code>```

Body:
I'm trying to send an email message using AWS SES with Python 3.7 lambda function.

When I'm trying to test the function and see if it sends the email message, I get a message that the task timed out.

It reaches to the code part where it sends the message, but the message isn't being sent at any time, and the task just got timeout.

This is the code that I'm using to send the message:

from __future__ import print_function
import boto3
import json
import decimal
from datetime import datetime
from boto3.dynamodb.conditions import Key, Attr
from botocore.exceptions import ClientError

ses = boto3.client(
    'ses',
    region_name = 'us-east-1',
    endpoint_url = 'https://email.us-east-1.amazonaws.com'
)

try:
    response = ses.send_email(
        Destination = {
            'ToAddresses': [
                email
            ],
        },
        Message = {
            'Body': {
                'Html': {
                    'Charset': CHARSET,
                    'Data': BODY_HTML
                },
                'Text': {
                    'Charset': CHARSET,
                    'Data': BODY_TEXT
                },
            },
            'Subject': {
                'Charset': CHARSET,
                'Data': SUBJECT
            }
        },
        Source = SENDER
    )
except ClientError as e:
    print(e)
else:
    print('Email sent! Message ID:'),
    print(response['MessageId'])

All of the above variables are hardcoded in the code, and definitely has value (I have printed it to the console to ensure it).

Body:
I tried unsuccessfully for a long time to get this to work with a boto3 SES client and just couldn't do it, even after creating an SES endpoint (my lambda is in a VPC). It kept timing out without any other error. In case this helps anyone else, I switched to using smtplib as explained here: https://www.authsmtp.com/python/index.html. I don't know if there is a downside to this approach; I only know that it worked for me.

import boto3
import json
from email.mime.multipart import MIMEMultipart
from email.mime.text import MIMEText
import smtplib

and then (I have the SES SMTP credentials stored in AWS Secrets Manager):

# get SES smtp email credentials from Secrets Manager
client2 = boto3.client('secretsmanager')
response = client2.get_secret_value(
SecretId='secretIDhere'
)
secretDict = json.loads(response['SecretString'])
smtpusername = secretDict['username']
smtppassword = secretDict['password']
smtphost = 'email-smtp.us-east-1.amazonaws.com:25'

and finally

# send email
msg = MIMEMultipart()
message = "Message text here"
msg['From'] = "email address here"
msg['To'] = "email address here"
msg['Subject'] = "Subject line here"
msg.attach(MIMEText(message, 'plain'))
server = smtplib.SMTP(smtphost)
server.starttls()
server.login(smtpusername, smtppassword)
server.sendmail(msg['From'], msg['To'], msg.as_string())
server.quit()


问:
Angular 9 Redirect if query param not provided
I have a route that should only be accessible to a user when they click a link from an email and are redirected with a valid query parameter in the url. If the parameter is not provided, I want to redirect to a 404 page not found component, like follows.

```<code>this.route.queryParams.subscribe(params => {
      if (params.myParam) {
        //do something
      } else {
        // redirect to 404
      }
    })
</code>```

The issue I have is that query params is initialized to an empty object in ngOnInit(). I have utilized rjxs to wait until the parameter is accessible like this

```<code>this.route.queryParamMap.pipe(
      filter(paramMap => paramMap.has('myParam')),
      map(paramMap => paramMap.get('myParam')),
      take(1)
    ).subscribe(myParam => doSomething(myParam));
</code>```

But I'm not very well versed in rxjs operators, and now I'm stuck on how to actually redirect if  paramMap has finished initializing and the parameter isn't found.

答:
this is a problem in many apps, as the params and query params are implemented as behavior subjects, get around this by waiting for the NavigationEnd event, where the query params will definitely already be set...

```<code>   const navEnd$ = this.router.events.pipe(filter(e => e instanceof NavigationEnd));
   navEnd$.pipe(withLatestFrom(this.route.queryParams)).subscribe(
     ([navEnd, queryParams]) => {
       if (queryParams.myParam) {
         // do the thing
       } else {
         // navigate
       }
     }
   )
</code>```

Body:
I have a route that should only be accessible to a user when they click a link from an email and are redirected with a valid query parameter in the url. If the parameter is not provided, I want to redirect to a 404 page not found component, like follows.

this.route.queryParams.subscribe(params => {
      if (params.myParam) {
        //do something
      } else {
        // redirect to 404
      }
    })

The issue I have is that query params is initialized to an empty object in ngOnInit(). I have utilized rjxs to wait until the parameter is accessible like this

this.route.queryParamMap.pipe(
      filter(paramMap => paramMap.has('myParam')),
      map(paramMap => paramMap.get('myParam')),
      take(1)
    ).subscribe(myParam => doSomething(myParam));

But I'm not very well versed in rxjs operators, and now I'm stuck on how to actually redirect if  paramMap has finished initializing and the parameter isn't found.

Body:
this is a problem in many apps, as the params and query params are implemented as behavior subjects, get around this by waiting for the NavigationEnd event, where the query params will definitely already be set...

   const navEnd$ = this.router.events.pipe(filter(e => e instanceof NavigationEnd));
   navEnd$.pipe(withLatestFrom(this.route.queryParams)).subscribe(
     ([navEnd, queryParams]) => {
       if (queryParams.myParam) {
         // do the thing
       } else {
         // navigate
       }
     }
   )


问:
-Namespace option is not recognized in Scaffold-DbContext
According to MS Docs, Scaffold-DbContext should accpect a optional parameter -Namespace. However, seems that Scaffold-DbContext cannot recognize this parameter. Did I miss something?

Command Error:

```<code>Scaffold-DbContext "Server=.\SQLExpress;Database=Stock;Trusted_Connection=True;User Id=sa;Password=P@ssw0rd;" Microsoft.EntityFrameworkCore.SqlServer -Namespace Stock.Models.DTO

Scaffold-DbContext : A parameter cannot be found that matches parameter name 'Namespace'.
At line:1 char:151
+ ... P@ssw0rd;" Microsoft.EntityFrameworkCore.SqlServer -Namespace Stock.M ...
+                                                        ~~~~~~~~~~
    + CategoryInfo          : InvalidArgument: (:) [Scaffold-DbContext], ParameterBindingException
    + FullyQualifiedErrorId : NamedParameterNotFound,Scaffold-DbContext

</code>```

Packages installed:

```<code>  <ItemGroup>
    <PackageReference Include="Microsoft.EntityFrameworkCore" Version="3.1.4" />
    <PackageReference Include="Microsoft.EntityFrameworkCore.Design" Version="3.1.4">
      <PrivateAssets>all</PrivateAssets>
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
    </PackageReference>
    <PackageReference Include="Microsoft.EntityFrameworkCore.Relational.Design" Version="1.1.6" />
    <PackageReference Include="Microsoft.EntityFrameworkCore.SqlServer" Version="3.1.4" />
    <PackageReference Include="Microsoft.EntityFrameworkCore.Tools" Version="3.1.4">
      <PrivateAssets>all</PrivateAssets>
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
    </PackageReference>
  </ItemGroup>
</code>```

答: -Namespace works EF Core 5.0 onwards. your version is lower than that. Please upgrade EF Core

Body:
According to MS Docs, Scaffold-DbContext should accpect a optional parameter -Namespace. However, seems that Scaffold-DbContext cannot recognize this parameter. Did I miss something?

Command Error:

Scaffold-DbContext "Server=.\SQLExpress;Database=Stock;Trusted_Connection=True;User Id=sa;Password=P@ssw0rd;" Microsoft.EntityFrameworkCore.SqlServer -Namespace Stock.Models.DTO

Scaffold-DbContext : A parameter cannot be found that matches parameter name 'Namespace'.
At line:1 char:151
+ ... P@ssw0rd;" Microsoft.EntityFrameworkCore.SqlServer -Namespace Stock.M ...
+                                                        ~~~~~~~~~~
    + CategoryInfo          : InvalidArgument: (:) [Scaffold-DbContext], ParameterBindingException
    + FullyQualifiedErrorId : NamedParameterNotFound,Scaffold-DbContext

Packages installed:

  <ItemGroup>
    <PackageReference Include="Microsoft.EntityFrameworkCore" Version="3.1.4" />
    <PackageReference Include="Microsoft.EntityFrameworkCore.Design" Version="3.1.4">
      <PrivateAssets>all</PrivateAssets>
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
    </PackageReference>
    <PackageReference Include="Microsoft.EntityFrameworkCore.Relational.Design" Version="1.1.6" />
    <PackageReference Include="Microsoft.EntityFrameworkCore.SqlServer" Version="3.1.4" />
    <PackageReference Include="Microsoft.EntityFrameworkCore.Tools" Version="3.1.4">
      <PrivateAssets>all</PrivateAssets>
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
    </PackageReference>
  </ItemGroup>


问:
insert 0 rows without error
Can an insert ever execute without error but not insert anything?  I am using sql server 2008 and wondering if I can get away with just checking @@error or is there something I am missing.  For an update I check @@error and @@rowcount.  Having @@rowcount = 0 for an insert just seems strange to me.

edit
@Gregory  I basically wondering how I should error check an insert statement.  Are there any strange boundary cases where an insert execute and @@error is 0?

答:
You can run an INSERT command using a select that returns an empty table.

The statement will succeed but no rows updated.

```<code>INSERT INTO myTable (col1, col2)
SELECT col1, col2
FROM myOtherTable
WHERE 1 = 2
</code>```

Title: insert 0 rows without error

Body:
You can run an INSERT command using a select that returns an empty table.

The statement will succeed but no rows updated.

INSERT INTO myTable (col1, col2)
SELECT col1, col2
FROM myOtherTable
WHERE 1 = 2


问:
What's the best way to do request multiple data from Firebase in React
I was wondering what would be the best way to get multiple data from Firebase in an async function to wait for some data from the first request. I'm using this code right now but it's not reliable and it breaks sometimes saying that it can't fetch the data for the second call as it's undefined.

```<code>function useOccasion() {
  const [occasionData, setOccasionData] = useState(null)
  const [friend, setFriend] = useState(null)
  let { occasion } = useParams()

  useEffect(() => {
    const unsubscribe = firestore.collection('occasions').doc(occasion)
      .onSnapshot(async eventData => {
        setOccasionData({id: eventData.id, ...eventData.data()})

        let friendData = await firestore.collection("friends").doc(eventData.data().friend).get();
        setFriend({id: friendData.id, ...friendData.data()});
      })
    return () => unsubscribe()
  }, [occasion])

  return [occasionData, friend]
}
</code>```

If there's a more robust way to achieve this that would be amazing.

答:
I'd suggest breaking your data fetches into two hooks, one for each collection occasion and friends.

I'm not sure how you've set firebase up, but I access it through context.

e.g.

```<code>const useOccasion = () => {
  const firebase = useContext(FirebaseContext)

  const [occasions, setOccasions] = useState([])
  const [loading, setLoading] = useState(true)

  useEffect(() => {
    const unsubscribe = firebase.db.collection('occasions')
    .onSnapshot(snapshot => {
      if (snapshot.size) {
        let occasionList = []
        snapshot.forEach(doc =>
          occasionList.push({ ...doc.data(), uid: doc.id }),
        )

        setOccasions(occasionList)
        setLoading(false)
      } else {
        setOccasions([])
        setLoading(false)
      }
    })

    return () => {
      unsubscribe()
    }
  }, [])

  return { occasions, loading }
}
</code>```

Finally, in your component where you require the data you can access this hook:

```<code>const { occasions, loading } = useOccasion()
</code>```

Body:
I was wondering what would be the best way to get multiple data from Firebase in an async function to wait for some data from the first request. I'm using this code right now but it's not reliable and it breaks sometimes saying that it can't fetch the data for the second call as it's undefined.

function useOccasion() {
  const [occasionData, setOccasionData] = useState(null)
  const [friend, setFriend] = useState(null)
  let { occasion } = useParams()

  useEffect(() => {
    const unsubscribe = firestore.collection('occasions').doc(occasion)
      .onSnapshot(async eventData => {
        setOccasionData({id: eventData.id, ...eventData.data()})

        let friendData = await firestore.collection("friends").doc(eventData.data().friend).get();
        setFriend({id: friendData.id, ...friendData.data()});
      })
    return () => unsubscribe()
  }, [occasion])

  return [occasionData, friend]
}

If there's a more robust way to achieve this that would be amazing.

Body:
I'd suggest breaking your data fetches into two hooks, one for each collection occasion and friends.

I'm not sure how you've set firebase up, but I access it through context.

e.g.

const useOccasion = () => {
  const firebase = useContext(FirebaseContext)

  const [occasions, setOccasions] = useState([])
  const [loading, setLoading] = useState(true)

  useEffect(() => {
    const unsubscribe = firebase.db.collection('occasions')
    .onSnapshot(snapshot => {
      if (snapshot.size) {
        let occasionList = []
        snapshot.forEach(doc =>
          occasionList.push({ ...doc.data(), uid: doc.id }),
        )

        setOccasions(occasionList)
        setLoading(false)
      } else {
        setOccasions([])
        setLoading(false)
      }
    })

    return () => {
      unsubscribe()
    }
  }, [])

  return { occasions, loading }
}

Finally, in your component where you require the data you can access this hook:

const { occasions, loading } = useOccasion()