问:
What is the difference between kubectl apply and kubectl replace
I am learning Kubernetes recently, and I am not very clear about the difference between "kubectl apply" and "kubectl replace". Is there any situation that we can only use one of them?</p>

答:
I have written up a thorough explanation of the differences between apply, replace, and patch: Kubernetes Apply vs. Replace vs. Patch. It includes an explanation that the current top-ranked answer to this question is wrong.

Briefly, kubectl apply uses the provided spec to create a resource if it does not exist and update, i.e., patch, it if it does.  The spec provided to apply need only contain the required parts of a spec, when creating a resource the API will use defaults for the rest and when updating a resource it will use its current values.

The kubectl replace completely replaces the existing resource with the one defined by the provided spec. replace wants a complete spec as input, including read-only properties supplied by the API like .metadata.resourceVersion, .spec.nodeName for pods, .spec.clusterIP for services, and .secrets for service accounts. kubectl has some internal tricks to help you get that right, but typically the use case for replace is getting a resource spec, changing a property, and then using that changed, complete spec to replace the existing resource.

The kubectl replace command has a --force option which actually does not use the replace, i.e., PUT, API endpoint. It forcibly deletes (DELETE) and then recreates, (POST) the resource using the provided spec.


来源: stackexchange

问:
Applying multiple filters at once with FFMPEG
I have the need to apply fadein and overlay filters to a video. Is it possible to apply 2 filters at once?

I got:

```<code>ffmpeg -i input.mpg -vf "movie=watermark.png [logo]; [in][logo] overlay=W-w-10:H-h-10 [out]" output.mpg
</code>```

I'm trying to add fade=in:0:20, but if I add a new -vf parameter, it will overwrite the preceding one, and if I add:

```<code>-vf "fade=in:0:20; movie=......"
</code>```

it won't work.

Is this possible or do I have to run FFmpeg twice?

答:
Okay, someone helped me somewhere.

I had to separate filters with commas:

```<code>ffmpeg -i input.mpg -vf "movie=watermark.png [logo]; [in][logo] overlay=W-w-10:H-h-10, fade=in:0:20 [out]" output.mpg
</code>```

This will apply fadein to both the watermark and the video.

ContentLicense: CC BY-SA 3.0

Body:
I have the need to apply fadein and overlay filters to a video. Is it possible to apply 2 filters at once?

I got:

ffmpeg -i input.mpg -vf "movie=watermark.png [logo]; [in][logo] overlay=W-w-10:H-h-10 [out]" output.mpg

I'm trying to add fade=in:0:20, but if I add a new -vf parameter, it will overwrite the preceding one, and if I add:

-vf "fade=in:0:20; movie=......"

it won't work.

Is this possible or do I have to run FFmpeg twice?

Body:
Okay, someone helped me somewhere.

I had to separate filters with commas:

ffmpeg -i input.mpg -vf "movie=watermark.png [logo]; [in][logo] overlay=W-w-10:H-h-10, fade=in:0:20 [out]" output.mpg

This will apply fadein to both the watermark and the video.


问:
How to run script with boost::process::system?
My script:

ps aux | awk '{s += $3} END {print s}'

So I'm trying to execute it

```<code>namespace bp = boost::process;

bp::ipstream out;
bp::system("ps aux | awk '{s += $3} END {print s}'", bp::std_out > out);

std::string line;
std::getline(out, line);
</code>```

Output:

```<code>error: garbage option
</code>```

What am I doing wrong?

答:
Actually you don't need manually piping. You just need to inform a shell of your command:

Live On Coliru

```<code>#include <boost/process.hpp>
#include <iostream>
#include <iomanip>

int main() {
    namespace bp = boost::process;

    bp::ipstream out;
    std::vector<std::string> args {
        "-c", "ps aux | awk '{s += $3} END {print s}'"
    };
    bp::system(bp::search_path("sh"), args, bp::std_out > out);

    for (std::string line; std::getline(out, line);) {
        std::cout << "output line: " << std::quoted(line) << "\n";
    }
}
</code>```

Prints, e.g.:

```<code>output line: "90.4"
</code>```

ContentLicense: CC BY-SA 4.0

Body:
My script:

ps aux | awk '{s += $3} END {print s}'

So I'm trying to execute it

namespace bp = boost::process;

bp::ipstream out;
bp::system("ps aux | awk '{s += $3} END {print s}'", bp::std_out > out);

std::string line;
std::getline(out, line);

Output:

error: garbage option

What am I doing wrong?

Body:
Actually you don't need manually piping. You just need to inform a shell of your command:

Live On Coliru

#include <boost/process.hpp>
#include <iostream>
#include <iomanip>

int main() {
    namespace bp = boost::process;

    bp::ipstream out;
    std::vector<std::string> args {
        "-c", "ps aux | awk '{s += $3} END {print s}'"
    };
    bp::system(bp::search_path("sh"), args, bp::std_out > out);

    for (std::string line; std::getline(out, line);) {
        std::cout << "output line: " << std::quoted(line) << "\n";
    }
}

Prints, e.g.:

output line: "90.4"


问:
Does my application "contain encryption"?
I'm uploading a binary for the first time. iTunes Connect has asked me:

Export laws require that products containing encryption be properly authorized for export.

  Failure to comply could result in severe penalties.

  For further information, click here.

  Does your product contain encryption?

I use https://, but only via NSURLConnection and UIWebView.

My reading of this is that my app doesn't "contain encryption," but I'm wondering if this is spelled out anywhere. "Severe penalties" doesn't sound pleasant at all, so "I think that's right" is a bit sketchy... an authoritative answer would be better.

Thanks.

答:
Simple answers are Yes(App has encryption) and Yes(App uses Exempt encryption).
In my application, I am just opening my company's website in WKWebView but as it uses "https", it will be considered as exempt encryption.
Apple document for more info: https://developer.apple.com/documentation/security/complying_with_encryption_export_regulations?language=objc

Alternatively, you can just add key "ITSAppUsesNonExemptEncryption" and value "NO" in your app's info.plist file. and this way iTunes connect won't ask you that questions anymore.
More info: https://developer.apple.com/documentation/bundleresources/information_property_list/itsappusesnonexemptencryption?language=objc

You can follow these 3 simple steps to verify if your application is exempt or not: https://help.apple.com/app-store-connect/#/dev63c95e436

You may need to submit this annual-self-classification to US gov. For more info: https://www.bis.doc.gov/index.php/policy-guidance/encryption/4-reports-and-reviews/a-annual-self-classification

ContentLicense: CC BY-SA 2.5


问:
Getting 502 gateway error when trying to Deploy to cloud hub
I am following the MuleSoft Development Fundamentals course for Mule 4 and i'm trying to deploy the project to CloudHub.

As I am using a work machine I do not have access to port 8081 so I do the development in 8080 however I know CloudHub uses 8081 so when I deploy the project I revert to 8081. It deploys successfully but when accessing the Api it gives me a 502 error. Not sure what I am doing wrong.

答:
It is very likely you are using the wrong port or the wrong URL. Without more details of the URL it is not possible to say. Are you trying to use HTTPS in the URL? Then it will not work because it is listening in the HTTP port for CloudHub apps (8081). Remember that CloudHub maps 8081 to the default HTTP port so your URL should not have a port number. Use the URL from the dashboard page if in doubt.

Be sure that the port is the correct one. It is better to use the property ${http.port} to be sure, because it is predefined in CloudHub. To test locally you can define it in Studio run configuration / Arguments / VM Arguments as -Dhttp.port=8080.


问:
Anti capture screen in C#
I want to write a program to stop the recording the screen, and I used the following code to do that.

```<code> const uint WDA_NONE = 0;
    const uint WDA_MONITOR = 1;

    [DllImport("user32.dll")]
    public static extern uint SetWindowDisplayAffinity(IntPtr hWnd, uint dwAffinity);
    private void Form4_Load(object sender, EventArgs e)
    {
        SetWindowDisplayAffinity(this.Handle, WDA_MONITOR);
    }
</code>```

However, this code does not work for Merlis Action software.

答:
As you can see in the code above, the program was running and the screen recording programs were doing their job.
However, the form in which the relevant code is executed is displayed in black and white in the recorded video, but Merlis َAction software avoids this.</p>

Title: Anti capture screen in C#

Body:
I want to write a program to stop the recording the screen, and I used the following code to do that.

 const uint WDA_NONE = 0;
    const uint WDA_MONITOR = 1;

    [DllImport("user32.dll")]
    public static extern uint SetWindowDisplayAffinity(IntPtr hWnd, uint dwAffinity);
    private void Form4_Load(object sender, EventArgs e)
    {
        SetWindowDisplayAffinity(this.Handle, WDA_MONITOR);
    }

However, this code does not work for Merlis Action software.


问:
Rename and Set a network printer as the default printer in powershell
printers need to be added due to print server migration. printers are renamed. need to set the default renamed printer. need help</p>

```<code>$oldprinters=Get-Printer -Name "\\eesfs2\eep*"
foreach ($oldprinter in $oldprinters){
$newprinter=$oldprinter.name.split("\")[-1] -replace "EEP","RUSP"
Add-Printer -ConnectionName "\\rus0005\$newprinter"
}
</code>```

答:
```<code>$OldPrinters = gwmi win32_printer | where {$_.Name -like "\\eesfs2\eep*"}
foreach ($oldprinter in $Oldprinters){
$newprinter=$oldprinter.name.split("\")[-1] -replace "EEP","RUSP"
$a="\\rus0005\$newprinter"
Add-Printer -ConnectionName $a
if ($oldprinter.Default) {
(New-Object -ComObject WScript.Network).SetDefaultPrinter($a)
}
}
</code>```

Body:
printers need to be added due to print server migration. printers are renamed. need to set the default renamed printer. need help

$oldprinters=Get-Printer -Name "\\eesfs2\eep*"
foreach ($oldprinter in $oldprinters){
$newprinter=$oldprinter.name.split("\")[-1] -replace "EEP","RUSP"
Add-Printer -ConnectionName "\\rus0005\$newprinter"
}


问:
How do I sort by the last letter of a value in a dictionay in python?
given the following list of dictionaries, how do I sort by the last letter of the name?

```<code>    list_of_dicts = [
    {'name': 'Jadis', 'year_of_birth': 0, 'gender': 'Female'},
    {'name': 'Peter', 'year_of_birth': 1927, 'gender': 'Male'},
    {'name': 'Susan', 'year_of_birth': 1928, 'gender': 'Female'},
    {'name': 'Edmund', 'year_of_birth': 1930, 'gender': 'Male'},
    {'name': 'Lucy', 'year_of_birth': 1932, 'gender': 'Female'}]
</code>```

答:
You can do it with the built-in function sorted(yourlist=, key=).

Where your_list is your list_of_dicts and with key= you define the logic how you want your input list to be sorted. In your case, the code is:

```<code>sorted_list=sorted(list_of_dicts, key=lambda x: x['name'][-1])
</code>```

The lambda function allows you to sort the last character of x['name'] for every x inside your original list.
It returns when print (sorted_list):

```<code>[{'name': 'Edmund', 'year_of_birth': 1930, 'gender': 'Male'}, {'name':
'Susan', 'year_of_birth': 1928, 'gender': 'Female'}, {'name': 'Peter',
'year_of_birth': 1927, 'gender': 'Male'}, {'name': 'Jadis', 'year_of_birth':
0, 'gender': 'Female'}, {'name': 'Lucy', 'year_of_birth': 1932, 'gender':
'Female'}]
</code>```

As you can see, the sorted list follows 'Edmund'->'Susan'->'Peter'->'Jadis'->'Lucy'

Body:
given the following list of dictionaries, how do I sort by the last letter of the name?

    list_of_dicts = [
    {'name': 'Jadis', 'year_of_birth': 0, 'gender': 'Female'},
    {'name': 'Peter', 'year_of_birth': 1927, 'gender': 'Male'},
    {'name': 'Susan', 'year_of_birth': 1928, 'gender': 'Female'},
    {'name': 'Edmund', 'year_of_birth': 1930, 'gender': 'Male'},
    {'name': 'Lucy', 'year_of_birth': 1932, 'gender': 'Female'}]

Body:
You can do it with the built-in function sorted(yourlist=, key=).

Where your_list is your list_of_dicts and with key= you define the logic how you want your input list to be sorted. In your case, the code is:

sorted_list=sorted(list_of_dicts, key=lambda x: x['name'][-1])

The lambda function allows you to sort the last character of x['name'] for every x inside your original list.
It returns when print (sorted_list):

[{'name': 'Edmund', 'year_of_birth': 1930, 'gender': 'Male'}, {'name':
'Susan', 'year_of_birth': 1928, 'gender': 'Female'}, {'name': 'Peter',
'year_of_birth': 1927, 'gender': 'Male'}, {'name': 'Jadis', 'year_of_birth':
0, 'gender': 'Female'}, {'name': 'Lucy', 'year_of_birth': 1932, 'gender':
'Female'}]

As you can see, the sorted list follows 'Edmund'->'Susan'->'Peter'->'Jadis'->'Lucy'


问:
asyncio.Queue as producer-consumer flow in a webserver like Quart
Is it possible to use asyncio.Queue with a webserver like Quart to communicate between the producer and consumer?

Here is what I am trying to do....

```<code>from quart import Quart, request
import asyncio

queue = asyncio.Queue()
producers = []
consumers = []

async def producer(mesg):
    print(f'produced {mesg}')
    await queue.put(mesg)
    await asyncio.sleep(1) # do some work

async def consumer():
    while True:
        token = await queue.get()
        await asyncio.sleep(1) # do some work
        queue.task_done()
        print(f'consumed {token}')

@app.route('/route', methods=['POST'])
async def index():
    mesg = await request.get_data()
    try:
        p = asyncio.create_task(producer(mesg))
        producers.append(p)
        c = asyncio.create_task(consumer())
        consumers.append(c)
        return f"published message {mesg}", 200
    except Exception as e:
        logger.exception("Failed tp publish message %s!", mesg)
        return f"Failed to publish message: {mesg}", 400

if __name__ == '__main__':
    PORT = int(os.getenv('PORT')) if os.getenv('PORT') else 8050
    app.run(host='0.0.0.0', port=PORT, debug=True)
</code>```

This works fine.
But I am not sure if this is a good practice because I am confused how (where in my code) to do the below steps.

```<code># Making sure all the producers have completed
await asyncio.gather(*producers)

#wait for the remaining tasks to be processed
await queue.join()

# cancel the consumers, which are now idle
for c in consumers:
    c.cancel()
</code>```

EDIT-1:

I have tried using @app.after_serving, with some logger.debug statements.

```<code>@app.after_serving
async def shutdown():
    logger.debug("Shutting down...")
    logger.debug("waiting for producers to finish...")
    await asyncio.gather(*producers)
    logger.debug("waiting for tasks to complete...")
    await queue.join()
    logger.debug("cancelling consumers...")
    for c in consumers:
        c.cancel()
</code>```

But the debug statements are not printed when hypercorn is gracefully shutting down. So, I am not sure whether the function(shutdown) decorated with @app.after_serving is actually called during a shutdown.

Here is the message from hypercorn during shutdown

```<code>appserver_1  | 2020-05-29 15:55:14,200 - base_events.py:1490 -        create_server - INFO - <Server sockets=(<asyncio.TransportSocket fd=14, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('0.0.0.0', 8080)>,)> is serving
appserver_1  | Running on 0.0.0.0:8080 over http (CTRL + C to quit)
Gracefully stopping... (press Ctrl+C again to force)
</code>```

I using a kill -SIGTERM <PID> to signal a graceful shutdown to the process.

答:
But I am not sure if this is a good practice

Global variables like the ones you've created in your example are not typically good practice in enterprise solutions. Especially in Python, there are some finicky behavior when it comes to globals.
Passing in variables to a function or class has been a cleaner approach from my experience.
However, I don't know how to do that in quart since I don't use that lib.

```<code># Making sure all the producers have completed
#wait for the remaining tasks to be processed
# cancel the consumers, which are now idle
</code>```

Typically clean up tasks are done upon exiting the event loop and before exiting the application.
I don't have knowledge of how quart works, but you might be able to put that logic after app.run() so that clean up tasks run after the event loop stops.
This might vary depending on how your application exits.
Check the documentation for some sort of a "on shutdown" event that you can hook into.

Body:
Is it possible to use asyncio.Queue with a webserver like Quart to communicate between the producer and consumer?

Here is what I am trying to do....

from quart import Quart, request
import asyncio

queue = asyncio.Queue()
producers = []
consumers = []

async def producer(mesg):
    print(f'produced {mesg}')
    await queue.put(mesg)
    await asyncio.sleep(1) # do some work

async def consumer():
    while True:
        token = await queue.get()
        await asyncio.sleep(1) # do some work
        queue.task_done()
        print(f'consumed {token}')

@app.route('/route', methods=['POST'])
async def index():
    mesg = await request.get_data()
    try:
        p = asyncio.create_task(producer(mesg))
        producers.append(p)
        c = asyncio.create_task(consumer())
        consumers.append(c)
        return f"published message {mesg}", 200
    except Exception as e:
        logger.exception("Failed tp publish message %s!", mesg)
        return f"Failed to publish message: {mesg}", 400

if __name__ == '__main__':
    PORT = int(os.getenv('PORT')) if os.getenv('PORT') else 8050
    app.run(host='0.0.0.0', port=PORT, debug=True)

This works fine.
But I am not sure if this is a good practice because I am confused how (where in my code) to do the below steps.

# Making sure all the producers have completed
await asyncio.gather(*producers)

#wait for the remaining tasks to be processed
await queue.join()

# cancel the consumers, which are now idle
for c in consumers:
    c.cancel()

EDIT-1:

I have tried using @app.after_serving, with some logger.debug statements.

@app.after_serving
async def shutdown():
    logger.debug("Shutting down...")
    logger.debug("waiting for producers to finish...")
    await asyncio.gather(*producers)
    logger.debug("waiting for tasks to complete...")
    await queue.join()
    logger.debug("cancelling consumers...")
    for c in consumers:
        c.cancel()

But the debug statements are not printed when hypercorn is gracefully shutting down. So, I am not sure whether the function(shutdown) decorated with @app.after_serving is actually called during a shutdown.

Here is the message from hypercorn during shutdown

appserver_1  | 2020-05-29 15:55:14,200 - base_events.py:1490 -        create_server - INFO - <Server sockets=(<asyncio.TransportSocket fd=14, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('0.0.0.0', 8080)>,)> is serving
appserver_1  | Running on 0.0.0.0:8080 over http (CTRL + C to quit)
Gracefully stopping... (press Ctrl+C again to force)

I using a kill -SIGTERM <PID> to signal a graceful shutdown to the process.

Body:
But I am not sure if this is a good practice

Global variables like the ones you've created in your example are not typically good practice in enterprise solutions. Especially in Python, there are some finicky behavior when it comes to globals.
Passing in variables to a function or class has been a cleaner approach from my experience.
However, I don't know how to do that in quart since I don't use that lib.

# Making sure all the producers have completed
#wait for the remaining tasks to be processed
# cancel the consumers, which are now idle

Typically clean up tasks are done upon exiting the event loop and before exiting the application.
I don't have knowledge of how quart works, but you might be able to put that logic after app.run() so that clean up tasks run after the event loop stops.
This might vary depending on how your application exits.
Check the documentation for some sort of a "on shutdown" event that you can hook into.


问:
Hidden Features of VHDL
The original question was:

What are some really useful but esoteric language features in VHDL that you've actually been able to employ to do useful work?

The original question was deleted, and I was just answering. I think this is an interesting question, especially when you compare VHDL to other (normal) programming languages.

Disclaimer: I only did few trivial examples in VHDL.

But what I especially liked, when compared to other programming languages (which is probably not what you are asking about, but I think just for these things every programmer should know VHDL a bit):

The ability to have several different implementations (architectures) of a single interface, and exchange them easily.

The low-overhead parallelism inherent to the language model. It sort of reminds me of dataflow languages.

答:
Sometimes, there are more than one way to do something. OK, most of the time, you can do things in many ways in VHDL. Look at situation where you want to assign different values to a signal, based on the value of another signal.

Selected Signal Assignment

The most specific way to do this is with as selected signal assignment. Based on several possible values of b, you assign a value to a. No redundancy in the code here.

```<code>with a select b <=
        "1000" when "00",
        "0100" when "01",
        "0010" when "10",
        "0001" when "11";
</code>```

Conditional Signal Assignment

The construct of a conditional signal assignment is a little more general. For each option, you have to give a condition. This means that you could write any boolean expression as a condition, which give you more freedom than equality checking. While this construct would give you more freedom, there is a bit more redundancy too. We had to write the equality check (a =) on every line. If you use a signal with a long name, this will make your code bulkier. Also, the separator that's used in the selected signal assingment was a comma. In the conditional signal assingment, you need the else keyword. More code for the same functionality.

```<code>b <= "1000" when a = "00" else
         "1000" when a = "01" else
         "1000" when a = "10" else
         "1000" when a = "11";
</code>```

Combinational Process with Case Statement

The most generally usable construct is a process. Inside this process, you can write a case statement, or a cascade of if statements. There is even more redundancy here. You the skeleton code for a process (begin, end) and the sensitivity list. That's not a big effort, but while I was drafting this, I had put b in the sensitivity list instead of a. Easy to make a small misstake. You also need to specify what happens in the other cases.

```<code>process(a)
begin
        case a is
                when "00" => b <= "1000";
                when "01" => b <= "0100";
                when "10" => b <= "0010";
                when "11" => b <= "0001";
                when others => assert "unreachable" severity failure;
        end case;
end process;
</code>```

While this last code snippet is the largest and perhaps most error-prone, it is probably also the most common. It uses two familiar and often-used constructs: the process and the case statements.

The problem with the selected and conditional signal assignments is that there is no system in their syntax. The meaning is almost identical, but the syntax is just different enough to throw you off. I know many engineers who permanently have a copy of the Doulos Golden Reference Guide to VHDL lying on their desks. Which is good for Doulos, because their name gets mentioned all the time.

Title: Hidden Features of VHDL

Body:
Sometimes, there are more than one way to do something. OK, most of the time, you can do things in many ways in VHDL. Look at situation where you want to assign different values to a signal, based on the value of another signal.

Selected Signal Assignment

The most specific way to do this is with as selected signal assignment. Based on several possible values of b, you assign a value to a. No redundancy in the code here.

with a select b <=
        "1000" when "00",
        "0100" when "01",
        "0010" when "10",
        "0001" when "11";

Conditional Signal Assignment

The construct of a conditional signal assignment is a little more general. For each option, you have to give a condition. This means that you could write any boolean expression as a condition, which give you more freedom than equality checking. While this construct would give you more freedom, there is a bit more redundancy too. We had to write the equality check (a =) on every line. If you use a signal with a long name, this will make your code bulkier. Also, the separator that's used in the selected signal assingment was a comma. In the conditional signal assingment, you need the else keyword. More code for the same functionality.

b <= "1000" when a = "00" else
         "1000" when a = "01" else
         "1000" when a = "10" else
         "1000" when a = "11";

Combinational Process with Case Statement

The most generally usable construct is a process. Inside this process, you can write a case statement, or a cascade of if statements. There is even more redundancy here. You the skeleton code for a process (begin, end) and the sensitivity list. That's not a big effort, but while I was drafting this, I had put b in the sensitivity list instead of a. Easy to make a small misstake. You also need to specify what happens in the other cases.

process(a)
begin
        case a is
                when "00" => b <= "1000";
                when "01" => b <= "0100";
                when "10" => b <= "0010";
                when "11" => b <= "0001";
                when others => assert "unreachable" severity failure;
        end case;
end process;

While this last code snippet is the largest and perhaps most error-prone, it is probably also the most common. It uses two familiar and often-used constructs: the process and the case statements.

The problem with the selected and conditional signal assignments is that there is no system in their syntax. The meaning is almost identical, but the syntax is just different enough to throw you off. I know many engineers who permanently have a copy of the Doulos Golden Reference Guide to VHDL lying on their desks. Which is good for Doulos, because their name gets mentioned all the time.


问:
What is the fastest way to draw circles in python and save as png?

Background

I'm optimizing a project.

Profiling the code I found that 50% of the time is spend on a function in which a set of circles (different radii, colors and locations) are drawn to a choosen sector of fixed size (white canvas) if their center corrdinates is within the sector bounds. Depending on the usage the function saves the figure as a png and returns the path or returns the image as an numpy array.

The build-in method matplotlib._png.write_png from savefig is the most expensive But there is also some overhead from creating the figures, etc.

Generally the code is used with multiprocessing / parallel programming.

Example output

Code

```<code>import matplotlib.pyplot as plt
import cv2
import os

    def get_top_view(sector, circles, file_path, save_image_flag):

        # get the sector bounds.
        x_low, y_low, x_high, y_high = get_sector_bounds(sector)

        # init figure
        fig, ax = plt.subplots()
        ax.set_xlim(y_low, y_high)
        ax.set_ylim(x_low, x_high)
        ax.set_yticklabels([])
        ax.set_xticklabels([])
        ax.set_yticks([])
        ax.set_xticks([])
        ax.set_aspect('equal')
        ax.axis('off')

        # c is a circle object with all relevant data (center coordinates,
        # radius, RGB color tuple)
        for c in circles:
           if x_low <= c.x_coord <= x_high and y_low <= c.y_coord <= y_high:
                    shape = plt.Circle((c.x_coord, c.y_coord), c.radius, color=c.color)
                    shape_plot = ax.add_artist(shape)
                    shapes.append(shape_plot)
        plt.gca().invert_yaxis()
        if save_image_flag:
            plt.savefig(file_path + '_cc.png', bbox_inches='tight', pad_inches=0.02)
            plt.close()
            return file_path
        else:
            ax.margins(0)
            fig.tight_layout()
            fig.canvas.draw()
            image_from_plot = np.frombuffer(fig.canvas.tostring_rgb(), dtype=np.uint8)
            image_from_plot = image_from_plot.reshape(
                                fig.canvas.get_width_height()[::-1] + (3,))
            image_from_plot = image_from_plot[:, 13:-14]
            resized = cv2.resize(image_from_plot, (499, 391))
            cropped = resized[78:-78]
            plt.close()
            return cropped
</code>```

Questions

There is the issue that the array version and the png image is slightly different. I think that relates to the DPI of the image. I want to fix that and I'm thinking about different options who to speedup this function.

Speedup the process and keeping matplotlib, similar to this example from Github.

Get rid of matplotlib and draw it with Pillow, e.g. some thing like:

```<code>from PIL import Image, ImageDraw

def get_top_view(sector, circles, file_path, save_image_flag):

    # get the sector bounds.
    x_low, y_low, x_high, y_high = get_sector_bounds(sector)

    im = Image.new('RGB', (499, 235), (255, 255, 255))
    draw = ImageDraw.Draw(im)

    # there needs to be some rescaling that the corrdinates match which
    # I don't account for at the moment.
    for c in circles:
           if x_low <= c.x_coord <= x_high and y_low <= c.y_coord <= y_high:
                 draw.ellipse((c.x_coord - c.radius, c.y_coord - c.radius,
                               c.x_coord + c.radius, c.y_coord + c.radius),
                               fill=c.color)

    if save_image_flag:
        im.save(file_path + '.png')
        return file_path

    else:
        image_as_array = convert_to_array()  # have to think about how I'll do that
        return image_as_array
</code>```

A different approach that is faster (and somehow convenient)...

I'd be glad for any feedback on the two issues.

答:
I'll share my findings here. I'm calling the function 2200 times for each output type  within the simulation framework. All computation is serial as the multiprocessing code is not part of the example.

There are 220 sectors with 200 circles randomly distributed amongst the sectors. The simulation runs for 10 steps, where the circles radii get updated and new figures are drawn. Hence, 2200 calls to the function.

To generate and save png images it took 293483ms previously.

To generate numpy arrays it took 92715ms previously.

Speedup with Matplotlib

Generating and saving images (save_image_flag = True) now takes 126485ms.

Generating numpy arrays now takes 57029ms.

```<code>import matplotlib.pyplot as plt
import os

def get_top_view(sector, circles, file_path, save_image_flag):

    # get the sector bounds.
    x_low, y_low, x_high, y_high = get_sector_bounds(sector)

    # init figure
    fig, ax = plt.subplots()
    ax.set_xlim(y_low, y_high)
    ax.set_ylim(x_low, x_high)
    # These were unnecessary
    # ax.set_yticklabels([])
    # ax.set_xticklabels([])
    # ax.set_yticks([])
    # ax.set_xticks([])
    ax.set_aspect('equal')
    ax.axis('off')

    # c is a circle object with all relevant data (center coordinates,
    # radius, RGB color tuple)
    for c in circles:
       if x_low <= c.x_coord <= x_high and y_low <= c.y_coord <= y_high:
                shape = plt.Circle((c.x_coord, c.y_coord), c.radius, color=c.color)
                shape_plot = ax.add_artist(shape)
                shapes.append(shape_plot)
    plt.gca().invert_yaxis()
    # I added this to get the discrepancies between the generated image and
    # the numpy array, as bbox_inches='tight' only applies to the saved image.
    bbox0 = fig.get_tightbbox(fig.canvas.get_renderer()).padded(0.02)
    if save_image_flag:
        plt.savefig(file_path + '_cc.png', bbox_inches=bbox0)
        plt.close()
        return file_path
    else:
        buf = io.BytesIO()
        fig.savefig(buf, format="rgba", dpi=100, bbox_inches=bbox0)
        buf.seek(0)
        img = np.reshape(np.frombuffer(buf.getvalue(), dtype=np.uint8),
                         newshape=(235, 499, -1))
        img = img[..., :3]
        buf.close()
        plt.close()
        return image
</code>```

Speedup without Matplotlib

Research is in progress...

Body:
Background

I'm optimizing a project.

Profiling the code I found that 50% of the time is spend on a function in which a set of circles (different radii, colors and locations) are drawn to a choosen sector of fixed size (white canvas) if their center corrdinates is within the sector bounds. Depending on the usage the function saves the figure as a png and returns the path or returns the image as an numpy array.

The build-in method matplotlib._png.write_png from savefig is the most expensive But there is also some overhead from creating the figures, etc.

Generally the code is used with multiprocessing / parallel programming.

Example output

Code

import matplotlib.pyplot as plt
import cv2
import os

    def get_top_view(sector, circles, file_path, save_image_flag):

        # get the sector bounds.
        x_low, y_low, x_high, y_high = get_sector_bounds(sector)

        # init figure
        fig, ax = plt.subplots()
        ax.set_xlim(y_low, y_high)
        ax.set_ylim(x_low, x_high)
        ax.set_yticklabels([])
        ax.set_xticklabels([])
        ax.set_yticks([])
        ax.set_xticks([])
        ax.set_aspect('equal')
        ax.axis('off')

        # c is a circle object with all relevant data (center coordinates,
        # radius, RGB color tuple)
        for c in circles:
           if x_low <= c.x_coord <= x_high and y_low <= c.y_coord <= y_high:
                    shape = plt.Circle((c.x_coord, c.y_coord), c.radius, color=c.color)
                    shape_plot = ax.add_artist(shape)
                    shapes.append(shape_plot)
        plt.gca().invert_yaxis()
        if save_image_flag:
            plt.savefig(file_path + '_cc.png', bbox_inches='tight', pad_inches=0.02)
            plt.close()
            return file_path
        else:
            ax.margins(0)
            fig.tight_layout()
            fig.canvas.draw()
            image_from_plot = np.frombuffer(fig.canvas.tostring_rgb(), dtype=np.uint8)
            image_from_plot = image_from_plot.reshape(
                                fig.canvas.get_width_height()[::-1] + (3,))
            image_from_plot = image_from_plot[:, 13:-14]
            resized = cv2.resize(image_from_plot, (499, 391))
            cropped = resized[78:-78]
            plt.close()
            return cropped

Questions

There is the issue that the array version and the png image is slightly different. I think that relates to the DPI of the image. I want to fix that and I'm thinking about different options who to speedup this function.

Speedup the process and keeping matplotlib, similar to this example from Github.

Get rid of matplotlib and draw it with Pillow, e.g. some thing like:

from PIL import Image, ImageDraw

def get_top_view(sector, circles, file_path, save_image_flag):

    # get the sector bounds.
    x_low, y_low, x_high, y_high = get_sector_bounds(sector)

    im = Image.new('RGB', (499, 235), (255, 255, 255))
    draw = ImageDraw.Draw(im)

    # there needs to be some rescaling that the corrdinates match which
    # I don't account for at the moment.
    for c in circles:
           if x_low <= c.x_coord <= x_high and y_low <= c.y_coord <= y_high:
                 draw.ellipse((c.x_coord - c.radius, c.y_coord - c.radius,
                               c.x_coord + c.radius, c.y_coord + c.radius),
                               fill=c.color)

    if save_image_flag:
        im.save(file_path + '.png')
        return file_path

    else:
        image_as_array = convert_to_array()  # have to think about how I'll do that
        return image_as_array

A different approach that is faster (and somehow convenient)...

I'd be glad for any feedback on the two issues.

Body:
I'll share my findings here. I'm calling the function 2200 times for each output type  within the simulation framework. All computation is serial as the multiprocessing code is not part of the example.

There are 220 sectors with 200 circles randomly distributed amongst the sectors. The simulation runs for 10 steps, where the circles radii get updated and new figures are drawn. Hence, 2200 calls to the function.

To generate and save png images it took 293483ms previously.

To generate numpy arrays it took 92715ms previously.

Speedup with Matplotlib

Generating and saving images (save_image_flag = True) now takes 126485ms.

Generating numpy arrays now takes 57029ms.

import matplotlib.pyplot as plt
import os

def get_top_view(sector, circles, file_path, save_image_flag):

    # get the sector bounds.
    x_low, y_low, x_high, y_high = get_sector_bounds(sector)

    # init figure
    fig, ax = plt.subplots()
    ax.set_xlim(y_low, y_high)
    ax.set_ylim(x_low, x_high)
    # These were unnecessary
    # ax.set_yticklabels([])
    # ax.set_xticklabels([])
    # ax.set_yticks([])
    # ax.set_xticks([])
    ax.set_aspect('equal')
    ax.axis('off')

    # c is a circle object with all relevant data (center coordinates,
    # radius, RGB color tuple)
    for c in circles:
       if x_low <= c.x_coord <= x_high and y_low <= c.y_coord <= y_high:
                shape = plt.Circle((c.x_coord, c.y_coord), c.radius, color=c.color)
                shape_plot = ax.add_artist(shape)
                shapes.append(shape_plot)
    plt.gca().invert_yaxis()
    # I added this to get the discrepancies between the generated image and
    # the numpy array, as bbox_inches='tight' only applies to the saved image.
    bbox0 = fig.get_tightbbox(fig.canvas.get_renderer()).padded(0.02)
    if save_image_flag:
        plt.savefig(file_path + '_cc.png', bbox_inches=bbox0)
        plt.close()
        return file_path
    else:
        buf = io.BytesIO()
        fig.savefig(buf, format="rgba", dpi=100, bbox_inches=bbox0)
        buf.seek(0)
        img = np.reshape(np.frombuffer(buf.getvalue(), dtype=np.uint8),
                         newshape=(235, 499, -1))
        img = img[..., :3]
        buf.close()
        plt.close()
        return image

Speedup without Matplotlib

Research is in progress...


问:
Concat multiple lists from a list element for element for the sublist
Given:

```<code>[['A', 'B', 'C'], ['1', '2', '3'], ['Z', 'X']]
</code>```

Is there a way to concatenate every element from the lists in the list for every index 1 to 1 not 1 to many without using a lot of for statements? And the output to be a list with every concatenation as its elements?

Expected output:

```<code>['A1Z','B2X', 'C3']
</code>```

答:
We want to zip the lists together. But if the lists are not of the same length, we need to pad them. itertools.zip_longest helps us with that:

```<code>from itertools import zip_longest

xss = [['A', 'B', 'C'], ['1', '2', '3'], ['Z', 'X']]
result = ["".join(xs) for xs in zip_longest(*xss, fillvalue="")]
</code>```

Which gives:

```<code>>>> result
['A1Z', 'B2X', 'C3']
</code>```

Body:
Given:

[['A', 'B', 'C'], ['1', '2', '3'], ['Z', 'X']]

Is there a way to concatenate every element from the lists in the list for every index 1 to 1 not 1 to many without using a lot of for statements? And the output to be a list with every concatenation as its elements?

Expected output:

['A1Z','B2X', 'C3']

Body:
We want to zip the lists together. But if the lists are not of the same length, we need to pad them. itertools.zip_longest helps us with that:

from itertools import zip_longest

xss = [['A', 'B', 'C'], ['1', '2', '3'], ['Z', 'X']]
result = ["".join(xs) for xs in zip_longest(*xss, fillvalue="")]

Which gives:

>>> result
['A1Z', 'B2X', 'C3']


问:
Custom tooltip with Rxjs
I'm trying to implement my custom tooltip on a diagram using vis-network library and RXJS.

The main principle is:

One observable diagramElementHover$ to show the tooltip

One observable hideTooltipObs$ to hide the tooltip (the emitted value is null)

I want the tooltip to wait 1000ms to check if the mouse pointer is still under the element to be displayed (clear the tooltip if the pointer has gone before that delay)

The tooltip display uses the *ngIf directive with tooltip value

For now, the problem is that I can't prevent the tooltip to be shown if the user hovers one element and stays less that 1 second (then the tooltip is not hidden, unless you enter another node)

Globally, my implementation looks like this:

this.sub = merge(
    diagramElementHover$.pipe(
        mergeMap(event => of(event).pipe(delay(1000))),
        repeat()
    ),
        hideTooltipObs$
    ).pipe(
        tap(tooltip => this.tooltip = tooltip)
    )
```

I tried to use the takeUntil() operator but without success...
Can you guide me through this?

Here is the stackblitz example.

答:
You can set the tooltip reference as null in the hide tooltip observable to ensure it will remove the tooltip on hide:

const hideTooltipObs$ = merge(
    new Observable<null>(obs => this.network.on('selectNode', () => obs.next(null))),
    new Observable<null>(obs => this.network.on('blurEdge', () => obs.next(null))),
    new Observable<null>(obs => this.network.on('blurNode', () => obs.next(null)))
).pipe(tap(() => {
  console.log('hide tooltip');
  this.tooltip = null;
}))
```

<ol start="2">
<li>Then you have to add inside the mergeMap delay the <code>takeUntil</code> operator. The idea here is to keep the reference of delaying to show the tooltip <strong>until</strong> the hide is trigerred.</li>
</ol>

<pre class="lang-js prettyprint-override"><code> mergeMap(event => of(event).pipe(
     delay(1000),
     takeUntil(hideTooltipObs$),
  )
),
</code>```

Take a look in this stackblitz:
https://stackblitz.com/edit/angular-ivy-gfj1gg

Title: Custom tooltip with Rxjs

Body:
I'm trying to implement my custom tooltip on a diagram using vis-network library and RXJS.

The main principle is:

One observable diagramElementHover$ to show the tooltip

One observable hideTooltipObs$ to hide the tooltip (the emitted value is null)

I want the tooltip to wait 1000ms to check if the mouse pointer is still under the element to be displayed (clear the tooltip if the pointer has gone before that delay)

The tooltip display uses the *ngIf directive with tooltip value

For now, the problem is that I can't prevent the tooltip to be shown if the user hovers one element and stays less that 1 second (then the tooltip is not hidden, unless you enter another node)

Globally, my implementation looks like this:

this.sub = merge(
    diagramElementHover$.pipe(
        mergeMap(event => of(event).pipe(delay(1000))),
        repeat()
    ),
        hideTooltipObs$
    ).pipe(
        tap(tooltip => this.tooltip = tooltip)
    )

I tried to use the takeUntil() operator but without success...
Can you guide me through this?

Here is the stackblitz example.

Body:
You can set the tooltip reference as null in the hide tooltip observable to ensure it will remove the tooltip on hide:

const hideTooltipObs$ = merge(
    new Observable<null>(obs => this.network.on('selectNode', () => obs.next(null))),
    new Observable<null>(obs => this.network.on('blurEdge', () => obs.next(null))),
    new Observable<null>(obs => this.network.on('blurNode', () => obs.next(null)))
).pipe(tap(() => {
  console.log('hide tooltip');
  this.tooltip = null;
}))

Then you have to add inside the mergeMap delay the takeUntil operator. The idea here is to keep the reference of delaying to show the tooltip until the hide is trigerred.

 mergeMap(event => of(event).pipe(
     delay(1000),
     takeUntil(hideTooltipObs$),
  )
),

Take a look in this stackblitz:
https://stackblitz.com/edit/angular-ivy-gfj1gg


问:
Getting the texture scale with assimp
I'm trying the texture scale from the .mtl file. Opening the file in notepad I can see that it has map_Kd -s 4.000000 4.000000 1.000000 crate_diffuse.dds

I am trying thte code below but it returns aiReturnFAILURE.

```<code>for (UINT j = 0; j < material->GetTextureCount(type); j++)
{
    aiString str;
    material->GetTexture(type, j, &str);

    aiUVTransform uvTransform;
    if (material->Get(AI_MATKEY_UVTRANSFORM(aiTextureType_DIFFUSE, j), uvTransform) == aiReturn_FAILURE)
    {
        std::cout << "Failed\n";
    }

    texture_diffuse = std::string(str.C_Str());
    std::cout << texture_diffuse << '\n';
}
</code>```

答: This will happen when not uvTransform was stored for your material. Without any uvTransform your scaling will be 1.0.</p>

Title: Getting the texture scale with assimp

Body:
I'm trying the texture scale from the .mtl file. Opening the file in notepad I can see that it has map_Kd -s 4.000000 4.000000 1.000000 crate_diffuse.dds

I am trying thte code below but it returns aiReturnFAILURE.

for (UINT j = 0; j < material->GetTextureCount(type); j++)
{
    aiString str;
    material->GetTexture(type, j, &str);

    aiUVTransform uvTransform;
    if (material->Get(AI_MATKEY_UVTRANSFORM(aiTextureType_DIFFUSE, j), uvTransform) == aiReturn_FAILURE)
    {
        std::cout << "Failed\n";
    }

    texture_diffuse = std::string(str.C_Str());
    std::cout << texture_diffuse << '\n';
}


问:
Trouble with nested ifelse statement in R
I'm trying to run a nested ifelse statement in R.  Here's a look at the structure of my data using the glimpse() function from the tidyverse:

```<code>Rows: 22,104
Columns: 9
$ `Formation/Locality`    <chr> "Montmartre", "Montmartre", "Montmartre", "Fur", "Me...
$ Location                <chr> "Ile-de-France Region, France", "Ile-de-France Regio...
$ Environment             <chr> "terrestrial", "terrestrial", "terrestrial", "offsho...
$ `Palaeolongitude(N/-S)` <dbl> 47.4, 47.4, 47.4, 52.3, 46.9, 42.9, 47.5, 46.9, 46.2...
$ `Palaeolatitude(E/-W)`  <dbl> 1.6, 1.6, 1.6, 5.4, 4.8, 1.9, -5.2, 4.8, -93.6, -111...
$ TaxonomicLevel          <chr> "Order", "Order", "Order", "Order", "Order", "Order"...
$ TaxonomicName           <chr> "Upupiformes", "Upupiformes", "Upupiformes", "Trogon...
$ MinMax                  <chr> "MaxMa", "MaxMa", "MaxMa", "MaxMa", "MaxMa", "MaxMa"...
$ Age                     <dbl> 37.2, 37.2, 37.2, 55.8, 48.6, 37.2, 48.6, 48.6, 55.8...
</code>```

I'm trying to get R to look at the Age column and if the value is within a certain range, for it to put the geological age name into a new column called AgeName.  If the value is not within the range, I want it to move on to the next age range and so on and so forth.  Here's my code so far:

```<code>pbdb_tidyish$AgeName <- ifelse(56>=pbdb_tidyish$Age&&47.8<pbdb_tidyish$Age,
                               "Ypresian",
                               ifelse(47.8>=pbdb_tidyish$Age&&41.2<pbdb_tidyish$Age,
                                      "Lutetian",
                                      ifelse(41.2>=pbdb_tidyish$Age&&37.8<pbdb_tidyish$Age,
                                             "Bartonian",
                                             ifelse(37.8>=pbdb_tidyish$Age&&33.9<=pbdb_tidyish$Age,
                                                    "Priabonian",NA))))
</code>```

When I run this code, it creates the new column but fills the whole column with "Priabonian" so the dataset now looks like this:

```<code>Rows: 22,104
Columns: 10
$ `Formation/Locality`    <chr> "Montmartre", "Montmartre", "Montmartre", "Fur", "Me...
$ Location                <chr> "Ile-de-France Region, France", "Ile-de-France Regio...
$ Environment             <chr> "terrestrial", "terrestrial", "terrestrial", "offsho...
$ `Palaeolongitude(N/-S)` <dbl> 47.4, 47.4, 47.4, 52.3, 46.9, 42.9, 47.5, 46.9, 46.2...
$ `Palaeolatitude(E/-W)`  <dbl> 1.6, 1.6, 1.6, 5.4, 4.8, 1.9, -5.2, 4.8, -93.6, -111...
$ TaxonomicLevel          <chr> "Order", "Order", "Order", "Order", "Order", "Order"...
$ TaxonomicName           <chr> "Upupiformes", "Upupiformes", "Upupiformes", "Trogon...
$ MinMax                  <chr> "MaxMa", "MaxMa", "MaxMa", "MaxMa", "MaxMa", "MaxMa"...
$ Age                     <dbl> 37.2, 37.2, 37.2, 55.8, 48.6, 37.2, 48.6, 48.6, 55.8...
$ AgeName                 <chr> "Priabonian", "Priabonian", "Priabonian", "Priabonia...
</code>```

Does anyone have any idea where I'm going wrong? I think it's just looking at the first Age value, running the ifelse statement then filling the whole column with the result of that as opposed to moving on to the next row.

Thanks,

Carolina

答:
You're already using the tidyverse, you should make yourself familiar with case_when:

```<code>pbdb_tidyish <- pbdb_tidyish %>%
  mutate(AgeName = case_when(
    (Age >= 33.9 & Age <= 37.8) ~ 'Priabonian',
    (Age > 37.8 & Age <= 41.2) ~ 'Bartonian',
    (Age > 41.2 & Age <= 47.8) ~ 'Lutetian',
    (Age > 47.8 & Age <= 56) ~ 'Ypresian',
  ))
</code>```

Body:
I'm trying to run a nested ifelse statement in R.  Here's a look at the structure of my data using the glimpse() function from the tidyverse:

Rows: 22,104
Columns: 9
$ `Formation/Locality`    <chr> "Montmartre", "Montmartre", "Montmartre", "Fur", "Me...
$ Location                <chr> "Ile-de-France Region, France", "Ile-de-France Regio...
$ Environment             <chr> "terrestrial", "terrestrial", "terrestrial", "offsho...
$ `Palaeolongitude(N/-S)` <dbl> 47.4, 47.4, 47.4, 52.3, 46.9, 42.9, 47.5, 46.9, 46.2...
$ `Palaeolatitude(E/-W)`  <dbl> 1.6, 1.6, 1.6, 5.4, 4.8, 1.9, -5.2, 4.8, -93.6, -111...
$ TaxonomicLevel          <chr> "Order", "Order", "Order", "Order", "Order", "Order"...
$ TaxonomicName           <chr> "Upupiformes", "Upupiformes", "Upupiformes", "Trogon...
$ MinMax                  <chr> "MaxMa", "MaxMa", "MaxMa", "MaxMa", "MaxMa", "MaxMa"...
$ Age                     <dbl> 37.2, 37.2, 37.2, 55.8, 48.6, 37.2, 48.6, 48.6, 55.8...

I'm trying to get R to look at the Age column and if the value is within a certain range, for it to put the geological age name into a new column called AgeName.  If the value is not within the range, I want it to move on to the next age range and so on and so forth.  Here's my code so far:

pbdb_tidyish$AgeName <- ifelse(56>=pbdb_tidyish$Age&&47.8<pbdb_tidyish$Age,
                               "Ypresian",
                               ifelse(47.8>=pbdb_tidyish$Age&&41.2<pbdb_tidyish$Age,
                                      "Lutetian",
                                      ifelse(41.2>=pbdb_tidyish$Age&&37.8<pbdb_tidyish$Age,
                                             "Bartonian",
                                             ifelse(37.8>=pbdb_tidyish$Age&&33.9<=pbdb_tidyish$Age,
                                                    "Priabonian",NA))))

When I run this code, it creates the new column but fills the whole column with "Priabonian" so the dataset now looks like this:

Rows: 22,104
Columns: 10
$ `Formation/Locality`    <chr> "Montmartre", "Montmartre", "Montmartre", "Fur", "Me...
$ Location                <chr> "Ile-de-France Region, France", "Ile-de-France Regio...
$ Environment             <chr> "terrestrial", "terrestrial", "terrestrial", "offsho...
$ `Palaeolongitude(N/-S)` <dbl> 47.4, 47.4, 47.4, 52.3, 46.9, 42.9, 47.5, 46.9, 46.2...
$ `Palaeolatitude(E/-W)`  <dbl> 1.6, 1.6, 1.6, 5.4, 4.8, 1.9, -5.2, 4.8, -93.6, -111...
$ TaxonomicLevel          <chr> "Order", "Order", "Order", "Order", "Order", "Order"...
$ TaxonomicName           <chr> "Upupiformes", "Upupiformes", "Upupiformes", "Trogon...
$ MinMax                  <chr> "MaxMa", "MaxMa", "MaxMa", "MaxMa", "MaxMa", "MaxMa"...
$ Age                     <dbl> 37.2, 37.2, 37.2, 55.8, 48.6, 37.2, 48.6, 48.6, 55.8...
$ AgeName                 <chr> "Priabonian", "Priabonian", "Priabonian", "Priabonia...

Does anyone have any idea where I'm going wrong? I think it's just looking at the first Age value, running the ifelse statement then filling the whole column with the result of that as opposed to moving on to the next row.

Thanks,

Carolina

Body:
You're already using the tidyverse, you should make yourself familiar with case_when:

pbdb_tidyish <- pbdb_tidyish %>%
  mutate(AgeName = case_when(
    (Age >= 33.9 & Age <= 37.8) ~ 'Priabonian',
    (Age > 37.8 & Age <= 41.2) ~ 'Bartonian',
    (Age > 41.2 & Age <= 47.8) ~ 'Lutetian',
    (Age > 47.8 & Age <= 56) ~ 'Ypresian',
  ))


问:
How to call ComponentDidMount when testing with Jest in React
I've a component , where ComponentDidMount is trying to call a API function

```<code>componentDidMount(){

   this.getTypes(this.props.user).then(Response => {this.setState({List: Response.Types })});

  }

async getTypes(userLogin){
    try{
      res = await API.get(......... );
      }
    catch(error) {
       ....
       }
}

</code>```

Now i would like to invoke componentDidMount & getTypes and will mock API.get.

So to do this, can anyone tell me how to invoke both componentDidMount & getTypes using Jest?

EDIT : Tried to do in this way . But i can see that its not invoking the ComponentDidMount .

```<code> it('renders ComponentDidMount',  () => {
               const wrapper = shallow(<HoldVendor region={"aaa"} applier={"xxx"} />);
               API.get = jest.fn().mockImplementation(
                () => {
                       return {"Types": [{}]};
               });

 expect(API.get).toHaveBeenCalled();
});

</code>```

答:
use Enzyme to test ur components.

In tests, shallow render your component as

shallow(<ComponentName {...props} />)

this will automatically call componentDidMount.

Now to mock the api call, you can use nock or even mock the entire Api.get method before you render the component in your tests.

Body:
I've a component , where ComponentDidMount is trying to call a API function

componentDidMount(){

   this.getTypes(this.props.user).then(Response => {this.setState({List: Response.Types })});

  }

async getTypes(userLogin){
    try{
      res = await API.get(......... );
      }
    catch(error) {
       ....
       }
}

Now i would like to invoke componentDidMount & getTypes and will mock API.get.

So to do this, can anyone tell me how to invoke both componentDidMount & getTypes using Jest?

EDIT : Tried to do in this way . But i can see that its not invoking the ComponentDidMount .

 it('renders ComponentDidMount',  () => {
               const wrapper = shallow(<HoldVendor region={"aaa"} applier={"xxx"} />);
               API.get = jest.fn().mockImplementation(
                () => {
                       return {"Types": [{}]};
               });

 expect(API.get).toHaveBeenCalled();
});


问:
Why is it possible to update UI using Handler created in a background Thread?
I have a worker class which has a Handler created in it. When I try to update my UI using the handler created in background thread its updating the UI and there is no crash here. Ideally it should crash as its running in the background thread. Please find the code below:

```<code>class WorkerThread(val mainThreadHandler: Handler, handlerThreadName: String) :
  HandlerThread(handlerThreadName) {

  lateinit var workerThreadHandler: Handler

override fun onLooperPrepared() {
    super.onLooperPrepared()
    Log.d("LLoyd", "onLooperPrepared")

}

override fun run() {
    super.run()
    Log.d("LLoyd", "run method called")

}

fun startTask(){
    workerThreadHandler = object : Handler(this.looper) {
        override fun handleMessage(msg: Message) {
            super.handleMessage(msg)
            Log.d(
                "Lloyd",
                "Message from Main Thread received " + msg.what + "  Thread name  " +
currentThread().name
            )
            val message = Message()
            message.what = msg.what
            mainThreadHandler.sendMessage(message)
        }
    }

 }
}
</code>```

This startTask() is called from activity on click of a button as below:

```<code>val workerThread = WorkerThread(mainThreadHandler, "Lloyd")
    workerThread.start()
    btn_task1.setOnClickListener {
        workerThread.startTask()
        val message = Message()
        message.what = FROM_MAINTHREAD
        workerThread.workerThreadHandler.sendMessage(message)
        val message1 = Message()
        message.what = 55
        Log.d("Lloyd ", "Outside Thread name " + Thread.currentThread().name)

        workerThread.workerThreadHandler.post {
            Log.d("Lloyd ", "Runnable Thread name " + currentThread().name)
            btn_task1.text = "Task 2"
            tv.text= "YAAAAAAAAAAAAAAAAAAAAAAy"
        }
    }
</code>```

Any kind of help is Appreciated!

答:
This was a problem for background tasks before. But Handler class is specifically create for updating UI in background tasks as Handler is in UI thread itself. For example, when you accomplish heavy task and need to update the ui you can do

```<code>new Thread(new Runnable() {
         @Override
         public void run() {

               // here you perform background operation
               //Update the value background thread to UI thread
               mHandler.post(new Runnable() {
                  @Override
                  public void run() {
                     // here you can update ui
                  }
               });
            }
         }
      }).start();
</code>```

for more information you can also learn here

Body:
I have a worker class which has a Handler created in it. When I try to update my UI using the handler created in background thread its updating the UI and there is no crash here. Ideally it should crash as its running in the background thread. Please find the code below:

class WorkerThread(val mainThreadHandler: Handler, handlerThreadName: String) :
  HandlerThread(handlerThreadName) {

  lateinit var workerThreadHandler: Handler

override fun onLooperPrepared() {
    super.onLooperPrepared()
    Log.d("LLoyd", "onLooperPrepared")

}

override fun run() {
    super.run()
    Log.d("LLoyd", "run method called")

}

fun startTask(){
    workerThreadHandler = object : Handler(this.looper) {
        override fun handleMessage(msg: Message) {
            super.handleMessage(msg)
            Log.d(
                "Lloyd",
                "Message from Main Thread received " + msg.what + "  Thread name  " +
currentThread().name
            )
            val message = Message()
            message.what = msg.what
            mainThreadHandler.sendMessage(message)
        }
    }

 }
}

This startTask() is called from activity on click of a button as below:

val workerThread = WorkerThread(mainThreadHandler, "Lloyd")
    workerThread.start()
    btn_task1.setOnClickListener {
        workerThread.startTask()
        val message = Message()
        message.what = FROM_MAINTHREAD
        workerThread.workerThreadHandler.sendMessage(message)
        val message1 = Message()
        message.what = 55
        Log.d("Lloyd ", "Outside Thread name " + Thread.currentThread().name)

        workerThread.workerThreadHandler.post {
            Log.d("Lloyd ", "Runnable Thread name " + currentThread().name)
            btn_task1.text = "Task 2"
            tv.text= "YAAAAAAAAAAAAAAAAAAAAAAy"
        }
    }

Any kind of help is Appreciated!

Body:
This was a problem for background tasks before. But Handler class is specifically create for updating UI in background tasks as Handler is in UI thread itself. For example, when you accomplish heavy task and need to update the ui you can do

new Thread(new Runnable() {
         @Override
         public void run() {

               // here you perform background operation
               //Update the value background thread to UI thread
               mHandler.post(new Runnable() {
                  @Override
                  public void run() {
                     // here you can update ui
                  }
               });
            }
         }
      }).start();

for more information you can also learn here


问:
Visual Studio 2019 New Project does not include SSIS, whcih is installed
I can open, edit and deploy existing SSIS projects in Visual Studio 2019 (16.6.0), but cannot see SQL Server Integration Services or related items in the list of available templates when I add a new project. I do not know why this is, since SQL Server Integration Services Projects is installed, and I think that is all I need. I am seeking Advice on how to make SSIS appear in the list of templates.

Edit: I also searched the following terms with no luck: SSIS, SSDT, Business, SQL, Intelligence

Demonstration that SQL Server Integration Services Projects is installed:

Demonstration that I cannot see the related templates in the list when adding a new project:

答: I don't think you will find it by searching for Integration, did you try SSIS? Microsoft has a steps process here: https://learn.microsoft.com/en-us/sql/integration-services/lesson-1-1-creating-a-new-integration-services-project?view=sql-server-ver15


问:
Is this unchecked wrapper acceptable given the checked error will not be thrown in my program
I am using an implementation of the Consumer Functional Interface which wraps the lambda and seemingly catches any checked exception and throws a Runtime unchecked exception instead. I really like this because it cleans up my lambdas and I am using it in a situation where the checked error in this case an InterruptedException will never be thrown anyway. Is this acceptable and if not why? The following is the Interface implementation:

```<code>@FunctionalInterface
public interface ThrowingConsumer<T, E extends Throwable> {

void accept(T t) throws E;

static <T, E extends Throwable> Consumer<T> unchecked(ThrowingConsumer<T, E> c){
    return t -> {
        try{
            c.accept(t);
        } catch (Throwable e){
            throw new RuntimeException();
        }
    };
}
</code>```

}

The following is one example of its use. To give context it takes the EntrySet from a ConcurrentHashMap which contains for the key a user identifier and the value an Optional BlockingArrayQueue of type Packet used as a means of communication with each respective user. In this scenario I am sending every user a packet making them aware the relay sever will shutdown shortly to give them the opportunity to close cleanly before their threads are interrupted. Because the thread that spawned them was Main and has since finished execution and the Interrupt came from a ShutdownHook there is seemingly no way another interrupt could occur.

```<code>public class NetworkThread implements Runnable {

private SecureSocketManager secureSocketManager;
private final int tlsPort;
private final ConcurrentHashMap<String, Optional<BlockingQueue<Packet>>> channelMap;

public NetworkThread(SecureSocketManager secureSocketManager, ConcurrentHashMap<String, Optional<BlockingQueue<Packet>>> channelMap, int tlsPort) {
    this.secureSocketManager = secureSocketManager;
    this.tlsPort = tlsPort;
    this.channelMap = channelMap;
}

public void run() {

    ExecutorService clientThreads = Executors.newCachedThreadPool();
    // SETUP A SECURESOCKETMANAGER INSTANCE AND ESTABLISH A SERVER SOCKET ON DESIRED PORT

    try {
        SSLServerSocket sslServerSocket = secureSocketManager.getSslServerSocket(tlsPort);

        // BEGIN RECEIVING NEW CONNECTION INSTANCES ON THAT PORT AND LOOP

        while (!interrupted()) {

            try {
                SSLSocket sslSocket = (SSLSocket) sslServerSocket.accept();
                clientThreads.execute(new ClientThread(sslSocket, channelMap));

            } catch (IOException e) {
                e.printStackTrace();
            }
        }

//------------------------------RELEVANT CODE----------------------------------------------
    clientThreads.shutdown();

        // There should be no second interrupt so having this unchecked lambda wrapper is acceptable
       channelMap.entrySet()
               .stream()
               .filter(e -> e.getValue().isPresent())
               .forEach(ThrowingConsumer.unchecked(e -> e.getValue().get() // WRAPPER USE
               .put(new Packet(e.getKey(), "",Type.RELAY_SHUTDOWN, "", "" ))));
       if(!clientThreads.awaitTermination(3, TimeUnit.MINUTES))
           clientThreads.shutdownNow();

//------------------------------------------------------------------------------------------

    } catch (IOException | InterruptedException e) {
        clientThreads.shutdownNow();
    }

}

}
</code>```

Thanks

答:
Is this acceptable ...

Ask the person who has to accept it.  (Or decide for yourself!)

... and if not why?

It is unfortunate if you have to deal with checked exceptions in a stream.  But if you have to then you have to.

But I have a couple of problems with the way you are doing this:

You are catching all exceptions, not just checked exceptions.  If you were just catching Exception that would be bad.  But you are also catching Error.

You should wrap the original exception.  What you are currently doing is throwing it away.

Your wrapper exception needs a message.

I think that this addresses the above problems, as well as is possible:

```<code>@FunctionalInterface
public interface ThrowingConsumer<T, E extends Exception> {

    void accept(T t) throws E;

    static <TT, EE extends Exception> Consumer<TT> unchecked(ThrowingConsumer<TT, EE> c){
        return t -> {
            try {
                c.accept(t);
            } catch (Exception e) {
                if (e instanceof RuntimeException) {
                    throw e;
                } else {
                    throw new RuntimeException("Unexpected exception", e);
                }
            }
        };
    }
}
</code>```

Even this makes me feel a bit uneasy, because you are relying on generic type safety to ensure that you catch / wrap only a specific checked exception and it subclasses.  The problem is if you use unchecked conversions, etc, you can violate generic type safety.  In this case, it could result in the "wrong" checked exception being caught and wrapped.  Which is fine ... unless you try to unwrap it and cast it to the expected type.

Ideally you would use runtime type checking to ensure that you only wrap the expected exception.  But that requires you to pass the checked exception Class object to the unchecked method.

Body:
I am using an implementation of the Consumer Functional Interface which wraps the lambda and seemingly catches any checked exception and throws a Runtime unchecked exception instead. I really like this because it cleans up my lambdas and I am using it in a situation where the checked error in this case an InterruptedException will never be thrown anyway. Is this acceptable and if not why? The following is the Interface implementation:

@FunctionalInterface
public interface ThrowingConsumer<T, E extends Throwable> {

void accept(T t) throws E;

static <T, E extends Throwable> Consumer<T> unchecked(ThrowingConsumer<T, E> c){
    return t -> {
        try{
            c.accept(t);
        } catch (Throwable e){
            throw new RuntimeException();
        }
    };
}

}

The following is one example of its use. To give context it takes the EntrySet from a ConcurrentHashMap which contains for the key a user identifier and the value an Optional BlockingArrayQueue of type Packet used as a means of communication with each respective user. In this scenario I am sending every user a packet making them aware the relay sever will shutdown shortly to give them the opportunity to close cleanly before their threads are interrupted. Because the thread that spawned them was Main and has since finished execution and the Interrupt came from a ShutdownHook there is seemingly no way another interrupt could occur.

public class NetworkThread implements Runnable {

private SecureSocketManager secureSocketManager;
private final int tlsPort;
private final ConcurrentHashMap<String, Optional<BlockingQueue<Packet>>> channelMap;

public NetworkThread(SecureSocketManager secureSocketManager, ConcurrentHashMap<String, Optional<BlockingQueue<Packet>>> channelMap, int tlsPort) {
    this.secureSocketManager = secureSocketManager;
    this.tlsPort = tlsPort;
    this.channelMap = channelMap;
}

public void run() {

    ExecutorService clientThreads = Executors.newCachedThreadPool();
    // SETUP A SECURESOCKETMANAGER INSTANCE AND ESTABLISH A SERVER SOCKET ON DESIRED PORT

    try {
        SSLServerSocket sslServerSocket = secureSocketManager.getSslServerSocket(tlsPort);

        // BEGIN RECEIVING NEW CONNECTION INSTANCES ON THAT PORT AND LOOP

        while (!interrupted()) {

            try {
                SSLSocket sslSocket = (SSLSocket) sslServerSocket.accept();
                clientThreads.execute(new ClientThread(sslSocket, channelMap));

            } catch (IOException e) {
                e.printStackTrace();
            }
        }

//------------------------------RELEVANT CODE----------------------------------------------
    clientThreads.shutdown();

        // There should be no second interrupt so having this unchecked lambda wrapper is acceptable
       channelMap.entrySet()
               .stream()
               .filter(e -> e.getValue().isPresent())
               .forEach(ThrowingConsumer.unchecked(e -> e.getValue().get() // WRAPPER USE
               .put(new Packet(e.getKey(), "",Type.RELAY_SHUTDOWN, "", "" ))));
       if(!clientThreads.awaitTermination(3, TimeUnit.MINUTES))
           clientThreads.shutdownNow();

//------------------------------------------------------------------------------------------

    } catch (IOException | InterruptedException e) {
        clientThreads.shutdownNow();
    }

}

}

Thanks

Body:
Is this acceptable ...

Ask the person who has to accept it.  (Or decide for yourself!)

... and if not why?

It is unfortunate if you have to deal with checked exceptions in a stream.  But if you have to then you have to.

But I have a couple of problems with the way you are doing this:

You are catching all exceptions, not just checked exceptions.  If you were just catching Exception that would be bad.  But you are also catching Error.

You should wrap the original exception.  What you are currently doing is throwing it away.

Your wrapper exception needs a message.

I think that this addresses the above problems, as well as is possible:

@FunctionalInterface
public interface ThrowingConsumer<T, E extends Exception> {

    void accept(T t) throws E;

    static <TT, EE extends Exception> Consumer<TT> unchecked(ThrowingConsumer<TT, EE> c){
        return t -> {
            try {
                c.accept(t);
            } catch (Exception e) {
                if (e instanceof RuntimeException) {
                    throw e;
                } else {
                    throw new RuntimeException("Unexpected exception", e);
                }
            }
        };
    }
}

Even this makes me feel a bit uneasy, because you are relying on generic type safety to ensure that you catch / wrap only a specific checked exception and it subclasses.  The problem is if you use unchecked conversions, etc, you can violate generic type safety.  In this case, it could result in the "wrong" checked exception being caught and wrapped.  Which is fine ... unless you try to unwrap it and cast it to the expected type.

Ideally you would use runtime type checking to ensure that you only wrap the expected exception.  But that requires you to pass the checked exception Class object to the unchecked method.


问:
Hosting multiple Angular apps on one Azure Blob Storage
At work we are hosting multiple Angular JS apps on one Azure Blob Storage. There is an Azure Function which acts as a proxy and navigates to the suitable app folder and html file.

Now I want to replace some of the Angular JS apps with Angular 9 apps.

But the above-mentioned approach which works fine for the Angular JS apps does not work with the Angular 9 app.

If I host the Angular 9 app as static website on a different storage, it works fine. But as it is not possible to host more than one app as static website on an Azure Storage Account, I'm looking for a solution for this issue.

Is it possible to host more than one Angular 9 app on one Azure Blob Storage?
If yes: What should I do to make this work?

Thank you in advance!

答:
You can deploy static angular apps in subfolders and have multiple at the same time on different subfolders. It will require some configuration changes, possible some code changes because the default configuration will fail to load.

There are 2 main issues; baseHref and LocationStrategy.

The first part of the fix: The base tag

When the base tag is misconfigured, the app fails to load and the browser console displays 404 - Not Found errors for the missing files. Look at where it tried to find those files and adjust the base tag appropriately.

On subfolder structured production server, you might configure this to prevent errors. For example, when the URL to load the app is something like http://www.example.com/my/app/, the subfolder is my/app/ and you should add  to the server version of the index.html.

You can configure while building like this;

ng build --prod --output-path docs --base-href /my/app/
```
<p>You can also configure in angular.json;</p>
<pre class="lang-json prettyprint-override"><code>{
  //...
  "projects": {
    "app": {
      //...
      "architect": {
        "build": {
          //...
          "configurations": {
            "production": {
              "baseHref": "/my/app/",
              //...
</code>```

It is possible to set baseHref to . to achieve a generic app which works in all subfolders by relative paths but all used asset should follow a relative path within the app to comply.

The second part of the fix: LocationStrategy

When the location strategy is misconfigured, the app fails to load any sub-routes and the browser console displays 404 - Not Found.  HashLocationStrategy is another strategy that is immune to this subfolder – subroute mismatch.

HashLocationStrategy is a LocationStrategy used to configure the Location service to represent its state in the hash fragment of the browser’s URL.

To enable HashLocationStrategy in an Angular application we pass {useHash: true} when we are providing our routes with RouterModule, like so:

RouterModule.forRoot(routes, {useHash: true})
```

With LocationStrategy and baseHref set properly, you can simply build and copy everything within the output folder (dist/ by default) to the subfolder on the server.

For more information: https://celilsemi.erkiner.com/blog/static-angular-deployment-to-a-subfolder/

Body:
You can deploy static angular apps in subfolders and have multiple at the same time on different subfolders. It will require some configuration changes, possible some code changes because the default configuration will fail to load.

There are 2 main issues; baseHref and LocationStrategy.

The first part of the fix: The base tag

When the base tag is misconfigured, the app fails to load and the browser console displays 404 - Not Found errors for the missing files. Look at where it tried to find those files and adjust the base tag appropriately.

On subfolder structured production server, you might configure this to prevent errors. For example, when the URL to load the app is something like http://www.example.com/my/app/, the subfolder is my/app/ and you should add  to the server version of the index.html.

You can configure while building like this;

ng build --prod --output-path docs --base-href /my/app/

You can also configure in angular.json;

{
  //...
  "projects": {
    "app": {
      //...
      "architect": {
        "build": {
          //...
          "configurations": {
            "production": {
              "baseHref": "/my/app/",
              //...

It is possible to set baseHref to . to achieve a generic app which works in all subfolders by relative paths but all used asset should follow a relative path within the app to comply.

The second part of the fix: LocationStrategy

When the location strategy is misconfigured, the app fails to load any sub-routes and the browser console displays 404 - Not Found.  HashLocationStrategy is another strategy that is immune to this subfolder – subroute mismatch.

HashLocationStrategy is a LocationStrategy used to configure the Location service to represent its state in the hash fragment of the browser’s URL.

To enable HashLocationStrategy in an Angular application we pass {useHash: true} when we are providing our routes with RouterModule, like so:

RouterModule.forRoot(routes, {useHash: true})

With LocationStrategy and baseHref set properly, you can simply build and copy everything within the output folder (dist/ by default) to the subfolder on the server.

For more information: https://celilsemi.erkiner.com/blog/static-angular-deployment-to-a-subfolder/


问:
Error while reading avro file in spark code in Scala IDE
I am creating a data frame by reading avro file but getting error while reading the file in spark application in scala IDE.

```<code>package dataFrameBasics

import org.apache.spark.sql.SparkSession

object WorkingWithAvroFile {
  def main(ar : Array[String]): Unit={
    val ss= SparkSession.builder().master("local")
    .appName("Working with Avro File")
    .getOrCreate()

    val avroDF= ss.read
    .format("com.databricks.spark.avro")
    .load("C:/Spark_Files/userdata1.avro")

    avroDF.printSchema()
    avroDF.show(10)
    println("Count:"+avroDF.count())

  }

}
</code>```

On Console ,giving below error:
Exception in thread "main" java.lang.ClassNotFoundException: Failed to find data source: org.apache.spark.sql.avro.AvroFileFormat. Please find packages at http://spark.apache.org/third-party-projects.html

On Problems tab, giving below error:

spark-avro_2.11-3.2.0.jar of SparkCourseAsMavenProject build path is cross-compiled with an incompatible version of Scala (2.11.0). In case this report is mistaken, this check can be disabled in the compiler preference page.

In pom.xml,added dependency:

```<code><dependency>
        <groupId>com.databricks</groupId>
<artifactId>spark-avro_2.11</artifactId>
<version>3.2.0</version>
</dependency>
</code>```

Tried different versions of this library , but still its giving the same error.

答:
```<code>@QuickSilver--- Continuing stack trace(3)

***************
Caused by: java.io.IOException: (null) entry in command string: null chmod 0644 C:\Users\santosh\Scala\Workspace\SparkCourseAsMavenProject\output_destination\avro_file\_temporary\0\_temporary\attempt_20200528195351_0003_m_000000_3\part-00000-370dab6e-2c60-4b7c-82d6-e6c3645b538c-c000.avro
        at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:762)
        at org.apache.hadoop.util.Shell.execCommand(Shell.java:859)
        at org.apache.hadoop.util.Shell.execCommand(Shell.java:842)
        at
org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:661)
        at org.apache.hadoop.fs.ChecksumFileSystem$1.apply(ChecksumFileSystem.java:501)
        at org.apache.hadoop.fs.ChecksumFileSystem$FsOperation.run(ChecksumFileSystem.java:482)
        at org.apache.hadoop.fs.ChecksumFileSystem.setPermission(ChecksumFileSystem.java:498)
        at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:467)
        at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:433)
        at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:908)
        at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:889)
        at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:786)
        at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:775)
        at org.apache.spark.sql.avro.AvroOutputWriter$$anon$1.getAvroFileOutputStream(AvroOutputWriter.scala:58)
        at org.apache.avro.mapreduce.AvroKeyOutputFormat.getRecordWriter(AvroKeyOutputFormat.java:105)
        at org.apache.spark.sql.avro.AvroOutputWriter.<init>(AvroOutputWriter.scala:61)
        at org.apache.spark.sql.avro.AvroOutputWriterFactory.newInstance(AvroOutputWriterFactory.scala:43)
        at org.apache.spark.sql.execution.datasources.SingleDirectoryDataWriter.newOutputWriter(FileFormatDataWriter.scala:123)
        at org.apache.spark.sql.execution.datasources.SingleDirectoryDataWriter.<init>(FileFormatDataWriter.scala:108)
        at org.apache.spark.sql.execution.datasources.FileFormatWriter$.executeTask(FileFormatWriter.scala:236)
        at org.apache.spark.sql.execution.datasources.FileFormatWriter$.$anonfun$write$15(FileFormatWriter.scala:177)
        at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
        at org.apache.spark.scheduler.Task.run(Task.scala:123)
        at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:411)
        at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
        at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
    20/05/28 19:53:52 INFO SparkContext: Invoking stop() from shutdown hook
    20/05/28 19:53:52 INFO SparkUI: Stopped Spark web UI at http://Lenovo-PC:4040
    20/05/28 19:53:52 INFO MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
    20/05/28 19:53:53 INFO MemoryStore: MemoryStore cleared
    20/05/28 19:53:53 INFO BlockManager: BlockManager stopped
    20/05/28 19:53:53 INFO BlockManagerMaster: BlockManagerMaster stopped
    20/05/28 19:53:53 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
    20/05/28 19:53:53 INFO SparkContext: Successfully stopped SparkContext
    20/05/28 19:53:53 INFO ShutdownHookManager: Shutdown hook called
    20/05/28 19:53:53 INFO ShutdownHookManager: Deleting directory C:\Users\santosh\AppData\Local\Temp\spark-5a9bc571-30e8-4a94-8f95-f62c6fc9023c
</code>```

Body:
I am creating a data frame by reading avro file but getting error while reading the file in spark application in scala IDE.

package dataFrameBasics

import org.apache.spark.sql.SparkSession

object WorkingWithAvroFile {
  def main(ar : Array[String]): Unit={
    val ss= SparkSession.builder().master("local")
    .appName("Working with Avro File")
    .getOrCreate()

    val avroDF= ss.read
    .format("com.databricks.spark.avro")
    .load("C:/Spark_Files/userdata1.avro")

    avroDF.printSchema()
    avroDF.show(10)
    println("Count:"+avroDF.count())

  }

}

On Console ,giving below error:
Exception in thread "main" java.lang.ClassNotFoundException: Failed to find data source: org.apache.spark.sql.avro.AvroFileFormat. Please find packages at http://spark.apache.org/third-party-projects.html

On Problems tab, giving below error:

spark-avro_2.11-3.2.0.jar of SparkCourseAsMavenProject build path is cross-compiled with an incompatible version of Scala (2.11.0). In case this report is mistaken, this check can be disabled in the compiler preference page.

In pom.xml,added dependency:

<dependency>
        <groupId>com.databricks</groupId>
<artifactId>spark-avro_2.11</artifactId>
<version>3.2.0</version>
</dependency>

Tried different versions of this library , but still its giving the same error.


问:
React Redux,Updation of particular item in state
The below Reducer supposed to update quantity of particular object in state when I dispatch {"id": 14, 'quantity':3} or {"id": 14, 'quantity':2}.

```<code>
const initialState={ items = [ {"id": 14, 'quantity':1},{"id": 15,'quantity':1},{"id": 25,'quantity':1}] }

const cart =(state=initialState,action)=>{
    switch(action.type){
        case 'UPDATECART':{

             //need updating code here

                 }
        default: return state
    }
}

export default cart
</code>```

How can I update particular object in state's array, keeping other values unaltered ?

The updation function increment quantity of existing object by 1 and id will be permanent.

Proposed work flow

```<code> CURRENT STATE : items = [ {"id": 14, 'quantity':1},{"id": 15,'quantity':1},{"id": 25,'quantity':1}]
    INCOMING OBJECT : {"id": 15,'quantity':2}
    EXPECTED RESULTANT STATE : items = [ {"id": 14, 'quantity':1},{"id": 15,'quantity':3},{"id": 25,'quantity':1}]
</code>```

答:
Try this: </p>

```<code>case 'UPDATECART':{

         //need updating code here
     return state.map( cart => cart.id === INCOMING_OBJECT.id ? {...cart, quantity:
     cart.quantity + INCOMING_OBJECT.quantity}: cart )

             }
</code>```

Body:
The below Reducer supposed to update quantity of particular object in state when I dispatch {"id": 14, 'quantity':3} or {"id": 14, 'quantity':2}.

const initialState={ items = [ {"id": 14, 'quantity':1},{"id": 15,'quantity':1},{"id": 25,'quantity':1}] }

const cart =(state=initialState,action)=>{
    switch(action.type){
        case 'UPDATECART':{

             //need updating code here

                 }
        default: return state
    }
}

export default cart

How can I update particular object in state's array, keeping other values unaltered ?

The updation function increment quantity of existing object by 1 and id will be permanent.

Proposed work flow

 CURRENT STATE : items = [ {"id": 14, 'quantity':1},{"id": 15,'quantity':1},{"id": 25,'quantity':1}]
    INCOMING OBJECT : {"id": 15,'quantity':2}
    EXPECTED RESULTANT STATE : items = [ {"id": 14, 'quantity':1},{"id": 15,'quantity':3},{"id": 25,'quantity':1}]

Body:
Try this:

case 'UPDATECART':{

         //need updating code here
     return state.map( cart => cart.id === INCOMING_OBJECT.id ? {...cart, quantity:
     cart.quantity + INCOMING_OBJECT.quantity}: cart )

             }


问:
Discord API: How to Block a Specific Discord Channel?
I did a lot of research on how to block a channel from using commands, like Rythm's blackslist, but I didn't find it, could someone help me?</p>

答:
The concept of a "channel blacklist" for commands lives entirely outside of the Discord API. There's no way to have the bot tell Discord to stop receiving messages for a channel (aside from, of course, editing the channel's permission overrides to deny the bot permissions to see the channel).

However, this is a very simple task regardless. You undoubtedly have a message handler of some sort. Something that looks like this:

```<code>client.on('message', message => {

  //check commands and stuff

});
</code>```

If you want your bot to ignore a channel, it's the same thing as just not running your message handler in those channels. A naive implementation would be something like this:

```<code>client.on('message', message => {
  if(client.blacklist.includes(message.channel.id)) return;

  //check commands and stuff

});
</code>```

Of course, you may prefer to use a Map or a database call instead of just some array, and those options would be better for sure. But this is the basic pattern you need.

Body:
The concept of a "channel blacklist" for commands lives entirely outside of the Discord API. There's no way to have the bot tell Discord to stop receiving messages for a channel (aside from, of course, editing the channel's permission overrides to deny the bot permissions to see the channel).

However, this is a very simple task regardless. You undoubtedly have a message handler of some sort. Something that looks like this:

client.on('message', message => {

  //check commands and stuff

});

If you want your bot to ignore a channel, it's the same thing as just not running your message handler in those channels. A naive implementation would be something like this:

client.on('message', message => {
  if(client.blacklist.includes(message.channel.id)) return;

  //check commands and stuff

});

Of course, you may prefer to use a Map or a database call instead of just some array, and those options would be better for sure. But this is the basic pattern you need.


问:
TextField in Listview.builder {Flutter}
I'm trying to create an app that takes so much values and gives me back some results. For that I need to create a ListView.builder that holds bunch of TextFields. But I can't seem to find any solution.
Please explain the solution thoroughly. Thank you</p>

答:
ListView.builder is nothing but helps you build N number of a single widget which you will pass into the ListView.builder. To give you more insight on that, I would suggest two picked up references which will help in you great extent, these are:

ListView.builder Flutter

Flutter - Displaying Dynamic Contents

How this works is, you need to pass on some important aspects, like

The number of widget you want to display

Scroll Direction [If you want to have a horizontal, it is vertical by default]

This is how you create a ListView.builder

```<code>ListView.builder(
  itemCount: litems.length,
  itemBuilder: (BuildContext ctxt, int index) {
    return new TextField();
  }
)
</code>```

Above data will give you clarifications on this widget. Also, I would suggest to follow StackOverFlow Policy of asking question. I would definitely suggest you to follow the below steps before you ask question:

Research on some documentation, in this case, Flutter Dev Docs will help you in great extent

Google it, you will get it for sure

Give out the things which you have tried so far, to the developers here, to help them understand what is your real requirement, do not ask it without having tried anything

I hope this might help you in a great way. Feel free to drop in a comment, if you would like to add something to it. Happy Learning, and welcome to the Flutter Community :)

Title: TextField in Listview.builder {Flutter}

Body:
ListView.builder is nothing but helps you build N number of a single widget which you will pass into the ListView.builder. To give you more insight on that, I would suggest two picked up references which will help in you great extent, these are:

ListView.builder Flutter

Flutter - Displaying Dynamic Contents

How this works is, you need to pass on some important aspects, like

The number of widget you want to display

Scroll Direction [If you want to have a horizontal, it is vertical by default]

This is how you create a ListView.builder

ListView.builder(
  itemCount: litems.length,
  itemBuilder: (BuildContext ctxt, int index) {
    return new TextField();
  }
)

Above data will give you clarifications on this widget. Also, I would suggest to follow StackOverFlow Policy of asking question. I would definitely suggest you to follow the below steps before you ask question:

Research on some documentation, in this case, Flutter Dev Docs will help you in great extent

Google it, you will get it for sure

Give out the things which you have tried so far, to the developers here, to help them understand what is your real requirement, do not ask it without having tried anything

I hope this might help you in a great way. Feel free to drop in a comment, if you would like to add something to it. Happy Learning, and welcome to the Flutter Community :)


问:
Django pre_save, instance is None
I am writing the pre_save method to create the slug for model, for some reason my instance is none.

```<code>@receiver(pre_save, sender=Employee)
def pre_save_employee_receiver(sender, instance, *args, **kwargs):
    slug = slugify(" ".join([instance.name, instance.surname, instance.id]))
    instance.slug = slug
</code>```

Error is :

```<code>File "E:\Work\hire_handler\employees\models.py", line 60, in pre_save_employee_receiver
    slug = slugify(" ".join([instance.name, instance.surname, instance.id]))
TypeError: sequence item 2: expected str instance, NoneType found
</code>```

答:
No, the instance is not None, the primary key (instance.id) is None.

That makes perfect sense, before creating the object at the database side, the item has no primary key, since database dispatches a primary key.

If you thus want to work with a primary key, you need to work with a post_save item, furthermore likely the id is an int, so it will still not work, since ' '.join() can not join integers.

You can use a post_save trigger, and save it again, but that will result in making two queries when you create a new Employee:

```<code>@receiver(post_save, sender=Employee)
def post_save_employee_receiver(sender, instance, created, *args, **kwargs):
    if <b>created</b>:
        instance.slug = slugify(f'{instance.name} {instance.surname} {instance.pk}')
        instance<b>.save()</b></code>```

Furthermore signals are usually a bit of an antipattern, especially since when you for example perform .bulk_create(), .update(), etc. the signals are not triggered.

Title: Django pre_save, instance is None

Body:
I am writing the pre_save method to create the slug for model, for some reason my instance is none.

@receiver(pre_save, sender=Employee)
def pre_save_employee_receiver(sender, instance, *args, **kwargs):
    slug = slugify(" ".join([instance.name, instance.surname, instance.id]))
    instance.slug = slug

Error is :

File "E:\Work\hire_handler\employees\models.py", line 60, in pre_save_employee_receiver
    slug = slugify(" ".join([instance.name, instance.surname, instance.id]))
TypeError: sequence item 2: expected str instance, NoneType found

Body:
No, the instance is not None, the primary key (instance.id) is None.

That makes perfect sense, before creating the object at the database side, the item has no primary key, since database dispatches a primary key.

If you thus want to work with a primary key, you need to work with a post_save item, furthermore likely the id is an int, so it will still not work, since ' '.join() can not join integers.

You can use a post_save trigger, and save it again, but that will result in making two queries when you create a new Employee:

@receiver(post_save, sender=Employee)
def post_save_employee_receiver(sender, instance, created, *args, **kwargs):
    if created:
        instance.slug = slugify(f'{instance.name} {instance.surname} {instance.pk}')
        instance.save()

Furthermore signals are usually a bit of an antipattern, especially since when you for example perform .bulk_create(), .update(), etc. the signals are not triggered.


问:
Can someone explain this regular expression?
```<code>(.*)([0-9]{6})|(.*)([0-9]{4})
</code>```

<p>Does it mean pick up a number that is either 6 digits or 4 digits ? Is there a concise way to write it ? </p>

答:
literally: any number of characters which ends with exactly 6 numbers or exactly 4 numbers.

technically:
The wierd thing is that the .* would allow anything that has any number of numbers prior to that, so anything that ends with 4 numbers or more would match?

Body:
(.*)([0-9]{6})|(.*)([0-9]{4})

Does it mean pick up a number that is either 6 digits or 4 digits ? Is there a concise way to write it ?


问:
Python struct to convert int to IEEE float
I implemented two lambda functions using struct

```<code>int_to_float = lambda n: struct.unpack('@f', struct.pack('@I', n))[0]
float_to_int = lambda f: struct.unpack('@I', struct.pack('@f', f))[0]
</code>```

Then I test:

```<code>f1 = 1.90826472023605
i = float_to_int(f1)
f2 = int_to_float(i)
print(f1, f2, i)
print(f1-f2)
</code>```

The print results are:

```<code>1.90826472023605   1.9082647562026978   1072972293
-3.596664766014612e-08
</code>```

Does this means struct has an issue of pack/unpack?

答:
You're not using enough bits to retain your original value, which is a 64-bit double.  Use the d and Q types instead.

```<code>def int_to_float(n):
    return struct.unpack('@d', struct.pack('@Q', n))[0]

def float_to_int(f):
    return struct.unpack('@Q', struct.pack('@d', f))[0]
</code>```

Body:
I implemented two lambda functions using struct

int_to_float = lambda n: struct.unpack('@f', struct.pack('@I', n))[0]
float_to_int = lambda f: struct.unpack('@I', struct.pack('@f', f))[0]

Then I test:

f1 = 1.90826472023605
i = float_to_int(f1)
f2 = int_to_float(i)
print(f1, f2, i)
print(f1-f2)

The print results are:

1.90826472023605   1.9082647562026978   1072972293
-3.596664766014612e-08

Does this means struct has an issue of pack/unpack?

Body:
You're not using enough bits to retain your original value, which is a 64-bit double.  Use the d and Q types instead.

def int_to_float(n):
    return struct.unpack('@d', struct.pack('@Q', n))[0]

def float_to_int(f):
    return struct.unpack('@Q', struct.pack('@d', f))[0]