问:
Sending enable interrupts (0xF4) command to ps2 keyboard results in system crash. OSDEV
I have just finished initializing the IDT and started the ps2 stuff. I got the ps2 controller initializer and now I want to enable the interrupts in the first ps2 port or the keyboard. Here is my code for initializing the ps2 controller:

```<code>
uint8_t initPS2()
{
    // Start by disabling devices

    disableFirstPS2Port();
    disableSecondPS2Port();

    // Flush the input buffer

    inb(PS2DATA);

    // Set the state of the status register.

    outb(PS2COMMAND, 0x20);
    uint8_t status = inb(PS2DATA);
    uint8_t dual0 = status & 0b00100000 == 0b00100000; // if dual = 0, there is no second channel
    status |= 0b01000011;
    outb(PS2COMMAND, 0x60);
    outb(PS2DATA, status);

    // Perform self test

    outb(PS2COMMAND, 0xAA);
    uint8_t response = inb(PS2DATA);
    if (response == 0xFC)
        return 0xFC;
    outb(PS2COMMAND, 0x60);
    outb(PS2DATA, status);

    // test dual channel controller
    uint8_t dual = 0xFF;

    if (dual0 == 1)
    {
        outb(PS2COMMAND, 0xAE);
        outb(PS2COMMAND, 0x20);
        status = inb(PS2DATA);
        uint8_t dual1 = status & 0b00100000 == 0b00100000;
        // if dual1 == 0, dual channel exists
        outb(PS2COMMAND, 0xAD);
        dual = !dual1;
    }
    else
    {
        dual = 0;
    }

    // Perform interface tests

    outb(PS2COMMAND, 0xAB);
    uint8_t responseIT1 = inb(PS2DATA);
    uint8_t responseIT2;
    if (dual == 1)
    {
        outb(PS2COMMAND, 0xA9);
        responseIT2 = inb(PS2DATA);
    }

    if (responseIT1 != 0x0)
        return responseIT1;
    if (responseIT2 != 0x0 && dual == 1)
        return responseIT2;

    // Enable devices

    enableFirstPS2Port();
    if (dual == 1)
        enableSecondPS2Port();

    writeToFirstPS2Port(0xFF);
    response = readPS2DataPort();
    if (response == 0xFC)
        return 0xFC;
    writeToSecondPS2Port(0xFF);
    response = readPS2DataPort();
    if (response == 0xFC)
        return 0xFC;

        return 0;
}

</code>```

I follow the tutorial on OSDev wiki for initializing the ps2 controller.

Now I try to enable scanning on the keyboard with the 0xf4 command:

```<code>uint8_t commandKeyboard(unsigned char byt)
{

    writeToFirstPS2Port(byt);
    uint8_t response = readPS2DataPort();

    if (response == 0xFE)
    {
        writeToFirstPS2Port(byt);
        response = readPS2DataPort();

        if (response == 0xFE)
        {
            writeToFirstPS2Port(byt);
            response = readPS2DataPort();
        }
    }

    return response;
}

uint8_t initKeyboard()
{
    uint8_t res = commandKeyboard(0xF4);
    return res;
}
</code>```

Note that the writeToFirstPS2Port and likewise other ps2 function do have a proper controllerWait function which makes sure that the data port is writable/readable.

I call this code before doing sti after I load my idt:

```<code>    idt_flush(&currentIDT);

    uint8_t ps2init = initPS2();
    printk("PS2 Initalized, returned with 0x%x\n", ps2init);
    while (1)
        ;
    uint8_t keyboardinit = initKeyboard();
    printk("PS2 Initalized, returned with 0x%x\n", keyboardinit);

    outb(PIC1_DATA, 0);
    outb(PIC2_DATA, 0);

    //int d = 5 / 0;
    asm volatile("sti");
</code>```

Also here is my IRQ Handler:

```<code>void irq1_handler(interrupt_frame_t *frame)
{
    inb(0x60);
    default_irq_handler(frame);
}
</code>```

I think the keyboard controller required for me to read 0x60 port before sending another interrupt so I thought that may help.

With this code here is what happens. I successfully initialize the IDT and then call the initializePS2 which returns with 0, meaning it succeeded. After that I start the initKeyboard call which doesn't even return let alone succeed. When I send the command to the keyboard, the system crashes. I am running this using qemu and I first thought this was a triple fault so I added -d int to see what kind of exceptions were raised but it seems that no exception is raised before the system closes itself.

Here is the github link to the project:

https://github.com/Danyy427/NEWOSDEV/tree/master/new%2064%20bit

The files are under src/Kernel/Hardware/PS2 and .../Hardware/Keyboard

The IDT is under src/Kernel/IDT

Any help is appreciated. Thank you.

答:
First add the interrupt attribute on your interrupt handler :

```<code> void __attribute__((interrupt)) Irq1Handler(interrupt_frame* frame);
</code>```

This will cause the compiler to get parameters from stack rather than getting them using __cdecl calling convention. And also performing an iret (Interrupt Return) rather than a normal ret (Return). Also add a parameter to gcc when compiling your interrupts file "-mgeneral-regs-only". This will prevent the compiler from using sse registers and generating a warning about that.

Body:
I have just finished initializing the IDT and started the ps2 stuff. I got the ps2 controller initializer and now I want to enable the interrupts in the first ps2 port or the keyboard. Here is my code for initializing the ps2 controller:

uint8_t initPS2()
{
    // Start by disabling devices

    disableFirstPS2Port();
    disableSecondPS2Port();

    // Flush the input buffer

    inb(PS2DATA);

    // Set the state of the status register.

    outb(PS2COMMAND, 0x20);
    uint8_t status = inb(PS2DATA);
    uint8_t dual0 = status & 0b00100000 == 0b00100000; // if dual = 0, there is no second channel
    status |= 0b01000011;
    outb(PS2COMMAND, 0x60);
    outb(PS2DATA, status);

    // Perform self test

    outb(PS2COMMAND, 0xAA);
    uint8_t response = inb(PS2DATA);
    if (response == 0xFC)
        return 0xFC;
    outb(PS2COMMAND, 0x60);
    outb(PS2DATA, status);

    // test dual channel controller
    uint8_t dual = 0xFF;

    if (dual0 == 1)
    {
        outb(PS2COMMAND, 0xAE);
        outb(PS2COMMAND, 0x20);
        status = inb(PS2DATA);
        uint8_t dual1 = status & 0b00100000 == 0b00100000;
        // if dual1 == 0, dual channel exists
        outb(PS2COMMAND, 0xAD);
        dual = !dual1;
    }
    else
    {
        dual = 0;
    }

    // Perform interface tests

    outb(PS2COMMAND, 0xAB);
    uint8_t responseIT1 = inb(PS2DATA);
    uint8_t responseIT2;
    if (dual == 1)
    {
        outb(PS2COMMAND, 0xA9);
        responseIT2 = inb(PS2DATA);
    }

    if (responseIT1 != 0x0)
        return responseIT1;
    if (responseIT2 != 0x0 && dual == 1)
        return responseIT2;

    // Enable devices

    enableFirstPS2Port();
    if (dual == 1)
        enableSecondPS2Port();

    writeToFirstPS2Port(0xFF);
    response = readPS2DataPort();
    if (response == 0xFC)
        return 0xFC;
    writeToSecondPS2Port(0xFF);
    response = readPS2DataPort();
    if (response == 0xFC)
        return 0xFC;

        return 0;
}

I follow the tutorial on OSDev wiki for initializing the ps2 controller.

Now I try to enable scanning on the keyboard with the 0xf4 command:

uint8_t commandKeyboard(unsigned char byt)
{

    writeToFirstPS2Port(byt);
    uint8_t response = readPS2DataPort();

    if (response == 0xFE)
    {
        writeToFirstPS2Port(byt);
        response = readPS2DataPort();

        if (response == 0xFE)
        {
            writeToFirstPS2Port(byt);
            response = readPS2DataPort();
        }
    }

    return response;
}

uint8_t initKeyboard()
{
    uint8_t res = commandKeyboard(0xF4);
    return res;
}

Note that the writeToFirstPS2Port and likewise other ps2 function do have a proper controllerWait function which makes sure that the data port is writable/readable.

I call this code before doing sti after I load my idt:

    idt_flush(&currentIDT);

    uint8_t ps2init = initPS2();
    printk("PS2 Initalized, returned with 0x%x\n", ps2init);
    while (1)
        ;
    uint8_t keyboardinit = initKeyboard();
    printk("PS2 Initalized, returned with 0x%x\n", keyboardinit);

    outb(PIC1_DATA, 0);
    outb(PIC2_DATA, 0);

    //int d = 5 / 0;
    asm volatile("sti");

Also here is my IRQ Handler:

void irq1_handler(interrupt_frame_t *frame)
{
    inb(0x60);
    default_irq_handler(frame);
}

I think the keyboard controller required for me to read 0x60 port before sending another interrupt so I thought that may help.

With this code here is what happens. I successfully initialize the IDT and then call the initializePS2 which returns with 0, meaning it succeeded. After that I start the initKeyboard call which doesn't even return let alone succeed. When I send the command to the keyboard, the system crashes. I am running this using qemu and I first thought this was a triple fault so I added -d int to see what kind of exceptions were raised but it seems that no exception is raised before the system closes itself.

Here is the github link to the project:

https://github.com/Danyy427/NEWOSDEV/tree/master/new%2064%20bit

The files are under src/Kernel/Hardware/PS2 and .../Hardware/Keyboard

The IDT is under src/Kernel/IDT

Any help is appreciated. Thank you.

Body:
First add the interrupt attribute on your interrupt handler :

 void __attribute__((interrupt)) Irq1Handler(interrupt_frame* frame);

This will cause the compiler to get parameters from stack rather than getting them using __cdecl calling convention. And also performing an iret (Interrupt Return) rather than a normal ret (Return). Also add a parameter to gcc when compiling your interrupts file "-mgeneral-regs-only". This will prevent the compiler from using sse registers and generating a warning about that.


来源: stackexchange

问:
Options for systat (iostat/sar) in cygwin?
Going through cygwin's search, I cannot find linux sysstat apps (e.g., iostat or sar).

Searching the web, I found some old messages of people unsuccessfully trying to compile sysstat from source on cygwin.

Has anyone succeeded in adding sysstat apps to cygwin?

Environment:

CYGWIN_NT-6.1 t43 1.7.22(0.268/5/3) 2013-07-22 17:06 i686 Cygwin

on Windows 7

答:
in cygwin bash under Windows 10+:

```<code>powershell -Command "& {Get-Counter -Counter '\processor(_total)\% processor time' -Continuous}"
</code>```

Not exactly the same, but gives you access to all Windows performance counters.

ContentLicense: CC BY-SA 3.0

Body:
in cygwin bash under Windows 10+:

powershell -Command "& {Get-Counter -Counter '\processor(_total)\% processor time' -Continuous}"

Not exactly the same, but gives you access to all Windows performance counters.

ContentLicense: CC BY-SA 4.0


问:
(Closed) Discord bot leave message with author and thumbnail of user
I am trying to make my bot create embed leave messages which is mostly figured out but I'm struggling with author and thumbnail because I want them to have the person who left pfp. I also want to how to set the author user as the person who left but specifically in the format of bot#0001.

client.on('guildMemberRemove', member => {
  let embed = new Discord.MessageEmbed()
    .setColor('#78a6dd')
    .setDescription(`<@${member.user.id}> left`)
});```

答:
Try use this code</p>
```<code>    client.on("guildMemberRemove", (member) => {
    let goodbyembed = new Discord.MessageEmbed()
.setAuthor(${member.user.tag} + " just left!", member.user.avatarURL())
.setDescription( /* Message here */)
.setColor('#78a6dd');
member.guild.channels.cache.get("samechannelid").send(goodbyembed)

        .catch((err) => console.log(err));
});
</code>```

Body:
Try use this code

    client.on("guildMemberRemove", (member) => {
    let goodbyembed = new Discord.MessageEmbed()
.setAuthor(${member.user.tag} + " just left!", member.user.avatarURL())
.setDescription( /* Message here */)
.setColor('#78a6dd');
member.guild.channels.cache.get("samechannelid").send(goodbyembed)

        .catch((err) => console.log(err));
});


问:
using a modules method in Pyspark map
I have heard that it is available to call a method of another module in python to bring some calculations that is not implemented in spark and of course it is inefficient to do that.
I need a method to compute eigenvector centrality of a graph (as it is not available in graphframes module) .
I am aware that there is a way to do that in Scala using sparkling graph, but I need python to include everything.
I am newbie to spark RDD and I am wondering what is the wrong with the code below or even if this is a proper way of doing this

```<code>import networkx as nx

def func1(dt):
    G = nx.Graph()
    src = dt.Provider
    dest = dt.AttendingPhysician
    gr = src.zip(dest)
    G = nx.from_edgelist(gr)
    deg =nx.eigenvector_centrality(G)
    return deg

rdd2=inpatient.rdd.map(lambda x: func1(x))'
rdd2.collect()
</code>```

inpatient is a dataframe read from a CSV file which I am looking forward to make a graph that is directed from nodes in column Provider to nodes in column AttendingPhysician

there is an error that I am encountered with which is:

```<code>AttributeError: 'str' object has no attribute 'zip'
</code>```

答:
What you need is to understand the so called user defined functions functionality. However plain python UDFs are not very efficient.

You can use UDFs efficiently via the provided pandas_udf GROUPED_MAP functionality.

from the documentation:

Grouped map Pandas UDFs first splits a Spark DataFrame into groups based on the conditions specified in the groupby operator, applies a user-defined function (pandas.DataFrame -> pandas.DataFrame) to each group, combines and returns the results as a new Spark DataFrame.

An example for networkx eigenvector centrality running on pyspark is below. I group per cluster_id which is the result from graphframes connected components function:

```<code>def eigencentrality(
    sparkdf, src="src", dst="dst", cluster_id_colname="cluster_id",
):

    """
    Args:
        sparkdf: imput edgelist Spark DataFrame
        src: src column name
        dst: dst column name
        distance_colname: distance column name
        cluster_id_colname: Graphframes-created connected components created cluster_id
    Returns:
        node_id:
        eigen_centrality: eigenvector centrality of cluster cluster_id
        cluster_id: cluster_id corresponding to the node_id

    """
    ecschema = StructType(
        [
            StructField("node_id", StringType()),
            StructField("eigen_centrality", DoubleType()),
            StructField(cluster_id_colname, LongType()),
        ]
    )

    psrc = src
    pdst = dst

    @pandas_udf(ecschema, PandasUDFType.GROUPED_MAP)
    def eigenc(pdf: pd.DataFrame) -> pd.DataFrame:
        nxGraph = nx.Graph()
        nxGraph = nx.from_pandas_edgelist(pdf, psrc, pdst)
        ec = eigenvector_centrality(nxGraph, tol=1e-03)
        out_df = (
            pd.DataFrame.from_dict(ec, orient="index", columns=["eigen_centrality"])
            .reset_index()
            .rename(
                columns={"index": "node_id", "eigen_centrality": "eigen_centrality"}
            )
        )

        cluster_id = pdf[cluster_id_colname][0]
        out_df[cluster_id_colname] = cluster_id
        return out_df

    out = sparkdf.groupby(cluster_id_colname).apply(eigenc)
    return out
</code>```

NB. I have created the splink_graph package in order to run parallelised networkx graph operations like the above on Pyspark efficiently. This is where this code comes from. If you are interested have a look there to see how other graph metrics have been implemented...

Title: using a modules method in Pyspark map

Body:
I have heard that it is available to call a method of another module in python to bring some calculations that is not implemented in spark and of course it is inefficient to do that.
I need a method to compute eigenvector centrality of a graph (as it is not available in graphframes module) .
I am aware that there is a way to do that in Scala using sparkling graph, but I need python to include everything.
I am newbie to spark RDD and I am wondering what is the wrong with the code below or even if this is a proper way of doing this

import networkx as nx

def func1(dt):
    G = nx.Graph()
    src = dt.Provider
    dest = dt.AttendingPhysician
    gr = src.zip(dest)
    G = nx.from_edgelist(gr)
    deg =nx.eigenvector_centrality(G)
    return deg

rdd2=inpatient.rdd.map(lambda x: func1(x))'
rdd2.collect()

inpatient is a dataframe read from a CSV file which I am looking forward to make a graph that is directed from nodes in column Provider to nodes in column AttendingPhysician

there is an error that I am encountered with which is:

AttributeError: 'str' object has no attribute 'zip'

Body:
What you need is to understand the so called user defined functions functionality. However plain python UDFs are not very efficient.

You can use UDFs efficiently via the provided pandas_udf GROUPED_MAP functionality.

from the documentation:

Grouped map Pandas UDFs first splits a Spark DataFrame into groups based on the conditions specified in the groupby operator, applies a user-defined function (pandas.DataFrame -> pandas.DataFrame) to each group, combines and returns the results as a new Spark DataFrame.

An example for networkx eigenvector centrality running on pyspark is below. I group per cluster_id which is the result from graphframes connected components function:

def eigencentrality(
    sparkdf, src="src", dst="dst", cluster_id_colname="cluster_id",
):

    """
    Args:
        sparkdf: imput edgelist Spark DataFrame
        src: src column name
        dst: dst column name
        distance_colname: distance column name
        cluster_id_colname: Graphframes-created connected components created cluster_id
    Returns:
        node_id:
        eigen_centrality: eigenvector centrality of cluster cluster_id
        cluster_id: cluster_id corresponding to the node_id

    """
    ecschema = StructType(
        [
            StructField("node_id", StringType()),
            StructField("eigen_centrality", DoubleType()),
            StructField(cluster_id_colname, LongType()),
        ]
    )

    psrc = src
    pdst = dst

    @pandas_udf(ecschema, PandasUDFType.GROUPED_MAP)
    def eigenc(pdf: pd.DataFrame) -> pd.DataFrame:
        nxGraph = nx.Graph()
        nxGraph = nx.from_pandas_edgelist(pdf, psrc, pdst)
        ec = eigenvector_centrality(nxGraph, tol=1e-03)
        out_df = (
            pd.DataFrame.from_dict(ec, orient="index", columns=["eigen_centrality"])
            .reset_index()
            .rename(
                columns={"index": "node_id", "eigen_centrality": "eigen_centrality"}
            )
        )

        cluster_id = pdf[cluster_id_colname][0]
        out_df[cluster_id_colname] = cluster_id
        return out_df

    out = sparkdf.groupby(cluster_id_colname).apply(eigenc)
    return out

NB. I have created the splink_graph package in order to run parallelised networkx graph operations like the above on Pyspark efficiently. This is where this code comes from. If you are interested have a look there to see how other graph metrics have been implemented...


问:
Upload file using HttpClient
i need upload any file vedio,image and doc by api from client to server
This code works for me, but for files smaller than 2MB
I can't download files up to 10 , 20MB
I tried to change the code and style
I also converted it to a byte[] and split it into several files, but it didn't work
Please Help

this code for client

```<code>uri = new Uri(WebAPIUrl + "setimg/");
        try
        {
            var content = new MultipartFormDataContent();
            if(mediafile!=null)
            {

                var streamContent = new StreamContent(mediafile);
                streamContent.Headers.Add("Content-Type", "application/octet-stream");
                content.Add(streamContent, "\"img\"", $"\"{mediafile.Name}\"");

            }

            string json = JsonConvert.SerializeObject(u);
            StringContent content1 = new StringContent(json, Encoding.UTF8, "application/json");
            content.Add(content1, "value");
            bool fin = false;
            do
            {
                fin = true;
                var response = await client.PostAsync(uri, content);
                if (response.IsSuccessStatusCode)
                {
                    var result = await response.Content.ReadAsStringAsync();
                    try
                    {
                        var Items = JsonConvert.DeserializeObject<List<string>>(result);
                        return Items;
                    }
                    catch
                    {

                    }
</code>```

}

this code for server

```<code>    public List<string> upload()
    {
        var httpRequset = HttpContext.Current.Request;
        string imageslink = "";
        if (httpRequset.Files.Count > 0)
        {
            foreach (string file in httpRequset.Files)
            {
                var postedfile = httpRequset.Files[file];
                var filename = postedfile.FileName.Split('\\').LastOrDefault().Split('/').LastOrDefault();
                // var filepath = HttpContext.Current.Server.MapPath("~/Uploads/" + filename);
                imageslink += "D:\\casting\\" + filename + ";";
                var filepath = "D:\\casting\\" + filename;
                postedfile.SaveAs(filepath);
            }
</code>```

please help

答:
After communicating with a friend
After searching a lot, I found the cause of the problem is the server itself iis
As it gives a default value for the maximum upload limit of 2 megabytes

Solve the problem if the server is on iis

https://stackoverflow.com/a/58566617/8967661

IIS -> web. config :

```<code><?xml version="1.0" encoding="utf-8"?>
<configuration>
  <system.web>
    <!-- ~ 2GB -->
    <httpRuntime maxRequestLength="2147483647" /> // kbytes
  </system.web>
  <system.webServer>
    <security>
      <requestFiltering>
        <!-- ~ 4GB -->
        <requestLimits maxAllowedContentLength="4294967295" /> // bytes
      </requestFiltering>
    </security>
  </system.webServer>
</configuration>
</code>```

Title: Upload file using HttpClient

Body:
i need upload any file vedio,image and doc by api from client to server
This code works for me, but for files smaller than 2MB
I can't download files up to 10 , 20MB
I tried to change the code and style
I also converted it to a byte[] and split it into several files, but it didn't work
Please Help

this code for client

uri = new Uri(WebAPIUrl + "setimg/");
        try
        {
            var content = new MultipartFormDataContent();
            if(mediafile!=null)
            {

                var streamContent = new StreamContent(mediafile);
                streamContent.Headers.Add("Content-Type", "application/octet-stream");
                content.Add(streamContent, "\"img\"", $"\"{mediafile.Name}\"");

            }

            string json = JsonConvert.SerializeObject(u);
            StringContent content1 = new StringContent(json, Encoding.UTF8, "application/json");
            content.Add(content1, "value");
            bool fin = false;
            do
            {
                fin = true;
                var response = await client.PostAsync(uri, content);
                if (response.IsSuccessStatusCode)
                {
                    var result = await response.Content.ReadAsStringAsync();
                    try
                    {
                        var Items = JsonConvert.DeserializeObject<List<string>>(result);
                        return Items;
                    }
                    catch
                    {

                    }

}

this code for server

    public List<string> upload()
    {
        var httpRequset = HttpContext.Current.Request;
        string imageslink = "";
        if (httpRequset.Files.Count > 0)
        {
            foreach (string file in httpRequset.Files)
            {
                var postedfile = httpRequset.Files[file];
                var filename = postedfile.FileName.Split('\\').LastOrDefault().Split('/').LastOrDefault();
                // var filepath = HttpContext.Current.Server.MapPath("~/Uploads/" + filename);
                imageslink += "D:\\casting\\" + filename + ";";
                var filepath = "D:\\casting\\" + filename;
                postedfile.SaveAs(filepath);
            }

please help

Body:
After communicating with a friend
After searching a lot, I found the cause of the problem is the server itself iis
As it gives a default value for the maximum upload limit of 2 megabytes

Solve the problem if the server is on iis

https://stackoverflow.com/a/58566617/8967661

IIS -> web. config :

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <system.web>
    <!-- ~ 2GB -->
    <httpRuntime maxRequestLength="2147483647" /> // kbytes
  </system.web>
  <system.webServer>
    <security>
      <requestFiltering>
        <!-- ~ 4GB -->
        <requestLimits maxAllowedContentLength="4294967295" /> // bytes
      </requestFiltering>
    </security>
  </system.webServer>
</configuration>


问:
tfrecords: encoding images results in distorted images
I am trying to make a tfrecords from image_dataset_from_directory; but when I try to visualize images to check if the encoding was correct, the images turn out to be distorted of some kind.

How I created the tfrecord:

Step 1: create dataset using image_dataset_from_directory

```<code>data_dir = 'path to JPG dataset'

load_split = partial(
    tf.keras.preprocessing.image_dataset_from_directory,
    data_dir,
    validation_split=0.2,
    shuffle=True,
    seed=123,
    image_size=(IMG_HEIGHT, IMG_WIDTH),
    batch_size=1,
)

ds_train = load_split(subset='training')
ds_valid = load_split(subset='validation')
</code>```

Step 2: encoding functions

```<code>def process_image(image, label):
    image = tf.image.convert_image_dtype(image, dtype=tf.uint8)
    image = tf.io.encode_jpeg(image)

    label = tf.one_hot(label, NUM_CLASSES)

    return image, label

def make_example(encoded_image, label):
    image_feature = Feature(
        bytes_list=BytesList(value=[
            encoded_image,
        ]),
    )
    label_feature = Feature(
        float_list=FloatList(value=label)
    )

    features = Features(feature={
        'image': image_feature,
        'label': label_feature,
    })

    example = Example(features=features)

    return example.SerializeToString()
</code>```

Step 3: encoding and creating tfrecord

```<code>ds_train_encoded = (
    ds_train
    .unbatch()
    .map(process_image)
)

ds_valid_encoded = (
    ds_valid
    .unbatch()
    .map(process_image)
)

ds_train_encoded_iter = (
    ds_train_encoded
    .as_numpy_iterator()
)
with tf.io.TFRecordWriter(path='train.tfrecord') as f: # you can pass gs:// path here :)
    for encoded_image, label in ds_train_encoded_iter:
        example = make_example(encoded_image, label)
        f.write(example)

ds_valid_encoded_iter = (
    ds_valid_encoded
    .as_numpy_iterator()
)
with tf.io.TFRecordWriter(path='/home/et/medai/images/tfrecords/test.tfrecord') as f:
    for encoded_image, label in ds_valid_encoded_iter:
        example = make_example(encoded_image, label)
        f.write(example)
</code>```

How I tried to visualize the images in the tfrecords

Step 1: decoding functions

```<code>def _parse_image_function(example):
    image_feature_description = {
        'image': tf.io.FixedLenFeature([], tf.string),
        'label': tf.io.FixedLenFeature([40], tf.float32),
    }

    features = tf.io.parse_single_example(example, image_feature_description)
    image = tf.image.decode_jpeg(features['image'], channels=3)
    image = tf.image.resize(image, [IMG_SIZE, IMG_SIZE])
    # image = features['image']
    label = features['label']

    return image, label

def read_dataset(filename, batch_size):
    dataset = tf.data.TFRecordDataset(filename)
    dataset = dataset.map(_parse_image_function, num_parallel_calls=tf.data.experimental.AUTOTUNE)
    dataset = dataset.shuffle(500)
    dataset = dataset.batch(batch_size, drop_remainder=True)
    # dataset = dataset.repeat()
    dataset = dataset.prefetch(buffer_size=tf.data.experimental.AUTOTUNE)

    return dataset
</code>```

Step 2: decode and display

```<code>x = read_dataset('/home/et/medai/images/tfrecords/tests_train.tfrecord', 32)

plt.figure(figsize=(10, 10))
batch_size = 32
for images, labels in x.take(1):
    for i in range(batch_size):
        # display.display(display.Image(data=images[i].numpy()))

        ax = plt.subplot(6, 6, i + 1)
        plt.imshow(images[i].numpy().astype("uint8"))
        plt.axis("off")

</code>```

The result is something distorted:
https://i.stack.imgur.com/tCAik.jpg

I am not quite sure where this distortion comes from. Original images look like this:

https://i.stack.imgur.com/Zi4HG.png

Any ideas?

答:
I had a similar issue with you. I fixed my problem with image normalization in image processing (for your case, in process_image).

When you use 0~255 as pixel data, it tends to break up while manipulating image data, such as changing into byte and resizing because these manipulations round its pixel value.
So, I want you to try normalizing your image pixel data into 0. to 1. of float values.

I used OpenCV to fix this, and I hope you can figure out your problem in a similar way as did refer to the code I posted below.

```<code># This line distorted my images.
img = cv2.normalize(img, None, alpha=0, beta=255, norm_type=cv2.NORM_MINMAX)

# I changed to this line, and it worked.
img = cv2.normalize(img, None, alpha=0., beta=1., norm_type=cv2.NORM_MINMAX, dtype=cv2.CV_32F)
</code>```

Body:
I am trying to make a tfrecords from image_dataset_from_directory; but when I try to visualize images to check if the encoding was correct, the images turn out to be distorted of some kind.

How I created the tfrecord:

Step 1: create dataset using image_dataset_from_directory

data_dir = 'path to JPG dataset'

load_split = partial(
    tf.keras.preprocessing.image_dataset_from_directory,
    data_dir,
    validation_split=0.2,
    shuffle=True,
    seed=123,
    image_size=(IMG_HEIGHT, IMG_WIDTH),
    batch_size=1,
)

ds_train = load_split(subset='training')
ds_valid = load_split(subset='validation')

Step 2: encoding functions

def process_image(image, label):
    image = tf.image.convert_image_dtype(image, dtype=tf.uint8)
    image = tf.io.encode_jpeg(image)

    label = tf.one_hot(label, NUM_CLASSES)

    return image, label

def make_example(encoded_image, label):
    image_feature = Feature(
        bytes_list=BytesList(value=[
            encoded_image,
        ]),
    )
    label_feature = Feature(
        float_list=FloatList(value=label)
    )

    features = Features(feature={
        'image': image_feature,
        'label': label_feature,
    })

    example = Example(features=features)

    return example.SerializeToString()

Step 3: encoding and creating tfrecord

ds_train_encoded = (
    ds_train
    .unbatch()
    .map(process_image)
)

ds_valid_encoded = (
    ds_valid
    .unbatch()
    .map(process_image)
)

ds_train_encoded_iter = (
    ds_train_encoded
    .as_numpy_iterator()
)
with tf.io.TFRecordWriter(path='train.tfrecord') as f: # you can pass gs:// path here :)
    for encoded_image, label in ds_train_encoded_iter:
        example = make_example(encoded_image, label)
        f.write(example)

ds_valid_encoded_iter = (
    ds_valid_encoded
    .as_numpy_iterator()
)
with tf.io.TFRecordWriter(path='/home/et/medai/images/tfrecords/test.tfrecord') as f:
    for encoded_image, label in ds_valid_encoded_iter:
        example = make_example(encoded_image, label)
        f.write(example)

How I tried to visualize the images in the tfrecords

Step 1: decoding functions

def _parse_image_function(example):
    image_feature_description = {
        'image': tf.io.FixedLenFeature([], tf.string),
        'label': tf.io.FixedLenFeature([40], tf.float32),
    }

    features = tf.io.parse_single_example(example, image_feature_description)
    image = tf.image.decode_jpeg(features['image'], channels=3)
    image = tf.image.resize(image, [IMG_SIZE, IMG_SIZE])
    # image = features['image']
    label = features['label']

    return image, label

def read_dataset(filename, batch_size):
    dataset = tf.data.TFRecordDataset(filename)
    dataset = dataset.map(_parse_image_function, num_parallel_calls=tf.data.experimental.AUTOTUNE)
    dataset = dataset.shuffle(500)
    dataset = dataset.batch(batch_size, drop_remainder=True)
    # dataset = dataset.repeat()
    dataset = dataset.prefetch(buffer_size=tf.data.experimental.AUTOTUNE)

    return dataset

Step 2: decode and display

x = read_dataset('/home/et/medai/images/tfrecords/tests_train.tfrecord', 32)

plt.figure(figsize=(10, 10))
batch_size = 32
for images, labels in x.take(1):
    for i in range(batch_size):
        # display.display(display.Image(data=images[i].numpy()))

        ax = plt.subplot(6, 6, i + 1)
        plt.imshow(images[i].numpy().astype("uint8"))
        plt.axis("off")

The result is something distorted:
https://i.stack.imgur.com/tCAik.jpg

I am not quite sure where this distortion comes from. Original images look like this:

https://i.stack.imgur.com/Zi4HG.png

Any ideas?

Body:
I had a similar issue with you. I fixed my problem with image normalization in image processing (for your case, in process_image).

When you use 0~255 as pixel data, it tends to break up while manipulating image data, such as changing into byte and resizing because these manipulations round its pixel value.
So, I want you to try normalizing your image pixel data into 0. to 1. of float values.

I used OpenCV to fix this, and I hope you can figure out your problem in a similar way as did refer to the code I posted below.

# This line distorted my images.
img = cv2.normalize(img, None, alpha=0, beta=255, norm_type=cv2.NORM_MINMAX)

# I changed to this line, and it worked.
img = cv2.normalize(img, None, alpha=0., beta=1., norm_type=cv2.NORM_MINMAX, dtype=cv2.CV_32F)


问:
numpy vectorization(accumulate variable)
I have a dataframe.
Witch contain a few columns. It's look like this:

A
B
C
D

1
10
a
Nan

2
11
b
Nan

3
12
c
Nan

So if I have 'b' in column C, I should do A+B.
In other cases A*B.
But with that I have variable that accumulate value(You will see code and it will be cleare).
So I write this code

```<code>z = 0
for i, row in df.iterrows():
    a = df['A']
    b = df['B']
    c = df['C']
    if c == 'b':
        d = a + b + z
        z = z + 2
    else:
        d = a*b
    df.at[i, 'D'] = d
</code>```

But df.iterrows() is antipattern and I should avoid this string in my code.
Because if my data set increase it will be a problem
I have tried to use vectorization but I can't figure out how to accumulate.
Code look like this:

```<code>z = 0
con = (df['C'] == 'b',
      df['C'] != 'b')
choise = (
    (df['A'] + dfs['B'], z + 2),
    (df['A'] * dfs['B'], )
)

dfs['D'], z = np.select(con, choise)
</code>```

Can someone help me with that? How to accumulate variable z?

答:
Why not:</p>
```<code>z = 0
new_D = []

for row in df.itertuples():
    if row.C == 'b':
        new_D.append(row.A + row.B + z)
        z += 2
    else:
        new_D.append(row.A * row.B)

df['D'] = new_D
</code>```

Body:
I have a dataframe.
Witch contain a few columns. It's look like this:

A
B
C
D

1
10
a
Nan

2
11
b
Nan

3
12
c
Nan

So if I have 'b' in column C, I should do A+B.
In other cases A*B.
But with that I have variable that accumulate value(You will see code and it will be cleare).
So I write this code

z = 0
for i, row in df.iterrows():
    a = df['A']
    b = df['B']
    c = df['C']
    if c == 'b':
        d = a + b + z
        z = z + 2
    else:
        d = a*b
    df.at[i, 'D'] = d

But df.iterrows() is antipattern and I should avoid this string in my code.
Because if my data set increase it will be a problem
I have tried to use vectorization but I can't figure out how to accumulate.
Code look like this:

z = 0
con = (df['C'] == 'b',
      df['C'] != 'b')
choise = (
    (df['A'] + dfs['B'], z + 2),
    (df['A'] * dfs['B'], )
)

dfs['D'], z = np.select(con, choise)

Can someone help me with that? How to accumulate variable z?

Body:
Why not:

z = 0
new_D = []

for row in df.itertuples():
    if row.C == 'b':
        new_D.append(row.A + row.B + z)
        z += 2
    else:
        new_D.append(row.A * row.B)

df['D'] = new_D


问:
Detecting peaks in noisy data-set using C#
I would like to find a specific set of maximum points in a collection of data I create.
Visually it looks something like this:
Where the blue points are my data, the green line is the plot, and in yellow I have marked which max points I want to find.

So far I have tried to "smooth" the function via local average and a rolling window but that modifies the max values and does not quite smooth it out enough to see each of those noisy peaks as one single max value.

Is there some transform I could do in code to this array of number to allow for easier extraction of such values?

I am writing in C#.
The closest posts I could find to what I am looking for were for R which I know nothing about unfortunately.

答: You can set a threshold to remove all the points below a certain value. In your case you could keep all the points above 1.2 for example. Once you have those points you can use some clustering technique based on the horizontal distance between points to find each cluster representing a different peak, and then all you have to do is to find the maximum value for each peak.</p>


问:
How to show snippets of code in razor page
I'm building the documentation page of Xamarin app in Blazor and I've encountered a problem.
I'd like to show code snippets on the page using razor page. So far, I don't have that big problem with C# code, for example of App.xaml.cs:

```<code>public App()
   {
     InitializeComponent();
   }
</code>```

For something like that - I don't have any problems. I can correctly style it as  tag, with CSS or with MudBlazor (which I'm also using). Also, IDE is not recognizing it as inside-code.
The problem however is when:

a) I'm trying to add XAML code / tags

b) I'm trying to add some C# methods from Xamarin and they include tags, for example:

_container.Resolve<WebService> ...

So the IDE is recognizing <WebService> as something inside Blazor app.

Of course, I was already trying with  blocks with no success. Is there any way to make it work? I really need to show those in documentation so I'm a the dead point right now.

答:
You have to escape the quotes. I also added some \r\n and a \t to demonstrate formatting with ```</code></p>
```@page "/"
```
    <code>@someString</code>
```
@code {
      string someString = "<Application.Resources>\r\n\t<Color x:Key=\"PrimaryText\">#777777</Color><Application.Resources>";
}
```

Fiddle

Body:
I'm building the documentation page of Xamarin app in Blazor and I've encountered a problem.
I'd like to show code snippets on the page using razor page. So far, I don't have that big problem with C# code, for example of App.xaml.cs:

public App()
   {
     InitializeComponent();
   }

For something like that - I don't have any problems. I can correctly style it as  tag, with CSS or with MudBlazor (which I'm also using). Also, IDE is not recognizing it as inside-code.
The problem however is when:

a) I'm trying to add XAML code / tags

b) I'm trying to add some C# methods from Xamarin and they include tags, for example:

_container.Resolve<WebService> ...

So the IDE is recognizing <WebService> as something inside Blazor app.

Of course, I was already trying with  blocks with no success. Is there any way to make it work? I really need to show those in documentation so I'm a the dead point right now.

Body:
You have to escape the quotes. I also added some \r\n and a \t to demonstrate formatting with <pre>

@page "/"
<pre>
    <code>@someString</code>
</pre>
@code {
      string someString = "<Application.Resources>\r\n\t<Color x:Key=\"PrimaryText\">#777777</Color><Application.Resources>";
}

Fiddle


问:
How to write a loop in R to perform a statistical test multiple times on two sets of randomly generated values
Using R,
I have two vectors a and b:

```<code>a <- rnorm(3, mean = 6, sd = 2)
b <- rnorm(3, mean = 8, sd = 2)
</code>```

I want to generate a and b a specified number of times and each time perform a t.test comparing the mean of a and the mean of b

```<code>t.test(a, b, paired = FALSE)
</code>```

How do I code repeating the rnorm and t.test functions a specified number of times and retrieving the full output of the t.test() or just the p.value

答:
To expand on your comment:

```<code>pvals <- replicate(20, t.test(rnorm(3, mean = 6, sd = 2), rnorm(3, mean = 8, sd = 2))$p.value)
sum(pvals < .05)
# [1] 2
hist(pvals)
</code>```

To get all of the results

```<code>results <- replicate(20, t.test(rnorm(3, mean = 6, sd = 2), rnorm(3, mean = 8, sd = 2)), simplify=FALSE)
results[[1]]
#
#   Welch Two Sample t-test
#
# data:  rnorm(3, mean = 6, sd = 2) and rnorm(3, mean = 8, sd = 2)
# t = 0.052882, df = 2.6225, p-value = 0.9616
# alternative hypothesis: true difference in means is not equal to 0
# 95 percent confidence interval:
#  -8.880021  9.155836
# sample estimates:
# mean of x mean of y
#  7.156974  7.019067
</code>```

With the other results being results[[2]], . . . , results[[20]]. The simplify=FALSE argument is needed to preserve the list structure of each run.

Body:
Using R,
I have two vectors a and b:

a <- rnorm(3, mean = 6, sd = 2)
b <- rnorm(3, mean = 8, sd = 2)

I want to generate a and b a specified number of times and each time perform a t.test comparing the mean of a and the mean of b

t.test(a, b, paired = FALSE)

How do I code repeating the rnorm and t.test functions a specified number of times and retrieving the full output of the t.test() or just the p.value

Body:
To expand on your comment:

pvals <- replicate(20, t.test(rnorm(3, mean = 6, sd = 2), rnorm(3, mean = 8, sd = 2))$p.value)
sum(pvals < .05)
# [1] 2
hist(pvals)

To get all of the results

results <- replicate(20, t.test(rnorm(3, mean = 6, sd = 2), rnorm(3, mean = 8, sd = 2)), simplify=FALSE)
results[[1]]
#
#   Welch Two Sample t-test
#
# data:  rnorm(3, mean = 6, sd = 2) and rnorm(3, mean = 8, sd = 2)
# t = 0.052882, df = 2.6225, p-value = 0.9616
# alternative hypothesis: true difference in means is not equal to 0
# 95 percent confidence interval:
#  -8.880021  9.155836
# sample estimates:
# mean of x mean of y
#  7.156974  7.019067

With the other results being results[[2]], . . . , results[[20]]. The simplify=FALSE argument is needed to preserve the list structure of each run.


问:
Java compiler forcing raw use of paramatized interface
I'm trying to implement a generic interface with concrete types, but the java compiler is forcing me to instantiate it as raw. This is a mock-up of my code:

```<code>public static void main(String[] args) {
    Map<String, String> map = new HashMap<>();
    map.put("thing", "extra");
    Worker<String, String> worker = new Worker<>();
    worker.execute(map);
}

public class Worker<K, V> {
    public void execute(Map<K, V> map) {
        IDao<K, V> dao = new Dao();
        dao.doThing(map);
    }
}

public interface IDao<K, V> {
    void doThing(Map<K, V> map);
}

public class Dao implements IDao<String, String> {
    public void doThing(Map<String, String> map) {
        for (Map.Entry<String, String> entry : map.entrySet()) {
            String key = entry.getKey();
            String newKey = key.toUpperCase();
            System.out.println(newKey + " " + entry.getValue());
        }
    }
}
</code>```

This code works as expected if I change IDao<K, V> dao = new Dao(); to IDao dao = new Dao(), but this isn't good practice and I'd like to avoid it. I don't understand why it's illegal to declare the types at instantiation like this.

答:
This...

```<code>public class Worker<K, V> {
    public void execute(Map<K, V> map) {
        IDao<K, V> dao = new Dao();
        dao.doThing(map);
    }
}
</code>```

seems wrong to me.

The Worker class allows for generic key/value parameters for the Map, but, you're instantiating an instance of Dao which is constrained to <String, String>.  This seems like a basic violation of the contract.

Personally, I might change this so that the instance of IDao is injected into the worker, for example...

```<code>public static class Worker<K, V> {

    private IDao<K, V> dao;

    public Worker(IDao<K, V> dao) {
        this.dao = dao;
    }

    public void execute(Map<K, V> map) {
        dao.doThing(map);
    }
}
</code>```

nb: This could be an abstract factory, which had a abstract method something like abstract protected IDao<K, V> getDao(); and then you'd have to provide some kind concrete implementation to support it, but it's basically boiling down to the same thing

Then you'd simply be able to call it doing something like...

```<code>Map<String, String> map = new HashMap<>();
map.put("thing", "extra");
Worker<String, String> worker = new Worker<>(new Dao());
worker.execute(map);
</code>```

Of course, if you didn't want to do that, then you'd have to remove the generic constraints for Worker and require that any caller pass it a Map constrained to <String, String>

```<code>public static class Worker {
    public void execute(Map<String, String> map) {
        IDao<String, String> dao = new Dao();
        dao.doThing(map);
    }
}
</code>```

But now, where's the benefit?

Body:
I'm trying to implement a generic interface with concrete types, but the java compiler is forcing me to instantiate it as raw. This is a mock-up of my code:

public static void main(String[] args) {
    Map<String, String> map = new HashMap<>();
    map.put("thing", "extra");
    Worker<String, String> worker = new Worker<>();
    worker.execute(map);
}

public class Worker<K, V> {
    public void execute(Map<K, V> map) {
        IDao<K, V> dao = new Dao();
        dao.doThing(map);
    }
}

public interface IDao<K, V> {
    void doThing(Map<K, V> map);
}

public class Dao implements IDao<String, String> {
    public void doThing(Map<String, String> map) {
        for (Map.Entry<String, String> entry : map.entrySet()) {
            String key = entry.getKey();
            String newKey = key.toUpperCase();
            System.out.println(newKey + " " + entry.getValue());
        }
    }
}

This code works as expected if I change IDao<K, V> dao = new Dao(); to IDao dao = new Dao(), but this isn't good practice and I'd like to avoid it. I don't understand why it's illegal to declare the types at instantiation like this.

Body:
This...

public class Worker<K, V> {
    public void execute(Map<K, V> map) {
        IDao<K, V> dao = new Dao();
        dao.doThing(map);
    }
}

seems wrong to me.

The Worker class allows for generic key/value parameters for the Map, but, you're instantiating an instance of Dao which is constrained to <String, String>.  This seems like a basic violation of the contract.

Personally, I might change this so that the instance of IDao is injected into the worker, for example...

public static class Worker<K, V> {

    private IDao<K, V> dao;

    public Worker(IDao<K, V> dao) {
        this.dao = dao;
    }

    public void execute(Map<K, V> map) {
        dao.doThing(map);
    }
}

nb: This could be an abstract factory, which had a abstract method something like abstract protected IDao<K, V> getDao(); and then you'd have to provide some kind concrete implementation to support it, but it's basically boiling down to the same thing

Then you'd simply be able to call it doing something like...

Map<String, String> map = new HashMap<>();
map.put("thing", "extra");
Worker<String, String> worker = new Worker<>(new Dao());
worker.execute(map);

Of course, if you didn't want to do that, then you'd have to remove the generic constraints for Worker and require that any caller pass it a Map constrained to <String, String>

public static class Worker {
    public void execute(Map<String, String> map) {
        IDao<String, String> dao = new Dao();
        dao.doThing(map);
    }
}

But now, where's the benefit?


问:
Flutter resize image to fill available space
I would like to have this image fill the available space between the top element and the bottom element. Also I would like the bottom text to be always at the bottom.

This is how I implemented it right now:

```<code>Column(
      mainAxisAlignment: MainAxisAlignment.spaceBetween,
      children: [
        Column(children: topWidgets),
        Container(
         color: Colors.red,
         child: Column(children: [
             Image.asset("assets/empty_state/illustration_01.png", height: 200),
             Text("No matches so far")
           ]),
         )
     ]
)
</code>```

I have tried to use the fit property of the Image without luck. If I don't use height: 200 I wouldn't be able to see the bottom text because it would go out of the screen (at the bottom)

I don't know if having a Column wrapping text and image is a good approach but I think it is the best among the ones I tried so far.

答:
```<code>Container(
  height: MediaQuery.of(context).size.height, //for max height
  width: MediaQuery.of(context).size.width,//for max width
  child: Column(
    children: [
      Expanded(
        flex: 1, // flex is used to expand less/more the contained widget
          child: topWidgets),
      Expanded(
        flex: 3,
          child: Container( //for red background
        color: Colors.red,
        child: Column(
          mainAxisAlignment: MainAxisAlignment.end,
          children: [
            Expanded(
              flex: 3, //increase to fill more space of picture
                child: FittedBox(
              fit: BoxFit.fill, // the picture will acquire all of the parent space.
              child: Image.asset("assets/empty_state/illustration_01.png"),
            )),
            Expanded(
              flex: 1, //by default flex is 1. so don't do this and save a line
              child: Text("No matches so far"),)
          ],
        ),
      ))
    ],
  ),
)
</code>```

Body:
I would like to have this image fill the available space between the top element and the bottom element. Also I would like the bottom text to be always at the bottom.

This is how I implemented it right now:

Column(
      mainAxisAlignment: MainAxisAlignment.spaceBetween,
      children: [
        Column(children: topWidgets),
        Container(
         color: Colors.red,
         child: Column(children: [
             Image.asset("assets/empty_state/illustration_01.png", height: 200),
             Text("No matches so far")
           ]),
         )
     ]
)

I have tried to use the fit property of the Image without luck. If I don't use height: 200 I wouldn't be able to see the bottom text because it would go out of the screen (at the bottom)

I don't know if having a Column wrapping text and image is a good approach but I think it is the best among the ones I tried so far.


问:
get unix style path join in python
While making a wsgi app I am in need of joining route names like /apple,  apple/buy etc.

```<code>    import os
    path = os.path.join("/apple/", "/buy")
    print("route path:", path)

</code>```

Now the problem is it works fine in *nix systems but on dos ones the ouput could become:

```<code>    route path: apple\buy
</code>```

cuz what is wanted was unix style paths like - apple/buy

is their a way that I can always get unix style paths? should I use something other than os module?

答:
os.path is a alias to posixpath or ntpath. You can import one of those modules directly to get the path rules you want. So, import posixpath and use functions like posixpath.join.

```<code>>>> import os
>>> os.path.__name__
'posixpath'
>>> import posixpath
>>> posixpath.join is os.path.join
True
>>>
</code>```

You could

```<code>import os
import posixpath
os.path = posixpath
</code>```

But that can be a bad choice because it would also affect any code that wants to work with local platform paths. Its usually best to use posixpath directly as needed.

Title: get unix style path join in python

Body:
While making a wsgi app I am in need of joining route names like /apple,  apple/buy etc.

    import os
    path = os.path.join("/apple/", "/buy")
    print("route path:", path)

Now the problem is it works fine in *nix systems but on dos ones the ouput could become:

    route path: apple\buy

cuz what is wanted was unix style paths like - apple/buy

is their a way that I can always get unix style paths? should I use something other than os module?

Body:
os.path is a alias to posixpath or ntpath. You can import one of those modules directly to get the path rules you want. So, import posixpath and use functions like posixpath.join.

>>> import os
>>> os.path.__name__
'posixpath'
>>> import posixpath
>>> posixpath.join is os.path.join
True
>>>

You could

import os
import posixpath
os.path = posixpath

But that can be a bad choice because it would also affect any code that wants to work with local platform paths. Its usually best to use posixpath directly as needed.


问:
Manifest Merger failed with multiple errors in Android Studio
So, I am a beginner into Android and Java. I just began learning. While I was experimenting with Intent today, I incurred an error.

```<code>Error:Execution failed for task ':app:processDebugManifest'.
> Manifest merger failed with multiple errors, see logs
</code>```

I found some solutions here and tried to implement them, but it did not work.

This is my build.gradle :

```<code>apply plugin: 'com.android.application'

android {
compileSdkVersion 23
buildToolsVersion "23.0.0"

defaultConfig {
    applicationId "com.example.rohan.petadoptionthing"
    minSdkVersion 10
    targetSdkVersion 23
    versionCode 1
    versionName "1.0"
}
buildTypes {
    release {
        minifyEnabled false
        proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'
    }
}
}

dependencies {
compile fileTree(dir: 'libs', include: ['*.jar'])
compile 'com.android.support:appcompat-v7:23.0.0'
}
</code>```

This is my AndroidManifest :

```<code><?xml version="1.0" encoding="utf-8"?>
</code>```

```<code>package="com.example.rohan.petadoptionthing" >

<application

    android:allowBackup="true"
    android:icon="@mipmap/ic_launcher"
    android:label="@string/app_name"
    android:theme="@style/AppTheme" >
    <activity
        android:name=".MainActivity"
        android:label="@string/app_name" >
        <intent-filter>
            <action android:name="android.intent.action.MAIN" />

            <category android:name="android.intent.category.LAUNCHER" />
        </intent-filter>
    </activity>

    <activity android:name=".Second"
        />

    <activity android:name=".third"/>
    <activity android:name=".MainActivity"/>

</application>
</code>```

This is my first week with coding, I am sorry if this is a really silly thing. I am really new to this and did not find any other place to ask. Sorry if I broke any rules

答: check if you've any duplicate activity declared in the manifest. That was my mistake for this error and problem solved, after deleting the duplicate.</p>

Body:
So, I am a beginner into Android and Java. I just began learning. While I was experimenting with Intent today, I incurred an error.

Error:Execution failed for task ':app:processDebugManifest'.
> Manifest merger failed with multiple errors, see logs

I found some solutions here and tried to implement them, but it did not work.

This is my build.gradle :

apply plugin: 'com.android.application'

android {
compileSdkVersion 23
buildToolsVersion "23.0.0"

defaultConfig {
    applicationId "com.example.rohan.petadoptionthing"
    minSdkVersion 10
    targetSdkVersion 23
    versionCode 1
    versionName "1.0"
}
buildTypes {
    release {
        minifyEnabled false
        proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'
    }
}
}

dependencies {
compile fileTree(dir: 'libs', include: ['*.jar'])
compile 'com.android.support:appcompat-v7:23.0.0'
}

This is my AndroidManifest :

<?xml version="1.0" encoding="utf-8"?>

package="com.example.rohan.petadoptionthing" >

<application

    android:allowBackup="true"
    android:icon="@mipmap/ic_launcher"
    android:label="@string/app_name"
    android:theme="@style/AppTheme" >
    <activity
        android:name=".MainActivity"
        android:label="@string/app_name" >
        <intent-filter>
            <action android:name="android.intent.action.MAIN" />

            <category android:name="android.intent.category.LAUNCHER" />
        </intent-filter>
    </activity>

    <activity android:name=".Second"
        />

    <activity android:name=".third"/>
    <activity android:name=".MainActivity"/>

</application>

This is my first week with coding, I am sorry if this is a really silly thing. I am really new to this and did not find any other place to ask. Sorry if I broke any rules


问:
How to avoid hard coding bootstrap server name/IP and port
This is how a producer/consumer connects to broker:

```<code>// create instance for properties to access producer configs
      Properties props = new Properties();

      //Assign localhost id
      props.put("bootstrap.servers", “localhost:9092");
</code>```

Obviously, using localhost/IP/DNS name doesn't look to be highly available solution to me. Even port number for that matter. How can this hard coding be avoided?

Even something like Consul or Vault may not be good enough.

答:
Not sure I understand the problem. You'll need to provide one or more broker addresses for the bootstrap protocol. Then all brokers advertised.listeners will be returned. The highly available solution is to give multiple addresses or give one address that will be available.

Using a singular load balancer address could work. Consul could work as well. The address isn't a secret, so I wouldn't suggest using Vault unless you have SSL keys.

Body:
This is how a producer/consumer connects to broker:

// create instance for properties to access producer configs
      Properties props = new Properties();

      //Assign localhost id
      props.put("bootstrap.servers", “localhost:9092");

Obviously, using localhost/IP/DNS name doesn't look to be highly available solution to me. Even port number for that matter. How can this hard coding be avoided?

Even something like Consul or Vault may not be good enough.


问:
Getting File Directories as nested Array object
I want to get my file directories as nested array object but i can't seem to figure out how to convert

```<code>[
  'routes/files',
  'routes/files/emptyfolder',
  'routes/files/somefolder',
  'routes/files/somefolder/example.docx',
  'routes/files/test.docx',
  'routes/randomdata.json'
]
</code>```

to

```<code>[
    {
      title: 'routes',
      content: [
        {
          title: 'files',
          content: [
            {
                title: 'empty folder',
                content: []
            },
            {
                title: 'somefolder',
                content: [
                    {
                        title: 'example.docx',
                    },
                ]
            },
            {
                title: 'test.docx',
            }
          ],
        },
        {
            title: 'randomdata.json'
        }
    ],
}
]
</code>```

it looks impossible problem for me to solve.
I would love to know how to solve it.

Thank you.

答:
Here is how I solved it:
Not the best solution, but works.

const arr = [
  "routes/files",
  "routes/files/emptyfolder",
  "routes/files/somefolder",
  "routes/files/somefolder/example.docx",
  "routes/files/test.docx",
  "routes/randomdata.json",
];

const arr2 = arr.map((p) => p.split("/"));

const setNestedObjectField = (
  obj,
  props,
  value
) => {
  if (!Array.isArray(obj)) {
    if (!obj.content) {
      obj.content = [];
    }
    obj = obj.content;
  }
  for (const propName of props) {
    const next = obj.find((el) => el.title === propName);
    if (!next) {
      console.assert(props.at(-1) === propName);
      // last propName
      obj.push(value);
    } else {
      if (!next.content) {
        next.content = [];
      }
      obj = next.content;
    }
  }
};

const rez = [];
let index = 0;
while (arr2.some((s) => s[index] !== undefined)) {
  // arr2 = arr2.filter((a) => a.length);
  const layer = arr2.reduce((acc, pathArr) => {
    if (pathArr[index] === undefined) return acc;
    acc.add(pathArr.slice(0, index + 1).join("/"));
    return acc;
  }, new Set());

  // console.log({ layer });

  for (const key of layer) {
    setNestedObjectField(rez, key.split("/"), { title: key.split("/").at(-1) });
  }
  index++;
}

console.log(rez);
```

Body:
I want to get my file directories as nested array object but i can't seem to figure out how to convert

[
  'routes/files',
  'routes/files/emptyfolder',
  'routes/files/somefolder',
  'routes/files/somefolder/example.docx',
  'routes/files/test.docx',
  'routes/randomdata.json'
]

to

[
    {
      title: 'routes',
      content: [
        {
          title: 'files',
          content: [
            {
                title: 'empty folder',
                content: []
            },
            {
                title: 'somefolder',
                content: [
                    {
                        title: 'example.docx',
                    },
                ]
            },
            {
                title: 'test.docx',
            }
          ],
        },
        {
            title: 'randomdata.json'
        }
    ],
}
]

it looks impossible problem for me to solve.
I would love to know how to solve it.

Thank you.


问:
Beginner Taxicab Distances
I am a beginner Python programmer enrolled in a Python course. The prompt is to essentially calculate the taxicab distance between a series of points in a city, however I am getting silly results. The prompt sets up the city as 26 row, 9 column, using a capital letter and numeric to describe its location. For instance, A1 is the bottom left corner of city. A2 is the bottom row, second column. The function I wrote to calculate the distances seems to work on its own, but once I incorporate it into my main while loop, it throws values that are way too high. The example prompt my teacher gives is as follows:

I've tried many different statements in the while loop, and I thought I had finally found one that worked using a temporary variable to store the previous location, but it just won't work. The loop is meant to end when the user types 'home' and the distance should be recalculated to find the distance to get home as well.

```<code>def calculate_distance(p1, p2):
    x1 = ord(p1[0])
    y1 = ord(p1[1])
    x2 = ord(p2[0])
    y2 = ord(p2[1])
    d = abs(x2 - x1) + abs(y2 - y1)
    return d

def main():
    home = input('Home? ')
    point = input('Next? ')
    distance = calculate_distance(home, point)
    while point != 'home':
        tmp = point
        point = input('Next? ')
        distance += calculate_distance(tmp, point)
    distance += calculate_distance(home, tmp)
    print(distance)

if __name__ == '__main__':
    main()
</code>```

Here is my code. Don't worry about the extra details on the example prompt, those will be easy once this part works. Also the first two inputs are assumed to never be 'home'. I used the prompt my teacher gave above and got 107.

答:
One significant issue is that within the while loop, your code executes the statement distance += calculate_distance(tmp, point) when point has the value 'home'. This will give garbage output from your calculate_distance() function (ascii arithmetic subtracting letters from numbers, or vice versa).

Another issue is that the line before you print the result reads:

```<code>distance += calculate_distance(home, tmp)
</code>```

But the most recent coordinate the user entered before typing 'home' was stored in point, not in tmp. Try replacing tmp with point in the above line as shown below:

```<code>distance += calculate_distance(home, point)
</code>```

A working version correcting the above two issues is:

```<code>def main():
    home = input('Home? ')
    point = input('Next? ')
    distance = calculate_distance(home, point)
    tmp = point
    point = input('Next? ')
    while point != 'home':
        distance += calculate_distance(tmp, point)
        tmp = point
        point = input('Next? ')
    distance += calculate_distance(tmp, home)
    print(distance)
</code>```

Title: Beginner Taxicab Distances

Body:
I am a beginner Python programmer enrolled in a Python course. The prompt is to essentially calculate the taxicab distance between a series of points in a city, however I am getting silly results. The prompt sets up the city as 26 row, 9 column, using a capital letter and numeric to describe its location. For instance, A1 is the bottom left corner of city. A2 is the bottom row, second column. The function I wrote to calculate the distances seems to work on its own, but once I incorporate it into my main while loop, it throws values that are way too high. The example prompt my teacher gives is as follows:

I've tried many different statements in the while loop, and I thought I had finally found one that worked using a temporary variable to store the previous location, but it just won't work. The loop is meant to end when the user types 'home' and the distance should be recalculated to find the distance to get home as well.

def calculate_distance(p1, p2):
    x1 = ord(p1[0])
    y1 = ord(p1[1])
    x2 = ord(p2[0])
    y2 = ord(p2[1])
    d = abs(x2 - x1) + abs(y2 - y1)
    return d

def main():
    home = input('Home? ')
    point = input('Next? ')
    distance = calculate_distance(home, point)
    while point != 'home':
        tmp = point
        point = input('Next? ')
        distance += calculate_distance(tmp, point)
    distance += calculate_distance(home, tmp)
    print(distance)

if __name__ == '__main__':
    main()

Here is my code. Don't worry about the extra details on the example prompt, those will be easy once this part works. Also the first two inputs are assumed to never be 'home'. I used the prompt my teacher gave above and got 107.

Body:
One significant issue is that within the while loop, your code executes the statement distance += calculate_distance(tmp, point) when point has the value 'home'. This will give garbage output from your calculate_distance() function (ascii arithmetic subtracting letters from numbers, or vice versa).

Another issue is that the line before you print the result reads:

distance += calculate_distance(home, tmp)

But the most recent coordinate the user entered before typing 'home' was stored in point, not in tmp. Try replacing tmp with point in the above line as shown below:

distance += calculate_distance(home, point)

A working version correcting the above two issues is:

def main():
    home = input('Home? ')
    point = input('Next? ')
    distance = calculate_distance(home, point)
    tmp = point
    point = input('Next? ')
    while point != 'home':
        distance += calculate_distance(tmp, point)
        tmp = point
        point = input('Next? ')
    distance += calculate_distance(tmp, home)
    print(distance)


问:
How do I provide an informal string representation of a python Class (not instance)
I understand how I can provide an informal representation of an instance of the object, but I am interested in providing an informal string representation of the Class name.

So specifically, I want to override what is returned when I print the Class (__main__.SomeClass).

```<code>>>> class SomeClass:
...   def __str__(self):
...     return 'I am a SomeClass instance.'
...
>>> SomeClass
<class __main__.SomeClass at 0x2ba2f0fd3b30>
>>> print SomeClass
__main__.SomeClass
>>>
>>> x = SomeClass()
>>> x
<__main__.SomeClass instance at 0x2ba2f0ff3f38>
>>> print x
I am a SomeClass instance.
</code>```

答:
you would have to override the __str__ method of the metaclass.  I don't really know why you would want to do this, but here it is anyway.

```<code>>>> class InformalType(type):
...     def __str__(self):
...             return self.__name__
...
>>> class MyFoo(object):
...     __metaclass__ = InformalType
...     pass
...
>>> MyFoo
<class '__main__.MyFoo'>
>>> print MyFoo
MyFoo
>>> foo = MyFoo()
>>> foo
<__main__.MyFoo object at 0x7fdf9581f910>
>>> print foo
<__main__.MyFoo object at 0x7fdf9581f910>
>>>
</code>```

Body:
I understand how I can provide an informal representation of an instance of the object, but I am interested in providing an informal string representation of the Class name.

So specifically, I want to override what is returned when I print the Class (__main__.SomeClass).

>>> class SomeClass:
...   def __str__(self):
...     return 'I am a SomeClass instance.'
...
>>> SomeClass
<class __main__.SomeClass at 0x2ba2f0fd3b30>
>>> print SomeClass
__main__.SomeClass
>>>
>>> x = SomeClass()
>>> x
<__main__.SomeClass instance at 0x2ba2f0ff3f38>
>>> print x
I am a SomeClass instance.

Body:
you would have to override the __str__ method of the metaclass.  I don't really know why you would want to do this, but here it is anyway.

>>> class InformalType(type):
...     def __str__(self):
...             return self.__name__
...
>>> class MyFoo(object):
...     __metaclass__ = InformalType
...     pass
...
>>> MyFoo
<class '__main__.MyFoo'>
>>> print MyFoo
MyFoo
>>> foo = MyFoo()
>>> foo
<__main__.MyFoo object at 0x7fdf9581f910>
>>> print foo
<__main__.MyFoo object at 0x7fdf9581f910>
>>>


问:
How do I complete K&R Exercise 2-4?
I'm learning how to write programs in C using the k&r book (The C Programming Language) and I have a problem with one of the exercises. It's asking me to detect and remove a character in string s1, which matches any characters in the string s2.

So, say s1 = "A";

And s2 = "AABAACAADAAE"

I want it to return "BCDE"

I know I'm on the right path towards it, i just don't know how to design programs very well, could you give me any additional tips. I tried to read about the binary search tree algorithm, but felt it was a little too advanced for this mundane task.

Thanks everyone!

```<code>/* An alternate version of squeeze(s1, s2) that deletes each character in
 * s1 that matches any character in the string s2
 *
 * Angie@odfx.org
 */

#include <stdio.h>
#include <string.h>

void squeeze(char s[], char t[]);

char string[] = "BAD";
char sstring[] = "ABC";

int
main(void)
{
    squeeze(string, sstring);
    return 0;
}

void
squeeze(char s[], char t[])
{
    int i, j, d;

    d = 0;
    if(strstr(s, t) == NULL)
        printf("%c", s[i]);
    s[j] = '\0';
}
</code>```

答:
Here is my very clear and simple answer with some logical explanation.

```<code>#include<stdio.h>
void squeeze();
</code>```

Inside main we type our tested string and wantbedelete string that has the chars that we want to delete from yourstring.

```<code>int main()
{

    char yourstring[] = "AABAACAADAAE";
    char wantbedelete[] = "A";

    squeeze(yourstring, wantbedelete);
    printf("%s", yourstring);

    return 0;
}
</code>```

The logic inside the squeeze function is as follows,

loop inside the wantbedelete char by char

then, for any char found in wantbedelete we loop through the whole yourstring

then, we compare if the two founded char does not match, we update (save) that char from yourstring to yourstring so basically here just we define a new index(k) to keep track and update the only wanted characters

now the trick here is that we want to reset our index each time we go back again to the outer loop that is looping through each char we intend to delete so that we end up with the only wanted string.

I highly recommend using the debugger and following each line inside the squeeze function so that you would be able to understand the logic more clearly.

```<code>void squeeze(char s1[], char s2[])
{
    int i, j, k;

    k = 0;
    for (i = 0 ; s2[i] != '\0' ; i++) /* loop for each char we want to delete with index i */
    {
        for (j = 0 ; s1[j] != '\0'; j++) /* loop for each char we want to keep with index j */
        {
            if (s2[i] != s1[j]) /* if the two chars do not match */
            {
                s1[k++] = s1[j]; /* update s1 with the char that we want to keep using index k */
            }
        }
        s1[k] = '\0'; /* since we update all char that we want to keep, the last char of index k must be empty */
        k = 0; /* reset index k so we will be ready for the next char that we want to delete from s1 */
    }
}
</code>```

OwnerDisplayName: ang

LastEditorDisplayName: Jason

Title: How do I complete K&R Exercise 2-4?

Body:
I'm learning how to write programs in C using the k&r book (The C Programming Language) and I have a problem with one of the exercises. It's asking me to detect and remove a character in string s1, which matches any characters in the string s2.

So, say s1 = "A";

And s2 = "AABAACAADAAE"

I want it to return "BCDE"

I know I'm on the right path towards it, i just don't know how to design programs very well, could you give me any additional tips. I tried to read about the binary search tree algorithm, but felt it was a little too advanced for this mundane task.

Thanks everyone!

/* An alternate version of squeeze(s1, s2) that deletes each character in
 * s1 that matches any character in the string s2
 *
 * Angie@odfx.org
 */

#include <stdio.h>
#include <string.h>

void squeeze(char s[], char t[]);

char string[] = "BAD";
char sstring[] = "ABC";

int
main(void)
{
    squeeze(string, sstring);
    return 0;
}

void
squeeze(char s[], char t[])
{
    int i, j, d;

    d = 0;
    if(strstr(s, t) == NULL)
        printf("%c", s[i]);
    s[j] = '\0';
}

Body:
Here is my very clear and simple answer with some logical explanation.

#include<stdio.h>
void squeeze();

Inside main we type our tested string and wantbedelete string that has the chars that we want to delete from yourstring.

int main()
{

    char yourstring[] = "AABAACAADAAE";
    char wantbedelete[] = "A";

    squeeze(yourstring, wantbedelete);
    printf("%s", yourstring);

    return 0;
}

The logic inside the squeeze function is as follows,

loop inside the wantbedelete char by char

then, for any char found in wantbedelete we loop through the whole yourstring

then, we compare if the two founded char does not match, we update (save) that char from yourstring to yourstring so basically here just we define a new index(k) to keep track and update the only wanted characters

now the trick here is that we want to reset our index each time we go back again to the outer loop that is looping through each char we intend to delete so that we end up with the only wanted string.

I highly recommend using the debugger and following each line inside the squeeze function so that you would be able to understand the logic more clearly.

void squeeze(char s1[], char s2[])
{
    int i, j, k;

    k = 0;
    for (i = 0 ; s2[i] != '\0' ; i++) /* loop for each char we want to delete with index i */
    {
        for (j = 0 ; s1[j] != '\0'; j++) /* loop for each char we want to keep with index j */
        {
            if (s2[i] != s1[j]) /* if the two chars do not match */
            {
                s1[k++] = s1[j]; /* update s1 with the char that we want to keep using index k */
            }
        }
        s1[k] = '\0'; /* since we update all char that we want to keep, the last char of index k must be empty */
        k = 0; /* reset index k so we will be ready for the next char that we want to delete from s1 */
    }
}


问:
does windows handle the opcode different?
(btw this ist my first question here, so I dont know any guidelines)

Im trying to learn reading the pure opcodes (I do not know the exact term) of Programms on x-86-architectur (intel i5-7400) but if I look at an .exe-file there are huge rows with the same (hexadicimal) numbers and it doesnt make much sense. Last I created a .exe-file from a simple .cpp-file with the gnu compiler on the cmd. It doesnt make thrifty things but writes a word in the console and ask a word from the user to write in the console:

```<code>#include<iostream>

int main(int argc,char*argv[]){
    std::cout<<"aaahhhhhhhhh"<<std::endl;
    char a;
    std::cin>>a;
    return 0;
}
</code>```

the .exe-file looks as follows in binary form:
4d 5a 90 20 03 20 20 20 04 20 20 20 ff ff 20 20 b8 20 20 20 20 20 20 20 40 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 80 20 20 20 0e 1f ba 0e 20 b4 09 cd 21 b8 01 4c cd 21 54 68 69 73 20 70 72 6f 67 72 61 6d 20 63 61 6e 6e 6f 74 20 62 65 20 72 75 6e 20 69 6e 20 44 4f 53 20 6d 6f 64 65 2e 0d 0d 0a 24 20 20 20 20 20 20 20 50 45 20 20 64 86 13 20 24 97 09 62 20 86 01 20 b6 05 20 20 f0 20 26 20 0b 02 02 25 20 18 20 20 20 40 20 20 20 02 20 20 d0 14 20 20 20 10 20 20 20 20 20 40 01 20 20 20 20 10 20 20 20 02 20 20 04 20 20 20 20 20 20 20 05 20 02 20 20 20 20 20 20 40 02 20 20 06 20 20 cf ea 02 20 03 20 60 01 20 20 20 20 20 20 20 20 20 10 20 20 20 20 20 20
...

and is very long of course (there is much left).

I was reading the first lines. As you can see (or let traduce; I took this to examine if I was correct and if I wasnt it was always the more correct one: the link) it first pops the r10 from the stack. In the rex byte 01001101 the 3th bit is set, but not used because there isnt any modrm byte in this kind of the pop-operation. Then the opcodes make strange thing with and and the register and it do not look like instruction but like a queue of numbers. There are nops and many false opcodes interrupts. So I now wonder, how this comes since the code starts with the first bit. Below it begins to do what it have to but the first lines? Why?

```<code>
0:  4d 5a                   rex.WRB pop r10
2:  90                      nop
3:  20 03                   and    BYTE PTR [rbx],al
5:  20 20                   and    BYTE PTR [rax],ah
7:  20 04 20                and    BYTE PTR [rax+riz*1],al
a:  20 20                   and    BYTE PTR [rax],ah
c:  ff                      (bad)
d:  ff 20                   jmp    QWORD PTR [rax]
f:  20 b8 20 20 20 20       and    BYTE PTR [rax+0x20202020],bh
15: 20 20                   and    BYTE PTR [rax],ah
17: 20 40 20                and    BYTE PTR [rax+0x20],al
1a: 20 20                   and    BYTE PTR [rax],ah
1c: 20 20                   and    BYTE PTR [rax],ah
1e: 20 20                   and    BYTE PTR [rax],ah
20: 20 20                   and    BYTE PTR [rax],ah
22: 20 20                   and    BYTE PTR [rax],ah
24: 20 20                   and    BYTE PTR [rax],ah
26: 20 20                   and    BYTE PTR [rax],ah
28: 20 20                   and    BYTE PTR [rax],ah
2a: 20 20                   and    BYTE PTR [rax],ah
2c: 20 20                   and    BYTE PTR [rax],ah
2e: 20 20                   and    BYTE PTR [rax],ah
30: 20 20                   and    BYTE PTR [rax],ah
32: 20 20                   and    BYTE PTR [rax],ah
34: 20 20                   and    BYTE PTR [rax],ah
36: 20 20                   and    BYTE PTR [rax],ah
38: 20 20                   and    BYTE PTR [rax],ah
3a: 20 20                   and    BYTE PTR [rax],ah
3c: 80 20 20                and    BYTE PTR [rax],0x20
3f: 20 0e                   and    BYTE PTR [rsi],cl
41: 1f                      (bad)
42: ba 0e 20 b4 09          mov    edx,0x9b4200e
47: cd 21                   int    0x21
49: b8 01 4c cd 21          mov    eax,0x21cd4c01
4e: 54                      push   rsp
4f: 68 69 73 20 70          push   0x70207369
54: 72 6f                   jb     0xc5
56: 67 72 61                addr32 jb 0xba
59: 6d                      ins    DWORD PTR es:[rdi],dx
5a: 20 63 61                and    BYTE PTR [rbx+0x61],ah
5d: 6e                      outs   dx,BYTE PTR ds:[rsi]
5e: 6e                      outs   dx,BYTE PTR ds:[rsi]
5f: 6f                      outs   dx,DWORD PTR ds:[rsi]
60: 74 20                   je     0x82
62: 62                      (bad)
63: 65 20 72 75             and    BYTE PTR gs:[rdx+0x75],dh
67: 6e                      outs   dx,BYTE PTR ds:[rsi]
68: 20 69 6e                and    BYTE PTR [rcx+0x6e],ch
6b: 20 44 4f 53             and    BYTE PTR [rdi+rcx*2+0x53],al
6f: 20 6d 6f                and    BYTE PTR [rbp+0x6f],ch
72: 64 65 2e 0d 0d 0a 24    fs gs cs or eax,0x20240a0d
79: 20
7a: 20 20                   and    BYTE PTR [rax],ah
7c: 20 20                   and    BYTE PTR [rax],ah
7e: 20 20                   and    BYTE PTR [rax],ah
80: 50                      push   rax
81: 45 20 20                and    BYTE PTR [r8],r12b
84: 64 86 13                xchg   BYTE PTR fs:[rbx],dl
87: 20 24 97                and    BYTE PTR [rdi+rdx*4],ah
8a: 09 62 20                or     DWORD PTR [rdx+0x20],esp
8d: 86 01                   xchg   BYTE PTR [rcx],al
8f: 20 b6 05 20 20 f0       and    BYTE PTR [rsi-0xfdfdffb],dh
95: 20 26                   and    BYTE PTR [rsi],ah
97: 20 0b                   and    BYTE PTR [rbx],cl
99: 02 02                   add    al,BYTE PTR [rdx]
9b: 25 20 18 20 20          and    eax,0x20201820
a0: 20 40 20                and    BYTE PTR [rax+0x20],al
a3: 20 20                   and    BYTE PTR [rax],ah
a5: 02 20                   add    ah,BYTE PTR [rax]
a7: 20 d0                   and    al,dl
a9: 14 20                   adc    al,0x20
ab: 20 20                   and    BYTE PTR [rax],ah
ad: 10 20                   adc    BYTE PTR [rax],ah
af: 20 20                   and    BYTE PTR [rax],ah
b1: 20 20                   and    BYTE PTR [rax],ah
b3: 40 01 20                rex add DWORD PTR [rax],esp
b6: 20 20                   and    BYTE PTR [rax],ah
b8: 20 10                   and    BYTE PTR [rax],dl
ba: 20 20                   and    BYTE PTR [rax],ah
bc: 20 02                   and    BYTE PTR [rdx],al
be: 20 20                   and    BYTE PTR [rax],ah
c0: 04 20                   add    al,0x20
c2: 20 20                   and    BYTE PTR [rax],ah
c4: 20 20                   and    BYTE PTR [rax],ah
c6: 20 20                   and    BYTE PTR [rax],ah
c8: 05 20 02 20 20          add    eax,0x20200220
cd: 20 20                   and    BYTE PTR [rax],ah
cf: 20 20                   and    BYTE PTR [rax],ah
d1: 40 02 20                add    spl,BYTE PTR [rax]
d4: 20 06                   and    BYTE PTR [rsi],al
d6: 20 20                   and    BYTE PTR [rax],ah
d8: cf                      iret
d9: ea                      (bad)
da: 02 20                   add    ah,BYTE PTR [rax]
dc: 03 20                   add    esp,DWORD PTR [rax]
de: 60                      (bad)
df: 01 20                   add    DWORD PTR [rax],esp
e1: 20 20                   and    BYTE PTR [rax],ah
e3: 20 20                   and    BYTE PTR [rax],ah
e5: 20 20                   and    BYTE PTR [rax],ah
e7: 20 20                   and    BYTE PTR [rax],ah
e9: 10 20                   adc    BYTE PTR [rax],ah
eb: 20 20                   and    BYTE PTR [rax],ah
ed: 20 20                   and    BYTE PTR [rax],ah
ef: 20 20                   and    BYTE PTR [rax],ah
f1: 20 10                   and    BYTE PTR [rax],dl
f3: 20 20                   and    BYTE PTR [rax],ah
f5: 20 20                   and    BYTE PTR [rax],ah
f7: 20 20                   and    BYTE PTR [rax],ah
f9: 10 20                   adc    BYTE PTR [rax],ah
fb: 20 20                   and    BYTE PTR [rax],ah
fd: 20 20                   and    BYTE PTR [rax],ah
ff: 20                      .byte 0x20
</code>```

答:
Not all bytes in the .exe are meant to be executed.  The .exe file format consists of several tables that tell the OS how to load the executable into memory, and how to start it.  What you're seeing in the listing above are the data tables that are meant for the OS to read and analyze, but not execute.

Specifically, each Windows executable is also a DOS executable.  This is why the first two bytes of each file are M and Z -- those are initials of Mark Zbikowski -- the lead of the DOS 2.0 development team who designed the format.  The DOS part of your executable only outputs a message that this program is designed to work on Windows, and exits.  This way, if your executable is started on a DOS machine, the user will know what's going on.

However, if your executable is started on a Windows machine, the Windows loader knows how to bypass the DOS part, verify that the Windows part can be loaded (e.g. it's 32-bit or 64-bit, it's for Intel or ARM or PowerPC architecture, etc.), and then load the parts of the file into the memory and start running it.

Body:
(btw this ist my first question here, so I dont know any guidelines)

Im trying to learn reading the pure opcodes (I do not know the exact term) of Programms on x-86-architectur (intel i5-7400) but if I look at an .exe-file there are huge rows with the same (hexadicimal) numbers and it doesnt make much sense. Last I created a .exe-file from a simple .cpp-file with the gnu compiler on the cmd. It doesnt make thrifty things but writes a word in the console and ask a word from the user to write in the console:

#include<iostream>

int main(int argc,char*argv[]){
    std::cout<<"aaahhhhhhhhh"<<std::endl;
    char a;
    std::cin>>a;
    return 0;
}

the .exe-file looks as follows in binary form:
4d 5a 90 20 03 20 20 20 04 20 20 20 ff ff 20 20 b8 20 20 20 20 20 20 20 40 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 80 20 20 20 0e 1f ba 0e 20 b4 09 cd 21 b8 01 4c cd 21 54 68 69 73 20 70 72 6f 67 72 61 6d 20 63 61 6e 6e 6f 74 20 62 65 20 72 75 6e 20 69 6e 20 44 4f 53 20 6d 6f 64 65 2e 0d 0d 0a 24 20 20 20 20 20 20 20 50 45 20 20 64 86 13 20 24 97 09 62 20 86 01 20 b6 05 20 20 f0 20 26 20 0b 02 02 25 20 18 20 20 20 40 20 20 20 02 20 20 d0 14 20 20 20 10 20 20 20 20 20 40 01 20 20 20 20 10 20 20 20 02 20 20 04 20 20 20 20 20 20 20 05 20 02 20 20 20 20 20 20 40 02 20 20 06 20 20 cf ea 02 20 03 20 60 01 20 20 20 20 20 20 20 20 20 10 20 20 20 20 20 20
...

and is very long of course (there is much left).

I was reading the first lines. As you can see (or let traduce; I took this to examine if I was correct and if I wasnt it was always the more correct one: the link) it first pops the r10 from the stack. In the rex byte 01001101 the 3th bit is set, but not used because there isnt any modrm byte in this kind of the pop-operation. Then the opcodes make strange thing with and and the register and it do not look like instruction but like a queue of numbers. There are nops and many false opcodes interrupts. So I now wonder, how this comes since the code starts with the first bit. Below it begins to do what it have to but the first lines? Why?

0:  4d 5a                   rex.WRB pop r10
2:  90                      nop
3:  20 03                   and    BYTE PTR [rbx],al
5:  20 20                   and    BYTE PTR [rax],ah
7:  20 04 20                and    BYTE PTR [rax+riz*1],al
a:  20 20                   and    BYTE PTR [rax],ah
c:  ff                      (bad)
d:  ff 20                   jmp    QWORD PTR [rax]
f:  20 b8 20 20 20 20       and    BYTE PTR [rax+0x20202020],bh
15: 20 20                   and    BYTE PTR [rax],ah
17: 20 40 20                and    BYTE PTR [rax+0x20],al
1a: 20 20                   and    BYTE PTR [rax],ah
1c: 20 20                   and    BYTE PTR [rax],ah
1e: 20 20                   and    BYTE PTR [rax],ah
20: 20 20                   and    BYTE PTR [rax],ah
22: 20 20                   and    BYTE PTR [rax],ah
24: 20 20                   and    BYTE PTR [rax],ah
26: 20 20                   and    BYTE PTR [rax],ah
28: 20 20                   and    BYTE PTR [rax],ah
2a: 20 20                   and    BYTE PTR [rax],ah
2c: 20 20                   and    BYTE PTR [rax],ah
2e: 20 20                   and    BYTE PTR [rax],ah
30: 20 20                   and    BYTE PTR [rax],ah
32: 20 20                   and    BYTE PTR [rax],ah
34: 20 20                   and    BYTE PTR [rax],ah
36: 20 20                   and    BYTE PTR [rax],ah
38: 20 20                   and    BYTE PTR [rax],ah
3a: 20 20                   and    BYTE PTR [rax],ah
3c: 80 20 20                and    BYTE PTR [rax],0x20
3f: 20 0e                   and    BYTE PTR [rsi],cl
41: 1f                      (bad)
42: ba 0e 20 b4 09          mov    edx,0x9b4200e
47: cd 21                   int    0x21
49: b8 01 4c cd 21          mov    eax,0x21cd4c01
4e: 54                      push   rsp
4f: 68 69 73 20 70          push   0x70207369
54: 72 6f                   jb     0xc5
56: 67 72 61                addr32 jb 0xba
59: 6d                      ins    DWORD PTR es:[rdi],dx
5a: 20 63 61                and    BYTE PTR [rbx+0x61],ah
5d: 6e                      outs   dx,BYTE PTR ds:[rsi]
5e: 6e                      outs   dx,BYTE PTR ds:[rsi]
5f: 6f                      outs   dx,DWORD PTR ds:[rsi]
60: 74 20                   je     0x82
62: 62                      (bad)
63: 65 20 72 75             and    BYTE PTR gs:[rdx+0x75],dh
67: 6e                      outs   dx,BYTE PTR ds:[rsi]
68: 20 69 6e                and    BYTE PTR [rcx+0x6e],ch
6b: 20 44 4f 53             and    BYTE PTR [rdi+rcx*2+0x53],al
6f: 20 6d 6f                and    BYTE PTR [rbp+0x6f],ch
72: 64 65 2e 0d 0d 0a 24    fs gs cs or eax,0x20240a0d
79: 20
7a: 20 20                   and    BYTE PTR [rax],ah
7c: 20 20                   and    BYTE PTR [rax],ah
7e: 20 20                   and    BYTE PTR [rax],ah
80: 50                      push   rax
81: 45 20 20                and    BYTE PTR [r8],r12b
84: 64 86 13                xchg   BYTE PTR fs:[rbx],dl
87: 20 24 97                and    BYTE PTR [rdi+rdx*4],ah
8a: 09 62 20                or     DWORD PTR [rdx+0x20],esp
8d: 86 01                   xchg   BYTE PTR [rcx],al
8f: 20 b6 05 20 20 f0       and    BYTE PTR [rsi-0xfdfdffb],dh
95: 20 26                   and    BYTE PTR [rsi],ah
97: 20 0b                   and    BYTE PTR [rbx],cl
99: 02 02                   add    al,BYTE PTR [rdx]
9b: 25 20 18 20 20          and    eax,0x20201820
a0: 20 40 20                and    BYTE PTR [rax+0x20],al
a3: 20 20                   and    BYTE PTR [rax],ah
a5: 02 20                   add    ah,BYTE PTR [rax]
a7: 20 d0                   and    al,dl
a9: 14 20                   adc    al,0x20
ab: 20 20                   and    BYTE PTR [rax],ah
ad: 10 20                   adc    BYTE PTR [rax],ah
af: 20 20                   and    BYTE PTR [rax],ah
b1: 20 20                   and    BYTE PTR [rax],ah
b3: 40 01 20                rex add DWORD PTR [rax],esp
b6: 20 20                   and    BYTE PTR [rax],ah
b8: 20 10                   and    BYTE PTR [rax],dl
ba: 20 20                   and    BYTE PTR [rax],ah
bc: 20 02                   and    BYTE PTR [rdx],al
be: 20 20                   and    BYTE PTR [rax],ah
c0: 04 20                   add    al,0x20
c2: 20 20                   and    BYTE PTR [rax],ah
c4: 20 20                   and    BYTE PTR [rax],ah
c6: 20 20                   and    BYTE PTR [rax],ah
c8: 05 20 02 20 20          add    eax,0x20200220
cd: 20 20                   and    BYTE PTR [rax],ah
cf: 20 20                   and    BYTE PTR [rax],ah
d1: 40 02 20                add    spl,BYTE PTR [rax]
d4: 20 06                   and    BYTE PTR [rsi],al
d6: 20 20                   and    BYTE PTR [rax],ah
d8: cf                      iret
d9: ea                      (bad)
da: 02 20                   add    ah,BYTE PTR [rax]
dc: 03 20                   add    esp,DWORD PTR [rax]
de: 60                      (bad)
df: 01 20                   add    DWORD PTR [rax],esp
e1: 20 20                   and    BYTE PTR [rax],ah
e3: 20 20                   and    BYTE PTR [rax],ah
e5: 20 20                   and    BYTE PTR [rax],ah
e7: 20 20                   and    BYTE PTR [rax],ah
e9: 10 20                   adc    BYTE PTR [rax],ah
eb: 20 20                   and    BYTE PTR [rax],ah
ed: 20 20                   and    BYTE PTR [rax],ah
ef: 20 20                   and    BYTE PTR [rax],ah
f1: 20 10                   and    BYTE PTR [rax],dl
f3: 20 20                   and    BYTE PTR [rax],ah
f5: 20 20                   and    BYTE PTR [rax],ah
f7: 20 20                   and    BYTE PTR [rax],ah
f9: 10 20                   adc    BYTE PTR [rax],ah
fb: 20 20                   and    BYTE PTR [rax],ah
fd: 20 20                   and    BYTE PTR [rax],ah
ff: 20                      .byte 0x20


问:
How to authenticate azure function for SharePoint online CSOM
I have a C# code which I want to run once a day as Azure function. The function is supposed to connect to my SharePoint Site and convert some documents. The code I altered a bit to fit my needs. The part I struggle with is how to authenticate while receiving the same context and avoiding refactoring my function/app. If I choose to use

```<code>AuthenticationManager am = new AuthenticationManager();

                using (var cc = am.GetSharePointOnlineAuthenticatedContextTenant(SiteUrl, userName, securePassword))
</code>```

for authentication there is a problem because of the 2FA. They are some posts on the internet recommending the following approach, but it is working only if I ran the app manually.

```<code>using (var cc = authManager.GetWebLoginClientContext(SiteUrl))
</code>```

From what I understood I probably need a certificate. However I can't understand how the SharePoint site will know about this certificate and grant the access. Also a lot of the examples are about using rest API which is fine but in my case this means refactoring the whole code. So is there a way to authenticate my function while not changing much of the CSOM code I currently use and not using REST API's and tokens?

答:
From what I understood I probably need a certificate. However I can't understand how the SharePoint site will know about this certificate and grant the access.

Here is an sample code of Authentication of SharePoint Online CSOM using Azure Function class script called csomHelper.csx.

```<code>using Microsoft.IdentityModel.Clients.ActiveDirectory;
using Microsoft.SharePoint.Client;
using System.Security.Cryptography.X509Certificates;

public static class csomHelper {
private static string ClientId = "(Application ID)";
private static string Cert = "(filename).pfx";
private static string CertPassword = "(password)";

private static string Authority = "[https://login.windows.net/](https://login.windows.net/)(tenantName).onmicrosoft.com/";
private static string Resource = "[https://](https:)(tenantName).sharepoint.com/";
public async static Task<ClientContext> GetClientContext(string siteUrl)
{

var authenticationContext = new AuthenticationContext(Authority, false);
var certPath = Path.Combine(Environment.GetEnvironmentVariable("HOME"), "site\\wwwroot\\<FunctionName>\\", Cert);
var cert = new X509Certificate2(System.IO.File.ReadAllBytes(certPath),

CertPassword,
X509KeyStorageFlags.Exportable |
X509KeyStorageFlags.MachineKeySet |
X509KeyStorageFlags.PersistKeySet);

var authenticationResult = await authenticationContext.AcquireTokenAsync(Resource, new ClientAssertionCertificate(ClientId, cert));
var token = authenticationResult.AccessToken;
var ctx = new ClientContext(siteUrl);

ctx.ExecutingWebRequest += (s, e) =>
{
e.WebRequestExecutor.RequestHeaders["Authorization"] ="Bearer "` `+ authenticationResult.AccessToken;
};
return ctx;
    }
}
</code>```

In the above code, we have App ID as ClientId, certificate file name, password and the tenant name to get the wright login and resource URL.

The line var certPath open the certificate file.

The line var authenticationResult uses the cert to initiate an authentication flow to SharePoint. If this works, the application receives an access token that it can use to access SharePoint.

The code block starts from var ctx hook up an event handler which will run whenever CSOM executes a request; it will insert the access token into the HTTP header so the request is authorized by SharePoint.

For more information, please visit this article.

Body:
I have a C# code which I want to run once a day as Azure function. The function is supposed to connect to my SharePoint Site and convert some documents. The code I altered a bit to fit my needs. The part I struggle with is how to authenticate while receiving the same context and avoiding refactoring my function/app. If I choose to use

AuthenticationManager am = new AuthenticationManager();

                using (var cc = am.GetSharePointOnlineAuthenticatedContextTenant(SiteUrl, userName, securePassword))

for authentication there is a problem because of the 2FA. They are some posts on the internet recommending the following approach, but it is working only if I ran the app manually.

using (var cc = authManager.GetWebLoginClientContext(SiteUrl))

From what I understood I probably need a certificate. However I can't understand how the SharePoint site will know about this certificate and grant the access. Also a lot of the examples are about using rest API which is fine but in my case this means refactoring the whole code. So is there a way to authenticate my function while not changing much of the CSOM code I currently use and not using REST API's and tokens?

Body:
From what I understood I probably need a certificate. However I can't understand how the SharePoint site will know about this certificate and grant the access.

Here is an sample code of Authentication of SharePoint Online CSOM using Azure Function class script called csomHelper.csx.

using Microsoft.IdentityModel.Clients.ActiveDirectory;
using Microsoft.SharePoint.Client;
using System.Security.Cryptography.X509Certificates;

public static class csomHelper {
private static string ClientId = "(Application ID)";
private static string Cert = "(filename).pfx";
private static string CertPassword = "(password)";

private static string Authority = "[https://login.windows.net/](https://login.windows.net/)(tenantName).onmicrosoft.com/";
private static string Resource = "[https://](https:)(tenantName).sharepoint.com/";
public async static Task<ClientContext> GetClientContext(string siteUrl)
{

var authenticationContext = new AuthenticationContext(Authority, false);
var certPath = Path.Combine(Environment.GetEnvironmentVariable("HOME"), "site\\wwwroot\\&lt;FunctionName&gt;\\", Cert);
var cert = new X509Certificate2(System.IO.File.ReadAllBytes(certPath),

CertPassword,
X509KeyStorageFlags.Exportable |
X509KeyStorageFlags.MachineKeySet |
X509KeyStorageFlags.PersistKeySet);

var authenticationResult = await authenticationContext.AcquireTokenAsync(Resource, new ClientAssertionCertificate(ClientId, cert));
var token = authenticationResult.AccessToken;
var ctx = new ClientContext(siteUrl);

ctx.ExecutingWebRequest += (s, e) =>
{
e.WebRequestExecutor.RequestHeaders["Authorization"] ="Bearer "` `+ authenticationResult.AccessToken;
};
return ctx;
    }
}

In the above code, we have App ID as ClientId, certificate file name, password and the tenant name to get the wright login and resource URL.

The line var certPath open the certificate file.

The line var authenticationResult uses the cert to initiate an authentication flow to SharePoint. If this works, the application receives an access token that it can use to access SharePoint.

The code block starts from var ctx hook up an event handler which will run whenever CSOM executes a request; it will insert the access token into the HTTP header so the request is authorized by SharePoint.

For more information, please visit this article.