问:
How to merge two json array which has same value in react native in a javascript
I have a two json objects. i need to merge those in a single object which has same id.

Example

```<code> const [orderdetails, setOrderdetails] = useState([]);
  const [ordertracking, setOrdertracking] = useState([]);

  useEffect(() => {
    fetch('first api')
      .then((response) => response.json())
      .then((json) => setOrderdetails(json))
      .catch((error) => console.error(error))
      .finally(() => setLoading(false));
  }, []);

  useEffect(() => {
    fetch('second api')
      .then((response) => response.json())
      .then((json) => setOrdertracking(json))
      .catch((error) => console.error(error))
      .finally(() => setLoading(false));
  }, []);
</code>```

Sample data

First api response in orderdetails

```<code>{
    "statusCode": "200",
    "message": "success",
    "content": [
         { subject_id: 711, topics: ["Test", "Test2"] },
         { subject_id: 712, topics: ["topic1", "Topic2"] }
]
}
</code>```

Second api response in ordertracking

```<code>{
        "statusCode": "200",
        "message": "success",
        "content": [
              {subject_id: 711, subject_name: "Science"},
               {subject_id: 712, subject_name: "Maths"}
    ]
    }
</code>```

I want the merged result to be:

```<code>const result = [
  {
    subject_id: "711",
    subjectName: "Science",
    topics: [ "Test", "Test2" ]
  },
  {
    subject_id: "712",
    subjectName: "Maths",
    topics: [ "topic1", "Topic2" ]
  }
];
</code>```

As you can see there are two api response in a josn format I need to merge those json object which is inside a content which has same 'subject_id'  and rest of the values as a single data.

I need a solution in javascript for my react native app if you can suggest me a react native code

答:
I guess you are looking for this answer</p>
```<code>let arr1 = [
         { subject_id: 711, topics: ["Test", "Test2"] },
         { subject_id: 712, topics: ["topic1", "Topic2"] }
];

let arr2 = [
         {subject_id: 711, subject_name: "Science"},
         {subject_id: 712, subject_name: "Maths"}
];

let res = arr1.map((item, i) => Object.assign({}, item, arr2[i]));

// Or the same with es6 syntax
let resEs6 = arr1.map((item, i) => {
    return { ...item, ...arr2[i] };
  });

</code>```

Body:
I have a two json objects. i need to merge those in a single object which has same id.

Example

 const [orderdetails, setOrderdetails] = useState([]);
  const [ordertracking, setOrdertracking] = useState([]);

  useEffect(() => {
    fetch('first api')
      .then((response) => response.json())
      .then((json) => setOrderdetails(json))
      .catch((error) => console.error(error))
      .finally(() => setLoading(false));
  }, []);

  useEffect(() => {
    fetch('second api')
      .then((response) => response.json())
      .then((json) => setOrdertracking(json))
      .catch((error) => console.error(error))
      .finally(() => setLoading(false));
  }, []);

Sample data

First api response in orderdetails

{
    "statusCode": "200",
    "message": "success",
    "content": [
         { subject_id: 711, topics: ["Test", "Test2"] },
         { subject_id: 712, topics: ["topic1", "Topic2"] }
]
}

Second api response in ordertracking

{
        "statusCode": "200",
        "message": "success",
        "content": [
              {subject_id: 711, subject_name: "Science"},
               {subject_id: 712, subject_name: "Maths"}
    ]
    }

I want the merged result to be:

const result = [
  {
    subject_id: "711",
    subjectName: "Science",
    topics: [ "Test", "Test2" ]
  },
  {
    subject_id: "712",
    subjectName: "Maths",
    topics: [ "topic1", "Topic2" ]
  }
];

As you can see there are two api response in a josn format I need to merge those json object which is inside a content which has same 'subject_id'  and rest of the values as a single data.

I need a solution in javascript for my react native app if you can suggest me a react native code

Body:
I guess you are looking for this answer

let arr1 = [
         { subject_id: 711, topics: ["Test", "Test2"] },
         { subject_id: 712, topics: ["topic1", "Topic2"] }
];

let arr2 = [
         {subject_id: 711, subject_name: "Science"},
         {subject_id: 712, subject_name: "Maths"}
];

let res = arr1.map((item, i) => Object.assign({}, item, arr2[i]));

// Or the same with es6 syntax
let resEs6 = arr1.map((item, i) => {
    return { ...item, ...arr2[i] };
  });


来源: stackexchange

问:
Spring - WEB services Problem in Instantiation
HI ALL

I am trying to expose spring webservice i am facing one problem My Service Object is not getting initialized

---------------End point Class--------------

```<code>import org.springframework.ws.server.endpoint.AbstractMarshallingPayloadEndpoint;

import com.mt.service.CalculatorService;

public abstract class AbstractCalculatorEndpoint  extends AbstractMarshallingPayloadEndpoint  {
    protected CalculatorService calculatorService;

    public void setCalcService(CalculatorService calculatorService) {
        this.calculatorService = calculatorService;
    }

    protected abstract Object invokeInternal(Object request) throws Exception;

}
</code>```

-----------AddNumberEndpoint Class------------

```<code>    package com.mt.endpoint;

import com.mt.calculator.schema.AddNumberRequest;

public class AddNumberEndpoint extends AbstractCalculatorEndpoint {
    @Override
    protected Object invokeInternal(Object request) throws Exception {
        AddNumberRequest addNumberRequest = (AddNumberRequest) request;
        if( calculatorService==null )
            System.out.println("------------------- I AM NULL    -------    :(   ");
        return calculatorService.addTwoNumber(addNumberRequest.getFirstNumber(),
                addNumberRequest.getSecondNumber());
    }
}
</code>```

My Servlet Configuration

```<code><?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:oxm="http://www.springframework.org/schema/oxm"
    xsi:schemaLocation="http://www.springframework.org/schema/beans
    http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
        http://www.springframework.org/schema/oxm
    http://www.springframework.org/schema/oxm/spring-oxm-1.5.xsd">

    <bean id="calculatorService" class="com.mt.service.CalculatorServiceImpl"
        init-method="initialize" />

    <bean id="Calculator"
        class="org.springframework.ws.wsdl.wsdl11.DefaultWsdl11Definition">
        <property name="schema" ref="schema" />
        <property name="portTypeName" value="Calculator" />
        <property name="locationUri" value="/services" />
        <property name="targetNamespace" value="http://www.mt.com/calculator/schema" />
    </bean>

    <bean id="schema" class="org.springframework.xml.xsd.SimpleXsdSchema">
        <property name="xsd" value="/WEB-INF/calculator.xsd" />
    </bean>

    <bean id="validatingInterceptor"
        class="org.springframework.ws.soap.server.endpoint.interceptor.PayloadValidatingInterceptor">
        <property name="xsdSchema" ref="schema" />
        <property name="validateRequest" value="true" />
        <property name="validateResponse" value="true" />
    </bean>

    <bean id="loggingInterceptor"
        class="org.springframework.ws.server.endpoint.interceptor.PayloadLoggingInterceptor" />

    <oxm:jaxb2-marshaller id="marshaller"
        contextPath="com.mt.calculator.schema" />
    <oxm:jaxb2-marshaller id="unmarshaller"
        contextPath="com.mt.calculator.schema" />

    <bean id="addNumberEndpoint" class=" com.mt.endpoint.AddNumberEndpoint"
        autowire="byName" />

    <bean name="endpointMapping"
        class="org.springframework.ws.server.endpoint.mapping.PayloadRootQNameEndpointMapping">
        <property name="interceptors">
            <list>
                <ref local="loggingInterceptor" />
                <ref local="validatingInterceptor" />
            </list>
        </property>
        <property name="mappings">
            <props>
                <prop key="{http://www.mt.com/calculator/schema}AddNumberRequest">
                    addNumberEndpoint</prop>
            </props>
        </property>
    </bean>

    <bean id="exceptionResolver"
        class="org.springframework.ws.soap.server.endpoint.SoapFaultMappingExceptionResolver">
        <property name="defaultFault" value="SERVER" />
        <property name="exceptionMappings">
            <props>
                <prop key="org.springframework.oxm.ValidationFailureException">CLIENT,Invalid request</prop>
                <prop key="com.mt.service.CalculatorException">SERVER</prop>
            </props>
        </property>
    </bean>
</beans>
</code>```

Plese Let me know if you need any other information

Thanks A Lot

Thanks A Lot
 Previous Exception gone Now I Am Getting
--------

```<code><SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">
   <SOAP-ENV:Header/>
   <SOAP-ENV:Body>
      <SOAP-ENV:Fault>
         <faultcode>SOAP-ENV:Server</faultcode>
         <faultstring xml:lang="en">JAXB marshalling exception; nested exception is javax.xml.bind.MarshalException
 - with linked exception:
[com.sun.istack.SAXException2: unable to marshal type "java.lang.Integer" as an element because it is missing an @XmlRootElement annotation]</faultstring>
      </SOAP-ENV:Fault>
   </SOAP-ENV:Body>
</SOAP-ENV:Envelope>
</code>```

答:
You are autowiring addNumberEndpoint by name so Spring will look for calcService to bind to it (because of the setter name) but the id of CalculatorService is calculatorService.

Change setCalcService(...) to setCalculatorService(...) in AbstractCalculatorEndpoint

ContentLicense: CC BY-SA 3.0

Body:
HI ALL

I am trying to expose spring webservice i am facing one problem My Service Object is not getting initialized

---------------End point Class--------------

import org.springframework.ws.server.endpoint.AbstractMarshallingPayloadEndpoint;

import com.mt.service.CalculatorService;

public abstract class AbstractCalculatorEndpoint  extends AbstractMarshallingPayloadEndpoint  {
    protected CalculatorService calculatorService;

    public void setCalcService(CalculatorService calculatorService) {
        this.calculatorService = calculatorService;
    }

    protected abstract Object invokeInternal(Object request) throws Exception;

}

-----------AddNumberEndpoint Class------------

    package com.mt.endpoint;

import com.mt.calculator.schema.AddNumberRequest;

public class AddNumberEndpoint extends AbstractCalculatorEndpoint {
    @Override
    protected Object invokeInternal(Object request) throws Exception {
        AddNumberRequest addNumberRequest = (AddNumberRequest) request;
        if( calculatorService==null )
            System.out.println("------------------- I AM NULL    -------    :(   ");
        return calculatorService.addTwoNumber(addNumberRequest.getFirstNumber(),
                addNumberRequest.getSecondNumber());
    }
}

My Servlet Configuration

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:oxm="http://www.springframework.org/schema/oxm"
    xsi:schemaLocation="http://www.springframework.org/schema/beans
    http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
        http://www.springframework.org/schema/oxm
    http://www.springframework.org/schema/oxm/spring-oxm-1.5.xsd">

    <bean id="calculatorService" class="com.mt.service.CalculatorServiceImpl"
        init-method="initialize" />

    <bean id="Calculator"
        class="org.springframework.ws.wsdl.wsdl11.DefaultWsdl11Definition">
        <property name="schema" ref="schema" />
        <property name="portTypeName" value="Calculator" />
        <property name="locationUri" value="/services" />
        <property name="targetNamespace" value="http://www.mt.com/calculator/schema" />
    </bean>

    <bean id="schema" class="org.springframework.xml.xsd.SimpleXsdSchema">
        <property name="xsd" value="/WEB-INF/calculator.xsd" />
    </bean>

    <bean id="validatingInterceptor"
        class="org.springframework.ws.soap.server.endpoint.interceptor.PayloadValidatingInterceptor">
        <property name="xsdSchema" ref="schema" />
        <property name="validateRequest" value="true" />
        <property name="validateResponse" value="true" />
    </bean>

    <bean id="loggingInterceptor"
        class="org.springframework.ws.server.endpoint.interceptor.PayloadLoggingInterceptor" />

    <oxm:jaxb2-marshaller id="marshaller"
        contextPath="com.mt.calculator.schema" />
    <oxm:jaxb2-marshaller id="unmarshaller"
        contextPath="com.mt.calculator.schema" />

    <bean id="addNumberEndpoint" class=" com.mt.endpoint.AddNumberEndpoint"
        autowire="byName" />

    <bean name="endpointMapping"
        class="org.springframework.ws.server.endpoint.mapping.PayloadRootQNameEndpointMapping">
        <property name="interceptors">
            <list>
                <ref local="loggingInterceptor" />
                <ref local="validatingInterceptor" />
            </list>
        </property>
        <property name="mappings">
            <props>
                <prop key="{http://www.mt.com/calculator/schema}AddNumberRequest">
                    addNumberEndpoint</prop>
            </props>
        </property>
    </bean>

    <bean id="exceptionResolver"
        class="org.springframework.ws.soap.server.endpoint.SoapFaultMappingExceptionResolver">
        <property name="defaultFault" value="SERVER" />
        <property name="exceptionMappings">
            <props>
                <prop key="org.springframework.oxm.ValidationFailureException">CLIENT,Invalid request</prop>
                <prop key="com.mt.service.CalculatorException">SERVER</prop>
            </props>
        </property>
    </bean>
</beans>

Plese Let me know if you need any other information

Thanks A Lot

Thanks A Lot
 Previous Exception gone Now I Am Getting
--------

<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">
   <SOAP-ENV:Header/>
   <SOAP-ENV:Body>
      <SOAP-ENV:Fault>
         <faultcode>SOAP-ENV:Server</faultcode>
         <faultstring xml:lang="en">JAXB marshalling exception; nested exception is javax.xml.bind.MarshalException
 - with linked exception:
[com.sun.istack.SAXException2: unable to marshal type "java.lang.Integer" as an element because it is missing an @XmlRootElement annotation]</faultstring>
      </SOAP-ENV:Fault>
   </SOAP-ENV:Body>
</SOAP-ENV:Envelope>


问:
How to find an element by href value using selenium python?
I have href value of an anchor tag which only have href value as attribute. Now I want to find the element in the page which have same value as my href value and click it. I am unable to find any way of doing this using standard selenium methods.How can I do this?
Basically these are the functions I found but it seems that I can't use any of these:</p>

```<code>find_element_by_id
find_element_by_name
find_element_by_xpath
find_element_by_link_text
find_element_by_partial_link_text
find_element_by_tag_name
find_element_by_class_name
find_element_by_css_selector
</code>```

答:
When the button looks like

```<code><a href="server_export.php" class="tab"> Export</a>
</code>```

and because possibly the text is translated, we cannot use the search for the text Export, so we have to search e.g. for server_export within the href and can e.g. click it with

```<code>driver.find_element(By.XPATH,'//a[contains(@href,"server_export")]').click()
</code>```

Note: Using the recommended find_element(By.xxxxx, ...) format.

Body:
I have href value of an anchor tag which only have href value as attribute. Now I want to find the element in the page which have same value as my href value and click it. I am unable to find any way of doing this using standard selenium methods.How can I do this?
Basically these are the functions I found but it seems that I can't use any of these:

find_element_by_id
find_element_by_name
find_element_by_xpath
find_element_by_link_text
find_element_by_partial_link_text
find_element_by_tag_name
find_element_by_class_name
find_element_by_css_selector

Body:
When the button looks like

<a href="server_export.php" class="tab">&nbsp;Export</a>

and because possibly the text is translated, we cannot use the search for the text Export, so we have to search e.g. for server_export within the href and can e.g. click it with

driver.find_element(By.XPATH,'//a[contains(@href,"server_export")]').click()

Note: Using the recommended find_element(By.xxxxx, ...) format.

ContentLicense: CC BY-SA 4.0


问:
UUID as primary key in DynamoDB -- good or bad idea?
In a new DynamoDB table, my use cases are already fulfilled by the following key schema design:

partition key: user_id

sort key: entity_id

Basically, access patterns are:

Get specific post by a specific user.

Get specific comment by a specific user.

List all posts by specific user.

List all comments by specific user.

List all entities (post or comment) by a specific user.

What benefits do I get if I use a more random ID as partition key instead and simply use GSIs for my access patterns above?

partition key: pseudo_random_id (This is going to be a UUID in reality. Please ignore that this is NOT a UUID in the illustration).

GSI:

partition key: user_id

sort key: entity_id

答: if you are looking at unique + sequence number in DynamoDB, worth to read Atomic Counters as a option. That maintains a counter in a table. But could be a problem for a high load application that requests IDs. Because the UpdateItem is a synchronized per tuple.</p>


问:
Components not included in Strapi api response
I decided today that I'm going to use Strapi as my headless CMS for my portfolio, I've bumped into some issues though, which I just seem to not be able to find a solution to online. Maybe I'm just too clueless to actually find the real issue.

I have set up a schema for my projects that will be stored in Strapi (everything done in the web), but I've had some issues with my custom components, and that is, they are not part of the API responses when I run it through Postman. (Not just empty keys but not included in the response at all). All other fields, that are not components, are filled out as expected.

At first I thought it might have to do with the permissions, but everything is enabled so it can't be that, I also tried looking into the API in the code, but that logging the answer there didn't include the components either.

Here is an image of some of the fields in the schema, but more importantly the components that are not included in the response.

So my question is, do I need to create some sort of a parser or anything in the project to be able to include these fields, or why are they not included?

答: Gatsbyjs solution!!!: I am using single types and I have a component inside of that single type called for example header settings where users fill stuff like phone number location and so on... and I am also using Gatsbyjs for my front-end and here strapi.js Screenshot is a solution for that too.gatsby-config.js Screenshot Just in case someone has the same issue


问:
Targeting auto-generated elements with node.js puppeteer
I am using puppeteer for node.js v13.3.1, I am building a bot that will apply to jobs on linkedIn.
So far I have the following code.

```<code>const puppeteer = require('puppeteer');
   const SEARCHPARAM = "react" // change this for search param
require("dotenv").config();
(async () => {
    const browser = await puppeteer.launch({headless:false});
  const page = await browser.newPage();
  await page.goto('https://www.linkedin.com/login?fromSignIn=true&trk=guest_homepage-basic_nav-header-signin')
  await page.setViewport({ width: 1920, height: 1080});
  await page.type("#username", process.env.NAMEUSERNAME)
  await page.type("#password", process.env.PASSWORD)
  await page.click('button[type="submit"]')
  //works fine
  await Promise.all([
    await page.waitForNavigation(),
    await page.click('#global-nav > div > nav > ul > li:nth-child(3)'),
    await page.type('input', SEARCHPARAM ),
    await page.click('button[type="button"]'),
    await page.click('#ember1052 > span') //Will not hit the button
  ]);

})();
</code>```

So far the bot is logging in and searching with the SEARCHPARAM just fine, however  I am having trouble getting the bot to hit the easy apply button on the job posting. Whenever I target this button I get the id as ember(number) and this number is never the same on different browser windows. I have tried targeting the button by class but node is telling me that it cannot find that selector.

Puppeteer docs for reference docs

答:
Please see this link for more detailed answer:
 How to click element in Puppeteer using xPath

```<code>const elements = await page.$x('<xPath>')
await elements[0].click()
</code>```

And your XPath selector should be like this:

```<code>const element = await page.$x('//button/span[text()="Easy Apply"]')
</code>```

Code syntax for:

Button element that contain span element which has text inside "Easy Apply"

Body:
I am using puppeteer for node.js v13.3.1, I am building a bot that will apply to jobs on linkedIn.
So far I have the following code.

const puppeteer = require('puppeteer');
   const SEARCHPARAM = "react" // change this for search param
require("dotenv").config();
(async () => {
    const browser = await puppeteer.launch({headless:false});
  const page = await browser.newPage();
  await page.goto('https://www.linkedin.com/login?fromSignIn=true&trk=guest_homepage-basic_nav-header-signin')
  await page.setViewport({ width: 1920, height: 1080});
  await page.type("#username", process.env.NAMEUSERNAME)
  await page.type("#password", process.env.PASSWORD)
  await page.click('button[type="submit"]')
  //works fine
  await Promise.all([
    await page.waitForNavigation(),
    await page.click('#global-nav > div > nav > ul > li:nth-child(3)'),
    await page.type('input', SEARCHPARAM ),
    await page.click('button[type="button"]'),
    await page.click('#ember1052 > span') //Will not hit the button
  ]);

})();

So far the bot is logging in and searching with the SEARCHPARAM just fine, however  I am having trouble getting the bot to hit the easy apply button on the job posting. Whenever I target this button I get the id as ember(number) and this number is never the same on different browser windows. I have tried targeting the button by class but node is telling me that it cannot find that selector.

Puppeteer docs for reference docs

Body:
Please see this link for more detailed answer:
 How to click element in Puppeteer using xPath

const elements = await page.$x('<xPath>')
await elements[0].click()

And your XPath selector should be like this:

const element = await page.$x('//button/span[text()="Easy Apply"]')

Code syntax for:

Button element that contain span element which has text inside "Easy Apply"


问:
Show duplicate entries removed by a set
I expect the below program to print the duplicate entries, which were removed by the Set.
But it always prints a empty array. What could be wrong?</p>
```<code>
public class HelloWorld{
    public static void main(String[] args){
        String str="HelloWorld";
        String[] strChar=str.split("");

        //Remove the duplicates
        Set<String> strSet=new LinkedHashSet<String>(Arrays.asList(strChar));

        List<String> strList=new ArrayList<String>(Arrays.asList(strChar));
        System.out.println(strList);
        System.out.println(strSet);

        //Show the duplicate entries that were removed by the set
        strList.removeAll(strSet);

        System.out.println(strList);
    }
}```
</code>```

答:
Because sets are more efficient for removal I would do it like this.

```<code>String str="HelloWorld";
String[] strChar=str.toLowerCase().split("");
Set<String> dups = new HashSet<>();
Set<String> nonDups = new HashSet<>();
for (String v : strChar) {
    nonDups.add(v);
    if(!dups.add(v)) {
        nonDups.remove(v);
    }
}
dups.removeAll(nonDups);
System.out.println(dups);
</code>```

prints

```<code>[l, o]
</code>```

If would like to get a count of the duplicates, you can do it like this.

create a Map<Character, Integer> to count occurrence of each character

iterate over the characters and if the value is null, set to 1, otherwise, add 1 to existing value.

then simply iterate the entrySet and print those whose count is > 1

```<code>String str="HelloWorld";
Map<Character, Integer> counts = new HashMap<>();
for (char ch : str.toCharArray()) {
    counts.compute(ch, (k,v)->v ==null ? 1 : v+1);
}
counts.entrySet().forEach(e-> {
    if (e.getValue() > 1) {
        System.out.println(e);
    }
});
</code>```

prints

```<code>l=3
o=2
</code>```

Title: Show duplicate entries removed by a set

Body:
I expect the below program to print the duplicate entries, which were removed by the Set.
But it always prints a empty array. What could be wrong?

public class HelloWorld{
    public static void main(String[] args){
        String str="HelloWorld";
        String[] strChar=str.split("");

        //Remove the duplicates
        Set<String> strSet=new LinkedHashSet<String>(Arrays.asList(strChar));

        List<String> strList=new ArrayList<String>(Arrays.asList(strChar));
        System.out.println(strList);
        System.out.println(strSet);

        //Show the duplicate entries that were removed by the set
        strList.removeAll(strSet);

        System.out.println(strList);
    }
}```

Body:
Because sets are more efficient for removal I would do it like this.

String str="HelloWorld";
String[] strChar=str.toLowerCase().split("");
Set<String> dups = new HashSet<>();
Set<String> nonDups = new HashSet<>();
for (String v : strChar) {
    nonDups.add(v);
    if(!dups.add(v)) {
        nonDups.remove(v);
    }
}
dups.removeAll(nonDups);
System.out.println(dups);

prints

[l, o]

If would like to get a count of the duplicates, you can do it like this.

create a Map<Character, Integer> to count occurrence of each character

iterate over the characters and if the value is null, set to 1, otherwise, add 1 to existing value.

then simply iterate the entrySet and print those whose count is > 1

String str="HelloWorld";
Map<Character, Integer> counts = new HashMap<>();
for (char ch : str.toCharArray()) {
    counts.compute(ch, (k,v)->v ==null ? 1 : v+1);
}
counts.entrySet().forEach(e-> {
    if (e.getValue() > 1) {
        System.out.println(e);
    }
});

prints

l=3
o=2


问:
How does the address translation between Main Memory and Disk storage work?
This question comes up when I learn about virtual memory and memory management.

The following describes what I understand so far:

The memory hierarchy, which benefits from locality principle, briefly includes (from top to bottom):

register

cache (SRAM)

main memory (DRAM)

disk storage

A page table provides the address translation from virtual address to physical address

The virtual memory provides an abstraction for physical memory

Page(Frame) is the basic unit when memory management unit (MMU) manipulates memory between main memory and disk.

A process can only understand the addresses inside the virtual address space.

Each process has its own virtual address space.

Each process has its own page table, and all page tables are maintained by the kernel.

a physical address describes a location inside main memory

Translation Lookaside Buffer (TLB) is introduced as a cache-version page table.

Cache stores a tag field for each cache line to determine its mapping to main memory

Cache line is the basic unit when MMU manipulates memory between main memory and cache.

In each cache line, it stores a tag field and a valid bit to determine what range of physical addresses (e.g. what part of main memory) resides in the cache line.

After tranlsating virtual address into physical address with TLB or page table, MMU compares the physical address with cache line tag field, and finds the desired memory content (assuming that cache hits).

I believe there's an address translation mechanism between the main memory and the disk for 2 reasons:

The main memory is the locality principle result for the disk.

To reduce main memory miss rate, the main memory applies the full associative placement policy.

However, I only find few material which possiblely relates to the mechanism.

wiki: frame table data says:

Frame table data

The simplest page table systems often maintain a frame table and a page table. The frame table holds information about which frames are mapped.

and wiki: LBA says:

Logical block addressing (LBA) is a common scheme used for specifying the location of blocks of data stored on computer storage devices, generally secondary storage systems such as hard disk drives.

So I guess there's a frame table to store the address translation between physical address and LBA, and MMU would refer to the frame table when page fault occurs.

Please help to point out how does the address translation between Main Memory and Disk storage work.

Thanks for help!

答:
Hard-disk addressing and main memory addressing is done much differently. The CPU doesn't support hard-disk addressing by default. What modern CPUs support is PCI devices that can present an interface to hard-disks like NVME or SATA.

PCI devices have registers that are memory mapped in RAM. The position of these registers is specified in the MCFG which is an ACPI table. ACPI is a convention to represent hardware for software (the os) to be able to determine what is present on the motherboard that it needs to drive. ACPI is also a power management convention which is required for software to even shutdown the computer.

With that said, you can take example on Linux and how it does things to understand how a modern os does to make the link between pages on the hard-disk and the pages in main memory. On the swap management chapter of the kernel.org documentation (https://www.kernel.org/doc/gorman/html/understand/understand014.html), you can read the following:

11.2  Mapping Page Table Entries to Swap Entries

When a page is swapped out, Linux uses the corresponding PTE to store enough information to locate the page on disk again. Obviously a PTE is not large enough in itself to store precisely where on disk the page is located, but it is more than enough to store an index into the swap_info array and an offset within the swap_map and this is precisely what Linux does.

Each PTE, regardless of architecture, is large enough to store a swp_entry_t which is declared as follows in <linux/shmem_fs.h>

```<code>16 typedef struct {
17     unsigned long val;
18 } swp_entry_t;
</code>```

Two macros are provided for the translation of PTEs to swap entries and vice versa. They are pte_to_swp_entry() and swp_entry_to_pte() respectively.

You should probably read the link above along with one of my answers on cs.stackexchange.com: https://cs.stackexchange.com/questions/142525/data-transfer-between-cpu-ram-and-secondary-storage/142553#142553. This will probably provide a fair understanding of what is going on.

Everything is PCI today. You can think of all graphics cards, Intel HD Audio, the AHCI for SATA, the xHCI to drive USB or network cards. That pretty much sums what a current modern computer supports which is audio, USB, SATA, network and graphics. Understanding PCI is thus the key to understand how low level drivers work. The higher level implementation detail is unimportant and varies between os.

Body:
Hard-disk addressing and main memory addressing is done much differently. The CPU doesn't support hard-disk addressing by default. What modern CPUs support is PCI devices that can present an interface to hard-disks like NVME or SATA.

PCI devices have registers that are memory mapped in RAM. The position of these registers is specified in the MCFG which is an ACPI table. ACPI is a convention to represent hardware for software (the os) to be able to determine what is present on the motherboard that it needs to drive. ACPI is also a power management convention which is required for software to even shutdown the computer.

With that said, you can take example on Linux and how it does things to understand how a modern os does to make the link between pages on the hard-disk and the pages in main memory. On the swap management chapter of the kernel.org documentation (https://www.kernel.org/doc/gorman/html/understand/understand014.html), you can read the following:

11.2  Mapping Page Table Entries to Swap Entries

When a page is swapped out, Linux uses the corresponding PTE to store enough information to locate the page on disk again. Obviously a PTE is not large enough in itself to store precisely where on disk the page is located, but it is more than enough to store an index into the swap_info array and an offset within the swap_map and this is precisely what Linux does.

Each PTE, regardless of architecture, is large enough to store a swp_entry_t which is declared as follows in <linux/shmem_fs.h>

16 typedef struct {
17     unsigned long val;
18 } swp_entry_t;

Two macros are provided for the translation of PTEs to swap entries and vice versa. They are pte_to_swp_entry() and swp_entry_to_pte() respectively.

You should probably read the link above along with one of my answers on cs.stackexchange.com: https://cs.stackexchange.com/questions/142525/data-transfer-between-cpu-ram-and-secondary-storage/142553#142553. This will probably provide a fair understanding of what is going on.

Everything is PCI today. You can think of all graphics cards, Intel HD Audio, the AHCI for SATA, the xHCI to drive USB or network cards. That pretty much sums what a current modern computer supports which is audio, USB, SATA, network and graphics. Understanding PCI is thus the key to understand how low level drivers work. The higher level implementation detail is unimportant and varies between os.


问:
TypeError: Field 'id' expected a number but got <django.db.models.fields.related_descriptors
I have a simple view for showing all the current users that are conditionally filtered based on the user type that is requesting the data.

models.py

```<code>class User(AbstractBaseUser, PermissionsMixin):

    class SEX(models.TextChoices):
        MALE = "MALE", "Male"
        FEMALE = "FEMALE", "Female"

    class TYPES(models.TextChoices):
        ...

    type = models.CharField(
        _("User Type"), max_length=50, choices=TYPES.choices, default=None, blank=True, null=True)
    sex = models.CharField(
        _("Sex Type"), max_length=50, choices=SEX.choices, default=None, blank=True, null=True)
    email = models.EmailField(
        _("Your Email"), max_length=254, unique=True, default=None, blank=True, null=True)
    student_id = models.CharField(
        _("student id"), unique=True, max_length=10, blank=True, null=True)
    firstname = models.CharField(max_length=250, blank=False, null=True)
    middlename = models.CharField(max_length=250, blank=True, null=True)
    lastname = models.CharField(max_length=250, blank=False, null=True)
    # grade = models.ForeignKey(Grade, verbose_name=_(
    #     "grade"), on_delete=models.CASCADE, blank=True, null=True)
    room = models.ForeignKey(Class, verbose_name=_(
        "class"), on_delete=models.CASCADE, blank=True, null=True)
    phone = models.CharField(
        _("phone number"), max_length=13, blank=True, null=True)
    age = IntegerRangeField(min_value=1, max_value=99,
                            default=25, blank=True, null=True)

    owns = models.ManyToManyField(School, verbose_name=_(
        "School_owner"), related_name='Schoolowner', blank=True)

    workes = models.ForeignKey(School, verbose_name=_(
        "School_workes"), related_name='Schoolworkes', on_delete=models.SET_NULL, blank=True, null=True)

    learns = models.ForeignKey(School, verbose_name=_(
        "School_learns"), related_name='Schoollearns', on_delete=models.SET_NULL, blank=True, null=True)

    image = models.ImageField(
        _("photo"), upload_to='user/pp', max_length=None, blank=False, null=True)
    joined_date = models.DateTimeField(
        _("created at"), auto_now=False, auto_now_add=True)
    is_active = models.BooleanField(default=False)
    is_staff = models.BooleanField(default=False)
    is_superuser = models.BooleanField(default=False)
    last_login = models.DateTimeField(
        _("last login"), auto_now=True, auto_now_add=False)
</code>```

serializers.py

```<code>  class MyAccountSerializer(serializers.ModelSerializer):
        image = serializers.ImageField()

        class Meta:
            model = User
            fields = ['id', 'sex', 'type', 'phone', 'image', 'email', 'student_id', 'firstname', 'middlename', 'lastname',
                      'image', 'joined_date', 'is_active', 'is_superuser', 'is_staff', 'last_login']
</code>```

views.py

```<code>class TeacherList(generics.ListAPIView):
permission_classes = [IsAuthenticated, ]
serializer_class = ListUsersSerializer
queryset = User.objects.filter(type=User.TYPES.TEACHER)
</code>```

A simple filter like this works without any errors but since I need to filter the response based on the requesting user

this is what I want to use

```<code>class TeacherList(generics.ListAPIView):
    permission_classes = [IsAuthenticated, ]
    serializer_class = ListUsersSerializer
    # queryset = User.objects.filter(type=User.TYPES.TEACHER)

    def get_queryset(self):
        request = self.request.user
        if request.type == User.TYPES.OWNER:
            queryset = User.objects.filter(
                type=User.TYPES.TEACHER, workes=request.owns)
        elif request.type in [User.TYPES.STAFF, User.TYPES.DIRECTOR, User.TYPES.VICE_DIR]:
            queryset = User.objects.filter(
                type=User.TYPES.TEACHER, workes=request.workes)
        return queryset
</code>```

however this shows me an error that goes like this:

```<code>Field 'id' expected a number but got <django.db.models.fields.related_descriptors.create_forward_many_to_many_manager.<locals>.ManyRelatedManager object at 0x0000019D64AE7D60>.
</code>```

I suspect it has something to do with a many-to-many field of which I have one named owns which is what I'm using to filter in 1 of the conditions but don't know why it throws error

答:
I think instead of

```<code>queryset = User.objects.filter(
            type=User.TYPES.TEACHER, workes=request.owns)
</code>```

You should try something like:

```<code>queryset = User.objects.filter(
            type=User.TYPES.TEACHER, workes__in=request.owns.all())
</code>```

or

```<code>queryset = User.objects.filter(
            type=User.TYPES.TEACHER, workes__in=request.owns.values_list("id", flat=True))
</code>```

It should match all Users that have all ids from request.owns ids.

Body:
I have a simple view for showing all the current users that are conditionally filtered based on the user type that is requesting the data.

models.py

class User(AbstractBaseUser, PermissionsMixin):

    class SEX(models.TextChoices):
        MALE = "MALE", "Male"
        FEMALE = "FEMALE", "Female"

    class TYPES(models.TextChoices):
        ...

    type = models.CharField(
        _("User Type"), max_length=50, choices=TYPES.choices, default=None, blank=True, null=True)
    sex = models.CharField(
        _("Sex Type"), max_length=50, choices=SEX.choices, default=None, blank=True, null=True)
    email = models.EmailField(
        _("Your Email"), max_length=254, unique=True, default=None, blank=True, null=True)
    student_id = models.CharField(
        _("student id"), unique=True, max_length=10, blank=True, null=True)
    firstname = models.CharField(max_length=250, blank=False, null=True)
    middlename = models.CharField(max_length=250, blank=True, null=True)
    lastname = models.CharField(max_length=250, blank=False, null=True)
    # grade = models.ForeignKey(Grade, verbose_name=_(
    #     "grade"), on_delete=models.CASCADE, blank=True, null=True)
    room = models.ForeignKey(Class, verbose_name=_(
        "class"), on_delete=models.CASCADE, blank=True, null=True)
    phone = models.CharField(
        _("phone number"), max_length=13, blank=True, null=True)
    age = IntegerRangeField(min_value=1, max_value=99,
                            default=25, blank=True, null=True)

    owns = models.ManyToManyField(School, verbose_name=_(
        "School_owner"), related_name='Schoolowner', blank=True)

    workes = models.ForeignKey(School, verbose_name=_(
        "School_workes"), related_name='Schoolworkes', on_delete=models.SET_NULL, blank=True, null=True)

    learns = models.ForeignKey(School, verbose_name=_(
        "School_learns"), related_name='Schoollearns', on_delete=models.SET_NULL, blank=True, null=True)

    image = models.ImageField(
        _("photo"), upload_to='user/pp', max_length=None, blank=False, null=True)
    joined_date = models.DateTimeField(
        _("created at"), auto_now=False, auto_now_add=True)
    is_active = models.BooleanField(default=False)
    is_staff = models.BooleanField(default=False)
    is_superuser = models.BooleanField(default=False)
    last_login = models.DateTimeField(
        _("last login"), auto_now=True, auto_now_add=False)

serializers.py

  class MyAccountSerializer(serializers.ModelSerializer):
        image = serializers.ImageField()

        class Meta:
            model = User
            fields = ['id', 'sex', 'type', 'phone', 'image', 'email', 'student_id', 'firstname', 'middlename', 'lastname',
                      'image', 'joined_date', 'is_active', 'is_superuser', 'is_staff', 'last_login']

views.py

class TeacherList(generics.ListAPIView):
permission_classes = [IsAuthenticated, ]
serializer_class = ListUsersSerializer
queryset = User.objects.filter(type=User.TYPES.TEACHER)

A simple filter like this works without any errors but since I need to filter the response based on the requesting user

this is what I want to use

class TeacherList(generics.ListAPIView):
    permission_classes = [IsAuthenticated, ]
    serializer_class = ListUsersSerializer
    # queryset = User.objects.filter(type=User.TYPES.TEACHER)

    def get_queryset(self):
        request = self.request.user
        if request.type == User.TYPES.OWNER:
            queryset = User.objects.filter(
                type=User.TYPES.TEACHER, workes=request.owns)
        elif request.type in [User.TYPES.STAFF, User.TYPES.DIRECTOR, User.TYPES.VICE_DIR]:
            queryset = User.objects.filter(
                type=User.TYPES.TEACHER, workes=request.workes)
        return queryset

however this shows me an error that goes like this:

Field 'id' expected a number but got <django.db.models.fields.related_descriptors.create_forward_many_to_many_manager.<locals>.ManyRelatedManager object at 0x0000019D64AE7D60>.

I suspect it has something to do with a many-to-many field of which I have one named owns which is what I'm using to filter in 1 of the conditions but don't know why it throws error

Body:
I think instead of

queryset = User.objects.filter(
            type=User.TYPES.TEACHER, workes=request.owns)

You should try something like:

queryset = User.objects.filter(
            type=User.TYPES.TEACHER, workes__in=request.owns.all())

or

queryset = User.objects.filter(
            type=User.TYPES.TEACHER, workes__in=request.owns.values_list("id", flat=True))

It should match all Users that have all ids from request.owns ids.


问:
Django channels websocket disconnects after handshake
I am building a simple chat room following examples of django channels. Everything worked like a charm yesterday and I managed to create a chatroom and even managed to chat in there. All of a sudden without any change in my code the Websocket started  disconnecting immediately after connection and handshake.

```<code>My setup:
Django == 1.10.5
Python == 2.7
channels == 1.1.8
asgi-redis == 1.4.2
daphne == 1.3.0
</code>```

My consumers.py looks like this:

```<code>consumers.py:
@channel_session
def ws_connect(message):
    room = message.content['path'].strip("/")
    message.channel_session['room'] = room
    Group("chat").add(message.reply_channel)
    message.reply_channel.send({"accept": True})
</code>```

And the frontend part :

```<code> $(function() {
        // When we're using HTTPS, use WSS too.
        var ws_scheme = window.location.protocol = "ws";
        var chatsock = new WebSocket(ws_scheme + '://' + window.location.host + window.location.pathname);

        chatsock.onmessage = function(message) {
            var data = JSON.parse(message.data);
            var chat = $("#chat");
            var ele = $('<tr></tr>');
            console.log(data);

            ele.append(
                $("<td></td>").text(data.timestamp)
            );
            ele.append(
                $("<td></td>").text(data.handle)
            );
            ele.append(
                $("<td></td>").text(data.message)
            );

            chat.append(ele)
        };

        $("#chatform").on("submit", function(event) {
            var time = new Date();
            var string = time.getHours() + ":" + time.getMinutes() + ":" + time.getSeconds();
            // var timestamp = time.getHourMinuteSecond();
            var message = {
                timestamp: string,
                handle: $('#handle').val(),
                message: $('#message').val()
            };
            console.log("submit");
            chatsock.send(JSON.stringify(message));
            $("#message").val('').focus();
            return false;
        });
    });
</code>```

Maybe an update in some technology is out. I am struggling to find out why this happens. In settings.py I have the following configuration for the redis channel layer:

```<code>CHANNEL_LAYERS = {
    "default": {
        "BACKEND": "asgi_redis.RedisChannelLayer",
        "CONFIG": {
            "hosts": [("localhost", 6379)],
        },
        "ROUTING": "config.routing.channel_routing",
    },
}
</code>```

I have another websocket logic for notifications which is also disconnecting right after handshake. I tried updating Django to 1.11. but no luck. In routing.py of the chat app:

```<code>chat_routing = [
    route("websocket.connect", consumers.ws_connect),
    route("websocket.receive", consumers.ws_message),
    route("websocket.disconnect", consumers.ws_disconnect),
]
</code>```

I am running on Linux Ubuntu 16.04 and the only thing I updated in the last few days is Chrome version. So any ideas what should I do?

答:
The error can be fixed if you add a slash +'/' to the end of the URL:

```<code>var chatsock = new WebSocket(ws_scheme + '://' + window.location.host + window.location.pathname + '/');
</code>```

In most cases / missing at the end causes problems in Chrome browser, but works fine in Firefox

Body:
I am building a simple chat room following examples of django channels. Everything worked like a charm yesterday and I managed to create a chatroom and even managed to chat in there. All of a sudden without any change in my code the Websocket started  disconnecting immediately after connection and handshake.

My setup:
Django == 1.10.5
Python == 2.7
channels == 1.1.8
asgi-redis == 1.4.2
daphne == 1.3.0

My consumers.py looks like this:

consumers.py:
@channel_session
def ws_connect(message):
    room = message.content['path'].strip("/")
    message.channel_session['room'] = room
    Group("chat").add(message.reply_channel)
    message.reply_channel.send({"accept": True})

And the frontend part :

 $(function() {
        // When we're using HTTPS, use WSS too.
        var ws_scheme = window.location.protocol = "ws";
        var chatsock = new WebSocket(ws_scheme + '://' + window.location.host + window.location.pathname);

        chatsock.onmessage = function(message) {
            var data = JSON.parse(message.data);
            var chat = $("#chat");
            var ele = $('

');
            console.log(data);

            ele.append(
                $("	").text(data.timestamp)
            );
            ele.append(
                $("	").text(data.handle)
            );
            ele.append(
                $("	").text(data.message)
            );

            chat.append(ele)
        };

        $("#chatform").on("submit", function(event) {
            var time = new Date();
            var string = time.getHours() + ":" + time.getMinutes() + ":" + time.getSeconds();
            // var timestamp = time.getHourMinuteSecond();
            var message = {
                timestamp: string,
                handle: $('#handle').val(),
                message: $('#message').val()
            };
            console.log("submit");
            chatsock.send(JSON.stringify(message));
            $("#message").val('').focus();
            return false;
        });
    });

Maybe an update in some technology is out. I am struggling to find out why this happens. In settings.py I have the following configuration for the redis channel layer:

CHANNEL_LAYERS = {
    "default": {
        "BACKEND": "asgi_redis.RedisChannelLayer",
        "CONFIG": {
            "hosts": [("localhost", 6379)],
        },
        "ROUTING": "config.routing.channel_routing",
    },
}

I have another websocket logic for notifications which is also disconnecting right after handshake. I tried updating Django to 1.11. but no luck. In routing.py of the chat app:

chat_routing = [
    route("websocket.connect", consumers.ws_connect),
    route("websocket.receive", consumers.ws_message),
    route("websocket.disconnect", consumers.ws_disconnect),
]

I am running on Linux Ubuntu 16.04 and the only thing I updated in the last few days is Chrome version. So any ideas what should I do?

Body:
The error can be fixed if you add a slash +'/' to the end of the URL:

var chatsock = new WebSocket(ws_scheme + '://' + window.location.host + window.location.pathname + '/');

In most cases / missing at the end causes problems in Chrome browser, but works fine in Firefox


问:
Python if...in.. statement. Check for vowel in word
I'm trying to check if the user input contains a vowel or not. However, I've only found how to check for one vowel at a time, but  not all.

```<code>vowel = ("a")

word = input("type a word: ")

if vowel in word:
 print (f"There is the vowel {vowel} in your word")
else:
 print ("There is no vowel in your word")
</code>```

This seems to work but I get a error if I try to make the vowel variable into a list. ["a","e","i","o","u"]

any ideas how to check for e i o u at the same time?

答:
you have to iterate over the list.

```<code>vowels =  ["a","e","i","o","u"]

word = input("type a word: ")

for vowel in vowels:
  if vowel in word:
     print (f"There is the vowel {vowel} in your word")
  else:
     print ("There is no vowel in your word")
</code>```

iteration is the proces where you go through each item in a list.

for example.

```<code>list_a = ['a', 'b', 'c' ]

for item in list_a:
  print(item)

#output will be a b c
</code>```

since other user complained in a comment. If you want to stop the loop after vowel found, you should add break statment

```<code>vowels =  ["a","e","i","o","u"]

word = input("type a word: ")

for vowel in vowels:
  if vowel in word:
     print (f"There is the vowel {vowel} in your word")
     break
  else:
     print ("There is no vowel in your word")
</code>```

Body:
I'm trying to check if the user input contains a vowel or not. However, I've only found how to check for one vowel at a time, but  not all.

vowel = ("a")

word = input("type a word: ")

if vowel in word:
 print (f"There is the vowel {vowel} in your word")
else:
 print ("There is no vowel in your word")

This seems to work but I get a error if I try to make the vowel variable into a list. ["a","e","i","o","u"]

any ideas how to check for e i o u at the same time?

Body:
you have to iterate over the list.

vowels =  ["a","e","i","o","u"]

word = input("type a word: ")

for vowel in vowels:
  if vowel in word:
     print (f"There is the vowel {vowel} in your word")
  else:
     print ("There is no vowel in your word")

iteration is the proces where you go through each item in a list.

for example.

list_a = ['a', 'b', 'c' ]

for item in list_a:
  print(item)

#output will be a b c

since other user complained in a comment. If you want to stop the loop after vowel found, you should add break statment

vowels =  ["a","e","i","o","u"]

word = input("type a word: ")

for vowel in vowels:
  if vowel in word:
     print (f"There is the vowel {vowel} in your word")
     break
  else:
     print ("There is no vowel in your word")


问:
MySQL High CPU usage and persistent links
I am having very high CPU spikes on mysqld process (greater than 100%, and even saw a 300% at one point).  My load average is around: .25, .34, .28.

I read this great post about this issue: MySQL high CPU usage

One of the main things to do is disable persistent connections.  So I checked my php.ini and mysql.allow_persistent = on and mysql.max_persistent = -1 -- which means no limit.

This raises a few questions for me before changing anything just to be sure:

If my mysqld process is spiking over 100% every couple seconds shouldn't my load average be higher then they are?

What will disabling persistent links do - will my scripts continue to function as is?

If I turn this off and reload php what does this mean for my current users as there will be many active users.

EDIT:

CPU Info: Core2Quad q9400 2.6 Ghz

答:
Persistent connections won't use any CPU by themselves - if nothing's using a connection, it's just sitting idle and only consumes a bit of memory and occupies a socket.

Load averages are just that - averages. If you have a process that alternates between 0% and 100% 10 times a second, you'd get a load average of 0.5. They're good for figuring out long-term persistent high cpu, but by their nature hide/obliterate signs of spikes.

Persistent connections with mysql are usually not needed. MySQL has a relatively fast connection protocol and any time savings from using persistent connections is fairly minimal. The downside is that once a connection goes persistent, it can be left in an inconsistent state. e.g. If an app using the connection dies unexpectedly, MySQL will not see that and start cleaning up. This means that any server-side variables created by the app, any locks, any transactions, etc... will be left at the state they were in when the app crashed.

When the connection gets re-used by another app, you'll start out with what amounts to dirty dishes in the sink and an unflushed toilet. It can quite easily cause deadlocks because of the dangling transactions/locks - the new app won't know about them, and the old app is no longer around to relinquish those.


问:
How to change the float value in a script?
Hello im trying to figure out how to change this float value within a script to trigger a animation any tips on how to?

答:
```<code>void Start()
{
    m_Animator = gameObject.GetComponent<Animator>();
}

void YourFunction()
{
    m_Animator.SetFloat("Speed", YourValueAsAVariable);
}
</code>```


问:
Npx hardhat could not determine executable to run
I am trying to follow this guide and I am struggling running hardhat. After following the commands in the article:

```<code>mkdir hardhat-tutorial
cd hardhat-tutorial
npm init --yes
npm install --save-dev hardhat
npx hardhat
</code>```

I get an error stating npm ERR!  could not determine executable to run. The hardhat tool can be run by navigating to the npm_modules/hardhat directory however this seems to be interfering with the commands later in the tutorial due to the tool being in a different directory to the files it is being used on.

I'm not sure if I have fundamentally misunderstood how hardhat is supposed to work. Are you supposed to install it in each project and have your project directory be within npm_modules/hardhat or have I messed up the install/configuartion?

There appears to have already been a question covering this issue but sadly it was deleted and I have not managed to find another solution.

答:
To address all issues (including breaking changes), run:

```<code>npm audit fix --force
</code>```

Run npm audit for details.

```<code>dee-MacBook-Pro-2 polygon-hardhat % npm audit fix --force
npm WARN using --force Recommended protections disabled.
npm WARN audit Updating hardhat to 0.0.7,which is a SemVer major change.
npm WARN deprecated mkdirp@0.3.5: Legacy versions of mkdirp are no longer supported. Please update to mkdirp 1.x. (Note that the API surface has changed to use Promises in 1.x.)

added 4 packages, removed 333 packages, changed 5 packages, and audited 10 packages in 4s

found 0 vulnerabilities
dee@Dee-MacBook-Pro-2 polygon-hardhat % npx hardhat
npm ERR! could not determine executable to run

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/dee/.npm/_logs/2022-02-13T16_54_02_999Z-debug-0.log
dee@Dee-MacBook-Pro-2 polygon-hardhat %
</code>```

Body:
I am trying to follow this guide and I am struggling running hardhat. After following the commands in the article:

mkdir hardhat-tutorial
cd hardhat-tutorial
npm init --yes
npm install --save-dev hardhat
npx hardhat

I get an error stating npm ERR!  could not determine executable to run. The hardhat tool can be run by navigating to the npm_modules/hardhat directory however this seems to be interfering with the commands later in the tutorial due to the tool being in a different directory to the files it is being used on.

I'm not sure if I have fundamentally misunderstood how hardhat is supposed to work. Are you supposed to install it in each project and have your project directory be within npm_modules/hardhat or have I messed up the install/configuartion?

There appears to have already been a question covering this issue but sadly it was deleted and I have not managed to find another solution.

Body:
To address all issues (including breaking changes), run:

npm audit fix --force

Run npm audit for details.

dee-MacBook-Pro-2 polygon-hardhat % npm audit fix --force
npm WARN using --force Recommended protections disabled.
npm WARN audit Updating hardhat to 0.0.7,which is a SemVer major change.
npm WARN deprecated mkdirp@0.3.5: Legacy versions of mkdirp are no longer supported. Please update to mkdirp 1.x. (Note that the API surface has changed to use Promises in 1.x.)

added 4 packages, removed 333 packages, changed 5 packages, and audited 10 packages in 4s

found 0 vulnerabilities
dee@Dee-MacBook-Pro-2 polygon-hardhat % npx hardhat
npm ERR! could not determine executable to run

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/dee/.npm/_logs/2022-02-13T16_54_02_999Z-debug-0.log
dee@Dee-MacBook-Pro-2 polygon-hardhat %


问:
Python convert child dicts with same key name to csv (DictWriter)
I have a list of json file like below:

```<code>[
{"A":{"value":1}, "B":{"value":2}},
{"A":{"value":9}, "B":{"value":3}}
]
</code>```

Which I want to turn to csv like so:

```<code>A.value,B.value
1,2
9,3
</code>```

The issue is that I have nested keys which have the same name : value but should be in a separate column. I could not find an elegant solution to this anywhere yet. I would like to be able to do something like:

```<code>data = json.load(open(file, 'r'))
with open("output.csv", "w") as f:

        columns = ["A.value","B.value"]
        cw = csv.DictWriter(f, columns)
        cw.writeheader()
        cw.writerows(data)
</code>```

Which I know would work if I did not have any nested keys. I found other questions similar to this but I don't think this applies to my situation.

As an extra challenge:

I'd rather keep a generic approach. Later I might have a list of jsons like:

```<code>  [
    {"A":{"value":1}, "B":{"value":2}, "key":90},
    {"A":{"value":9}, "B":{"value":3}, "key":91}
    ]
</code>```

Meaning not all keys I want to add to csv will have a nested value key!

**output ^ **

```<code> A.value,B.value,key
    1,2,90
    9,3,91
</code>```

答:
Flattening the dicts worked. Since there is a list of dicts, the flattening has to be done for each dict:

```<code>import collections

def flatten(d, parent_key='', sep='.'):
    items = []
    for k, v in d.items():
        new_key = parent_key + sep + k if parent_key else k
        if isinstance(v, collections.MutableMapping):
            items.extend(flatten(v, new_key, sep=sep).items())
        else:
            items.append((new_key, v))
    return dict(items)

def flatten_json_list(list):
    flattened_list = []
    for d in list:
        flattened_list.append(flatten(d))

    return flattened_list
</code>```

Then the code should work as implemented in the question:

```<code>with open("out.csv","w") as f:

     columns = ["A.value","B.value","key"]
     cw = csv.DictWriter(f, columns)
     cw.writeheader()
     cw.writerows(data)
</code>```

Body:
I have a list of json file like below:

[
{"A":{"value":1}, "B":{"value":2}},
{"A":{"value":9}, "B":{"value":3}}
]

Which I want to turn to csv like so:

A.value,B.value
1,2
9,3

The issue is that I have nested keys which have the same name : value but should be in a separate column. I could not find an elegant solution to this anywhere yet. I would like to be able to do something like:

data = json.load(open(file, 'r'))
with open("output.csv", "w") as f:

        columns = ["A.value","B.value"]
        cw = csv.DictWriter(f, columns)
        cw.writeheader()
        cw.writerows(data)

Which I know would work if I did not have any nested keys. I found other questions similar to this but I don't think this applies to my situation.

As an extra challenge:

I'd rather keep a generic approach. Later I might have a list of jsons like:

  [
    {"A":{"value":1}, "B":{"value":2}, "key":90},
    {"A":{"value":9}, "B":{"value":3}, "key":91}
    ]

Meaning not all keys I want to add to csv will have a nested value key!

**output ^ **

 A.value,B.value,key
    1,2,90
    9,3,91

Body:
Flattening the dicts worked. Since there is a list of dicts, the flattening has to be done for each dict:

import collections

def flatten(d, parent_key='', sep='.'):
    items = []
    for k, v in d.items():
        new_key = parent_key + sep + k if parent_key else k
        if isinstance(v, collections.MutableMapping):
            items.extend(flatten(v, new_key, sep=sep).items())
        else:
            items.append((new_key, v))
    return dict(items)

def flatten_json_list(list):
    flattened_list = []
    for d in list:
        flattened_list.append(flatten(d))

    return flattened_list

Then the code should work as implemented in the question:

with open("out.csv","w") as f:

     columns = ["A.value","B.value","key"]
     cw = csv.DictWriter(f, columns)
     cw.writeheader()
     cw.writerows(data)


答:
You should use the method Animator.SetFloat

Here is an example from the mentioned link

```<code>Animator m_Animator;
float m_HorizontalMovement;

void Start()
{
    //Get the animator, which you attach to the GameObject you are intending to animate.
    m_Animator = gameObject.GetComponent<Animator>();
}

void Update()
{
    //Translate the left and right button presses or the horizontal joystick movements to a float
    m_HorizontalMovement = Input.GetAxis("Horizontal");
    //Sends the value from the horizontal axis input to the animator. Change the settings in the
    //Animator to define when the character is walking or running
    m_Animator.SetFloat("horizontalSpeed", m_HorizontalMovement);
}
</code>```

Title: How to change the float value in a script?

Body: Hello im trying to figure out how to change this float value within a script to trigger a animation any tips on how to?

Body:
You should use the method Animator.SetFloat

Here is an example from the mentioned link

Animator m_Animator;
float m_HorizontalMovement;

void Start()
{
    //Get the animator, which you attach to the GameObject you are intending to animate.
    m_Animator = gameObject.GetComponent<Animator>();
}

void Update()
{
    //Translate the left and right button presses or the horizontal joystick movements to a float
    m_HorizontalMovement = Input.GetAxis("Horizontal");
    //Sends the value from the horizontal axis input to the animator. Change the settings in the
    //Animator to define when the character is walking or running
    m_Animator.SetFloat("horizontalSpeed", m_HorizontalMovement);
}


问:
Ghidra decompile windows is greyed backgound
For some methods, Ghidra's decompiler background window is greyed out and I can't rename the function nor the local variables.
Why?

It works fine for methods with a "white background".

Example:

Matching code

```<code>        004d49dd cc              ??         CCh
        004d49de cc              ??         CCh
        004d49df cc              ??         CCh
                             LAB_004d49e0                                    XREF[1]:     FUN_004d4ac0:004d4b0e(*)
  ==>   004d49e0 64 8b 0d        MOV        ECX,dword ptr FS:[0x2c]
                 2c 00 00 00
        004d49e7 a1 bc 39        MOV        EAX,[DAT_00d439bc]                               = ??
                 d4 00
        004d49ec 8b 14 81        MOV        EDX,dword ptr [ECX + EAX*0x4]
        004d49ef 8b 92 08        MOV        EDX,dword ptr [EDX + 0x8]
                 00 00 00
</code>```

答: You can only do rename in a fully defined function. The grey background means that Ghidra didn't properly create a function at this point. You can see this also in a disassembly where you only have a label at this location. If you think this is a function you can type F and define a function. It should enable all the edit options.

Body:
For some methods, Ghidra's decompiler background window is greyed out and I can't rename the function nor the local variables.
Why?

It works fine for methods with a "white background".

Example:

Matching code

        004d49dd cc              ??         CCh
        004d49de cc              ??         CCh
        004d49df cc              ??         CCh
                             LAB_004d49e0                                    XREF[1]:     FUN_004d4ac0:004d4b0e(*)
  ==>   004d49e0 64 8b 0d        MOV        ECX,dword ptr FS:[0x2c]
                 2c 00 00 00
        004d49e7 a1 bc 39        MOV        EAX,[DAT_00d439bc]                               = ??
                 d4 00
        004d49ec 8b 14 81        MOV        EDX,dword ptr [ECX + EAX*0x4]
        004d49ef 8b 92 08        MOV        EDX,dword ptr [EDX + 0x8]
                 00 00 00


问:
"await" in Widget build FLUTTER
I've been stuck for several hours with a problem on flutter. If you can help me that would be really nice.

I need to put "await" in my Widget build(BuildContext context){} but it's impossible to put "async".
How to do ?

When i test void _myAsyncMethod()async{} :

答:
To Fix your issue you can put async in the body of method like this

```<code>Before=> Widget build(BuildContext context) {

After=> Widget build(BuildContext context) async{
</code>```

Although this will not solve your problem as flutter wiill warn you as this is not the proper way to do it.

It's not a good practice to call await inside flutter's build method Because

Generally an apps need to run a 60 frames per second on an average hence flutter's build method we'll be called over and over to re-render the ui.

Another reason is that, doing calling await function() in build method will block your UI.

Solution

use FutureBuilder

call await auth.currentUser() in initState method

Another way to solve this is to use FutureBuilder

sample Code for 1

```<code>FutureBuilder(
      builder: (BuildContext ctx, AsyncSnapshot<userModel> snapshot) {
        if(ConnectionState.done == snapshot.connectionState) {
          return Text(snapshot.data.userId);
        } else {
          return CircularProgressIndicator();
        }
      },
      future: auth.currentUser(),
    );
</code>```

sample Code for 2(stateful widget)

```<code>late UserModel;
void initState() {
 UserModel user = await auth.currentUser();
}
</code>```

this is very basic code but it's enough for you to get started.

Note: I've assumed userModel mentioned above is response type of auth.currentUser() you can change it accordingly.

Title: "await" in Widget build FLUTTER

Body:
To Fix your issue you can put async in the body of method like this

Before=> Widget build(BuildContext context) {

After=> Widget build(BuildContext context) async{

Although this will not solve your problem as flutter wiill warn you as this is not the proper way to do it.

It's not a good practice to call await inside flutter's build method Because

Generally an apps need to run a 60 frames per second on an average hence flutter's build method we'll be called over and over to re-render the ui.

Another reason is that, doing calling await function() in build method will block your UI.

Solution

use FutureBuilder

call await auth.currentUser() in initState method

Another way to solve this is to use FutureBuilder

sample Code for 1

FutureBuilder(
      builder: (BuildContext ctx, AsyncSnapshot<userModel> snapshot) {
        if(ConnectionState.done == snapshot.connectionState) {
          return Text(snapshot.data.userId);
        } else {
          return CircularProgressIndicator();
        }
      },
      future: auth.currentUser(),
    );

sample Code for 2(stateful widget)

late UserModel;
void initState() {
 UserModel user = await auth.currentUser();
}

this is very basic code but it's enough for you to get started.

Note: I've assumed userModel mentioned above is response type of auth.currentUser() you can change it accordingly.


问:
What does an overflow mean?
While I debug my flutter app, at certain points I see warnings like this:

Bottom overflow by 1234 pixel

BoxConstraints have a negative value

I suppose, my concept of Flutter viewport / layout isn't correct:

Currently, I image an infinite drawing area, where I declaratively
define widgets.

Widgets, which are too large, get drawn outside the
viewport [or clipped].

Could you please refine my probably wrong concept?

答:
This happens when the size [height, width] exceeds the the total size of the screen.
For example : the aspect ratio height of mobile screen is 500 and the size.height of button is 600, then the Bottom overflow by 100 pixel.

Title: What does an overflow mean?


问:
Divide text in different groups using regex
I want to extract some data using regex. I'm currently using python and the data can be like this:

```<code>text [text1 & text2]
text [text1]
text [text2]
text
</code>```

There is a "text" value (can be one, or more words) and can be followed by [] or no. In case [] exists, the text inside can be two words separated by & or only one of them (at the moment exists only two possible words but I think is better to get the words by separator instead of matching the exact word)

So my goal is to get the data in two different groups:

Group1: text

Group2: [text1, text2] (if exists)

My regex is:

```<code>(^([^\[])+)(.+?(?=&|\]))?(.+?(?=\]))?
</code>```

I've tried this using findall which I think fits better with the result I want:

regex = "(^([^\[])+)(.+?(?=&|\]))?(.+?(?=\]))?"
result = re.findall(regex,text)
```
<p>But it returns:</p>
```[('text ', ' ', '[text1 ', '& text2')]
```
<p>And I want, something like:</p>
<pre class="lang-py prettyprint-override"><code>[('text'), ('text1', 'text2')]
</code>```

I've tried in this playground

How can I improve my regex pattern?

Thanks in advance.

答:
Try this regex:

```<code>(?x)            # Extended (whitespace and comments are ignored)
([^[]+)         # Group 1: 1 or more non-'[' characters
(?:             # Start of a non-capturing group 1
   \s*          # Match optional whitespace
   \[           # Match '['
   ([^&\]]+)    # Group 2: 1 or more non- '&' or ']' characters
   (?:          # Start of a non-capturing group 2
      \s*       # Match optional whitespace
      &         # Match '&'
      \s*       # Match optional whitespace
      ([^\]]+)  # Capture group 3: 1 or more non-']' characters
      \s*       # Match optional whitespace
   )*           # End of optional non-capturing group 2
   ]            # Match ']'
)*              # End of optional non-capturing group 1
</code>```

See Regex Demo

import re

tests = [
    'text [text1 & text2]',
    'text [text1]',
    'text [text2]',
    'text'
]

regex = r"""(?x)# Extended (whitespace and comments are ignored)
([^[]+)         # Group 1: 1 or more non-'[' characters
(?:             # Start of a non-capturing group 1
   \s*          # Match optional whitespace
   \[           # Match '['
   ([^&\]]+)    # Group 2: 1 or more non- '&' or ']' characters
   (?:          # Start of a non-capturing group 2
      \s*       # Match optional whitespace
      &         # Match '&'
      \s*       # Match optional whitespace
      ([^\]]+)  # Capture group 3: 1 or more non-']' characters
      \s*       # Match optional whitespace
   )*           # End of optional non-capturing group 2
   ]            # Match ']'
)*              # End of optional non-capturing group 1
"""
for test in tests:
    # Here I am doing a fullmatch, but you can do a search to be more lenient:
    m = re.fullmatch(regex, test)
    if m:
        results = m[1], m[2], m[3]
        print(test, '->', results)
```
<p>Prints:</p>
<pre class="lang-None prettyprint-override"><code>text [text1 & text2] -> ('text ', 'text1 ', 'text2')
text [text1] -> ('text ', 'text1', None)
text [text2] -> ('text ', 'text2', None)
text -> ('text', None, None)
</code>```

You can, of course, group the results any way you want, such as:

results = (m[1], (m[2] or '' , m[3] or ''))
```
<p>Prints:</p>
<pre class="lang-None prettyprint-override"><code>text [text1 & text2] -> ('text ', ('text1 ', 'text2'))
text [text1] -> ('text ', ('text1', ''))
text [text2] -> ('text ', ('text2', ''))
text -> ('text', ('', ''))
</code>```

Update

This regex will do a better job of automatically trimming whitespace if you know there is always whitespace before the '[' and '&' characters:

```<code>(?x)            # Extended (whitespace and comments are ignored)
([^[]+)         # Group 1: 1 or more non-'[' characters
(?:             # Start of a non-capturing group 1
   \s+          # Match whitespace
   \[           # Match '['
   ([^&\]]+)    # Group 2: 1 or more non- '&' or ']' characters
   (?:          # Start of a non-capturing group 2
      \s+       # Match whitespace
      &         # Match '&'
      \s*       # Match optional whitespace
      ([^\]]+)  # Capture group 3: 1 or more non-']' characters
      \s*       # Match optional whitespace
   )*           # End of optional non-capturing group 2
   ]            # Match ']'
)*              # End of optional non-capturing group 1
</code>```

Code:

import re

tests = [
    'text [text1 & text2]',
    'text [text1]',
    'text [text2]',
    'text'
]

regex = r"""(?x)# Extended (whitespace and comments are ignored)
([^[]+)         # Group 1: 1 or more non-'[' characters
(?:             # Start of a non-capturing group 1
   \s+          # Match whitespace
   \[           # Match '['
   ([^&\]]+)    # Group 2: 1 or more non- '&' or ']' characters
   (?:          # Start of a non-capturing group 2
      \s+       # Match whitespace
      &         # Match '&'
      \s*       # Match optional whitespace
      ([^\]]+)  # Capture group 3: 1 or more non-']' characters
      \s*       # Match optional whitespace
   )*           # End of optional non-capturing group 2
   ]            # Match ']'
)*              # End of optional non-capturing group 1
"""
for test in tests:
    # Here I am doing a fullmatch, but you can do a search to be more lenient:
    m = re.fullmatch(regex, test)
    if m:
        results = (m[1], (m[2] or '' , m[3] or ''))
        print(test, '->', results)
```
<p>Prints:</p>
<pre class="lang-None prettyprint-override"><code>text [text1] -> ('text', ('text1', ''))
text [text2] -> ('text', ('text2', ''))
text -> ('text', ('', ''))
</code>```

Body:
I want to extract some data using regex. I'm currently using python and the data can be like this:

text [text1 & text2]
text [text1]
text [text2]
text

There is a "text" value (can be one, or more words) and can be followed by [] or no. In case [] exists, the text inside can be two words separated by & or only one of them (at the moment exists only two possible words but I think is better to get the words by separator instead of matching the exact word)

So my goal is to get the data in two different groups:

Group1: text

Group2: [text1, text2] (if exists)

My regex is:

(^([^\[])+)(.+?(?=&|\]))?(.+?(?=\]))?

I've tried this using findall which I think fits better with the result I want:

regex = "(^([^\[])+)(.+?(?=&|\]))?(.+?(?=\]))?"
result = re.findall(regex,text)

But it returns:

[('text ', ' ', '[text1 ', '& text2')]

And I want, something like:

[('text'), ('text1', 'text2')]

I've tried in this playground

How can I improve my regex pattern?

Thanks in advance.

Body:
Try this regex:

(?x)            # Extended (whitespace and comments are ignored)
([^[]+)         # Group 1: 1 or more non-'[' characters
(?:             # Start of a non-capturing group 1
   \s*          # Match optional whitespace
   \[           # Match '['
   ([^&\]]+)    # Group 2: 1 or more non- '&' or ']' characters
   (?:          # Start of a non-capturing group 2
      \s*       # Match optional whitespace
      &         # Match '&'
      \s*       # Match optional whitespace
      ([^\]]+)  # Capture group 3: 1 or more non-']' characters
      \s*       # Match optional whitespace
   )*           # End of optional non-capturing group 2
   ]            # Match ']'
)*              # End of optional non-capturing group 1

See Regex Demo

import re

tests = [
    'text [text1 & text2]',
    'text [text1]',
    'text [text2]',
    'text'
]

regex = r"""(?x)# Extended (whitespace and comments are ignored)
([^[]+)         # Group 1: 1 or more non-'[' characters
(?:             # Start of a non-capturing group 1
   \s*          # Match optional whitespace
   \[           # Match '['
   ([^&\]]+)    # Group 2: 1 or more non- '&' or ']' characters
   (?:          # Start of a non-capturing group 2
      \s*       # Match optional whitespace
      &         # Match '&'
      \s*       # Match optional whitespace
      ([^\]]+)  # Capture group 3: 1 or more non-']' characters
      \s*       # Match optional whitespace
   )*           # End of optional non-capturing group 2
   ]            # Match ']'
)*              # End of optional non-capturing group 1
"""
for test in tests:
    # Here I am doing a fullmatch, but you can do a search to be more lenient:
    m = re.fullmatch(regex, test)
    if m:
        results = m[1], m[2], m[3]
        print(test, '->', results)

Prints:

text [text1 & text2] -> ('text ', 'text1 ', 'text2')
text [text1] -> ('text ', 'text1', None)
text [text2] -> ('text ', 'text2', None)
text -> ('text', None, None)

You can, of course, group the results any way you want, such as:

results = (m[1], (m[2] or '' , m[3] or ''))

Prints:

text [text1 & text2] -> ('text ', ('text1 ', 'text2'))
text [text1] -> ('text ', ('text1', ''))
text [text2] -> ('text ', ('text2', ''))
text -> ('text', ('', ''))

Update

This regex will do a better job of automatically trimming whitespace if you know there is always whitespace before the '[' and '&' characters:

(?x)            # Extended (whitespace and comments are ignored)
([^[]+)         # Group 1: 1 or more non-'[' characters
(?:             # Start of a non-capturing group 1
   \s+          # Match whitespace
   \[           # Match '['
   ([^&\]]+)    # Group 2: 1 or more non- '&' or ']' characters
   (?:          # Start of a non-capturing group 2
      \s+       # Match whitespace
      &         # Match '&'
      \s*       # Match optional whitespace
      ([^\]]+)  # Capture group 3: 1 or more non-']' characters
      \s*       # Match optional whitespace
   )*           # End of optional non-capturing group 2
   ]            # Match ']'
)*              # End of optional non-capturing group 1

Code:

import re

tests = [
    'text [text1 & text2]',
    'text [text1]',
    'text [text2]',
    'text'
]

regex = r"""(?x)# Extended (whitespace and comments are ignored)
([^[]+)         # Group 1: 1 or more non-'[' characters
(?:             # Start of a non-capturing group 1
   \s+          # Match whitespace
   \[           # Match '['
   ([^&\]]+)    # Group 2: 1 or more non- '&' or ']' characters
   (?:          # Start of a non-capturing group 2
      \s+       # Match whitespace
      &         # Match '&'
      \s*       # Match optional whitespace
      ([^\]]+)  # Capture group 3: 1 or more non-']' characters
      \s*       # Match optional whitespace
   )*           # End of optional non-capturing group 2
   ]            # Match ']'
)*              # End of optional non-capturing group 1
"""
for test in tests:
    # Here I am doing a fullmatch, but you can do a search to be more lenient:
    m = re.fullmatch(regex, test)
    if m:
        results = (m[1], (m[2] or '' , m[3] or ''))
        print(test, '->', results)

Prints:

text [text1] -> ('text', ('text1', ''))
text [text2] -> ('text', ('text2', ''))
text -> ('text', ('', ''))


问:
Run different validation depending on the HTTP Method of the request [Go Gin]
So I'm currently developing a simple API using Go and the Gin library and wanted to have some validation of the data that I receive from the client before further checking.
I have come across the usage of tags and this is a clear example of what I have so far:

```<code>type Model struct {
    ID           primitive.ObjectID `bson:"_id,omitempty"`
    Name         string             `json:"name" bson:"name" binding:"required"`
    ProductId    int                `json:"productId" bson:"productId" binding:"required,min=1"`
    Brand        string             `json:"brand" bson:"brand" binding:"required"`
    Model        string             `json:"model" bson:"model" binding:"required"`
    Weight       string             `json:"weight" bson:"weight"`
    Observations string             `json:"observations" bson:"observations"`
}
</code>```

Thanks to the binding tags I can check that these fields are included in the body when I get a POST request.
Now my question is... What happens if, for example, I don't want to require all of these fields for a PATCH request?
The user may want to update only the observations or only the Name without having to provide all of the rest of the things since they're already saved in the database.

What is the best approach here? Write a custom validator that uses the methods of the HTTP request? Create different structs to use in different HTTP methods?

Thank you so much for your help.

答: I think best pattern here is to decouple the HTTP interface from the underlying data model. So in this case, you should have separate structs for each route/method combination, and marshal those structs into a common model type after validation.</p>

Body:
So I'm currently developing a simple API using Go and the Gin library and wanted to have some validation of the data that I receive from the client before further checking.
I have come across the usage of tags and this is a clear example of what I have so far:

type Model struct {
    ID           primitive.ObjectID `bson:"_id,omitempty"`
    Name         string             `json:"name" bson:"name" binding:"required"`
    ProductId    int                `json:"productId" bson:"productId" binding:"required,min=1"`
    Brand        string             `json:"brand" bson:"brand" binding:"required"`
    Model        string             `json:"model" bson:"model" binding:"required"`
    Weight       string             `json:"weight" bson:"weight"`
    Observations string             `json:"observations" bson:"observations"`
}

Thanks to the binding tags I can check that these fields are included in the body when I get a POST request.
Now my question is... What happens if, for example, I don't want to require all of these fields for a PATCH request?
The user may want to update only the observations or only the Name without having to provide all of the rest of the things since they're already saved in the database.

What is the best approach here? Write a custom validator that uses the methods of the HTTP request? Create different structs to use in different HTTP methods?

Thank you so much for your help.


问:
'The node must be of type 'JsonObject'.'
I'm trying to parse this Json

With the code:

```<code> Dim streamData As Stream = Nothing
        Using http As New HttpClient
            Dim url As String = "https://api.hotbit.io/api/v1/market.deals?market=KIBA/USDT&limit=150&last_id=1521100930&api_key=44812d8f-66d3-01c0-94c3b29305040b03&sign=98EEC3D69D3F70F9BDFED901984B2AA4"

            Dim t As Task(Of Stream) = http.GetStreamAsync(url)
            streamData = t.Result
        End Using

        Dim jsonResponse As JsonNode = JsonNode.Parse(streamData)
        Dim result As JsonObject = jsonResponse("result").AsObject
        Dim c As String = String.Empty
        For Each kvp In result.AsEnumerable
            c &= kvp.Value("price").ToString & vbCr
        Next
        RichTextBox1.Text = c
    End Sub
</code>```

but I keep getting the error at debug

The node must be of type 'JsonObject'.'
on the line

```<code> Dim result As JsonObject = jsonResponse("result").AsObject
</code>```

How it comes it gives an error If I'm already trying to parse it as a Jsonobject?

Thanks

答:
Give a go at this..

Paste this into a new class file:

```<code>Namespace HotBit
    Partial Public Class TradeCollection
        <JsonProperty("error")>
        Public Property [Error] As Object
        <JsonProperty("result")>
        Public Property Result As List(Of Result)
        <JsonProperty("id")>
        Public Property Id As Long
    End Class

    Partial Public Class Result
        <JsonProperty("id")>
        Public Property Id As Long
        <JsonProperty("time")>
        Public Property Time As Long
        <JsonProperty("price")>
        Public Property Price As String
        <JsonProperty("amount")>
        <JsonConverter(GetType(ParseStringConverter))>
        Public Property Amount As Long
        <JsonProperty("type")>
        Public Property Type As TypeEnum
    End Class

    Public Enum TypeEnum
        Buy
        Sell
    End Enum

    Partial Public Class TradeCollection
        Public Shared Function FromJson(ByVal json As String) As TradeCollection
            Return JsonConvert.DeserializeObject(Of TradeCollection)(json, Settings)
        End Function
    End Class

    Public Module Serialize
        <Extension()>
        Public Function ToJson(ByVal self As TradeCollection) As String
            Return JsonConvert.SerializeObject(self, Settings)
        End Function
    End Module

    Friend Module Converter
        Public ReadOnly Settings As JsonSerializerSettings = New JsonSerializerSettings With {
            .MetadataPropertyHandling = MetadataPropertyHandling.Ignore,
            .DateParseHandling = DateParseHandling.None
        }
    End Module

    Friend Class ParseStringConverter
        Inherits JsonConverter

        Public Overrides Function CanConvert(ByVal t As Type) As Boolean
            Return t Is GetType(Long) OrElse t Is GetType(Long?)
        End Function

        Public Overrides Function ReadJson(ByVal reader As JsonReader, ByVal t As Type, ByVal existingValue As Object, ByVal serializer As JsonSerializer) As Object
            If reader.TokenType = JsonToken.Null Then Return Nothing
            Dim value = serializer.Deserialize(Of String)(reader)
            Dim l As Long

            If Long.TryParse(value, l) Then
                Return l
            End If

            Throw New Exception("Cannot unmarshal type long")
        End Function

        Public Overrides Sub WriteJson(ByVal writer As JsonWriter, ByVal untypedValue As Object, ByVal serializer As JsonSerializer)
            If untypedValue Is Nothing Then
                serializer.Serialize(writer, Nothing)
                Return
            End If

            Dim value = CLng(untypedValue)
            serializer.Serialize(writer, value.ToString())
            Return
        End Sub

        Public Shared ReadOnly Singleton As ParseStringConverter = New ParseStringConverter()
    End Class

    Friend Class TypeEnumConverter
        Inherits JsonConverter

        Public Overrides Function CanConvert(ByVal t As Type) As Boolean
            Return t Is GetType(TypeEnum) OrElse t Is GetType(TypeEnum?)
        End Function

        Public Overrides Function ReadJson(ByVal reader As JsonReader, ByVal t As Type, ByVal existingValue As Object, ByVal serializer As JsonSerializer) As Object
            If reader.TokenType = JsonToken.Null Then Return Nothing
            Dim value = serializer.Deserialize(Of String)(reader)

            Select Case value
                Case "buy"
                    Return TypeEnum.Buy
                Case "sell"
                    Return TypeEnum.Sell
            End Select

            Throw New Exception("Cannot unmarshal type TypeEnum")
        End Function

        Public Overrides Sub WriteJson(ByVal writer As JsonWriter, ByVal untypedValue As Object, ByVal serializer As JsonSerializer)
            If untypedValue Is Nothing Then
                serializer.Serialize(writer, Nothing)
                Return
            End If

            Dim value = CType(untypedValue, TypeEnum)

            Select Case value
                Case TypeEnum.Buy
                    serializer.Serialize(writer, "buy")
                    Return
                Case TypeEnum.Sell
                    serializer.Serialize(writer, "sell")
                    Return
            End Select

            Throw New Exception("Cannot marshal type TypeEnum")
        End Sub

        Public Shared ReadOnly Singleton As TypeEnumConverter = New TypeEnumConverter()
    End Class
End Namespace

</code>```

Then install Newtonsoft (if not already) and Imports it

Then Imports HotBit (or if you changed the Namespace of the "paste this" above, Imports that new namespace

Then do your request and query the result e.g.:

```<code>Sub Main(args As String())

        Dim s = New WebClient().DownloadString("https://api.hotbit.io/api/v1/market.deals?market=KIBA/USDT&limit=150&last_id=1521100930&api_key=44812d8f-66d3-01c0-94c3b29305040b03&sign=98EEC3D69D3F70F9BDFED901984B2AA4")

        Dim tc = TradeCollection.FromJson(s)

        Dim prices = String.Join(","c, tc.Result.Select(Function(r) r.Price))

    End Sub
</code>```

Body:
I'm trying to parse this Json

With the code:

 Dim streamData As Stream = Nothing
        Using http As New HttpClient
            Dim url As String = "https://api.hotbit.io/api/v1/market.deals?market=KIBA/USDT&limit=150&last_id=1521100930&api_key=44812d8f-66d3-01c0-94c3b29305040b03&sign=98EEC3D69D3F70F9BDFED901984B2AA4"

            Dim t As Task(Of Stream) = http.GetStreamAsync(url)
            streamData = t.Result
        End Using

        Dim jsonResponse As JsonNode = JsonNode.Parse(streamData)
        Dim result As JsonObject = jsonResponse("result").AsObject
        Dim c As String = String.Empty
        For Each kvp In result.AsEnumerable
            c &= kvp.Value("price").ToString & vbCr
        Next
        RichTextBox1.Text = c
    End Sub

but I keep getting the error at debug

The node must be of type 'JsonObject'.'
on the line

 Dim result As JsonObject = jsonResponse("result").AsObject

How it comes it gives an error If I'm already trying to parse it as a Jsonobject?

Thanks

Body:
Give a go at this..

Paste this into a new class file:

Namespace HotBit
    Partial Public Class TradeCollection
        <JsonProperty("error")>
        Public Property [Error] As Object
        <JsonProperty("result")>
        Public Property Result As List(Of Result)
        <JsonProperty("id")>
        Public Property Id As Long
    End Class

    Partial Public Class Result
        <JsonProperty("id")>
        Public Property Id As Long
        <JsonProperty("time")>
        Public Property Time As Long
        <JsonProperty("price")>
        Public Property Price As String
        <JsonProperty("amount")>
        <JsonConverter(GetType(ParseStringConverter))>
        Public Property Amount As Long
        <JsonProperty("type")>
        Public Property Type As TypeEnum
    End Class

    Public Enum TypeEnum
        Buy
        Sell
    End Enum

    Partial Public Class TradeCollection
        Public Shared Function FromJson(ByVal json As String) As TradeCollection
            Return JsonConvert.DeserializeObject(Of TradeCollection)(json, Settings)
        End Function
    End Class

    Public Module Serialize
        <Extension()>
        Public Function ToJson(ByVal self As TradeCollection) As String
            Return JsonConvert.SerializeObject(self, Settings)
        End Function
    End Module

    Friend Module Converter
        Public ReadOnly Settings As JsonSerializerSettings = New JsonSerializerSettings With {
            .MetadataPropertyHandling = MetadataPropertyHandling.Ignore,
            .DateParseHandling = DateParseHandling.None
        }
    End Module

    Friend Class ParseStringConverter
        Inherits JsonConverter

        Public Overrides Function CanConvert(ByVal t As Type) As Boolean
            Return t Is GetType(Long) OrElse t Is GetType(Long?)
        End Function

        Public Overrides Function ReadJson(ByVal reader As JsonReader, ByVal t As Type, ByVal existingValue As Object, ByVal serializer As JsonSerializer) As Object
            If reader.TokenType = JsonToken.Null Then Return Nothing
            Dim value = serializer.Deserialize(Of String)(reader)
            Dim l As Long

            If Long.TryParse(value, l) Then
                Return l
            End If

            Throw New Exception("Cannot unmarshal type long")
        End Function

        Public Overrides Sub WriteJson(ByVal writer As JsonWriter, ByVal untypedValue As Object, ByVal serializer As JsonSerializer)
            If untypedValue Is Nothing Then
                serializer.Serialize(writer, Nothing)
                Return
            End If

            Dim value = CLng(untypedValue)
            serializer.Serialize(writer, value.ToString())
            Return
        End Sub

        Public Shared ReadOnly Singleton As ParseStringConverter = New ParseStringConverter()
    End Class

    Friend Class TypeEnumConverter
        Inherits JsonConverter

        Public Overrides Function CanConvert(ByVal t As Type) As Boolean
            Return t Is GetType(TypeEnum) OrElse t Is GetType(TypeEnum?)
        End Function

        Public Overrides Function ReadJson(ByVal reader As JsonReader, ByVal t As Type, ByVal existingValue As Object, ByVal serializer As JsonSerializer) As Object
            If reader.TokenType = JsonToken.Null Then Return Nothing
            Dim value = serializer.Deserialize(Of String)(reader)

            Select Case value
                Case "buy"
                    Return TypeEnum.Buy
                Case "sell"
                    Return TypeEnum.Sell
            End Select

            Throw New Exception("Cannot unmarshal type TypeEnum")
        End Function

        Public Overrides Sub WriteJson(ByVal writer As JsonWriter, ByVal untypedValue As Object, ByVal serializer As JsonSerializer)
            If untypedValue Is Nothing Then
                serializer.Serialize(writer, Nothing)
                Return
            End If

            Dim value = CType(untypedValue, TypeEnum)

            Select Case value
                Case TypeEnum.Buy
                    serializer.Serialize(writer, "buy")
                    Return
                Case TypeEnum.Sell
                    serializer.Serialize(writer, "sell")
                    Return
            End Select

            Throw New Exception("Cannot marshal type TypeEnum")
        End Sub

        Public Shared ReadOnly Singleton As TypeEnumConverter = New TypeEnumConverter()
    End Class
End Namespace

Then install Newtonsoft (if not already) and Imports it

Then Imports HotBit (or if you changed the Namespace of the "paste this" above, Imports that new namespace

Then do your request and query the result e.g.:

Sub Main(args As String())

        Dim s = New WebClient().DownloadString("https://api.hotbit.io/api/v1/market.deals?market=KIBA/USDT&limit=150&last_id=1521100930&api_key=44812d8f-66d3-01c0-94c3b29305040b03&sign=98EEC3D69D3F70F9BDFED901984B2AA4")

        Dim tc = TradeCollection.FromJson(s)

        Dim prices = String.Join(","c, tc.Result.Select(Function(r) r.Price))

    End Sub


问:
My VBA code is extremely slow when running
Basically I want to loop through column H (with 430 rows) and if its value is "Tera" then I want to change the value of the cell in column E (i.e. 6th column) with the value in column F (7th column).

```<code>Sub R_Adj()
Sheets("Sheet5").Select
temp = 7
For Each i In Worksheets("Sheet5").Range("H8:H430").Cells
temp = temp + 1
Set column_to = Worksheets("Sheet5").Cells(temp, 6)
Set column_from = Worksheets("Sheet5").Cells(temp, 7)
If i.Value = "Tera" Then column_to.Value = column_from.Value
Next
End Sub
</code>```

The running time is about 10 seconds which is too much in my opinion for such an easy algorithm. Does anybody have any suggestions to improve the performance and running time of this code?

答:
Use an array.

Option Explicit
Sub RTB_justering()

    Dim rng As Range, ar, i As Long, n As Long
    With Sheets("Sheet5")
        Set rng = .Range("E8:H430")
        ar = rng.Value2
        For i = 1 To UBound(ar)
            If ar(i, 4) = "Tera" Then 'H
                ar(i, 1) = ar(i, 2) ' F->E
                n = n + 1
            End If
        Next
        rng.Value2 = ar
    End With
    MsgBox n & " cells changed", vbInformation

End Sub
```

Body:
Basically I want to loop through column H (with 430 rows) and if its value is "Tera" then I want to change the value of the cell in column E (i.e. 6th column) with the value in column F (7th column).

Sub R_Adj()
Sheets("Sheet5").Select
temp = 7
For Each i In Worksheets("Sheet5").Range("H8:H430").Cells
temp = temp + 1
Set column_to = Worksheets("Sheet5").Cells(temp, 6)
Set column_from = Worksheets("Sheet5").Cells(temp, 7)
If i.Value = "Tera" Then column_to.Value = column_from.Value
Next
End Sub

The running time is about 10 seconds which is too much in my opinion for such an easy algorithm. Does anybody have any suggestions to improve the performance and running time of this code?