问:
Vertex AI forecasting AutoML datatype mismatch
I could train the vertex AI AutoML forecating model but when I do batch prediction I get following error

Batch prediction job batch_prediction encountered the following
errors:

```<code>Column "sales" expects type: NUMBER, the actual type is: STRING.
</code>```

Below is a sample of test set I am passing for batch prediction in big query.

According to the documentation for batch prediction we have to send some training/historical data and forecasting dates. I did just that.

答:
Google recommend you to use the same input format for ingraining and prediction. Seams you have trained your model using a input format here the column sales were a numeric type, and now in the prediction you a using a BigQuery table with the sales column as string.

Delete this table and import the data again defining the schema manually, and set sales as a numeric field, as following:

```<code>date:DATE,
store_product_id:STRING,
sales:NUMERIC
</code>```

Body:
I could train the vertex AI AutoML forecating model but when I do batch prediction I get following error

Batch prediction job batch_prediction encountered the following
errors:

Column "sales" expects type: NUMBER, the actual type is: STRING.

Below is a sample of test set I am passing for batch prediction in big query.

According to the documentation for batch prediction we have to send some training/historical data and forecasting dates. I did just that.

Body:
Google recommend you to use the same input format for ingraining and prediction. Seams you have trained your model using a input format here the column sales were a numeric type, and now in the prediction you a using a BigQuery table with the sales column as string.

Delete this table and import the data again defining the schema manually, and set sales as a numeric field, as following:

date:DATE,
store_product_id:STRING,
sales:NUMERIC


来源: stackexchange

问:
C++ equivalent of Python properties
Is there an equivalent of Python properties in C++? Or would it just be better to do this using getters and setters?</p>

答: Yes, explicit getter and setters would be the closest construct in C++.</p>

ContentLicense: CC BY-SA 3.0

Title: C++ equivalent of Python properties


问:
Postgres run multithreaded call from function
I need to run one stored function from another in PostgreSQL server, but I need to run inner function in 4 threads to reduce in execution time.
I can do that by make 4 threaded calls from backend, but it will more complex than call inside PostgeSQL server.
I'm using plpgsql language.

I'd like to make something like following

```<code>create or replace function abc(some_data integer)
returns integer
language plpgsql
as $$
declare
  intermediate_data some_my_type;
  is_done boolean;
  result integer;
begin
  intermediate_data := result_of_synchronous_call(some_data);

  async perform asinchronous_call(intermediate_data.part1);
  async perform asinchronous_call(intermediate_data.part2);
  async perform asinchronous_call(intermediate_data.part3);
  async perform asinchronous_call(intermediate_data.part4);

  SELECT count(*) = 4 INTO is_done FROM results;
  while not is_done loop
    SELECT pg_sleep(10);
    SELECT count(*) = 4 INTO is_done FROM results;
  end loop;

  SELECT sum(result_field) INTO result FROM results;
  return result;
end;
$$
</code>```

答: There is no pretty way to do this. The best I can think of is to use the dblink extension and use dblink_send_query to send a query. Completion status can be checked with dblink_is_busy, and the results are ultimately collected with dblink_get_result.

ContentLicense: CC BY-SA 4.0

Body:
I need to run one stored function from another in PostgreSQL server, but I need to run inner function in 4 threads to reduce in execution time.
I can do that by make 4 threaded calls from backend, but it will more complex than call inside PostgeSQL server.
I'm using plpgsql language.

I'd like to make something like following

create or replace function abc(some_data integer)
returns integer
language plpgsql
as $$
declare
  intermediate_data some_my_type;
  is_done boolean;
  result integer;
begin
  intermediate_data := result_of_synchronous_call(some_data);

  async perform asinchronous_call(intermediate_data.part1);
  async perform asinchronous_call(intermediate_data.part2);
  async perform asinchronous_call(intermediate_data.part3);
  async perform asinchronous_call(intermediate_data.part4);

  SELECT count(*) = 4 INTO is_done FROM results;
  while not is_done loop
    SELECT pg_sleep(10);
    SELECT count(*) = 4 INTO is_done FROM results;
  end loop;

  SELECT sum(result_field) INTO result FROM results;
  return result;
end;
$$


问:
Hide and show the content of sibling elements on single click among them in ReactJS
Please click on the link https://cdn.dribbble.com/users/2079553/screenshots/6462890/accordion3.gif and see the GIF, which I am trying to create the same UI using ReactJs. Need to hide and show the contents of sibling p elements on basis of single click among them. Please find the code snippet below and result of the same on the following link https://codesandbox.io/s/vkij6. Please help me to create UI like the GIF. Thanks in Advance.

The code snippet:

App.js

```<code>import Toggle from './components/Toggle';

const cities = [
  {
    name: "Mysuru",
    about: "Mysore (or Mysuru), a city in India's southwestern Karnataka state, was the capital of the Kingdom of Mysore from 1399 to 1947. In its center is opulent Mysore Palace, seat of the former ruling Wodeyar dynasty."
  },
  {
    name: "Bengaluru",
    about: "Bengaluru (also called Bangalore) is the capital of India's southern Karnataka state. The center of India's high-tech industry, the city is also known for its parks and nightlife. By Cubbon Park, Vidhana Soudha is a Neo-Dravidian legislative building."
  },
  {
    name: "Udupi",
    about: "Udupi is a city in the southwest Indian state of Karnataka. It's known for its Hindu temples, including the huge, 13th-century Sri Krishna Temple, which houses a statue of Lord Krishna and attracts many pilgrims."
  },
  {
    name: "Mandya",
    about: "Mandya is a city in the state of Karnataka. It is the headquarter of Mandya district and is located 45 kilometres from Mysore and 100 kilometres from Bangalore"
  },
  {
    name: "Kodagu",
    about: "Kodagu, also known as Coorg, is a rural district in the southwest Indian state of Karnataka. In the area’s north, Madikeri Fort has 2 life-size elephant statues at its entrance, plus a Gothic-style church with a museum on its grounds."
  }
]

function App() {
  return (
    <div className="App">
      <div className="toggle-container">
        <div className="cities">
          {cities.map((city) => (
            <Toggle key={city.name.toString()} city={city} />
          ))}
        </div>
      </div>
    </div>
  );
}

export default App;
</code>```

Toggle.js

```<code>
export default function Toggle({ city }) {
    const [toggle, setToggle] = useState(false);

    const handleClick = () => {
        setToggle(!toggle)
    }

    return (
        <>
            <p className="city-name" onClick={handleClick}>{city.name}</p>
            <p className={toggle ? "city-about-show" : "city-about-hide"}>{city.about}</p>
        </>
    )
}
</code>```

style.css

```<code>  margin: 0;
  padding: 0;
  box-sizing:border-box
}

.App {
  /* text-align: center; */
  background-color: #000000;
  padding-top: 5px;
  padding-bottom: 5px;
  position: relative;
}

.toggle-container {
  background-color: hsla(245, 100%, 87%, 0.521);
  transition: background-color 300ms;
  width: 800px;
  height: 98.3vh;
  margin: auto;
  padding-top: 70px;
}

.cities {
  background-color: rgb(243, 244, 245);
  width: 450px;
  margin: auto;
  border-radius: 3px;
}

.city-name{
  font-family: sans-serif;
  color: rgb(105, 105, 105);
  padding: 15px 0 15px 15px;
  letter-spacing: 0.5px;
  cursor: pointer;
  border-bottom: 1.5px solid rgb(172, 169, 169);
}

.city-name-symbol {
  display: inline;
}

.city-about-hide{
  display: none;
}

.city-about-show {
  display: block;
  font-family: sans-serif;
  color: rgb(105, 105, 105);
  background-color: rgb(214, 231, 247);
  letter-spacing: 0.3px;
  font-size: 14px;
  padding: 20px 15px;
  line-height: 1.5;
}
</code>```

答:
You use an individual toggle state for each city. I suggest to use a common state object that holds a toggle state for each city and move this state object to a parent component to prevent passing around the props when you want to open a specific city and at the same time close others cities.

```<code>import { useState } from 'react';

const cities = [
  {
    name: "Mysuru",
    about: "Mysore (or Mysuru), a city in India's southwestern Karnataka state, was the capital of the Kingdom of Mysore from 1399 to 1947. In its center is opulent Mysore Palace, seat of the former ruling Wodeyar dynasty."
  },
  {
    name: "Bengaluru",
    about: "Bengaluru (also called Bangalore) is the capital of India's southern Karnataka state. The center of India's high-tech industry, the city is also known for its parks and nightlife. By Cubbon Park, Vidhana Soudha is a Neo-Dravidian legislative building."
  },
  {
    name: "Udupi",
    about: "Udupi is a city in the southwest Indian state of Karnataka. It's known for its Hindu temples, including the huge, 13th-century Sri Krishna Temple, which houses a statue of Lord Krishna and attracts many pilgrims."
  },
  {
    name: "Mandya",
    about: "Mandya is a city in the state of Karnataka. It is the headquarter of Mandya district and is located 45 kilometres from Mysore and 100 kilometres from Bangalore"
  },
  {
    name: "Kodagu",
    about: "Kodagu, also known as Coorg, is a rural district in the southwest Indian state of Karnataka. In the area’s north, Madikeri Fort has 2 life-size elephant statues at its entrance, plus a Gothic-style church with a museum on its grounds."
  }
]

function Toggle({ city, toggle, handleClick }) {
  return (
      <>
          <p className="city-name" onClick={() => handleClick(city.name)}>{city.name}</p>
          <p className={toggle[city.name] ? "city city-about-show" : "city city-about-hide"}>{city.about}</p>
      </>
  )
}

function App() {
  const [toggle, setToggle] = useState({
    Mysuru: false,
    Bengaluru: false,
    Udupi: false,
    Mandya: false,
    Kodagu: false,
  });

  const handleClick = (city) => {
      setToggle({
        Mysuru: false,
        Bengaluru: false,
        Udupi: false,
        Mandya: false,
        Kodagu: false,
        [city]: !toggle[city]
      })
  }

  return (
    <div className="App">
      <div className="toggle-container">
        <div className="cities">
          {cities.map((city) => (
            <Toggle key={city.name.toString()} city={city} toggle={toggle} handleClick={handleClick} />
          ))}
        </div>
      </div>
    </div>
  );
}
</code>```

CSS

```<code>.city {
  transition: height 1s;
  overflow: hidden;
}

.city-about-hide{
  height: 0;

}

.city-about-show {
  height: 100px;
  display: block;
  font-family: sans-serif;
  color: rgb(105, 105, 105);
  background-color: rgb(214, 231, 247);
  letter-spacing: 0.3px;
  font-size: 14px;
  padding: 20px 15px;
  line-height: 1.5;
}
</code>```

Body:
Please click on the link https://cdn.dribbble.com/users/2079553/screenshots/6462890/accordion3.gif and see the GIF, which I am trying to create the same UI using ReactJs. Need to hide and show the contents of sibling p elements on basis of single click among them. Please find the code snippet below and result of the same on the following link https://codesandbox.io/s/vkij6. Please help me to create UI like the GIF. Thanks in Advance.

The code snippet:

App.js

import Toggle from './components/Toggle';

const cities = [
  {
    name: "Mysuru",
    about: "Mysore (or Mysuru), a city in India's southwestern Karnataka state, was the capital of the Kingdom of Mysore from 1399 to 1947. In its center is opulent Mysore Palace, seat of the former ruling Wodeyar dynasty."
  },
  {
    name: "Bengaluru",
    about: "Bengaluru (also called Bangalore) is the capital of India's southern Karnataka state. The center of India's high-tech industry, the city is also known for its parks and nightlife. By Cubbon Park, Vidhana Soudha is a Neo-Dravidian legislative building."
  },
  {
    name: "Udupi",
    about: "Udupi is a city in the southwest Indian state of Karnataka. It's known for its Hindu temples, including the huge, 13th-century Sri Krishna Temple, which houses a statue of Lord Krishna and attracts many pilgrims."
  },
  {
    name: "Mandya",
    about: "Mandya is a city in the state of Karnataka. It is the headquarter of Mandya district and is located 45 kilometres from Mysore and 100 kilometres from Bangalore"
  },
  {
    name: "Kodagu",
    about: "Kodagu, also known as Coorg, is a rural district in the southwest Indian state of Karnataka. In the area’s north, Madikeri Fort has 2 life-size elephant statues at its entrance, plus a Gothic-style church with a museum on its grounds."
  }
]

function App() {
  return (

          {cities.map((city) => (
            <Toggle key={city.name.toString()} city={city} />
          ))}

  );
}

export default App;

Toggle.js

export default function Toggle({ city }) {
    const [toggle, setToggle] = useState(false);

    const handleClick = () => {
        setToggle(!toggle)
    }

    return (
        <>

{city.name}

{city.about}

        </>
    )
}

style.css

  margin: 0;
  padding: 0;
  box-sizing:border-box
}

.App {
  /* text-align: center; */
  background-color: #000000;
  padding-top: 5px;
  padding-bottom: 5px;
  position: relative;
}

.toggle-container {
  background-color: hsla(245, 100%, 87%, 0.521);
  transition: background-color 300ms;
  width: 800px;
  height: 98.3vh;
  margin: auto;
  padding-top: 70px;
}

.cities {
  background-color: rgb(243, 244, 245);
  width: 450px;
  margin: auto;
  border-radius: 3px;
}

.city-name{
  font-family: sans-serif;
  color: rgb(105, 105, 105);
  padding: 15px 0 15px 15px;
  letter-spacing: 0.5px;
  cursor: pointer;
  border-bottom: 1.5px solid rgb(172, 169, 169);
}

.city-name-symbol {
  display: inline;
}

.city-about-hide{
  display: none;
}

.city-about-show {
  display: block;
  font-family: sans-serif;
  color: rgb(105, 105, 105);
  background-color: rgb(214, 231, 247);
  letter-spacing: 0.3px;
  font-size: 14px;
  padding: 20px 15px;
  line-height: 1.5;
}

Body:
You use an individual toggle state for each city. I suggest to use a common state object that holds a toggle state for each city and move this state object to a parent component to prevent passing around the props when you want to open a specific city and at the same time close others cities.

import { useState } from 'react';

const cities = [
  {
    name: "Mysuru",
    about: "Mysore (or Mysuru), a city in India's southwestern Karnataka state, was the capital of the Kingdom of Mysore from 1399 to 1947. In its center is opulent Mysore Palace, seat of the former ruling Wodeyar dynasty."
  },
  {
    name: "Bengaluru",
    about: "Bengaluru (also called Bangalore) is the capital of India's southern Karnataka state. The center of India's high-tech industry, the city is also known for its parks and nightlife. By Cubbon Park, Vidhana Soudha is a Neo-Dravidian legislative building."
  },
  {
    name: "Udupi",
    about: "Udupi is a city in the southwest Indian state of Karnataka. It's known for its Hindu temples, including the huge, 13th-century Sri Krishna Temple, which houses a statue of Lord Krishna and attracts many pilgrims."
  },
  {
    name: "Mandya",
    about: "Mandya is a city in the state of Karnataka. It is the headquarter of Mandya district and is located 45 kilometres from Mysore and 100 kilometres from Bangalore"
  },
  {
    name: "Kodagu",
    about: "Kodagu, also known as Coorg, is a rural district in the southwest Indian state of Karnataka. In the area’s north, Madikeri Fort has 2 life-size elephant statues at its entrance, plus a Gothic-style church with a museum on its grounds."
  }
]

function Toggle({ city, toggle, handleClick }) {
  return (
      <>
          <p className="city-name" onClick={() => handleClick(city.name)}>{city.name}</p>
          <p className={toggle[city.name] ? "city city-about-show" : "city city-about-hide"}>{city.about}</p>
      </>
  )
}

function App() {
  const [toggle, setToggle] = useState({
    Mysuru: false,
    Bengaluru: false,
    Udupi: false,
    Mandya: false,
    Kodagu: false,
  });

  const handleClick = (city) => {
      setToggle({
        Mysuru: false,
        Bengaluru: false,
        Udupi: false,
        Mandya: false,
        Kodagu: false,
        [city]: !toggle[city]
      })
  }

  return (
    <div className="App">
      <div className="toggle-container">
        <div className="cities">
          {cities.map((city) => (
            <Toggle key={city.name.toString()} city={city} toggle={toggle} handleClick={handleClick} />
          ))}
        </div>
      </div>
    </div>
  );
}

CSS

.city {
  transition: height 1s;
  overflow: hidden;
}

.city-about-hide{
  height: 0;

}

.city-about-show {
  height: 100px;
  display: block;
  font-family: sans-serif;
  color: rgb(105, 105, 105);
  background-color: rgb(214, 231, 247);
  letter-spacing: 0.3px;
  font-size: 14px;
  padding: 20px 15px;
  line-height: 1.5;
}


问:
In Gradle, is there a better way to get Environment Variables?
In several Tasks, I reference jars in my home folder.

Is there a better way to get Environment Variables than

```<code>ENV = System.getenv()
HOME = ENV['HOME']

task copyToServer(dependsOn: 'jar', type: Copy) {

 from 'build/libs/'
 into HOME + "/something/plugins/"
}
</code>```

This sets $HOME but I was hoping that I missed some magic from the documentation.

答:
This is for Kotlin DSL (build.gradle.kts):

val myVariable = System.getenv("MY_VARIABLE_NAME") ?: "my default value"
```
<p>OR</p>
<pre class="lang-kotlin prettyprint-override"><code>val myVariable = System.getenv("MY_VARIABLE_NAME") ?: error("Env variable not found")
</code>```

OR

val environment = System.getenv()
val myVariable = environment["MY_VARIABLE_NAME"] ?: "my default value"
// OR val myVariable = environment["MY_VARIABLE_NAME"] ?: error("Env variable not found")
```

Body:
In several Tasks, I reference jars in my home folder.

Is there a better way to get Environment Variables than

ENV = System.getenv()
HOME = ENV['HOME']

task copyToServer(dependsOn: 'jar', type: Copy) {

 from 'build/libs/'
 into HOME + "/something/plugins/"
}

This sets $HOME but I was hoping that I missed some magic from the documentation.

Body:
This is for Kotlin DSL (build.gradle.kts):

val myVariable = System.getenv("MY_VARIABLE_NAME") ?: "my default value"

OR

val myVariable = System.getenv("MY_VARIABLE_NAME") ?: error("Env variable not found")

OR

val environment = System.getenv()
val myVariable = environment["MY_VARIABLE_NAME"] ?: "my default value"
// OR val myVariable = environment["MY_VARIABLE_NAME"] ?: error("Env variable not found")


问:
Save every single plot for every column from dataframe
I have a dataframe with 68 columns.

How can I make matplotlib scatter plots with every column and save every single plot?
Now I can only build plots one by one with changing the column's name in X value. (Y value is always the same, it is the target column)

For example, it is the code where I'm changing the X value, but I don't want to change it 68 times and save every plot one by one.

```<code>plt.figure(figsize=(9,5))
plt.scatter(output_weight['weight_ifas'], output_weight['target_sur'], color='darkred', s = 17)
plt.title('weight_ifas vs target_sur', fontsize=10)
plt.xlabel('weight_ifas', fontsize=10)
plt.ylabel('target_sur', fontsize=10)
plt.ticklabel_format(useOffset=False, style='plain')
plt.grid(True)
plt.show()
</code>```

Is there a good way? Thanks for the help!

答:
```<code>for column in output_weight.columns.values.tolist():
   fig = plt.figure(figsize=(9,5))
   plt.scatter(output_weight['weight_ifas'], output_weight[column], color='darkred', s = 17)
   plt.title('weight_ifas vs '+column, fontsize=10)
   plt.xlabel('weight_ifas', fontsize=10)
   plt.ylabel(column, fontsize=10)
   plt.ticklabel_format(useOffset=False, style='plain')
   plt.grid(True)
   plt.show()
   fig.savefig(column+'.png')
</code>```

Body:
I have a dataframe with 68 columns.

How can I make matplotlib scatter plots with every column and save every single plot?
Now I can only build plots one by one with changing the column's name in X value. (Y value is always the same, it is the target column)

For example, it is the code where I'm changing the X value, but I don't want to change it 68 times and save every plot one by one.

plt.figure(figsize=(9,5))
plt.scatter(output_weight['weight_ifas'], output_weight['target_sur'], color='darkred', s = 17)
plt.title('weight_ifas vs target_sur', fontsize=10)
plt.xlabel('weight_ifas', fontsize=10)
plt.ylabel('target_sur', fontsize=10)
plt.ticklabel_format(useOffset=False, style='plain')
plt.grid(True)
plt.show()

Is there a good way? Thanks for the help!


问:
ngFor Directive Not Found
I am using Angular 4.3.3 with the JIT compiler and get the error below when I run my application:

Property binding ngforOf not used by any directive on an embedded template.
Make sure that the property name is spelled correctly and all directives are listed in the @NgModule.declarations.

I have made sure to import the BrowserModule in my main app module and I've imported the CommonModule in my child module that this error is originating from.

Here is the template that throws the error:

```<code><div class="background"></div>
<div class="content normal-page">
    <ons-fab position="top right" ripple>+</ons-fab>
    <ons-list>
        <ons-list-item *ngFor="let item of items; let i = index">
            <div class="center">
                #{{i}} msg: {{item.msg}}
            </div>
        </ons-list-item>
    </ons-list>
</div>
</code>```

Since I am importing the proper modules into the right places what could be causing this error?

答:
You also need to ensure that the component you're working on is declared in your module.ts file.

e.g. In your Component while where you're using the *ngFor:

```<code>@Component({
  ...
  template: '<div *ngFor="let thing of things">..</div>'
})
export class MyComponent {...
</code>```

This component must also be declared in your module.ts file:

```<code>@NgModule({
  ...
  declarations: [
    MyComponent
  ]
</code>```

I use a VS Code extension that creates my components for me, but doesn't register them in the app module, and I often forget to do it manually, causing this problem.

Title: ngFor Directive Not Found

Body:
I am using Angular 4.3.3 with the JIT compiler and get the error below when I run my application:

Property binding ngforOf not used by any directive on an embedded template.
Make sure that the property name is spelled correctly and all directives are listed in the @NgModule.declarations.

I have made sure to import the BrowserModule in my main app module and I've imported the CommonModule in my child module that this error is originating from.

Here is the template that throws the error:

    <ons-fab position="top right" ripple>+</ons-fab>
    <ons-list>
        <ons-list-item *ngFor="let item of items; let i = index">

                #{{i}} msg: {{item.msg}}

        </ons-list-item>
    </ons-list>

Since I am importing the proper modules into the right places what could be causing this error?

Body:
You also need to ensure that the component you're working on is declared in your module.ts file.

e.g. In your Component while where you're using the *ngFor:

@Component({
  ...
  template: '<div *ngFor="let thing of things">..</div>'
})
export class MyComponent {...

This component must also be declared in your module.ts file:

@NgModule({
  ...
  declarations: [
    MyComponent
  ]

I use a VS Code extension that creates my components for me, but doesn't register them in the app module, and I often forget to do it manually, causing this problem.


问:
Count unique strings that only occur in a single group based on all possible groups
I have the following df

```<code>a = data.frame(PA = c("A", "A", "A", "B", "B"), Family = c("aa", "ab", "ac", "aa", "ad"))
</code>```

What I want to obtain is a count of unique 'Family' strings (aa, ab, ac, ad) in each PA (A or B) based on all possible PAs. For example, aa is a unique string for A and B, but since it occurs in both PAs I don't want it. On the other hand, ab and ac are unique for PA A and only occur in PA A: that's what I want.

Using dplyr I was doing something like:

```<code>df >%> group_by(PA) %>%
summarise(count_family = n_distinct(Family))
</code>```

But this only returns unique terms inside each PA — and I want unique Families that occur inside unique PAs based on all possible PAs

答:
Here's a tidyverse approach.

First remove all duplicated Family, then group_by(PA) and count.

```<code>library(tidyverse)

a %>% group_by(Family) %>%
  filter(n() == 1) %>%
  group_by(PA) %>%
  summarize(count_family = n())
</code>```

Output

```<code># A tibble: 2 x 2
  PA    count_family
  <chr>        <int>
1 A                2
2 B                1
</code>```

Output before summarise()

```<code># A tibble: 3 x 2
# Groups:   Family [3]
  PA    Family
  <chr> <chr>
1 A     ab
2 A     ac
3 B     ad
</code>```

Body:
I have the following df

a = data.frame(PA = c("A", "A", "A", "B", "B"), Family = c("aa", "ab", "ac", "aa", "ad"))

What I want to obtain is a count of unique 'Family' strings (aa, ab, ac, ad) in each PA (A or B) based on all possible PAs. For example, aa is a unique string for A and B, but since it occurs in both PAs I don't want it. On the other hand, ab and ac are unique for PA A and only occur in PA A: that's what I want.

Using dplyr I was doing something like:

df >%> group_by(PA) %>%
summarise(count_family = n_distinct(Family))

But this only returns unique terms inside each PA — and I want unique Families that occur inside unique PAs based on all possible PAs

Body:
Here's a tidyverse approach.

First remove all duplicated Family, then group_by(PA) and count.

library(tidyverse)

a %>% group_by(Family) %>%
  filter(n() == 1) %>%
  group_by(PA) %>%
  summarize(count_family = n())

Output

# A tibble: 2 x 2
  PA    count_family
  <chr>        <int>
1 A                2
2 B                1

Output before summarise()

# A tibble: 3 x 2
# Groups:   Family [3]
  PA    Family
  <chr> <chr>
1 A     ab
2 A     ac
3 B     ad


问:
adding jQuery address to a tab script
With the help of a friendly guy I've built a tab script that is very simple and doesn't require an additional script like jQuery UI.

Now, I've seen the address bar thing which is really cool so you actually have ACCESS to the tabs, be able to bookmark the tabs and use browsers back/forth buttons.

http://www.asual.com/jquery/address/

here is my JS fiddle: http://jsfiddle.net/BiBA/WEEFd/

I tried to add it into my existing JS snippet but it didnt' work.
Do you know how to add the address support into this tab script?
        $(document).ready(function(){
      initTabs();
    });

```<code>function initTabs() {
  $('#tabMenu a').bind('click',function(e) {
  e.preventDefault();
  var thref = $(this).attr("href").replace(/#/, '');
  $('#tabMenu a').removeClass('active');
  $(this).addClass('active');
  $('#tabContent div.content').removeClass('active');
  $('#'+thref).addClass('active');
  });
}
</code>```

答:
Use location.hash to retrieve the hash value of your url. You can pass this value to your initTabs function and check if there's a content/tab with the corresponding id/href value and activate it.

See also http://www.ezineasp.net/post/Javascript-Location-Hash-with-Example.aspx

Title: adding jQuery address to a tab script

Body:
With the help of a friendly guy I've built a tab script that is very simple and doesn't require an additional script like jQuery UI.

Now, I've seen the address bar thing which is really cool so you actually have ACCESS to the tabs, be able to bookmark the tabs and use browsers back/forth buttons.

http://www.asual.com/jquery/address/

here is my JS fiddle: http://jsfiddle.net/BiBA/WEEFd/

I tried to add it into my existing JS snippet but it didnt' work.
Do you know how to add the address support into this tab script?
        $(document).ready(function(){
      initTabs();
    });

function initTabs() {
  $('#tabMenu a').bind('click',function(e) {
  e.preventDefault();
  var thref = $(this).attr("href").replace(/#/, '');
  $('#tabMenu a').removeClass('active');
  $(this).addClass('active');
  $('#tabContent div.content').removeClass('active');
  $('#'+thref).addClass('active');
  });
}


问:
NgRX Effect is not workling
im trying to work with ngrx and angular. My Authenfication works but my product store doesn work because nothing happens:

I working with modules:

const routes: Routes = [
  {
    path: '',
    redirectTo: 'home',
    pathMatch: 'full'
  },
  {
    path: 'home',
    loadChildren: () => import('src/app/home/home.module')
      .then(module => module.HomeModule)
  }
]

@NgModule({
  declarations: [
    AppComponent
  ],
    imports: [
        BrowserModule,
        BrowserAnimationsModule,
        RouterModule.forRoot(routes),
        EffectsModule.forRoot([]),
        StoreModule.forRoot({}),
        StoreDevtoolsModule.instrument({
            maxAge: 25,
            logOnly: environment.production
        }),
        TopBarModule,
        AuthModule,
        HomeModule

    ],
  providers: [],
  bootstrap: [AppComponent]
})
export class AppModule { }```
</div>
</div>
</p>
<p>And in my home module i inject the effectservices:</p>
<p><div class="snippet" data-lang="js" data-hide="false" data-console="true" data-babel="false">
<div class="snippet-code">
<pre class="snippet-code-html lang-html prettyprint-override"><code>const routes: Routes = [
  {
    path: '',
    component: HomeComponent
  }
]
@NgModule({
  declarations: [
    HomeComponent
  ],
  imports: [
    CommonModule,
    RouterModule.forChild(routes),
    EffectsModule.forFeature([
      GetCategoryEffectsService,
      GetProductEffectService
    ]),
    StoreModule.forFeature('category', categoryReducers),
    StoreModule.forFeature('product', productReducers),
  ],
  providers: [
    GetCategoriesService,
    GetProductsService
  ]
})
export class HomeModule { }</code>```

My redux dont do anything with this module and my effect doesnt work:

@Injectable()
export class GetProductEffectService {

  constructor(
    private actions$: Actions,
    private getProductsService: GetProductsService
  ) { }

  getProducts$ = createEffect( () =>
    this.actions$.pipe(
      ofType(getProductAction),
      switchMap(() => {
        return this.getProductsService.getProducts().pipe(
          map((products) => getProductActionSuccess({products}))
        )
      })
    ))
}```
</div>
</div>
</p>
<p>if i subscribe the the method to get products without my store i gets all products...</p>
<p>my Actions look like this:</p>
<p><div class="snippet" data-lang="js" data-hide="false" data-console="true" data-babel="false">
<div class="snippet-code">
<pre class="snippet-code-html lang-html prettyprint-override"><code>export const getProductAction = createAction(
  ProductActionTypes.PRODUCT_ACTION
);

export const getProductActionSuccess = createAction(
  ProductActionTypes.PRODUCT_ACTION_SUCCESS,
  props<{products: ProductInterface[]}>()
);

export const getProductActionFailure = createAction(
  ProductActionTypes.PRODUCT_ACTION_FAILURE,
  props<{error: BackendErrorsInterface}>()
);</code>```

export enum ProductActionTypes {
  PRODUCT_ACTION = '[Product] product action',
  PRODUCT_ACTION_SUCCESS = '[Product] product action success',
  PRODUCT_ACTION_FAILURE = '[Product] product action failure',
}```
</div>
</div>

My reducers are similar to my working reducers but here is the code:</p>
<p><div class="snippet" data-lang="js" data-hide="false" data-console="true" data-babel="false">
<div class="snippet-code">
<pre class="snippet-code-html lang-html prettyprint-override"><code>const initialState: ProductStateInterface = {
  data: null,
  error: null,
  isLoading: false,
  isSubmitting: false
}

const reducers = createReducer(
  initialState,
  on(
    getProductAction,
    (state): ProductStateInterface => ({
      ...state,
      isLoading: true,
      error: null
    })
  ),

  on(
    getProductActionSuccess,
    (state, action): ProductStateInterface => ({
      ...state,
      data: action.products,
      isLoading: false,
      error: null
    })
  ),

  on(
    getProductActionFailure,
    (state, action): ProductStateInterface => ({
      ...state,
      data: null,
      isLoading: false,
      error: action.error
    })
  )
);
export function productReducers(state: ProductStateInterface, action: Action){
  return reducers(state, action);
}</code>```

i select the actions in my product selector File:

export const actionFeatureSelector = createFeatureSelector<
  AppStateInterface,ProductStateInterface>('product');

export const productIsLoadingSelector = createSelector(
  actionFeatureSelector,
  (productState: ProductStateInterface) => productState.isLoading
);

export const getProductsSelector = createSelector(
  actionFeatureSelector,
  (productState: ProductStateInterface) => productState.data
);

export const productErrorSelector = createSelector(
  actionFeatureSelector,
  (productState: ProductStateInterface) => productState.error
);

export const productIsSubmittingSelector = createSelector(
  actionFeatureSelector,
  (productState: ProductStateInterface) => productState.isSubmitting
);```
</div>
</div>
</p>
<p>Nothing happens with this store.
Here is my component:</p>
<p><div class="snippet" data-lang="js" data-hide="false" data-console="true" data-babel="false">
<div class="snippet-code">
<pre class="snippet-code-html lang-html prettyprint-override"><code>ngOnInit(): void {
    this.initializeValues();
  }

  private initializeValues(): void {

    this.store.pipe(select(getProductsSelector)) //.subscribe(test => console.log(test));

    this.categories$ = this.store.pipe(select(categorySelector));

    this.isLoading$ = this.store.pipe(select(isLoadingSelector));
  }</code>```

答:
You have an effect which triggers only when getProductAction is dispatched:

```<code>getProducts$ = createEffect( () =>
    this.actions$.pipe(
      ofType(getProductAction),
      switchMap(() => {
        return this.getProductsService.getProducts().pipe(
          map((products) => getProductActionSuccess({products}))
        )
      })
    ))
</code>```

However, you never actually dispatch the action, so the effect will never be triggered.

I suggest you dispatch the action in your component:

```<code>ngOnInit(): void {
    this.initializeValues();
    this.store.dispatch(getProductAction()); // <-- here you trigger the effect to fetch the products

  }

  private initializeValues(): void {

    this.store.pipe(select(getProductsSelector)) //.subscribe(test => console.log(test));

    this.categories$ = this.store.pipe(select(categorySelector));

    this.isLoading$ = this.store.pipe(select(isLoadingSelector));
  }
</code>```

Title: NgRX Effect is not workling

Body:
im trying to work with ngrx and angular. My Authenfication works but my product store doesn work because nothing happens:

I working with modules:

const routes: Routes = [
  {
    path: '',
    redirectTo: 'home',
    pathMatch: 'full'
  },
  {
    path: 'home',
    loadChildren: () => import('src/app/home/home.module')
      .then(module => module.HomeModule)
  }
]

@NgModule({
  declarations: [
    AppComponent
  ],
    imports: [
        BrowserModule,
        BrowserAnimationsModule,
        RouterModule.forRoot(routes),
        EffectsModule.forRoot([]),
        StoreModule.forRoot({}),
        StoreDevtoolsModule.instrument({
            maxAge: 25,
            logOnly: environment.production
        }),
        TopBarModule,
        AuthModule,
        HomeModule

    ],
  providers: [],
  bootstrap: [AppComponent]
})
export class AppModule { }

And in my home module i inject the effectservices:

const routes: Routes = [
  {
    path: '',
    component: HomeComponent
  }
]
@NgModule({
  declarations: [
    HomeComponent
  ],
  imports: [
    CommonModule,
    RouterModule.forChild(routes),
    EffectsModule.forFeature([
      GetCategoryEffectsService,
      GetProductEffectService
    ]),
    StoreModule.forFeature('category', categoryReducers),
    StoreModule.forFeature('product', productReducers),
  ],
  providers: [
    GetCategoriesService,
    GetProductsService
  ]
})
export class HomeModule { }

My redux dont do anything with this module and my effect doesnt work:

@Injectable()
export class GetProductEffectService {

  constructor(
    private actions$: Actions,
    private getProductsService: GetProductsService
  ) { }

  getProducts$ = createEffect( () =>
    this.actions$.pipe(
      ofType(getProductAction),
      switchMap(() => {
        return this.getProductsService.getProducts().pipe(
          map((products) => getProductActionSuccess({products}))
        )
      })
    ))
}

if i subscribe the the method to get products without my store i gets all products...

my Actions look like this:

export const getProductAction = createAction(
  ProductActionTypes.PRODUCT_ACTION
);

export const getProductActionSuccess = createAction(
  ProductActionTypes.PRODUCT_ACTION_SUCCESS,
  props<{products: ProductInterface[]}>()
);

export const getProductActionFailure = createAction(
  ProductActionTypes.PRODUCT_ACTION_FAILURE,
  props<{error: BackendErrorsInterface}>()
);

export enum ProductActionTypes {
  PRODUCT_ACTION = '[Product] product action',
  PRODUCT_ACTION_SUCCESS = '[Product] product action success',
  PRODUCT_ACTION_FAILURE = '[Product] product action failure',
}

My reducers are similar to my working reducers but here is the code:

const initialState: ProductStateInterface = {
  data: null,
  error: null,
  isLoading: false,
  isSubmitting: false
}

const reducers = createReducer(
  initialState,
  on(
    getProductAction,
    (state): ProductStateInterface => ({
      ...state,
      isLoading: true,
      error: null
    })
  ),

  on(
    getProductActionSuccess,
    (state, action): ProductStateInterface => ({
      ...state,
      data: action.products,
      isLoading: false,
      error: null
    })
  ),

  on(
    getProductActionFailure,
    (state, action): ProductStateInterface => ({
      ...state,
      data: null,
      isLoading: false,
      error: action.error
    })
  )
);
export function productReducers(state: ProductStateInterface, action: Action){
  return reducers(state, action);
}

i select the actions in my product selector File:

export const actionFeatureSelector = createFeatureSelector<
  AppStateInterface,ProductStateInterface>('product');

export const productIsLoadingSelector = createSelector(
  actionFeatureSelector,
  (productState: ProductStateInterface) => productState.isLoading
);

export const getProductsSelector = createSelector(
  actionFeatureSelector,
  (productState: ProductStateInterface) => productState.data
);

export const productErrorSelector = createSelector(
  actionFeatureSelector,
  (productState: ProductStateInterface) => productState.error
);

export const productIsSubmittingSelector = createSelector(
  actionFeatureSelector,
  (productState: ProductStateInterface) => productState.isSubmitting
);

Nothing happens with this store.
Here is my component:

ngOnInit(): void {
    this.initializeValues();
  }

  private initializeValues(): void {

    this.store.pipe(select(getProductsSelector)) //.subscribe(test => console.log(test));

    this.categories$ = this.store.pipe(select(categorySelector));

    this.isLoading$ = this.store.pipe(select(isLoadingSelector));
  }

Body:
You have an effect which triggers only when getProductAction is dispatched:

getProducts$ = createEffect( () =>
    this.actions$.pipe(
      ofType(getProductAction),
      switchMap(() => {
        return this.getProductsService.getProducts().pipe(
          map((products) => getProductActionSuccess({products}))
        )
      })
    ))

However, you never actually dispatch the action, so the effect will never be triggered.

I suggest you dispatch the action in your component:

ngOnInit(): void {
    this.initializeValues();
    this.store.dispatch(getProductAction()); // <-- here you trigger the effect to fetch the products

  }

  private initializeValues(): void {

    this.store.pipe(select(getProductsSelector)) //.subscribe(test => console.log(test));

    this.categories$ = this.store.pipe(select(categorySelector));

    this.isLoading$ = this.store.pipe(select(isLoadingSelector));
  }


问:
I s it possible to connect tableau data to populate in Googel sheets?
Can I link tableau data to google sheets? is that possible?

Thank you in advance,

答:
You can connect Tableau Desktop and Tableau Public to Google Sheets.

Desktop: Connect to Data/Google Sheets

Public: Connect to Data/Google Sheets


问:
How to fix the error "ModuleNotFoundError: No module named 'pygame'"?
I wanted to "compile" the project on pygame using pyinstaller, but when opening the .exe file, this error pops up:

答:
this could happen because either you havent installed pygame
do pip install pygame to fix that or might be because python is not added to you path in environment variable steps to add python to path
or it might be that you haven't installed python check here to install it


问:
divide a list of positive integers by the length of digits
I would like to divide a list of positive integers by the length of digits.

For example, int_list = [1,3,13,888,1004]
-> [1,3] , [13] , [888] , [1004]

I have implemented below:

def seperate_by_digit(int_list):
    len_set = set()  # track how many digits we have

    int_dict = {}  # key : len of digits  / val : list of integer
    for int_ele in int_list:
        n = len(str(int_ele))
        if n not in len_set:
            int_dict[n] = [int_ele]
            len_set.add(n)
        else:
            sub_int_list = int_dict[n]
            sub_int_list.append(int_ele)
            int_dict[n] = sub_int_list
    return int_dict
```

Is there any better, cleaner way to complete this task?

答:
Based on this linke converting to str have performance issue, we can do it as:

import math

def getIntegerPlaces(theNumber):
    if theNumber <= 999999999999997:
        return int(math.log10(theNumber)) + 1
    else:
        counter = 15
        while theNumber >= 10**counter:
            counter += 1
        return counter

def convert(l):
    m = {}
    for v in a:
        digitCount = getIntegerPlaces(v)
        if not digitCount in m:
            m[digitCount] = []
        m[digitCount].append(v)
    return list(m.values())

a = [1,3,13,888,1004]
print(convert(a))
```
<p>The output will be:</p>
<pre class="lang-python prettyprint-override"><code>[[1, 3], [13], [888], [1004]]
</code>```

Body:
I would like to divide a list of positive integers by the length of digits.

For example, int_list = [1,3,13,888,1004]
-> [1,3] , [13] , [888] , [1004]

I have implemented below:

def seperate_by_digit(int_list):
    len_set = set()  # track how many digits we have

    int_dict = {}  # key : len of digits  / val : list of integer
    for int_ele in int_list:
        n = len(str(int_ele))
        if n not in len_set:
            int_dict[n] = [int_ele]
            len_set.add(n)
        else:
            sub_int_list = int_dict[n]
            sub_int_list.append(int_ele)
            int_dict[n] = sub_int_list
    return int_dict

Is there any better, cleaner way to complete this task?

Body:
Based on this linke converting to str have performance issue, we can do it as:

import math

def getIntegerPlaces(theNumber):
    if theNumber <= 999999999999997:
        return int(math.log10(theNumber)) + 1
    else:
        counter = 15
        while theNumber >= 10**counter:
            counter += 1
        return counter

def convert(l):
    m = {}
    for v in a:
        digitCount = getIntegerPlaces(v)
        if not digitCount in m:
            m[digitCount] = []
        m[digitCount].append(v)
    return list(m.values())

a = [1,3,13,888,1004]
print(convert(a))

The output will be:

[[1, 3], [13], [888], [1004]]


问:
How to serialize an Object to Map by Moshi
I want to serialize an Object to Map by Moshi.Here is my codes by Gson

```<code>    public static Map<String, String> toMap(Object obj, Gson gson) {
    if (gson == null) {
        gson = new Gson();
    }
    String json = gson.toJson(obj);
    Map<String, String> map = gson.fromJson(json, new TypeToken<Map<String, String>>() {
    }.getType());
    return map;
}
</code>```

And how to write by Moshi ?

答:
In Kotlin:</p>
```<code>val type = Types.newParameterizedType(
            MutableMap::class.java,
            String::class.java,
            String::class.java
        )
        val adapter: JsonAdapter<Map<String, String>> = moshi.adapter(type)
        val map: Map<String, String> = adapter.fromJson(responseJson)
</code>```

Body:
I want to serialize an Object to Map by Moshi.Here is my codes by Gson

    public static Map<String, String> toMap(Object obj, Gson gson) {
    if (gson == null) {
        gson = new Gson();
    }
    String json = gson.toJson(obj);
    Map<String, String> map = gson.fromJson(json, new TypeToken<Map<String, String>>() {
    }.getType());
    return map;
}

And how to write by Moshi ?

Body:
In Kotlin:

val type = Types.newParameterizedType(
            MutableMap::class.java,
            String::class.java,
            String::class.java
        )
        val adapter: JsonAdapter<Map<String, String>> = moshi.adapter(type)
        val map: Map<String, String> = adapter.fromJson(responseJson)


问:
argocd app create in CI pipeline (GitHub Actions, Tekton, ...) throws "PermissionDenied desc = permission denied: applications, create, default/myapp"
From our Tekton pipeline we want to use ArgoCD CLI to do a argocd app create and argocd app sync dynamically based on the app that is build. We created a new user as described in the docs by adding a accounts.tekton: apiKey to the argocd-cm ConfigMap:

```<code>kubectl patch configmap argocd-cm -n argocd -p '{"data": {"accounts.tekton": "apiKey"}}'
</code>```

Then we created a token for the tekton user with:

```<code>argocd account generate-token --account tekton
</code>```

With this token as the password and the username tekton we did the argocd login like

```<code>argocd login $(kubectl get service argocd-server -n argocd --output=jsonpath='{.status.loadBalancer.ingress[0].hostname}') --username=tekton --password="$TOKEN";
</code>```

Now from within our Tekton pipeline (but we guess that would be the same for every other CI, given the usage of a non-admin user) we get the following error if we run argocd app create:

```<code>$ argocd app create microservice-api-spring-boot --repo https://gitlab.com/jonashackt/microservice-api-spring-boot-config.git --path deployment --dest-server https://kubernetes.default.svc --dest-namespace default --revision argocd --sync-policy auto
error rpc error: code = PermissionDenied desc = permission denied: applications, create, default/microservice-api-spring-boot, sub: tekton, iat: 2022-02-03T16:36:48Z
</code>```

答:
The problem is mentioned in Argo's useraccounts docs:

When you create local users, each of those users will need additional
RBAC rules set up, otherwise they will fall back to the default policy
specified by policy.default field of the argocd-rbac-cm ConfigMap.

But these additional RBAC rules could be setup the simplest using ArgoCD Projects. And with such a AppProject you don't even need to create a user like tekton in the ConfigMap argocd-cm. ArgoCD projects have the ability to define Project roles:

Projects include a feature called roles that enable automated access to a project's applications. These can be used to give a CI pipeline a restricted set of permissions. For example, a CI system may only be able to sync a single app (but not change its source or destination).

There are 2 solutions how to configure the AppProject, role & permissions incl. role token:

using argocd CLI

using a manifest YAML file

1.) Use argocd CLI to create AppProject, role & permissions incl. role token

So let's get our hands dirty and create a ArgoCD AppProject using the argocd CLI called apps2deploy:

```<code>argocd proj create apps2deploy -d https://kubernetes.default.svc,default --src "*"
</code>```

We create it with the --src "*" as a wildcard for any git repository (as described here).

Now we create a Project role called create-sync via:

```<code>argocd proj role create apps2deploy create-sync --description "project role to create and sync apps from a CI/CD pipeline"
</code>```

You can check the new role has been created with argocd proj role list apps2deploy.

Then we need to create a token for the new Project role create-sync, which can be created via:

```<code>argocd proj role create-token apps2deploy create-sync
</code>```

This token needs to be used for the argocd login command inside our Tekton / CI pipeline. There's also a --token-only parameter for the command, so we can create an environment variable via

```<code>ARGOCD_AUTH_TOKEN=$(argocd proj role create-token apps2deploy create-sync --token-only)
</code>```

The ARGOCD_AUTH_TOKEN will be automatically used by argo login.

Now we need to give permissions to the role, so it will be able to create and sync our application in ArgoCD from within Tekton or any other CI pipeline. As described in the docs we therefore add policies to our roles using the argocd CLI:

argocd proj role add-policy apps2deploy create-sync --action get --permission allow --object "*"
argocd proj role add-policy apps2deploy create-sync --action create --permission allow --object "*"
argocd proj role add-policy apps2deploy create-sync --action sync --permission allow --object "*"
argocd proj role add-policy apps2deploy create-sync --action update --permission allow --object "*"
argocd proj role add-policy apps2deploy create-sync --action delete --permission allow --object "*"
```
<p>Have a look on the role policies with <code>argocd proj role get apps2deploy create-sync</code>, which should look somehow like this:</p>
```$ argocd proj role get apps2deploy create-sync
Role Name:     create-sync
Description:   project role to create and sync apps from a CI/CD pipeline
Policies:
p, proj:apps2deploy:create-sync, projects, get, apps2deploy, allow
p, proj:apps2deploy:create-sync, applications, get, apps2deploy/*, allow
p, proj:apps2deploy:create-sync, applications, create, apps2deploy/*, allow
p, proj:apps2deploy:create-sync, applications, update, apps2deploy/*, allow
p, proj:apps2deploy:create-sync, applications, delete, apps2deploy/*, allow
p, proj:apps2deploy:create-sync, applications, sync, apps2deploy/*, allow
JWT Tokens:
ID          ISSUED-AT                                EXPIRES-AT
1644166189  2022-02-06T17:49:49+01:00 (2 hours ago)  <none>
```
<p>Finally we should have setup everything to do a successful <code>argocd app create</code>. All we need to do is to add the <code>--project apps2deploy</code> parameter:</p>
```argocd app create microservice-api-spring-boot --repo https://gitlab.com/jonashackt/microservice-api-spring-boot-config.git --path deployment --project apps2deploy --dest-server https://kubernetes.default.svc --dest-namespace default --revision argocd --sync-policy auto
```
<h2>2.) Use manifest YAML to create <code>AppProject</code>, role & permissions incl. role token</h2>
<p>As all those CLI based steps in solution 1.) are quite many, we could also using a manifest YAML file. Here's an example <code>argocd-appproject-apps2deploy.yml</code> which configures exactly the same as in solution a):</p>
```apiVersion: argoproj.io/v1alpha1
kind: AppProject
metadata:
  name: apps2deploy
  namespace: argocd
spec:
  destinations:
    - namespace: default
      server: https://kubernetes.default.svc
  sourceRepos:
    - '*'
  roles:
    - description: project role to create and sync apps from a CI/CD pipeline
      name: create-sync
      policies:
      - p, proj:apps2deploy:create-sync, applications, get, apps2deploy/*, allow
      - p, proj:apps2deploy:create-sync, applications, create, apps2deploy/*, allow
      - p, proj:apps2deploy:create-sync, applications, update, apps2deploy/*, allow
      - p, proj:apps2deploy:create-sync, applications, delete, apps2deploy/*, allow
      - p, proj:apps2deploy:create-sync, applications, sync, apps2deploy/*, allow
```
<p>There are only 2 steps left to be able to do a successful <code>argocd app create</code> from within Tekton (or other CI pipeline). We need to <code>apply</code> the manifest with</p>
```kubectl apply -f argocd-appproject-apps2deploy.yml
```
<p>And we need to need to create a role token, ideally assigning it directly to the <code>ARGOCD_AUTH_TOKEN</code> for the <code>argocd login</code> command (which also needs to be done afterwards):</p>
```ARGOCD_AUTH_TOKEN=$(argocd proj role create-token apps2deploy create-sync --token-only)
```
<p>The same <code>argocd app create</code> command as mentioned in solution 1.) should work now:</p>
```argocd app create microservice-api-spring-boot --repo https://gitlab.com/jonashackt/microservice-api-spring-boot-config.git --path deployment --project apps2deploy --dest-server https://kubernetes.default.svc --dest-namespace default --revision argocd --sync-policy auto
```

Body:
From our Tekton pipeline we want to use ArgoCD CLI to do a argocd app create and argocd app sync dynamically based on the app that is build. We created a new user as described in the docs by adding a accounts.tekton: apiKey to the argocd-cm ConfigMap:

kubectl patch configmap argocd-cm -n argocd -p '{"data": {"accounts.tekton": "apiKey"}}'

Then we created a token for the tekton user with:

argocd account generate-token --account tekton

With this token as the password and the username tekton we did the argocd login like

argocd login $(kubectl get service argocd-server -n argocd --output=jsonpath='{.status.loadBalancer.ingress[0].hostname}') --username=tekton --password="$TOKEN";

Now from within our Tekton pipeline (but we guess that would be the same for every other CI, given the usage of a non-admin user) we get the following error if we run argocd app create:

$ argocd app create microservice-api-spring-boot --repo https://gitlab.com/jonashackt/microservice-api-spring-boot-config.git --path deployment --dest-server https://kubernetes.default.svc --dest-namespace default --revision argocd --sync-policy auto
error rpc error: code = PermissionDenied desc = permission denied: applications, create, default/microservice-api-spring-boot, sub: tekton, iat: 2022-02-03T16:36:48Z

Body:
The problem is mentioned in Argo's useraccounts docs:

When you create local users, each of those users will need additional
RBAC rules set up, otherwise they will fall back to the default policy
specified by policy.default field of the argocd-rbac-cm ConfigMap.

But these additional RBAC rules could be setup the simplest using ArgoCD Projects. And with such a AppProject you don't even need to create a user like tekton in the ConfigMap argocd-cm. ArgoCD projects have the ability to define Project roles:

Projects include a feature called roles that enable automated access to a project's applications. These can be used to give a CI pipeline a restricted set of permissions. For example, a CI system may only be able to sync a single app (but not change its source or destination).

There are 2 solutions how to configure the AppProject, role & permissions incl. role token:

using argocd CLI

using a manifest YAML file

1.) Use argocd CLI to create AppProject, role & permissions incl. role token

So let's get our hands dirty and create a ArgoCD AppProject using the argocd CLI called apps2deploy:

argocd proj create apps2deploy -d https://kubernetes.default.svc,default --src "*"

We create it with the --src "*" as a wildcard for any git repository (as described here).

Now we create a Project role called create-sync via:

argocd proj role create apps2deploy create-sync --description "project role to create and sync apps from a CI/CD pipeline"

You can check the new role has been created with argocd proj role list apps2deploy.

Then we need to create a token for the new Project role create-sync, which can be created via:

argocd proj role create-token apps2deploy create-sync

This token needs to be used for the argocd login command inside our Tekton / CI pipeline. There's also a --token-only parameter for the command, so we can create an environment variable via

ARGOCD_AUTH_TOKEN=$(argocd proj role create-token apps2deploy create-sync --token-only)

The ARGOCD_AUTH_TOKEN will be automatically used by argo login.

Now we need to give permissions to the role, so it will be able to create and sync our application in ArgoCD from within Tekton or any other CI pipeline. As described in the docs we therefore add policies to our roles using the argocd CLI:

argocd proj role add-policy apps2deploy create-sync --action get --permission allow --object "*"
argocd proj role add-policy apps2deploy create-sync --action create --permission allow --object "*"
argocd proj role add-policy apps2deploy create-sync --action sync --permission allow --object "*"
argocd proj role add-policy apps2deploy create-sync --action update --permission allow --object "*"
argocd proj role add-policy apps2deploy create-sync --action delete --permission allow --object "*"

Have a look on the role policies with argocd proj role get apps2deploy create-sync, which should look somehow like this:

$ argocd proj role get apps2deploy create-sync
Role Name:     create-sync
Description:   project role to create and sync apps from a CI/CD pipeline
Policies:
p, proj:apps2deploy:create-sync, projects, get, apps2deploy, allow
p, proj:apps2deploy:create-sync, applications, get, apps2deploy/*, allow
p, proj:apps2deploy:create-sync, applications, create, apps2deploy/*, allow
p, proj:apps2deploy:create-sync, applications, update, apps2deploy/*, allow
p, proj:apps2deploy:create-sync, applications, delete, apps2deploy/*, allow
p, proj:apps2deploy:create-sync, applications, sync, apps2deploy/*, allow
JWT Tokens:
ID          ISSUED-AT                                EXPIRES-AT
1644166189  2022-02-06T17:49:49+01:00 (2 hours ago)  <none>

Finally we should have setup everything to do a successful argocd app create. All we need to do is to add the --project apps2deploy parameter:

argocd app create microservice-api-spring-boot --repo https://gitlab.com/jonashackt/microservice-api-spring-boot-config.git --path deployment --project apps2deploy --dest-server https://kubernetes.default.svc --dest-namespace default --revision argocd --sync-policy auto

2.) Use manifest YAML to create AppProject, role & permissions incl. role token

As all those CLI based steps in solution 1.) are quite many, we could also using a manifest YAML file. Here's an example argocd-appproject-apps2deploy.yml which configures exactly the same as in solution a):

apiVersion: argoproj.io/v1alpha1
kind: AppProject
metadata:
  name: apps2deploy
  namespace: argocd
spec:
  destinations:
    - namespace: default
      server: https://kubernetes.default.svc
  sourceRepos:
    - '*'
  roles:
    - description: project role to create and sync apps from a CI/CD pipeline
      name: create-sync
      policies:
      - p, proj:apps2deploy:create-sync, applications, get, apps2deploy/*, allow
      - p, proj:apps2deploy:create-sync, applications, create, apps2deploy/*, allow
      - p, proj:apps2deploy:create-sync, applications, update, apps2deploy/*, allow
      - p, proj:apps2deploy:create-sync, applications, delete, apps2deploy/*, allow
      - p, proj:apps2deploy:create-sync, applications, sync, apps2deploy/*, allow

There are only 2 steps left to be able to do a successful argocd app create from within Tekton (or other CI pipeline). We need to apply the manifest with

kubectl apply -f argocd-appproject-apps2deploy.yml

And we need to need to create a role token, ideally assigning it directly to the ARGOCD_AUTH_TOKEN for the argocd login command (which also needs to be done afterwards):

ARGOCD_AUTH_TOKEN=$(argocd proj role create-token apps2deploy create-sync --token-only)

The same argocd app create command as mentioned in solution 1.) should work now:

argocd app create microservice-api-spring-boot --repo https://gitlab.com/jonashackt/microservice-api-spring-boot-config.git --path deployment --project apps2deploy --dest-server https://kubernetes.default.svc --dest-namespace default --revision argocd --sync-policy auto


问:
So I have two html files and they are both linked to another another and share the same css/js file
So I have two  html files and they are both linked to another another and share the same css/js file. when I use a btn on one index.html and click it I want it to return append something to a section/div on index2.html. I have the code but for some reason when I run it I get an error so I console.logged it and it returns null. How do I access elements from another html file do I need to import that into js ?
index:

```<code><a href="index.html">home</a>
    <a href="test.html">test</a>

    <button class="btn" onclick="add()">addP</button>
</code>```

index2:

```<code> <a href="index.html">home</a>
    <a href="test.html">test</a>

    <h1>add p below </h1>

    <div class="pbox">

    </div>
</code>```

I want to access the div.pbox

答:
You are facing a few issues here. On the home page there is no pbox and on the test page there is no add button this is easy enough to solve with a conditional to see if they even exist or using 2 JavaScript files.

The next problem you face is how you store the information and get it from one page to another. Once the page reloads the information will just go away. The answer in this case would be local storage.

When the page loads it checks for the existence of a list in local storage. If it exists it gets that information and parses it. If it does not exist it is just an empty array. This happens on both pages because they both need this information.

When the test page loads it simply grabs the information in the list and maps over it to fill out the pbox with whatever is in the list at that time.

On the home page once the button is clicked it pushes the new item to the list array then updates the local storage.

Here is the Sandbox Link

Body:
So I have two  html files and they are both linked to another another and share the same css/js file. when I use a btn on one index.html and click it I want it to return append something to a section/div on index2.html. I have the code but for some reason when I run it I get an error so I console.logged it and it returns null. How do I access elements from another html file do I need to import that into js ?
index:

home
    test

    <button class="btn" onclick="add()">addP</button>

index2:

 home
    test

add p below

I want to access the div.pbox


问:
Why do I get nan from a new column consisting of the average of other columns?
I need to create a new column consisting of the average of other columns in the dataframe. If I add the columns to be averaged manually everything works, as in this case:

Case 1 - It works and give me a new column "BL_SFI_AV" with float

```<code>matrice_clean['BL_SFI_AV'] = matrice_clean[['BL_SFI_01','BL_SFI_02','BL_SFI_03']].mean(axis = 1)
</code>```

Unfortunately, when I want to use the column slice and then give a range of columns to take the result of the new column is nan. I can't figure out where the problem is. Here is the code I use for the slice:

Case 2 - It doesn't give me any error but the new column BL_SFI_AV is full of nan.

```<code>matrice_clean['BL_SFI_AV'] = matrice_clean.loc['BL_SFI_01':'BL_SFI_03'].mean (axis = 1)
</code>```

答:
First, I think your "case 2" example contains an error.  The columns are in the second axis, so you need to include a row indexer first:

```<code>matrice_clean.loc[:, 'BL_SFI_01':'BL_SFI_03'].mean (axis = 1)
#                 ^^
#                 Don't forget this
</code>```

Next, I have a theory: In what order to your columns appear?

If they are not stored as ['BL_SFI_01','BL_SFI_02','BL_SFI_03'], then using slicing notation ['BL_SFI_01':'BL_SFI_03'] will not have the desired effect.

For example, this works fine:

```<code>In [25]: cols = ['BL_SFI_01', 'BL_SFI_02', 'BL_SFI_03']
    ...: df = pd.DataFrame(np.ones((5, 3)), columns=cols)
    ...: df.loc[:, 'BL_SFI_01':'BL_SFI_03'].mean(axis=1)
Out[25]:
0    1.0
1    1.0
2    1.0
3    1.0
4    1.0
dtype: float64

</code>```

But this returns NaN, because the column order is reversed:

```<code>
In [26]: cols = ['BL_SFI_03', 'BL_SFI_02', 'BL_SFI_01']
    ...: df = pd.DataFrame(np.ones((5, 3)), columns=cols)
    ...: df.loc[:, 'BL_SFI_01':'BL_SFI_03'].mean(axis=1)
Out[26]:
0   NaN
1   NaN
2   NaN
3   NaN
4   NaN
dtype: float64
</code>```

Body:
I need to create a new column consisting of the average of other columns in the dataframe. If I add the columns to be averaged manually everything works, as in this case:

Case 1 - It works and give me a new column "BL_SFI_AV" with float

matrice_clean['BL_SFI_AV'] = matrice_clean[['BL_SFI_01','BL_SFI_02','BL_SFI_03']].mean(axis = 1)

Unfortunately, when I want to use the column slice and then give a range of columns to take the result of the new column is nan. I can't figure out where the problem is. Here is the code I use for the slice:

Case 2 - It doesn't give me any error but the new column BL_SFI_AV is full of nan.

matrice_clean['BL_SFI_AV'] = matrice_clean.loc['BL_SFI_01':'BL_SFI_03'].mean (axis = 1)

Body:
First, I think your "case 2" example contains an error.  The columns are in the second axis, so you need to include a row indexer first:

matrice_clean.loc[:, 'BL_SFI_01':'BL_SFI_03'].mean (axis = 1)
#                 ^^
#                 Don't forget this

Next, I have a theory: In what order to your columns appear?

If they are not stored as ['BL_SFI_01','BL_SFI_02','BL_SFI_03'], then using slicing notation ['BL_SFI_01':'BL_SFI_03'] will not have the desired effect.

For example, this works fine:

In [25]: cols = ['BL_SFI_01', 'BL_SFI_02', 'BL_SFI_03']
    ...: df = pd.DataFrame(np.ones((5, 3)), columns=cols)
    ...: df.loc[:, 'BL_SFI_01':'BL_SFI_03'].mean(axis=1)
Out[25]:
0    1.0
1    1.0
2    1.0
3    1.0
4    1.0
dtype: float64

But this returns NaN, because the column order is reversed:

In [26]: cols = ['BL_SFI_03', 'BL_SFI_02', 'BL_SFI_01']
    ...: df = pd.DataFrame(np.ones((5, 3)), columns=cols)
    ...: df.loc[:, 'BL_SFI_01':'BL_SFI_03'].mean(axis=1)
Out[26]:
0   NaN
1   NaN
2   NaN
3   NaN
4   NaN
dtype: float64


问:
How do I convert multiple columns into one from Binary data?
My data has 3 surveys per year (for 10 years) where 1 represents presence and 0s present absence. The subset looks like this

x <- structure(c(0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1,
                 0, 1, 1, 1, 0, 1, 1, 1, 0, 1, 1, 1,
                 0, 1, 1, 1, 0, 1, 1, 1, 0, 1, 1, 1,
                 0, 1, 1, 1, 0, 1, 1, 1, 0, 1, 1, 1),
               .Dim = c(4L, 3L, 4L))
```

I want to collapse these three columns into one in a way that every row that has 1 in any survey, shows 1 in the final otherwise shows 0.

答:
Collapse the second dimension of the array with apply:

apply(x, c(1L, 3L), function(y) as.integer(any(as.logical(y))))
##      [,1] [,2] [,3] [,4]
## [1,]    0    0    0    0
## [2,]    1    1    1    1
## [3,]    0    1    1    1
## [4,]    1    1    1    1
```

The result is a [site, year] matrix.

Body:
My data has 3 surveys per year (for 10 years) where 1 represents presence and 0s present absence. The subset looks like this

x <- structure(c(0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1,
                 0, 1, 1, 1, 0, 1, 1, 1, 0, 1, 1, 1,
                 0, 1, 1, 1, 0, 1, 1, 1, 0, 1, 1, 1,
                 0, 1, 1, 1, 0, 1, 1, 1, 0, 1, 1, 1),
               .Dim = c(4L, 3L, 4L))

I want to collapse these three columns into one in a way that every row that has 1 in any survey, shows 1 in the final otherwise shows 0.

Body:
Collapse the second dimension of the array with apply:

apply(x, c(1L, 3L), function(y) as.integer(any(as.logical(y))))
##      [,1] [,2] [,3] [,4]
## [1,]    0    0    0    0
## [2,]    1    1    1    1
## [3,]    0    1    1    1
## [4,]    1    1    1    1

The result is a [site, year] matrix.


问:
React locator example
I am trying to understand how the react selectors are working according to https://playwright.dev/docs/selectors#react-selectors . So I am trying some things in playwright sandbox. Seems that the react component cannot be found.

```<code>from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch()
    page = browser.new_page()
    page.goto("https://www.glassdoor.co.uk/Job/qa-engineer-jobs-SRCH_KO0,11.htm")
    page.locator("_react=q[key='1007467366491']").click()
    browser.close()
</code>```

Error:

```<code>    playwright._impl._api_types.TimeoutError: Timeout 30000ms exceeded.
=========================== logs ===========================
waiting for selector "_react=q[key='1007467366491']"
============================================================
</code>```

sandbox example

Are there any more detailed examples for react out there?

答:
Playwright does not support key filtering at the moment. But you can filter for the job.id which is part of the props:

```<code>from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch()
    page = browser.new_page()
    page.goto("https://www.glassdoor.co.uk/Job/qa-engineer-jobs-SRCH_KO0,11.htm")
    page.locator("_react=q[job.id=1007630619432]").click()
    browser.close()
</code>```

Title: React locator example

Body:
I am trying to understand how the react selectors are working according to https://playwright.dev/docs/selectors#react-selectors . So I am trying some things in playwright sandbox. Seems that the react component cannot be found.

from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch()
    page = browser.new_page()
    page.goto("https://www.glassdoor.co.uk/Job/qa-engineer-jobs-SRCH_KO0,11.htm")
    page.locator("_react=q[key='1007467366491']").click()
    browser.close()

Error:

    playwright._impl._api_types.TimeoutError: Timeout 30000ms exceeded.
=========================== logs ===========================
waiting for selector "_react=q[key='1007467366491']"
============================================================

sandbox example

Are there any more detailed examples for react out there?

Body:
Playwright does not support key filtering at the moment. But you can filter for the job.id which is part of the props:

from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch()
    page = browser.new_page()
    page.goto("https://www.glassdoor.co.uk/Job/qa-engineer-jobs-SRCH_KO0,11.htm")
    page.locator("_react=q[job.id=1007630619432]").click()
    browser.close()


问:
How can I speed up my python query (large sets of JSON with multiple API calls)
I need to run a script that will log into multiple instances of a product and pull all of the data into a centralized list. There are 4 regions and each region may have 1,000 devices.

Each region is setup identical. If I want to crawl the list of devices, I first make an API call to /api/device which only returns a URI and description.

```<code>"result_set":[
{
"URI":"/api/device/1",
"description":"ardmore"
},
{
"URI":"/api/device/10",
"description":"oban"
}
]
</code>```

I then have to make an API call to /api/device/ which has a whole lot more data, including more pointers (URIs)

```<code>{
"name":"ardmore",
"ip":"1.2.3.4",
"hostname":null,
"snmp_cred_id":"/api/credential/snmp/79",
"snmp_w_cred_id":null,
"class_type":"/api/device_class/7A126AD829FF5CD0B2525A658D4EEB17",
"organization":"/api/organization/518"
}
</code>```

So now I finally have the device 'name' and 'ip', but if I want the "Class Type" (contains two values, 'Vendor' and 'Model) and "Organization" it's two more API calls.

It seems to me there has to be a way to just 'suck in' all of these tables into some list, and then just query things as I go. I'm still quite new to Python so I'm sure it's a dumb idea and there's some much better way I've never considered.

Modified script is below, this is enough to give you the jist of things. There are even more API calls etc, but this should be enough to show the overall flow.

```<code>def list_assets(Region):
    url = 'https://db01.' +str(Region)+ '.myhost.net/api/device?limit=3000&filter.0.state.eq=Active'
    response = requests.get(url,data=data,headers=headers,verify=False)
    data = response.json()
    return(data)

def getDetails(Region,URI):
    url = 'https://db01.' +str(Region)+ '.myhost.net/' +str(URI)
    response = requests.get(url,data=data,headers=headers,verify=False)
    data = response.json()
    return(data)

def getClass(Region,URI):
    url = 'https://db01.' +str(Region)+ '.myhost.net/' +str(URI)
    response = requests.get(url,data=data,headers=headers,verify=False)
    data = response.json()
    vendor = data['class']
    model = data['description']
    return(vendor,model)

def getCompany(Region,URI):
    url = 'https://db01.' +str(Region)+ '.myhost.net/' +str(URI)
    response = requests.get(url,data=data,headers=headers,verify=False)
    data = response.json()
    Company = data['company']
    return(Company)

regions = {"APAC","EU","US","AU"}

for region in regions:
    devices = list_assets(region)
    for device in devices['result_set']:
        URI = device['URI']
        details = getDetails(region,URI)
        hostName = details['name']
        IP = details['ip']
        ClassURI = details['class_type']
        Vendor,Model = getClass(region,ClassURI)
        OrgURI = details['organization']
        Company = getCompany(region,OrgURI)
        csvOut = [hostName,IP,Vendor,Model,region,Company]
        writer.writerow(csvOut)
</code>```

答:
Here is a Minimal Reproducible Example :

import time
from math import log10, ceil

import tqdm  # used only for displaying throughput

ACTUAL_TIME_PER_DEVICE = 5 / 4  # 5 seconds, split between 4 API calls
DEMO_TIME_SPEEDUP = 1000  # to make things faster for demo purposes
DEVICES_PER_REGION = 1000

def fake_list_assets(region):
    # fetching is slow ...
    time.sleep(ACTUAL_TIME_PER_DEVICE / DEMO_TIME_SPEEDUP)
    # returning 1000 fake device's ids
    return {
        "result_set": [
            {"URI": f"{region!s}_device_{str(i).zfill(ceil(log10(DEVICES_PER_REGION - 1)))}"}
            for i in range(DEVICES_PER_REGION)
        ]
    }

def fake_get_details(region, uri):
    time.sleep(ACTUAL_TIME_PER_DEVICE / DEMO_TIME_SPEEDUP)

    return {
        "name": f"fake name for {region=!r} {uri=!r}",
        "ip": f"fake ip for {region=!r} {uri=!r}",
        "class_type": f"fake class_type for {region=!r} {uri=!r}",
        "organization": f"fake organization for {region=!r} {uri=!r}",
    }

def fake_get_class(region, uri):
    time.sleep(ACTUAL_TIME_PER_DEVICE / DEMO_TIME_SPEEDUP)
    return f"fake vendor for {region=!r} {uri=!r}",\
           f"fake model for {region=!r} {uri=!r}"

def fake_get_company(region, uri):
    time.sleep(ACTUAL_TIME_PER_DEVICE / DEMO_TIME_SPEEDUP)
    return f"fake company for {region=!r} {uri=!r}"

counter = tqdm.tqdm(unit="devices")

regions = ["APAC", "EU", "US", "AU"]
for region in regions:
    devices = fake_list_assets(region)
    for device in devices['result_set']:
        URI = device['URI']
        details = fake_get_details(region, URI)
        hostName = details['name']
        IP = details['ip']
        ClassURI = details['class_type']
        Vendor, Model = fake_get_class(region, ClassURI)
        OrgURI = details['organization']
        Company = fake_get_company(region, OrgURI)
        # print([hostName, IP, Vendor, Model, region, Company])
        counter.update(1)  # add 1 to the counter
```
<p>which gives me :</p>
```4000devices [00:16, 245.69devices/s]
```
<p>Around 16 seconds but with a <code>1000</code> speedup factor, it means it would have actually taken around 4 hours and 30 minutes.</p>
<p>The problem is that you run your code serially, with much of the time waiting for the server to answer. This is a situation called <code>I/O bound</code> because your code runs as fast as the I/O it receives, in this case the time to receive each request's answer.</p>
<p>Ideally, we would like to be <code>CPU-bound</code>, which means that the computer is going as fast as it can, but it just can not go as fast as the server is answering the requests.</p>
<p>To do that, you need to make your requests <strong>parallel</strong>, either using <code>threading</code> or <code>multiprocessing</code>. It happens to be just the situation that <code>asyncio</code> is meant to solve.<br />
I use concurrent.futures because it makes things simpler to write :</p>
<pre class="lang-py prettyprint-override"><code>import concurrent.futures

counter = tqdm.tqdm(unit="devices")

regions = ["APAC", "EU", "US", "AU"]

def fetch_all_info_for_one_device(region, device_uri):
    details = fake_get_details(region, device_uri)
    ClassURI = details['class_type']
    OrgURI = details['organization']
    Vendor, Model = fake_get_class(region, ClassURI)
    Company = fake_get_company(region, OrgURI)
    hostName = details['name']
    IP = details['ip']
    return hostName, IP, Vendor, Model, region, Company

NUMBER_OF_WORKERS = 40  # you can change this value

# fetch the few regions to prepare all the data
all_devices_uris_to_fetch_by_region = {}
for region in regions:
    region_devices = fake_list_assets(region)
    all_devices_uris_to_fetch_by_region[region] = tuple(device["URI"] for device in region_devices["result_set"])

# create an executor which will run all the tasks submitted with `map` to its workers
with concurrent.futures.ThreadPoolExecutor(max_workers=NUMBER_OF_WORKERS) as executor:
    all_params = tuple(
        (region, device_uri)
        for region, devices_uris in all_devices_uris_to_fetch_by_region.items()
        for device_uri in devices_uris
    )
    for future in concurrent.futures.as_completed(
        executor.submit(fetch_all_info_for_one_device, *params)
        for params in all_params
    ):
        device_result = future.result()
        # print(device_result)
        counter.update(1)
</code>```
```<code>4000devices [00:00, 9099.61devices/s]
</code>```

It went too fast ! Reducing the demo time speedup : DEMO_TIME_SPEEDUP = 10 gives instead

```<code>4000devices [00:38, 105.04devices/s]
</code>```

At 10x speedup only, it means the real code would run in about 6 minutes. Assuming that the server can handle the load.

If you don't want to go parallel, maybe the server provides other APIs ? Instead of fetching the data uri by uri, maybe it is possible to give a list of uris and and receive a list of results ? It would have a lot of impact too I think.

Body:
I need to run a script that will log into multiple instances of a product and pull all of the data into a centralized list. There are 4 regions and each region may have 1,000 devices.

Each region is setup identical. If I want to crawl the list of devices, I first make an API call to /api/device which only returns a URI and description.

"result_set":[
{
"URI":"/api/device/1",
"description":"ardmore"
},
{
"URI":"/api/device/10",
"description":"oban"
}
]

I then have to make an API call to /api/device/ which has a whole lot more data, including more pointers (URIs)

{
"name":"ardmore",
"ip":"1.2.3.4",
"hostname":null,
"snmp_cred_id":"/api/credential/snmp/79",
"snmp_w_cred_id":null,
"class_type":"/api/device_class/7A126AD829FF5CD0B2525A658D4EEB17",
"organization":"/api/organization/518"
}

So now I finally have the device 'name' and 'ip', but if I want the "Class Type" (contains two values, 'Vendor' and 'Model) and "Organization" it's two more API calls.

It seems to me there has to be a way to just 'suck in' all of these tables into some list, and then just query things as I go. I'm still quite new to Python so I'm sure it's a dumb idea and there's some much better way I've never considered.

Modified script is below, this is enough to give you the jist of things. There are even more API calls etc, but this should be enough to show the overall flow.

def list_assets(Region):
    url = 'https://db01.' +str(Region)+ '.myhost.net/api/device?limit=3000&filter.0.state.eq=Active'
    response = requests.get(url,data=data,headers=headers,verify=False)
    data = response.json()
    return(data)

def getDetails(Region,URI):
    url = 'https://db01.' +str(Region)+ '.myhost.net/' +str(URI)
    response = requests.get(url,data=data,headers=headers,verify=False)
    data = response.json()
    return(data)

def getClass(Region,URI):
    url = 'https://db01.' +str(Region)+ '.myhost.net/' +str(URI)
    response = requests.get(url,data=data,headers=headers,verify=False)
    data = response.json()
    vendor = data['class']
    model = data['description']
    return(vendor,model)

def getCompany(Region,URI):
    url = 'https://db01.' +str(Region)+ '.myhost.net/' +str(URI)
    response = requests.get(url,data=data,headers=headers,verify=False)
    data = response.json()
    Company = data['company']
    return(Company)

regions = {"APAC","EU","US","AU"}

for region in regions:
    devices = list_assets(region)
    for device in devices['result_set']:
        URI = device['URI']
        details = getDetails(region,URI)
        hostName = details['name']
        IP = details['ip']
        ClassURI = details['class_type']
        Vendor,Model = getClass(region,ClassURI)
        OrgURI = details['organization']
        Company = getCompany(region,OrgURI)
        csvOut = [hostName,IP,Vendor,Model,region,Company]
        writer.writerow(csvOut)

Body:
Here is a Minimal Reproducible Example :

import time
from math import log10, ceil

import tqdm  # used only for displaying throughput

ACTUAL_TIME_PER_DEVICE = 5 / 4  # 5 seconds, split between 4 API calls
DEMO_TIME_SPEEDUP = 1000  # to make things faster for demo purposes
DEVICES_PER_REGION = 1000

def fake_list_assets(region):
    # fetching is slow ...
    time.sleep(ACTUAL_TIME_PER_DEVICE / DEMO_TIME_SPEEDUP)
    # returning 1000 fake device's ids
    return {
        "result_set": [
            {"URI": f"{region!s}_device_{str(i).zfill(ceil(log10(DEVICES_PER_REGION - 1)))}"}
            for i in range(DEVICES_PER_REGION)
        ]
    }

def fake_get_details(region, uri):
    time.sleep(ACTUAL_TIME_PER_DEVICE / DEMO_TIME_SPEEDUP)

    return {
        "name": f"fake name for {region=!r} {uri=!r}",
        "ip": f"fake ip for {region=!r} {uri=!r}",
        "class_type": f"fake class_type for {region=!r} {uri=!r}",
        "organization": f"fake organization for {region=!r} {uri=!r}",
    }

def fake_get_class(region, uri):
    time.sleep(ACTUAL_TIME_PER_DEVICE / DEMO_TIME_SPEEDUP)
    return f"fake vendor for {region=!r} {uri=!r}",\
           f"fake model for {region=!r} {uri=!r}"

def fake_get_company(region, uri):
    time.sleep(ACTUAL_TIME_PER_DEVICE / DEMO_TIME_SPEEDUP)
    return f"fake company for {region=!r} {uri=!r}"

counter = tqdm.tqdm(unit="devices")

regions = ["APAC", "EU", "US", "AU"]
for region in regions:
    devices = fake_list_assets(region)
    for device in devices['result_set']:
        URI = device['URI']
        details = fake_get_details(region, URI)
        hostName = details['name']
        IP = details['ip']
        ClassURI = details['class_type']
        Vendor, Model = fake_get_class(region, ClassURI)
        OrgURI = details['organization']
        Company = fake_get_company(region, OrgURI)
        # print([hostName, IP, Vendor, Model, region, Company])
        counter.update(1)  # add 1 to the counter

which gives me :

4000devices [00:16, 245.69devices/s]

Around 16 seconds but with a 1000 speedup factor, it means it would have actually taken around 4 hours and 30 minutes.

The problem is that you run your code serially, with much of the time waiting for the server to answer. This is a situation called I/O bound because your code runs as fast as the I/O it receives, in this case the time to receive each request's answer.

Ideally, we would like to be CPU-bound, which means that the computer is going as fast as it can, but it just can not go as fast as the server is answering the requests.

To do that, you need to make your requests parallel, either using threading or multiprocessing. It happens to be just the situation that asyncio is meant to solve.

I use concurrent.futures because it makes things simpler to write :

import concurrent.futures

counter = tqdm.tqdm(unit="devices")

regions = ["APAC", "EU", "US", "AU"]

def fetch_all_info_for_one_device(region, device_uri):
    details = fake_get_details(region, device_uri)
    ClassURI = details['class_type']
    OrgURI = details['organization']
    Vendor, Model = fake_get_class(region, ClassURI)
    Company = fake_get_company(region, OrgURI)
    hostName = details['name']
    IP = details['ip']
    return hostName, IP, Vendor, Model, region, Company

NUMBER_OF_WORKERS = 40  # you can change this value

# fetch the few regions to prepare all the data
all_devices_uris_to_fetch_by_region = {}
for region in regions:
    region_devices = fake_list_assets(region)
    all_devices_uris_to_fetch_by_region[region] = tuple(device["URI"] for device in region_devices["result_set"])

# create an executor which will run all the tasks submitted with `map` to its workers
with concurrent.futures.ThreadPoolExecutor(max_workers=NUMBER_OF_WORKERS) as executor:
    all_params = tuple(
        (region, device_uri)
        for region, devices_uris in all_devices_uris_to_fetch_by_region.items()
        for device_uri in devices_uris
    )
    for future in concurrent.futures.as_completed(
        executor.submit(fetch_all_info_for_one_device, *params)
        for params in all_params
    ):
        device_result = future.result()
        # print(device_result)
        counter.update(1)

4000devices [00:00, 9099.61devices/s]

It went too fast ! Reducing the demo time speedup : DEMO_TIME_SPEEDUP = 10 gives instead

4000devices [00:38, 105.04devices/s]

At 10x speedup only, it means the real code would run in about 6 minutes. Assuming that the server can handle the load.

If you don't want to go parallel, maybe the server provides other APIs ? Instead of fetching the data uri by uri, maybe it is possible to give a list of uris and and receive a list of results ? It would have a lot of impact too I think.


问:
Which is better to use: $arrayName['literal'] or $arrayName[literal]?
Both are working fine. I'm just curious how php parse the two. Do they have difference on speed, efficiency, etc. Why does php allow us to use both?</p>

答:
if you are using just literal php recognizes this as a constant. So it try to find that constant, and if it fails - it just assumes that your desire was using string literal 'literal'. But, to indicate that it doesn't found the constant it raises Notice level error.

So, using just literal have tweo drawbacks:

If you have constant literal defined - you'll get it's value (and this is a correct usage), not string 'literal'

You'll receive a Notice if you don't have such constant.

So, don't use just literal unless you have a constant with that name defined.


问:
404 on inner pages in subfolder WP installations (htaccess issue)
I have weird problem concerning .htaccess files and 404 errors on inner pages in subfolders. Happened without any changes yesterday.

I have WP installations on root domain and subfolders us domain.com/es/ and domain.com/fr/. Permalink structure was %postname% in all these 3 installations working with no problems before. For some reason after one night I woke up with all inner pages in subfolder websites fr and es throwing 404 errors.

After multiple troubleshooting, restarting servers, web servers, backups and so on, some progress was made. When I change .htaccess file in a root website with plain (default) permalinks. Basically empty .htaccess file. Then pages in subfolder websites started showing again. If I change root .htaccess file to %postname& permalinks subfolder pages broke again. I guess somehow root .htaccess file catches subfolder urls and rewrites them. Then I have added a line to the root .htaccess file to exclude fr and es folders

```<code>RewriteCond %{REQUEST_URI} !(fr|es) [NC]
</code>```

This solves a problem partially. Now subfolder website pages works fine, but the root website with permalink structure postname has plenty of broken links. The weird part that not all, but just a part of links are broken on the root website. They even do not load a custom 404 error page, simply server side 404 that resource is not found. If I change it to default links, the same page works ok, but in this case all my external backlinks will be broken. What else I could try?

答:
An issue solved modifying .htaccess file line as

```<code>RewriteCond %{REQUEST_URI} !^/es/.*
</code>```

instead of

```<code>RewriteCond %{REQUEST_URI}  !(es) [NC]
</code>```

to prevent overwriting subfolder links.

Body:
I have weird problem concerning .htaccess files and 404 errors on inner pages in subfolders. Happened without any changes yesterday.

I have WP installations on root domain and subfolders us domain.com/es/ and domain.com/fr/. Permalink structure was %postname% in all these 3 installations working with no problems before. For some reason after one night I woke up with all inner pages in subfolder websites fr and es throwing 404 errors.

After multiple troubleshooting, restarting servers, web servers, backups and so on, some progress was made. When I change .htaccess file in a root website with plain (default) permalinks. Basically empty .htaccess file. Then pages in subfolder websites started showing again. If I change root .htaccess file to %postname& permalinks subfolder pages broke again. I guess somehow root .htaccess file catches subfolder urls and rewrites them. Then I have added a line to the root .htaccess file to exclude fr and es folders

RewriteCond %{REQUEST_URI} !(fr|es) [NC]

This solves a problem partially. Now subfolder website pages works fine, but the root website with permalink structure postname has plenty of broken links. The weird part that not all, but just a part of links are broken on the root website. They even do not load a custom 404 error page, simply server side 404 that resource is not found. If I change it to default links, the same page works ok, but in this case all my external backlinks will be broken. What else I could try?

Body:
An issue solved modifying .htaccess file line as

RewriteCond %{REQUEST_URI} !^/es/.*

instead of

RewriteCond %{REQUEST_URI}  !(es) [NC]

to prevent overwriting subfolder links.