问:
How to Remove selected item from listbox C#
i currently trying to view all the files and folder selected by the user in a listbox. At the Moment i am able to list what the user have chosen using the openfiledialogue HOWEVER i am now facing prob when i try to remove it form the listbox. i trying to allow the user to click on the checkbox beside the file and press the remove button to remove it

this is my code for remove button

```<code>      private void button2_Click(object sender, EventArgs e)
    {
        for (int i = listView1.SelectedItems.Count - 1; i >= 0; i--)
        {
            listView1.Items.Remove(listView1.SelectedItems[i]);
        }

    }
</code>```

this is the add file to listbox for reference jsut in case

```<code>    private void button1_Click(object sender, EventArgs e)
    {

        OpenFileDialog openfiledialog = new OpenFileDialog();
        // Display open file dialog
        openfiledialog.InitialDirectory = "C:\\";
        //openfiledialog.Multiselect = true;
        openfiledialog.Title = "Lock File";
        openfiledialog.Filter = "All Files | *.*";
        openfiledialog.ShowDialog();

        if (openfiledialog.FileName != "")
        {

        //move through FileInfo array and store in new array of fi
            listView1.Items.Clear();
            foreach (string file in openfiledialog.FileNames)
            {
                listView1.Items.Add(file);
            }
        }

    }
</code>```

and i pressed the remove button nothing happen and i saw some answer on google on the using of selectionmode but when i used that, my listbox does not have the property of selectionmode and have red lines underlined

答:
Your problem is because the SelectedItems property is actually a reference to Items collection and you change the collection while iterating through it.
try the following code

```<code>listView1.BeginUpdate();
ArrayList vSelectedItems = new ArrayList(listView1.SelectedItems);
foreach (string item in vSelectedItems)
{
   listView1.Items.Remove(item);
}
listView1.EndUpdate();
</code>```

the BeginUpdate() and EndUpdate() methods will optimize the performance of this action - the listView will not refresh itself during activities between these methods call.

Body:
i currently trying to view all the files and folder selected by the user in a listbox. At the Moment i am able to list what the user have chosen using the openfiledialogue HOWEVER i am now facing prob when i try to remove it form the listbox. i trying to allow the user to click on the checkbox beside the file and press the remove button to remove it

this is my code for remove button

      private void button2_Click(object sender, EventArgs e)
    {
        for (int i = listView1.SelectedItems.Count - 1; i >= 0; i--)
        {
            listView1.Items.Remove(listView1.SelectedItems[i]);
        }

    }

this is the add file to listbox for reference jsut in case

    private void button1_Click(object sender, EventArgs e)
    {

        OpenFileDialog openfiledialog = new OpenFileDialog();
        // Display open file dialog
        openfiledialog.InitialDirectory = "C:\\";
        //openfiledialog.Multiselect = true;
        openfiledialog.Title = "Lock File";
        openfiledialog.Filter = "All Files | *.*";
        openfiledialog.ShowDialog();

        if (openfiledialog.FileName != "")
        {

        //move through FileInfo array and store in new array of fi
            listView1.Items.Clear();
            foreach (string file in openfiledialog.FileNames)
            {
                listView1.Items.Add(file);
            }
        }

    }

and i pressed the remove button nothing happen and i saw some answer on google on the using of selectionmode but when i used that, my listbox does not have the property of selectionmode and have red lines underlined

Body:
Your problem is because the SelectedItems property is actually a reference to Items collection and you change the collection while iterating through it.
try the following code

listView1.BeginUpdate();
ArrayList vSelectedItems = new ArrayList(listView1.SelectedItems);
foreach (string item in vSelectedItems)
{
   listView1.Items.Remove(item);
}
listView1.EndUpdate();

the BeginUpdate() and EndUpdate() methods will optimize the performance of this action - the listView will not refresh itself during activities between these methods call.


来源: stackexchange

问:
How can I split dataframe with blank spaces
```<code>import pandas
import numpy

names = ['a', 'b', 'c']
df = pandas.DataFrame([1, 2, 3, numpy.nan, numpy.nan, 4, 5, 6, numpy.nan, numpy.nan, 7, 8, 9])
</code>```

For the above one, how will the condition change? Can someone please explain?
how can I get this,

```<code>df1 =
     0
0  1.0
1  2.0
2  3.0
</code>```
```<code>df2 =
     0
4  4.0
5  5.0
6  6.0
</code>```
```<code>df3 =
      0
8   7.0
9   8.0
10  9.0
</code>```

答:
You can generate a temporary column, remove NaNs, and group by the temporary column:

```<code>dataframes = {f'df{idx+1}': d for idx, (_, d) in enumerate(df.dropna().groupby(df.assign(cond=df.isna().cumsum()).dropna()['cond']))}
</code>```

Output:

```<code>>>> dataframes
{'df1':      0
 0  1.0
 1  2.0
 2  3.0,
 'df2':      0
 5  4.0
 6  5.0
 7  6.0,
 'df3':       0
 10  7.0
 11  8.0
 12  9.0}

>>> dataframes['df1']
     0
0  1.0
1  2.0
2  3.0

>>> dataframes['df2']
     0
5  4.0
6  5.0
7  6.0

>>> dataframes['df3']
      0
10  7.0
11  8.0
12  9.0
</code>```

ContentLicense: CC BY-SA 4.0

Body:
import pandas
import numpy

names = ['a', 'b', 'c']
df = pandas.DataFrame([1, 2, 3, numpy.nan, numpy.nan, 4, 5, 6, numpy.nan, numpy.nan, 7, 8, 9])

For the above one, how will the condition change? Can someone please explain?
how can I get this,

df1 =
     0
0  1.0
1  2.0
2  3.0

df2 =
     0
4  4.0
5  5.0
6  6.0

df3 =
      0
8   7.0
9   8.0
10  9.0

Body:
You can generate a temporary column, remove NaNs, and group by the temporary column:

dataframes = {f'df{idx+1}': d for idx, (_, d) in enumerate(df.dropna().groupby(df.assign(cond=df.isna().cumsum()).dropna()['cond']))}

Output:

>>> dataframes
{'df1':      0
 0  1.0
 1  2.0
 2  3.0,
 'df2':      0
 5  4.0
 6  5.0
 7  6.0,
 'df3':       0
 10  7.0
 11  8.0
 12  9.0}

>>> dataframes['df1']
     0
0  1.0
1  2.0
2  3.0

>>> dataframes['df2']
     0
5  4.0
6  5.0
7  6.0

>>> dataframes['df3']
      0
10  7.0
11  8.0
12  9.0


问:
How to check if two table include the same information in Big Query?
Assume we have a table table_a with two string and one int fields:

```<code>CREATE TABLE `YOUR_DATABASE.table_a` (
  `A1` STRING,
  `A2` INT64,
  `A3` STRING,
);
</code>```

table_a includes some data:

```<code>INSERT `YOUR_DATABASE.table_a` (A1, A2, A3)
VALUES("X111", 10, "2022-02-07 08:19:00"),
      ("Y222", 15, "2017-05-08 10:19:00"),
      ("X111", 0,"2011-08-02 02:30:00"),
      ("X111", 12, "2019-12-20 11:25:00"),
      ("Y222", 258, "2001-10-14 07:00:00"),
      ("Z333", 4, "2015-03-08 03:25:12")
</code>```

We have another table, table_b, which is kind of identical to table_a. In fact, table_b has the exact same fields and values as in table_a and it also has an extra DATETIME field. This extra DATETIME field, B4, is essentially same as A3 but converted to a standard DATETIME format.

```<code>CREATE TABLE `YOUR_DATABASE.table_b` (
  `B1` STRING,
  `B2` INT64,
  `B3` STRING,
  `B4` DATETIME,
);
</code>```

And the values in table_b are:

```<code>INSERT `YOUR_DATABASE.table_b` (B1, B2, B3, B4)
VALUES("X111", 10, "2022-02-07 08:19:00", DATETIME(PARSE_TIMESTAMP("%Y-%m-%y %H:%M:%S","2022-02-07 08:19:00"),"America/Los_Angeles")),
      ("Y222", 15, "2017-05-08 10:19:00", DATETIME(PARSE_TIMESTAMP("%Y-%m-%y %H:%M:%S","2017-05-08 10:19:00"),"America/Los_Angeles")),
      ("X111", 0, "2011-08-02 02:30:00", DATETIME(PARSE_TIMESTAMP("%Y-%m-%y %H:%M:%S","2011-08-02 02:30:00"),"America/Los_Angeles")),
      ("X111", 12, "2019-12-20 11:25:00", DATETIME(PARSE_TIMESTAMP("%Y-%m-%y %H:%M:%S","2019-12-20 11:25:00"),"America/Los_Angeles")),
      ("Y222", 258, "2001-10-14 07:00:00", DATETIME(PARSE_TIMESTAMP("%Y-%m-%y %H:%M:%S","2001-10-14 07:00:00"),"America/Los_Angeles")),
      ("Z333", 4, "2015-03-08 03:25:12", DATETIME(PARSE_TIMESTAMP("%Y-%m-%y %H:%M:%S","2015-03-08 03:25:12"),"America/Los_Angeles"))
</code>```

Note, table_a and table_b may or may not have a primary key. How can I check if table_a and table_b include the same information? I have a couple of table_a in my database and I have created equivalent table_b for each of them; now I want to double check and make sure the tables include same information.

答:
Try the following:

```<code>SELECT
    *
FROM table_a
FULL OUTER JOIN table_b
 on FARM_FINGERPRINT(ARRAY_TO_STRING([A1, CAST(A2 AS STRING), A3],"")) = FARM_FINGERPRINT(ARRAY_TO_STRING([B1, CAST(B2 AS STRING), B3],""))
WHERE A1 IS NULL
OR B1 IS NULL
</code>```

This process concatenates and hashes the row from each table then returns where there is not a match on each side.

With the sample data provided it yields:

Body:
Assume we have a table table_a with two string and one int fields:

CREATE TABLE `YOUR_DATABASE.table_a` (
  `A1` STRING,
  `A2` INT64,
  `A3` STRING,
);

table_a includes some data:

INSERT `YOUR_DATABASE.table_a` (A1, A2, A3)
VALUES("X111", 10, "2022-02-07 08:19:00"),
      ("Y222", 15, "2017-05-08 10:19:00"),
      ("X111", 0,"2011-08-02 02:30:00"),
      ("X111", 12, "2019-12-20 11:25:00"),
      ("Y222", 258, "2001-10-14 07:00:00"),
      ("Z333", 4, "2015-03-08 03:25:12")

We have another table, table_b, which is kind of identical to table_a. In fact, table_b has the exact same fields and values as in table_a and it also has an extra DATETIME field. This extra DATETIME field, B4, is essentially same as A3 but converted to a standard DATETIME format.

CREATE TABLE `YOUR_DATABASE.table_b` (
  `B1` STRING,
  `B2` INT64,
  `B3` STRING,
  `B4` DATETIME,
);

And the values in table_b are:

INSERT `YOUR_DATABASE.table_b` (B1, B2, B3, B4)
VALUES("X111", 10, "2022-02-07 08:19:00", DATETIME(PARSE_TIMESTAMP("%Y-%m-%y %H:%M:%S","2022-02-07 08:19:00"),"America/Los_Angeles")),
      ("Y222", 15, "2017-05-08 10:19:00", DATETIME(PARSE_TIMESTAMP("%Y-%m-%y %H:%M:%S","2017-05-08 10:19:00"),"America/Los_Angeles")),
      ("X111", 0, "2011-08-02 02:30:00", DATETIME(PARSE_TIMESTAMP("%Y-%m-%y %H:%M:%S","2011-08-02 02:30:00"),"America/Los_Angeles")),
      ("X111", 12, "2019-12-20 11:25:00", DATETIME(PARSE_TIMESTAMP("%Y-%m-%y %H:%M:%S","2019-12-20 11:25:00"),"America/Los_Angeles")),
      ("Y222", 258, "2001-10-14 07:00:00", DATETIME(PARSE_TIMESTAMP("%Y-%m-%y %H:%M:%S","2001-10-14 07:00:00"),"America/Los_Angeles")),
      ("Z333", 4, "2015-03-08 03:25:12", DATETIME(PARSE_TIMESTAMP("%Y-%m-%y %H:%M:%S","2015-03-08 03:25:12"),"America/Los_Angeles"))

Note, table_a and table_b may or may not have a primary key. How can I check if table_a and table_b include the same information? I have a couple of table_a in my database and I have created equivalent table_b for each of them; now I want to double check and make sure the tables include same information.

Body:
Try the following:

SELECT
    *
FROM table_a
FULL OUTER JOIN table_b
 on FARM_FINGERPRINT(ARRAY_TO_STRING([A1, CAST(A2 AS STRING), A3],"")) = FARM_FINGERPRINT(ARRAY_TO_STRING([B1, CAST(B2 AS STRING), B3],""))
WHERE A1 IS NULL
OR B1 IS NULL

This process concatenates and hashes the row from each table then returns where there is not a match on each side.

With the sample data provided it yields:


问:
MySQL MATCH AGAINST not find results if an underscore is attached to the search term
I use full text indexing to find results faster and it works well except when the term i search for is attached to an underscore inside the database record.

My database records:

```<code>article.title
++++++++++++++++++++++++++++++

My article 123456 created
------------------------------
My article new_123456 created
------------------------------
My article 123456_new created
</code>```

My match against query:

MATCH(article.title) AGAINST ( "123456*" IN BOOLEAN MODE )

This query return only the first record and ignore the others since the term "123456" is attached to an underscore ( _ ), either before or after the term, the query will ignore the records.

What is the thing I did wrong and how to fix this problem?

答:
There are many things that can mess up FULLTEXT:

Punctuation

"stop words"

min word "length"

Language

It is sometimes best to edit the data before storing it.  In your case, replacing "_" with " " might be the 'right' solution.  That could be done either in your application code as you insert strings, or by using MySQL's REPLACE() as the string is INSERTed.

Body:
I use full text indexing to find results faster and it works well except when the term i search for is attached to an underscore inside the database record.

My database records:

article.title
++++++++++++++++++++++++++++++

My article 123456 created
------------------------------
My article new_123456 created
------------------------------
My article 123456_new created

My match against query:

MATCH(article.title) AGAINST ( "123456*" IN BOOLEAN MODE )

This query return only the first record and ignore the others since the term "123456" is attached to an underscore ( _ ), either before or after the term, the query will ignore the records.

What is the thing I did wrong and how to fix this problem?


问:
How to fix "MetadataFetchFailedException: Missing an output location for shuffle"?
If I increase the model size of my word2vec model I start to get this kind of exception in my log:

```<code>org.apache.spark.shuffle.MetadataFetchFailedException: Missing an output location for shuffle 6
    at org.apache.spark.MapOutputTracker$$anonfun$org$apache$spark$MapOutputTracker$$convertMapStatuses$2.apply(MapOutputTracker.scala:542)
    at org.apache.spark.MapOutputTracker$$anonfun$org$apache$spark$MapOutputTracker$$convertMapStatuses$2.apply(MapOutputTracker.scala:538)
    at scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:772)
    at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
    at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:108)
    at scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:771)
    at org.apache.spark.MapOutputTracker$.org$apache$spark$MapOutputTracker$$convertMapStatuses(MapOutputTracker.scala:538)
    at org.apache.spark.MapOutputTracker.getMapSizesByExecutorId(MapOutputTracker.scala:155)
    at org.apache.spark.shuffle.BlockStoreShuffleReader.read(BlockStoreShuffleReader.scala:47)
    at org.apache.spark.rdd.ShuffledRDD.compute(ShuffledRDD.scala:98)
    at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
    at org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
    at org.apache.spark.rdd.CoalescedRDD$$anonfun$compute$1.apply(CoalescedRDD.scala:96)
    at org.apache.spark.rdd.CoalescedRDD$$anonfun$compute$1.apply(CoalescedRDD.scala:95)
    at scala.collection.Iterator$$anon$13.hasNext(Iterator.scala:371)
    at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:327)
    at scala.collection.Iterator$class.foreach(Iterator.scala:727)
    at scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
    at scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:48)
    at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:103)
    at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:47)
    at scala.collection.TraversableOnce$class.to(TraversableOnce.scala:273)
    at scala.collection.AbstractIterator.to(Iterator.scala:1157)
    at scala.collection.TraversableOnce$class.toBuffer(TraversableOnce.scala:265)
    at scala.collection.AbstractIterator.toBuffer(Iterator.scala:1157)
    at scala.collection.TraversableOnce$class.toArray(TraversableOnce.scala:252)
    at scala.collection.AbstractIterator.toArray(Iterator.scala:1157)
    at org.apache.spark.rdd.RDD$$anonfun$collect$1$$anonfun$12.apply(RDD.scala:927)
    at org.apache.spark.rdd.RDD$$anonfun$collect$1$$anonfun$12.apply(RDD.scala:927)
    at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1858)
    at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1858)
    at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
    at org.apache.spark.scheduler.Task.run(Task.scala:89)
    at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
</code>```

I tried to write my own "save model" version which looks like this:

  def save(model: Word2VecModel, sc: SparkContext, path: String): Unit = {

    println("Saving model as CSV ..")

    val vectorSize = model.getVectors.values.head.size

    println("vectorSize="+vectorSize)

    val SEPARATOR_TOKEN = " "
    val dataArray = model.getVectors.toSeq.map { case (w, v) => Data(w, v) }

    println("Got dataArray ..")
    println("parallelize(dataArray, 10)")
    val par = sc.parallelize(dataArray, 10)
          .map(d => {

            val sb = new mutable.StringBuilder()
            sb.append(d.word)
            sb.append(SEPARATOR_TOKEN)

            for(v <- d.vector) {
              sb.append(v)
              sb.append(SEPARATOR_TOKEN)
            }
            sb.setLength(sb.length - 1)
            sb.append("\n")
            sb.toString()
          })
    println("repartition(1)")
    val rep = par.repartition(1)
    println("collect()")
    val vectorsAsString = rep.collect()

    println("Collected serialized vectors ..")

    val cfile = new mutable.StringBuilder()

    cfile.append(vectorsAsString.length)
    cfile.append(" ")
    cfile.append(vectorSize)
    cfile.append("\n")

    val sb = new StringBuilder
    sb.append("word,")
    for(i <- 0 until vectorSize) {
      sb.append("v")
      sb.append(i.toString)
      sb.append(",")
    }
    sb.setLength(sb.length - 1)
    sb.append("\n")

    for(vectorString <- vectorsAsString) {
      sb.append(vectorString)
      cfile.append(vectorString)
    }

    println("Saving file to " + new Path(path, "data").toUri.toString)
    sc.parallelize(sb.toString().split("\n"), 1).saveAsTextFile(new Path(path+".csv", "data").toUri.toString)
    sc.parallelize(cfile.toString().split("\n"), 1).saveAsTextFile(new Path(path+".cs", "data").toUri.toString)
  }
```

Apparently it's working similar to their current implementation - it doesn't.

I'd like to get a word2vec model. It works with small files but not if the model gets larger.

答:
In short, configs might help:

```<code>--conf spark.blacklist.enabled=true # blacklist bad machine
--conf spark.reducer.maxReqsInFlight=10 # limit concurrent requests from reducer
--conf spark.shuffle.io.retryWait=10s # increase retry wait
--conf spark.shuffle.io.maxRetries=10 # increase retry times
--conf spark.shuffle.io.backLog=4096 # increase tcp connection wait queue length
</code>```

Long explainations as below.

For MetadataFetchFailedException, it usually happens when one executor suddenly being killed or terminated, but this executor has some shuffle output, then when another executor try to fetch metadata of this shuffle output, exception happens.

In most cases, this is caused by container killed by Yarn for
exceeding memory limits. So you need to double confirm this in the
logs.

The most common fix is to increase memoryOverhead, the default value
is 0.1 * executor memory. This is too small for most cases. I would
suggest to make it 0.2 * executor memory. If you have large amount
of executors or run another sub-process, you need bigger value for
this.

ContentLicense: CC BY-SA 3.0

Body:
If I increase the model size of my word2vec model I start to get this kind of exception in my log:

org.apache.spark.shuffle.MetadataFetchFailedException: Missing an output location for shuffle 6
    at org.apache.spark.MapOutputTracker$$anonfun$org$apache$spark$MapOutputTracker$$convertMapStatuses$2.apply(MapOutputTracker.scala:542)
    at org.apache.spark.MapOutputTracker$$anonfun$org$apache$spark$MapOutputTracker$$convertMapStatuses$2.apply(MapOutputTracker.scala:538)
    at scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:772)
    at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
    at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:108)
    at scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:771)
    at org.apache.spark.MapOutputTracker$.org$apache$spark$MapOutputTracker$$convertMapStatuses(MapOutputTracker.scala:538)
    at org.apache.spark.MapOutputTracker.getMapSizesByExecutorId(MapOutputTracker.scala:155)
    at org.apache.spark.shuffle.BlockStoreShuffleReader.read(BlockStoreShuffleReader.scala:47)
    at org.apache.spark.rdd.ShuffledRDD.compute(ShuffledRDD.scala:98)
    at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
    at org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
    at org.apache.spark.rdd.CoalescedRDD$$anonfun$compute$1.apply(CoalescedRDD.scala:96)
    at org.apache.spark.rdd.CoalescedRDD$$anonfun$compute$1.apply(CoalescedRDD.scala:95)
    at scala.collection.Iterator$$anon$13.hasNext(Iterator.scala:371)
    at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:327)
    at scala.collection.Iterator$class.foreach(Iterator.scala:727)
    at scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
    at scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:48)
    at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:103)
    at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:47)
    at scala.collection.TraversableOnce$class.to(TraversableOnce.scala:273)
    at scala.collection.AbstractIterator.to(Iterator.scala:1157)
    at scala.collection.TraversableOnce$class.toBuffer(TraversableOnce.scala:265)
    at scala.collection.AbstractIterator.toBuffer(Iterator.scala:1157)
    at scala.collection.TraversableOnce$class.toArray(TraversableOnce.scala:252)
    at scala.collection.AbstractIterator.toArray(Iterator.scala:1157)
    at org.apache.spark.rdd.RDD$$anonfun$collect$1$$anonfun$12.apply(RDD.scala:927)
    at org.apache.spark.rdd.RDD$$anonfun$collect$1$$anonfun$12.apply(RDD.scala:927)
    at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1858)
    at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1858)
    at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
    at org.apache.spark.scheduler.Task.run(Task.scala:89)
    at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

I tried to write my own "save model" version which looks like this:

  def save(model: Word2VecModel, sc: SparkContext, path: String): Unit = {

    println("Saving model as CSV ..")

    val vectorSize = model.getVectors.values.head.size

    println("vectorSize="+vectorSize)

    val SEPARATOR_TOKEN = " "
    val dataArray = model.getVectors.toSeq.map { case (w, v) => Data(w, v) }

    println("Got dataArray ..")
    println("parallelize(dataArray, 10)")
    val par = sc.parallelize(dataArray, 10)
          .map(d => {

            val sb = new mutable.StringBuilder()
            sb.append(d.word)
            sb.append(SEPARATOR_TOKEN)

            for(v <- d.vector) {
              sb.append(v)
              sb.append(SEPARATOR_TOKEN)
            }
            sb.setLength(sb.length - 1)
            sb.append("\n")
            sb.toString()
          })
    println("repartition(1)")
    val rep = par.repartition(1)
    println("collect()")
    val vectorsAsString = rep.collect()

    println("Collected serialized vectors ..")

    val cfile = new mutable.StringBuilder()

    cfile.append(vectorsAsString.length)
    cfile.append(" ")
    cfile.append(vectorSize)
    cfile.append("\n")

    val sb = new StringBuilder
    sb.append("word,")
    for(i <- 0 until vectorSize) {
      sb.append("v")
      sb.append(i.toString)
      sb.append(",")
    }
    sb.setLength(sb.length - 1)
    sb.append("\n")

    for(vectorString <- vectorsAsString) {
      sb.append(vectorString)
      cfile.append(vectorString)
    }

    println("Saving file to " + new Path(path, "data").toUri.toString)
    sc.parallelize(sb.toString().split("\n"), 1).saveAsTextFile(new Path(path+".csv", "data").toUri.toString)
    sc.parallelize(cfile.toString().split("\n"), 1).saveAsTextFile(new Path(path+".cs", "data").toUri.toString)
  }

Apparently it's working similar to their current implementation - it doesn't.

I'd like to get a word2vec model. It works with small files but not if the model gets larger.

Body:
In short, configs might help:

--conf spark.blacklist.enabled=true # blacklist bad machine
--conf spark.reducer.maxReqsInFlight=10 # limit concurrent requests from reducer
--conf spark.shuffle.io.retryWait=10s # increase retry wait
--conf spark.shuffle.io.maxRetries=10 # increase retry times
--conf spark.shuffle.io.backLog=4096 # increase tcp connection wait queue length

Long explainations as below.

For MetadataFetchFailedException, it usually happens when one executor suddenly being killed or terminated, but this executor has some shuffle output, then when another executor try to fetch metadata of this shuffle output, exception happens.

In most cases, this is caused by container killed by Yarn for
exceeding memory limits. So you need to double confirm this in the
logs.

The most common fix is to increase memoryOverhead, the default value
is 0.1 * executor memory. This is too small for most cases. I would
suggest to make it 0.2 * executor memory. If you have large amount
of executors or run another sub-process, you need bigger value for
this.


问:
How stop after running react-scripts start?
I started a React app with
npm start
with start defined in package.json:

{
  "name": "testreactapp",
  "version": "0.1.0",
  "private": true,
  "dependencies": {
    "react": "^15.6.1",
    "react-dom": "^15.6.1",
    "react-scripts": "1.0.10"
  },
  "scripts": {
    "start": "react-scripts start",
    "build": "react-scripts build",
    "test": "react-scripts test --env=jsdom",
    "eject": "react-scripts eject"
  }
}```
</div>
</div>
</p>
<p>I want to stop it now, without closing the terminal. How do I do that?</p>
<p>Tried:
npm stop testrectapp
but that throws error that it needs a script</p>
<p>Then tried:
npm run stop with script "stop": "pkill --signal SIGINT testreactapp"
that throws error 'pkill is not recognized as a command'</p>
<h1>Edit</h1>
<p>Running <code>ps</code> in <code>bash</code> shows:</p>
```      PID    PPID    PGID     WINPID   TTY         UID    STIME COMMAND
     6652       1    6652       6652  ?         197612 19:52:49 /usr/bin/mintty
     1092       1    1092       1092  ?         197612   Jul 25 /usr/bin/mintty
    11092    6652   11092      10060  pty1      197612 19:52:49 /usr/bin/bash
    13868    1092   13868        992  pty0      197612   Jul 25 /usr/bin/bash
    11428   13868   11428      17340  pty0      197612 12:48:27 /usr/bin/ps
    11672    1092    1092      11672  ?         197612   Jul 25 /usr/bin/mintty <defunct>
```

Don't see the app there?

答:
I am having the same problem on Mac with a terminal started in VS code.

CTRL C kills the node server however a vscode process remains attached to the port afterwards and prevents restarting on the same port.

The following workaround works for me on Mac

npx kill-port 3000

You can call this from another terminal and it should kill the node server and any other related vscode processes attached to the port and allow you to restart the server on the same port.

You can also add a script in package.json:

"stop": "npx kill-port 3000"

Then call yarn stop to stop your server

Body:
I started a React app with
npm start
with start defined in package.json:

{
  "name": "testreactapp",
  "version": "0.1.0",
  "private": true,
  "dependencies": {
    "react": "^15.6.1",
    "react-dom": "^15.6.1",
    "react-scripts": "1.0.10"
  },
  "scripts": {
    "start": "react-scripts start",
    "build": "react-scripts build",
    "test": "react-scripts test --env=jsdom",
    "eject": "react-scripts eject"
  }
}

I want to stop it now, without closing the terminal. How do I do that?

Tried:
npm stop testrectapp
but that throws error that it needs a script

Then tried:
npm run stop with script "stop": "pkill --signal SIGINT testreactapp"
that throws error 'pkill is not recognized as a command'

Edit

Running ps in bash shows:

      PID    PPID    PGID     WINPID   TTY         UID    STIME COMMAND
     6652       1    6652       6652  ?         197612 19:52:49 /usr/bin/mintty
     1092       1    1092       1092  ?         197612   Jul 25 /usr/bin/mintty
    11092    6652   11092      10060  pty1      197612 19:52:49 /usr/bin/bash
    13868    1092   13868        992  pty0      197612   Jul 25 /usr/bin/bash
    11428   13868   11428      17340  pty0      197612 12:48:27 /usr/bin/ps
    11672    1092    1092      11672  ?         197612   Jul 25 /usr/bin/mintty <defunct>

Don't see the app there?


问:
View button and routing in Angular
I think I have a road that is not good. In the Market > Value section, when I click on the Add button, I do not see the addition.component.html page. I have the impression that the button does not work?!

image

value.component.html

```<code><div class="text-center">
   <button type="submit" class="btn btn-primary" (click)="btnAjout()">Add</button>
</div>
</code>```

market-routing.module.ts

```<code>export const MARKET_ROUTES: Routes = [
  {
    path: '',
    pathMatch: 'full',
    redirectTo: '/value',
  },

  {
    path: 'value',
    component: ValueComponent,
  },
  {
    path: 'ajout',
    component: AjoutComponent,
  },

];
</code>```

market.module.ts

```<code>@NgModule({
  imports: [CommonModule, MarketRoutingModule,  PipesModule,],
  declarations: [MarketComponent, ValueComponent],
})
export class MarketModule { }
</code>```

value.module.ts

```<code>@NgModule({
  declarations: [
    AjoutComponent
  ],
  imports: [
    CommonModule
  ]
})
export class ValueModule { }
</code>```

Here is an illustration on Stackblitz.
User toto & Password 1234.

答:
In you code, when you navigateByUrl to ajout, it replaces the entire route with ajout. On click of the button use the below routing.

```<code> btnAjout(): void {
    this.router.navigateByUrl('dashboard/market/ajout');
  }
</code>```

Here is a Working StackBlitz

Title: View button and routing in Angular

Body:
I think I have a road that is not good. In the Market > Value section, when I click on the Add button, I do not see the addition.component.html page. I have the impression that the button does not work?!

image

value.component.html

   <button type="submit" class="btn btn-primary" (click)="btnAjout()">Add</button>

market-routing.module.ts

export const MARKET_ROUTES: Routes = [
  {
    path: '',
    pathMatch: 'full',
    redirectTo: '/value',
  },

  {
    path: 'value',
    component: ValueComponent,
  },
  {
    path: 'ajout',
    component: AjoutComponent,
  },

];

market.module.ts

@NgModule({
  imports: [CommonModule, MarketRoutingModule,  PipesModule,],
  declarations: [MarketComponent, ValueComponent],
})
export class MarketModule { }

value.module.ts

@NgModule({
  declarations: [
    AjoutComponent
  ],
  imports: [
    CommonModule
  ]
})
export class ValueModule { }

Here is an illustration on Stackblitz.
User toto & Password 1234.

Body:
In you code, when you navigateByUrl to ajout, it replaces the entire route with ajout. On click of the button use the below routing.

 btnAjout(): void {
    this.router.navigateByUrl('dashboard/market/ajout');
  }

Here is a Working StackBlitz


问:
'UFuncTypeError' regarding dtype('int64') and dtype('<m8[s]') working with numpy
I'm using this Python script:

```<code>import pandas as pd
import numpy as np
from utils import timedelta64_to_secs

if __name__ == '__main__':
    data_frame = pd.read_csv("/path/to/file.csv", nrows=1000000, skiprows=0, header=0)
    data_array = data_frame.iloc[:, 1]

    activations = get_activations(data_array, min_off_duration=30, min_on_duration=12, border=1, on_power_threshold=200)
</code>```

where get_activations() is defined here.

At line 954, the function timedelta64_to_secs() is called; it generates the error:

```<code>File "C:\Users\username\Documents\PyCharmProjects\NILMTK\nilmtk\utils.py", line 65, in timedelta64_to_secs
return timedelta / np.timedelta64(1, 's')
numpy.core._exceptions.UFuncTypeError: ufunc 'true_divide' cannot use operands with types dtype('int64') and dtype('<m8[s]')
</code>```

The definition of timedelta64_to_secs() (which can be found here) is:

```<code>def timedelta64_to_secs(timedelta):
    """Convert `timedelta` to seconds.

    Parameters | timedelta : np.timedelta64

    Returns | float : seconds
    """
    if len(timedelta) == 0:
       return np.array([])
    else:
        return timedelta / np.timedelta64(1, 's')
</code>```

where timedelta is a 712-element numpy array, while np.timedelta64(1, 's') is timedelta64 type.

How could I solve?

答:
The function should work with a proper timedelta input:

```<code>In [24]: x=np.array([3,2,1], 'datetime64[m]')
In [25]: x
Out[25]:
array(['1970-01-01T00:03', '1970-01-01T00:02', '1970-01-01T00:01'],
      dtype='datetime64[m]')
In [26]: y=x[:-1]-x[1:]
In [27]: y
Out[27]: array([1, 1], dtype='timedelta64[m]')
In [28]: y/np.timedelta64(1,'s')
Out[28]: array([60., 60.])
</code>```

your error:

```<code>In [29]: 1/np.timedelta64(1,'s')
Traceback (most recent call last):
  File "<ipython-input-29-d9febc47850f>", line 1, in <module>
    1/np.timedelta64(1,'s')
UFuncTypeError: ufunc 'true_divide' cannot use operands with types dtype('int64') and dtype('<m8[s]')
</code>```

another converter option:

```<code>In [31]: y.astype('timedelta64[s]')
Out[31]: array([60, 60], dtype='timedelta64[s]')
</code>```

Body:
I'm using this Python script:

import pandas as pd
import numpy as np
from utils import timedelta64_to_secs

if __name__ == '__main__':
    data_frame = pd.read_csv("/path/to/file.csv", nrows=1000000, skiprows=0, header=0)
    data_array = data_frame.iloc[:, 1]

    activations = get_activations(data_array, min_off_duration=30, min_on_duration=12, border=1, on_power_threshold=200)

where get_activations() is defined here.

At line 954, the function timedelta64_to_secs() is called; it generates the error:

File "C:\Users\username\Documents\PyCharmProjects\NILMTK\nilmtk\utils.py", line 65, in timedelta64_to_secs
return timedelta / np.timedelta64(1, 's')
numpy.core._exceptions.UFuncTypeError: ufunc 'true_divide' cannot use operands with types dtype('int64') and dtype('<m8[s]')

The definition of timedelta64_to_secs() (which can be found here) is:

def timedelta64_to_secs(timedelta):
    """Convert `timedelta` to seconds.

    Parameters | timedelta : np.timedelta64

    Returns | float : seconds
    """
    if len(timedelta) == 0:
       return np.array([])
    else:
        return timedelta / np.timedelta64(1, 's')

where timedelta is a 712-element numpy array, while np.timedelta64(1, 's') is timedelta64 type.

How could I solve?

Body:
The function should work with a proper timedelta input:

In [24]: x=np.array([3,2,1], 'datetime64[m]')
In [25]: x
Out[25]:
array(['1970-01-01T00:03', '1970-01-01T00:02', '1970-01-01T00:01'],
      dtype='datetime64[m]')
In [26]: y=x[:-1]-x[1:]
In [27]: y
Out[27]: array([1, 1], dtype='timedelta64[m]')
In [28]: y/np.timedelta64(1,'s')
Out[28]: array([60., 60.])

your error:

In [29]: 1/np.timedelta64(1,'s')
Traceback (most recent call last):
  File "<ipython-input-29-d9febc47850f>", line 1, in <module>
    1/np.timedelta64(1,'s')
UFuncTypeError: ufunc 'true_divide' cannot use operands with types dtype('int64') and dtype('<m8[s]')

another converter option:

In [31]: y.astype('timedelta64[s]')
Out[31]: array([60, 60], dtype='timedelta64[s]')


问:
How to use variables in dot notation like square bracket notation
I have been using square bracket notation in Javascript to create and call associative arrays.

In this example, I understand that square bracket notation allows you to use a variable to call a certain object in the array.

How would you do something like this in dot notation?

```<code>var item = {};
    item['1'] = 'pen';

var x = 1;

console.log(item[x]);  // console will show 'pen'
</code>```

答:
You can't use variables in dot notation (short of using eval, which you don't want to do). With dot notation the property name is essentially a constant.

```<code>myObj.propName
// is equivalent to
myObj["propName"]
</code>```

Body:
I have been using square bracket notation in Javascript to create and call associative arrays.

In this example, I understand that square bracket notation allows you to use a variable to call a certain object in the array.

How would you do something like this in dot notation?

var item = {};
    item['1'] = 'pen';

var x = 1;

console.log(item[x]);  // console will show 'pen'

Body:
You can't use variables in dot notation (short of using eval, which you don't want to do). With dot notation the property name is essentially a constant.

myObj.propName
// is equivalent to
myObj["propName"]


问:
Never[] is missing some properties
I got a simple JSON response with a single object. And I like to view this on the HTML page. But I got the error:

src/app/games/game.component.ts:35:7 - error TS2740: Type 'never[]' is missing the following properties from type 'GAMEDETAIL': id, title, releasedate, description, and 6 more

import { Component, OnInit } from '@angular/core';
import { Router, ActivatedRoute, ParamMap } from '@angular/router';
import { HttpClient } from '@angular/common/http';
import { Observable, throwError } from 'rxjs';

interface GAMEDETAIL {
  id: number,
  title: string
  releasedate: number
  description: string,
  adddate: string,
  changedate: string,
  pdffile: string,
  youtubelink: string,
  images: any,
  producer: any
}

@Component({
  selector: 'app-game',
  templateUrl: './game.component.html',
  styleUrls: ['./game.component.css']
})

export class GameComponent implements OnInit {

  private data:any = [];
  gamedetail:GAMEDETAIL;
  gameid: number = 0;

    constructor(private route: ActivatedRoute, private http:HttpClient) {
      **this.gamedetail = [];**
    }

    ngOnInit(): void {
      this.gameid = parseInt( this.route.snapshot.paramMap.get('id') as string );

      this.getJSON(this.gameid).subscribe(data => {
        console.log(this.gamedetail.title);

    });
    }

    getJSON(spielid: Number): Observable<GAMEDETAIL[]> {
      return this.http.get<GAMEDETAIL[]>("https://example.org/api/gamedetail/" + spielid);
    }

}
```
<p>I got in the console the following JSON-Data:</p>
```{
    "id":1,
    "title":"Ligretto",
    "releasedate":2000,
    "producer":[
    {
        "company":"Schmidt",
        "url":"https://example.org"
    }
    ],
    "pdffile":"https://example.org",
    "discription":"Das ist die Beschreibung zum Spiel",
    "images":[
    {
        "position":0,
        "imagefile":"https://example.org"
    },
    {
        "position":1,
        "imagefile":"https://example.org"
    }
    ],
    "youtubelink":"https://example.org",
    "adddate":"2021-12-22 22:22:44",
    "changedate":"2022-01-11 11:11:20"
}
```
<p>I like to check if I can log with the console the following line:</p>
<pre class="lang-js prettyprint-override"><code>console.log(this.gamedetail.title);
</code>```

But I can't compile it.

答:
From your attached JSON, it is a single object.

There are a few errors to be pointed out:

Error 1: gamedetail declared as GAMEDETAIL type, but assigned with [] (empty array).

You shouldn't assign gamedetail with [] as it conflicts with its type.

gamedetail:GAMEDETAIL;
```
<pre class="lang-js prettyprint-override"><code>constructor(private route: ActivatedRoute, private http:HttpClient) {
  **this.gamedetail = [];**
}
</code>```

Solution for Error 1

Remove the assigned value from constructor.

constructor(private route: ActivatedRoute, private http:HttpClient) { }
```
<p>Side note: You should perform initialization in <code>ngOnInit()</code>  instead of in <code>constructor</code>, check out this Difference between Constructor and ngOnInit.</p>
<hr />
<p><strong>Error 2: <code>http.get</code> should expect to receive <code>GAMEDETAIL</code> object.</strong></p>
<p>As your JSON returns single object, <code>http.get</code> shouldn't expect to receive an <code>GAMEDETAIL</code> array and return <code>Observable</code> of <code>GAMEDETAIL</code> array.</p>
<pre class="lang-js prettyprint-override"><code>getJSON(spielid: Number): Observable<GAMEDETAIL[]> {
  return this.http.get<GAMEDETAIL[]>("https://example.org/api/gamedetail/" + spielid);
}
</code>```

Solution for Error 2

Change the getJSON method as below:

getJSON(spielid: Number): Observable<GAMEDETAIL> {
  return this.http.get<GAMEDETAIL>("https://example.org/api/gamedetail/" + spielid);
}
```
<hr />
<p><strong>Error 3: Missing assign the value to <code>gamedetail</code> in <code>subscribe</code>.</strong></p>
<pre class="lang-js prettyprint-override"><code>this.getJSON(this.gameid).subscribe((data) => {
  console.log(this.gamedetail.title);
});
</code>```

Solution for Error 3

Add missing assign value to gamedetail.

this.getJSON(this.gameid).subscribe((data) => {
  this.gamedetail = data;
  console.log(this.gamedetail.title);
});
```

Sample Demo on StackBlitz

Title: Never[] is missing some properties

Body:
I got a simple JSON response with a single object. And I like to view this on the HTML page. But I got the error:

src/app/games/game.component.ts:35:7 - error TS2740: Type 'never[]' is missing the following properties from type 'GAMEDETAIL': id, title, releasedate, description, and 6 more

import { Component, OnInit } from '@angular/core';
import { Router, ActivatedRoute, ParamMap } from '@angular/router';
import { HttpClient } from '@angular/common/http';
import { Observable, throwError } from 'rxjs';

interface GAMEDETAIL {
  id: number,
  title: string
  releasedate: number
  description: string,
  adddate: string,
  changedate: string,
  pdffile: string,
  youtubelink: string,
  images: any,
  producer: any
}

@Component({
  selector: 'app-game',
  templateUrl: './game.component.html',
  styleUrls: ['./game.component.css']
})

export class GameComponent implements OnInit {

  private data:any = [];
  gamedetail:GAMEDETAIL;
  gameid: number = 0;

    constructor(private route: ActivatedRoute, private http:HttpClient) {
      **this.gamedetail = [];**
    }

    ngOnInit(): void {
      this.gameid = parseInt( this.route.snapshot.paramMap.get('id') as string );

      this.getJSON(this.gameid).subscribe(data => {
        console.log(this.gamedetail.title);

    });
    }

    getJSON(spielid: Number): Observable<GAMEDETAIL[]> {
      return this.http.get<GAMEDETAIL[]>("https://example.org/api/gamedetail/" + spielid);
    }

}

I got in the console the following JSON-Data:

{
    "id":1,
    "title":"Ligretto",
    "releasedate":2000,
    "producer":[
    {
        "company":"Schmidt",
        "url":"https://example.org"
    }
    ],
    "pdffile":"https://example.org",
    "discription":"Das ist die Beschreibung zum Spiel",
    "images":[
    {
        "position":0,
        "imagefile":"https://example.org"
    },
    {
        "position":1,
        "imagefile":"https://example.org"
    }
    ],
    "youtubelink":"https://example.org",
    "adddate":"2021-12-22 22:22:44",
    "changedate":"2022-01-11 11:11:20"
}

I like to check if I can log with the console the following line:

console.log(this.gamedetail.title);

But I can't compile it.

Body:
From your attached JSON, it is a single object.

There are a few errors to be pointed out:

Error 1: gamedetail declared as GAMEDETAIL type, but assigned with [] (empty array).

You shouldn't assign gamedetail with [] as it conflicts with its type.

gamedetail:GAMEDETAIL;

constructor(private route: ActivatedRoute, private http:HttpClient) {
  **this.gamedetail = [];**
}

Solution for Error 1

Remove the assigned value from constructor.

constructor(private route: ActivatedRoute, private http:HttpClient) { }

Side note: You should perform initialization in ngOnInit()  instead of in constructor, check out this Difference between Constructor and ngOnInit.

Error 2: http.get should expect to receive GAMEDETAIL object.

As your JSON returns single object, http.get shouldn't expect to receive an GAMEDETAIL array and return Observable of GAMEDETAIL array.

getJSON(spielid: Number): Observable<GAMEDETAIL[]> {
  return this.http.get<GAMEDETAIL[]>("https://example.org/api/gamedetail/" + spielid);
}

Solution for Error 2

Change the getJSON method as below:

getJSON(spielid: Number): Observable<GAMEDETAIL> {
  return this.http.get<GAMEDETAIL>("https://example.org/api/gamedetail/" + spielid);
}

Error 3: Missing assign the value to gamedetail in subscribe.

this.getJSON(this.gameid).subscribe((data) => {
  console.log(this.gamedetail.title);
});

Solution for Error 3

Add missing assign value to gamedetail.

this.getJSON(this.gameid).subscribe((data) => {
  this.gamedetail = data;
  console.log(this.gamedetail.title);
});

Sample Demo on StackBlitz


问:
How to deal with complex list and dictionary on python
I've been working on python to make a program which need to handle complex problem from list of dict. The thing is I need to transform this data into dictionary and sort it. The input for this function is come from trees. The code I share here is working, but takes a long time to run. In here I wanna ask is there any idea to make this function run more faster in python? I use python 3.7.3 if you ask. The reason I wanna improve this code is because when I tried to make input data for this function need around 3-4 hours, but to run this function need time around 21-22 hours (this really shock me).

here is the structure of data that I input on below:

```<code>[ # list of trees
    [ # list of leaf nodes
        { # dict of spg that contain in leaf
            (tau_1,tau_2):[(ent1,ent2),(ent1,ent3)]
        }
    ]
]
</code>```

this is the sample data that I work on:

```<code>[
    [
        {(7, 8): [(1, 28), (1, 29)]},
        {(7, 8): [(1, 30)]},
        {(7, 8): [(1, 32)]},
        {(7, 8): [(1, 21), (1, 22)]},
        {(7, 8): [(1, 18)]},
        {(7, 8): [(1, 19), (1, 31)]},
        {(7, 8): [(1, 13), (1, 14)]},
        {(7, 8): [(1, 16)]},
        {(7, 8): [(1, 15)]},
        {(7, 8): [(1, 11), (1, 12)]},
        {(7, 8): [(1, 17)]},
        {(7, 8): [(1, 23), (1, 26)]},
        {(7, 8): [(1, 20), (1, 27)]},
        {(7, 8): [(1, 7)]},
        {(7, 8): [(1, 9), (1, 10)]},
        {(7, 8): [(1, 8)]},
        {(7, 8): [(1, 5), (1, 6)]},
        {(7, 8): [(1, 24), (1, 25)]},
        {
            (0, 1): [(1, 2)],
            (1, 2): [(1, 2)],
            (2, 3): [(1, 2)],
            (3, 4): [(1, 2)],
            (4, 5): [(1, 2)],
            (5, 6): [(1, 2)],
            (6, 7): [(1, 2)],
            (7, 8): [(1, 2)],
        },
        {
            (0, 1): [(1, 3), (1, 4)],
            (1, 2): [(1, 3), (1, 4)],
            (2, 3): [(1, 3), (1, 4)],
            (3, 4): [(1, 3), (1, 4)],
            (4, 5): [(1, 3), (1, 4)],
            (5, 6): [(1, 3), (1, 4)],
            (6, 7): [(1, 3), (1, 4)],
            (7, 8): [(1, 3), (1, 4)],
        },
    ],
    [
        {(7, 8): [(2, 28), (2, 29)]},
        {(7, 8): [(2, 30)]},
        {(7, 8): [(2, 32)]},
        {(7, 8): [(2, 21), (2, 22)]},
        {(7, 8): [(2, 18)]},
        {(7, 8): [(2, 19), (2, 31)]},
        {(7, 8): [(2, 13), (2, 14)]},
        {(7, 8): [(2, 16)]},
        {(7, 8): [(2, 24), (2, 25)]},
        {(7, 8): [(2, 23)]},
        {(7, 8): [(2, 26), (2, 27)]},
        {(7, 8): [(2, 20)]},
        {(7, 8): [(2, 15)]},
        {(7, 8): [(2, 11), (2, 12)]},
        {(7, 8): [(2, 17)]},
        {(7, 8): [(2, 9), (2, 10)]},
        {(7, 8): [(2, 1)]},
        {(7, 8): [(2, 3), (2, 4)]},
        {(7, 8): [(2, 7), (2, 8)]},
        {(7, 8): [(2, 5), (2, 6)]},
    ]
]
</code>```

this is the parameters value in case you need to know:

```<code>tau_lenght: 50
ent: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42]
gamma: 1
tau_start: [1, 3, 5, 7, 9, 11, 13, 15, 17, 19, 21, 23, 25, 27, 29, 31, 33, 35, 37, 39, 41, 43, 45, 47, 49, 51, 53, 55, 57, 59, 61, 63, 65, 67, 69, 71, 73, 75, 77, 79, 81, 83, 85, 87, 89, 91, 93, 95, 97, 99]
tau_end: [2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 30, 32, 34, 36, 38, 40, 42, 44, 46, 48, 50, 52, 54, 56, 58, 60, 62, 64, 66, 68, 70, 72, 74, 76, 78, 80, 82, 84, 86, 88, 90, 92, 94, 96, 98, 100]
time_range: 2
json_output: 'output_json'
</code>```

Here the code:

```<code>import json
def prepare_output(data,tau_lenght,ent,gamma,tau_start,tau_end,time_range,json_output):
    # step 1:
    # change data to dict where pair of tau became key
    # list of group of entity into sorted based on o_r
    spgs = {} # collect all spg per 2 tau based on entity
    for tau in range(tau_lenght-1): # for every tau
        temp_tau = [] # containing result per 2 tau
        for o_r in ent: # for every o_r
            temp = [] # contain all SPG whic contain o_r on tau
            for lst in data: # lst is the result on every decision tree
                for leaf in lst: # leaf from decision tree
                    if (tau,tau+1) in leaf.keys(): # looking for pair tau existence
                        # if leaf contain entity o_r
                        if list_contain(leaf[(tau,tau+1)],o_r):
                            # adding new crew candidate
                            # helper.list_of_ent is working for change [(1,2),(2,3)] into [1,2,3]
                            temp.append(helper.list_of_ent(leaf[(tau,tau+1)]))
            if temp: # if temp not empty
                temp_tau.append(temp) # adding temp into temp_tau
        if temp_tau: # if temp_tau not empty
            # connect all candidate into one spg
            cur_tau = []
            for lst in temp_tau:
                for pairs in lst:
                    cur_tau.append(pairs)
            spgs[(tau,tau+1)] = set(tuple(i) for i in cur_tau) # menambahkan
            # temp_tau into res

    # step 2:
    # change into dict which include group of ent as key,
    # and list of (tau1,tau2) as value
    result_per_group = {}
    for tau in spgs.keys():
        for group in spgs[tau]:
            if group in result_per_group.keys():
                result_per_group[group].append(tau)
            else:
                result_per_group[group] = [tau]

    # step 3:
    # remove all candidate > gamma
    for group in result_per_group.keys():
        lst = result_per_group[group]
        for tau_idx in range(len(lst)-1):
            tau1_end = tau_end[tau_idx]
            tau2_start = tau_start[tau_idx+1]
            if (tau2_start - tau1_end) > (time_range * gamma):
                result_per_group.pop(group)
                break

    global output
    # helper.dict_str_key is fuction I made to convert key of dict into string
    output['output'] = helper.dict_str_key(result_per_group)
    # to convert into json file
    helper.to_json(json_output,output)

# this some function from helper that I use in here
def list_of_ent(lst):
    res = set()
    for e1,e2 in lst:
        res.add(e1)
        res.add(e2)
    return sorted(list(res))

def dict_str_key(data):
    data_str_key = {}
    for key in data.keys():
        data_str_key[str(key)] = data[key]
    return data_str_key

def to_json(name,data):
    name = 'output/'+name+'.json'
    with open(name, 'w') as file:
        json.dump(data, file)
</code>```

To make question clear, I put the code focus on three big loop as steps.

step 1 used to make output like this:

```<code>{
    (tau1, tau2): {
        (ent1,ent2),
        (ent8,ent19)
    }
}
</code>```

step 2 used to make the output like this:

```<code>{
    (ent1,ent2,ent3):[(tau1,tau2),(tau2,tau3)]
}
</code>```

step 3 have the same format for output step 2

p.s: tau is group of time (in this example I use 2 frame as 1 tau)

There's the question I wanna ask, sorry if I say it too long or the problem is kinda too complex. I hope my question is clear. I really look forward for any responds and support, thanks.

答:
Without having the full code to test outputs this is harder to do, but it seems that there are some redundant processes that you are adding elements to a list of lists only to flatten that list and add that to a dictionary as a set. You can increase some of the speed and memory by removing that and instead just adding it to the dictionary right away.

There are some other tweaks that can be done such as using f-strings instead of string concatenation, using list comprehension, and removing having to do the same math in the loop (time_range * gamma) and instead just reference it by memory.

But these are all minor tweaks compared to your step one process which looks to be the largest time sink (approx N^4 in time complexity). I am unsure if it is larger as I don't see the functions that you use inside that for loop, but tweaking that to reduce the number of calculations would provide the largest benefit to time savings.

```<code>import json
from collections import defaultdict

def prepare_output(data,tau_lenght,ent,gamma,tau_start,tau_end,time_range,json_output):
    # step 1:
    # change data to dict where pair of tau became key
    # list of group of entity into sorted based on o_r
    spgs = defaultdict(set) # collect all spg per 2 tau based on entity
    for tau in range(tau_lenght-1): # for every tau
        for o_r in ent: # for every o_r
            for lst in data: # lst is the result on every decision tree
                for leaf in lst: # leaf from decision tree
                    if (tau,tau+1) in leaf: # looking for pair tau existence
                        # if leaf contain entity o_r
                        if list_contain(leaf[(tau,tau+1)],o_r):
                            # adding new crew candidate
                            # helper.list_of_ent is working for change [(1,2),(2,3)] into [1,2,3]
                            list_unique_elements = helper.list_of_ent(leaf[(tau,tau+1)])
                            if (tau,tau+1) in spgs: # menambahkan
                                dict[(tau,tau+1)].add(tuple(list_unique_elements))
                            else:
                                dict[(tau,tau+1)] = {tuple(list_unique_elements)}

    # step 2:
    # change into dict which include group of ent as key,
    # and list of (tau1,tau2) as value
    result_per_group = {}
    for tau, value in spgs.items():
        for group in value:
            if group in result_per_group:
                result_per_group[group].append(tau)
            else:
                result_per_group[group] = [tau]

    # step 3:
    # remove all candidate > gamma
    time_gamma_multiplied = time_range * gamma
    for group,lst in result_per_group.items():
        for tau_idx in lst[:-1]:
            if (tau_start[tau_idx+1] - tau_end[tau_idx]) > time_gamma_multiplied:
                result_per_group.pop(group)
                break

    global output
    # helper.dict_str_key is fuction I made to convert key of dict into string
    output['output'] = helper.dict_str_key(result_per_group)
    # to convert into json file
    helper.to_json(json_output,output)

# this some function from helper that I use in here
def list_of_ent(lst):
    res = set()
    for e1,e2 in lst:
        res.add(e1)
        res.add(e2)
    return sorted(list(res))

def dict_str_key(data):
    return {data_str_key[str(key)]: value for key, value in data.items()}

def to_json(name,data):
    with open(f'output/{name}.json', 'w') as file:
        json.dump(data, file)
</code>```

Obviously profiling your code would give you the best indicator for what is taking the most time and to find processes that are either redundant or that take a large time and could be reduced with minor tweaks.

Body:
I've been working on python to make a program which need to handle complex problem from list of dict. The thing is I need to transform this data into dictionary and sort it. The input for this function is come from trees. The code I share here is working, but takes a long time to run. In here I wanna ask is there any idea to make this function run more faster in python? I use python 3.7.3 if you ask. The reason I wanna improve this code is because when I tried to make input data for this function need around 3-4 hours, but to run this function need time around 21-22 hours (this really shock me).

here is the structure of data that I input on below:

[ # list of trees
    [ # list of leaf nodes
        { # dict of spg that contain in leaf
            (tau_1,tau_2):[(ent1,ent2),(ent1,ent3)]
        }
    ]
]

this is the sample data that I work on:

[
    [
        {(7, 8): [(1, 28), (1, 29)]},
        {(7, 8): [(1, 30)]},
        {(7, 8): [(1, 32)]},
        {(7, 8): [(1, 21), (1, 22)]},
        {(7, 8): [(1, 18)]},
        {(7, 8): [(1, 19), (1, 31)]},
        {(7, 8): [(1, 13), (1, 14)]},
        {(7, 8): [(1, 16)]},
        {(7, 8): [(1, 15)]},
        {(7, 8): [(1, 11), (1, 12)]},
        {(7, 8): [(1, 17)]},
        {(7, 8): [(1, 23), (1, 26)]},
        {(7, 8): [(1, 20), (1, 27)]},
        {(7, 8): [(1, 7)]},
        {(7, 8): [(1, 9), (1, 10)]},
        {(7, 8): [(1, 8)]},
        {(7, 8): [(1, 5), (1, 6)]},
        {(7, 8): [(1, 24), (1, 25)]},
        {
            (0, 1): [(1, 2)],
            (1, 2): [(1, 2)],
            (2, 3): [(1, 2)],
            (3, 4): [(1, 2)],
            (4, 5): [(1, 2)],
            (5, 6): [(1, 2)],
            (6, 7): [(1, 2)],
            (7, 8): [(1, 2)],
        },
        {
            (0, 1): [(1, 3), (1, 4)],
            (1, 2): [(1, 3), (1, 4)],
            (2, 3): [(1, 3), (1, 4)],
            (3, 4): [(1, 3), (1, 4)],
            (4, 5): [(1, 3), (1, 4)],
            (5, 6): [(1, 3), (1, 4)],
            (6, 7): [(1, 3), (1, 4)],
            (7, 8): [(1, 3), (1, 4)],
        },
    ],
    [
        {(7, 8): [(2, 28), (2, 29)]},
        {(7, 8): [(2, 30)]},
        {(7, 8): [(2, 32)]},
        {(7, 8): [(2, 21), (2, 22)]},
        {(7, 8): [(2, 18)]},
        {(7, 8): [(2, 19), (2, 31)]},
        {(7, 8): [(2, 13), (2, 14)]},
        {(7, 8): [(2, 16)]},
        {(7, 8): [(2, 24), (2, 25)]},
        {(7, 8): [(2, 23)]},
        {(7, 8): [(2, 26), (2, 27)]},
        {(7, 8): [(2, 20)]},
        {(7, 8): [(2, 15)]},
        {(7, 8): [(2, 11), (2, 12)]},
        {(7, 8): [(2, 17)]},
        {(7, 8): [(2, 9), (2, 10)]},
        {(7, 8): [(2, 1)]},
        {(7, 8): [(2, 3), (2, 4)]},
        {(7, 8): [(2, 7), (2, 8)]},
        {(7, 8): [(2, 5), (2, 6)]},
    ]
]

this is the parameters value in case you need to know:

tau_lenght: 50
ent: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42]
gamma: 1
tau_start: [1, 3, 5, 7, 9, 11, 13, 15, 17, 19, 21, 23, 25, 27, 29, 31, 33, 35, 37, 39, 41, 43, 45, 47, 49, 51, 53, 55, 57, 59, 61, 63, 65, 67, 69, 71, 73, 75, 77, 79, 81, 83, 85, 87, 89, 91, 93, 95, 97, 99]
tau_end: [2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 30, 32, 34, 36, 38, 40, 42, 44, 46, 48, 50, 52, 54, 56, 58, 60, 62, 64, 66, 68, 70, 72, 74, 76, 78, 80, 82, 84, 86, 88, 90, 92, 94, 96, 98, 100]
time_range: 2
json_output: 'output_json'

Here the code:

import json
def prepare_output(data,tau_lenght,ent,gamma,tau_start,tau_end,time_range,json_output):
    # step 1:
    # change data to dict where pair of tau became key
    # list of group of entity into sorted based on o_r
    spgs = {} # collect all spg per 2 tau based on entity
    for tau in range(tau_lenght-1): # for every tau
        temp_tau = [] # containing result per 2 tau
        for o_r in ent: # for every o_r
            temp = [] # contain all SPG whic contain o_r on tau
            for lst in data: # lst is the result on every decision tree
                for leaf in lst: # leaf from decision tree
                    if (tau,tau+1) in leaf.keys(): # looking for pair tau existence
                        # if leaf contain entity o_r
                        if list_contain(leaf[(tau,tau+1)],o_r):
                            # adding new crew candidate
                            # helper.list_of_ent is working for change [(1,2),(2,3)] into [1,2,3]
                            temp.append(helper.list_of_ent(leaf[(tau,tau+1)]))
            if temp: # if temp not empty
                temp_tau.append(temp) # adding temp into temp_tau
        if temp_tau: # if temp_tau not empty
            # connect all candidate into one spg
            cur_tau = []
            for lst in temp_tau:
                for pairs in lst:
                    cur_tau.append(pairs)
            spgs[(tau,tau+1)] = set(tuple(i) for i in cur_tau) # menambahkan
            # temp_tau into res

    # step 2:
    # change into dict which include group of ent as key,
    # and list of (tau1,tau2) as value
    result_per_group = {}
    for tau in spgs.keys():
        for group in spgs[tau]:
            if group in result_per_group.keys():
                result_per_group[group].append(tau)
            else:
                result_per_group[group] = [tau]

    # step 3:
    # remove all candidate > gamma
    for group in result_per_group.keys():
        lst = result_per_group[group]
        for tau_idx in range(len(lst)-1):
            tau1_end = tau_end[tau_idx]
            tau2_start = tau_start[tau_idx+1]
            if (tau2_start - tau1_end) > (time_range * gamma):
                result_per_group.pop(group)
                break

    global output
    # helper.dict_str_key is fuction I made to convert key of dict into string
    output['output'] = helper.dict_str_key(result_per_group)
    # to convert into json file
    helper.to_json(json_output,output)

# this some function from helper that I use in here
def list_of_ent(lst):
    res = set()
    for e1,e2 in lst:
        res.add(e1)
        res.add(e2)
    return sorted(list(res))

def dict_str_key(data):
    data_str_key = {}
    for key in data.keys():
        data_str_key[str(key)] = data[key]
    return data_str_key

def to_json(name,data):
    name = 'output/'+name+'.json'
    with open(name, 'w') as file:
        json.dump(data, file)

To make question clear, I put the code focus on three big loop as steps.

step 1 used to make output like this:

{
    (tau1, tau2): {
        (ent1,ent2),
        (ent8,ent19)
    }
}

step 2 used to make the output like this:

{
    (ent1,ent2,ent3):[(tau1,tau2),(tau2,tau3)]
}

step 3 have the same format for output step 2

p.s: tau is group of time (in this example I use 2 frame as 1 tau)

There's the question I wanna ask, sorry if I say it too long or the problem is kinda too complex. I hope my question is clear. I really look forward for any responds and support, thanks.

Body:
Without having the full code to test outputs this is harder to do, but it seems that there are some redundant processes that you are adding elements to a list of lists only to flatten that list and add that to a dictionary as a set. You can increase some of the speed and memory by removing that and instead just adding it to the dictionary right away.

There are some other tweaks that can be done such as using f-strings instead of string concatenation, using list comprehension, and removing having to do the same math in the loop (time_range * gamma) and instead just reference it by memory.

But these are all minor tweaks compared to your step one process which looks to be the largest time sink (approx N^4 in time complexity). I am unsure if it is larger as I don't see the functions that you use inside that for loop, but tweaking that to reduce the number of calculations would provide the largest benefit to time savings.

import json
from collections import defaultdict

def prepare_output(data,tau_lenght,ent,gamma,tau_start,tau_end,time_range,json_output):
    # step 1:
    # change data to dict where pair of tau became key
    # list of group of entity into sorted based on o_r
    spgs = defaultdict(set) # collect all spg per 2 tau based on entity
    for tau in range(tau_lenght-1): # for every tau
        for o_r in ent: # for every o_r
            for lst in data: # lst is the result on every decision tree
                for leaf in lst: # leaf from decision tree
                    if (tau,tau+1) in leaf: # looking for pair tau existence
                        # if leaf contain entity o_r
                        if list_contain(leaf[(tau,tau+1)],o_r):
                            # adding new crew candidate
                            # helper.list_of_ent is working for change [(1,2),(2,3)] into [1,2,3]
                            list_unique_elements = helper.list_of_ent(leaf[(tau,tau+1)])
                            if (tau,tau+1) in spgs: # menambahkan
                                dict[(tau,tau+1)].add(tuple(list_unique_elements))
                            else:
                                dict[(tau,tau+1)] = {tuple(list_unique_elements)}

    # step 2:
    # change into dict which include group of ent as key,
    # and list of (tau1,tau2) as value
    result_per_group = {}
    for tau, value in spgs.items():
        for group in value:
            if group in result_per_group:
                result_per_group[group].append(tau)
            else:
                result_per_group[group] = [tau]

    # step 3:
    # remove all candidate > gamma
    time_gamma_multiplied = time_range * gamma
    for group,lst in result_per_group.items():
        for tau_idx in lst[:-1]:
            if (tau_start[tau_idx+1] - tau_end[tau_idx]) > time_gamma_multiplied:
                result_per_group.pop(group)
                break

    global output
    # helper.dict_str_key is fuction I made to convert key of dict into string
    output['output'] = helper.dict_str_key(result_per_group)
    # to convert into json file
    helper.to_json(json_output,output)

# this some function from helper that I use in here
def list_of_ent(lst):
    res = set()
    for e1,e2 in lst:
        res.add(e1)
        res.add(e2)
    return sorted(list(res))

def dict_str_key(data):
    return {data_str_key[str(key)]: value for key, value in data.items()}

def to_json(name,data):
    with open(f'output/{name}.json', 'w') as file:
        json.dump(data, file)

Obviously profiling your code would give you the best indicator for what is taking the most time and to find processes that are either redundant or that take a large time and could be reduced with minor tweaks.


问:
Rerendering page based on submission in React/Redux with Calling API
I am trying to call API with Redux, and it calls again based on a form submission.
which means if query is none, it returns all lists or it returns lists that match the query.

```<code>// List.tsx

import React, { useEffect, useState } from "react";
import { connect } from "react-redux";
import { getFeatures, increasePage } from "../../redux/featuresSlice";
import { FeatureItem } from "../../components/featureItem";

interface IFeatureProp {
  featureObject: any;
  getFeaturesWith: any;
}

const FeatureList = ({
  featureObject,
  getFeaturesWith,
}: // page,
IFeatureProp) => {
  const [keywords, setKeywords] = useState("");

  const fetchData = async () => {
    getFeaturesWith(featureObject.page, keywords);
  };

  const handleSubmit = (e: any) => {
    e.preventDefault();
    fetchData();
  };

  useEffect(() => {
    console.log(keywords);
    fetchData();
  }, []);

  return (
    <div className="max-w-screen-xl mx-auto mt-8">
      <div className="px-12">
        <div className="relative">
          <div className="relative">
            <form onSubmit={handleSubmit}>
              <div className="absolute top-0 bottom-0 left-0 flex items-center px-5">
              </div>
              <input
                type="text"
                placeholder="Search..."
                value={keywords}
                onChange={e => setKeywords(e.target.value)}
              />
            </form>
          </div>
          <ul>
            {featureObject.features
              .map((c: any) => {
                return (
                  <FeatureItem
                    id={c.id}
                    name={c.name}
                    desc={c.desc}
                  />
                );
              })}
          </ul>
        </div>
      </div>
    </div>
  );
};

function mapStateToProps(state: any) {
  return { featureObject: state.featuresReducer.explore };
}

function mapDispatchToProps(dispatch: any) {
  return {
    getFeaturesWith: (page: any, keyword: string) =>
      dispatch(getFeatures(page, keyword)),
    increasePageWith: () => dispatch(increasePage(1)),
  };
}

export default connect(mapStateToProps, mapDispatchToProps)(FeatureList);
</code>```
```<code>//featuresSlice.js

import { createSlice } from "@reduxjs/toolkit";
import api from "../api";

const featuresSlice = createSlice({
  name: "features",
  initialState: {
    explore: {
      page: 1,
      features: [],
    },
    favs: [],
  },
  reducers: {
    setExploreFeatures(state, action) {
      const { explore } = state;
      const { payload } = action;
      payload.features.forEach(payloadFeature => {
        const exists = explore.features.find(
          savedFeature => savedFeature.id === payloadFeature.id
        );
        if (!exists) {
          explore.features.push(payloadFeature);
        }
      });
      state.explore.page = payload.page;
    },
  },
});

export const { setExploreFeatures, increasePage, setFavs, setFav } =
  featuresSlice.actions;

export const getFeatures = (page, keyword) => async (dispatch, getState) => {
  const {
    usersReducer: { token },
  } = getState();

  try {
    const {
      data: { results },
    } = await api.features(token, page, keyword);
    dispatch(
      setExploreFeatures({
        features: results,
        page: 1,
      })
    );
  } catch (e) {
    console.warn(e);
  }
};

export default featuresSlice.reducer;

</code>```

when I submit keyword, it works as I expected at backend.
[07/Feb/2022 01:55:23] "GET /api/v1/features/?search=abcd HTTP/1.1" 200 3615

And I see only lists that match the query in redux-debugger.
But it doesn't re-render the page, which means I only see whole lists.

Is there what I can do for updating state?

答:
The issue is that your reducer is mutating the state, which should never be the case within the Redux ideology.

Instead, you should be creating a new state object (with your changes applied) and return it from the reducer. And if you don't want to modify the state - just return the original one kept intact. All your reducers should be like that.

```<code>reducers: {
    setExploreFeatures(state, action) {
      const { payload } = action;
      const newFeatures = payload.features.filter(x => !state.explore.features.some(y => y.id === x.id));
      return { ...state, explore: { ...state.explore, features: [...state.explore.features, ...newFeatures], page: payload.page } };
    },
  },
</code>```

along these lines ^^

Body:
I am trying to call API with Redux, and it calls again based on a form submission.
which means if query is none, it returns all lists or it returns lists that match the query.

// List.tsx

import React, { useEffect, useState } from "react";
import { connect } from "react-redux";
import { getFeatures, increasePage } from "../../redux/featuresSlice";
import { FeatureItem } from "../../components/featureItem";

interface IFeatureProp {
  featureObject: any;
  getFeaturesWith: any;
}

const FeatureList = ({
  featureObject,
  getFeaturesWith,
}: // page,
IFeatureProp) => {
  const [keywords, setKeywords] = useState("");

  const fetchData = async () => {
    getFeaturesWith(featureObject.page, keywords);
  };

  const handleSubmit = (e: any) => {
    e.preventDefault();
    fetchData();
  };

  useEffect(() => {
    console.log(keywords);
    fetchData();
  }, []);

  return (

            <form onSubmit={handleSubmit}>

              <input
                type="text"
                placeholder="Search..."
                value={keywords}
                onChange={e => setKeywords(e.target.value)}
              />
            </form>

            {featureObject.features
              .map((c: any) => {
                return (
                  <FeatureItem
                    id={c.id}
                    name={c.name}
                    desc={c.desc}
                  />
                );
              })}

  );
};

function mapStateToProps(state: any) {
  return { featureObject: state.featuresReducer.explore };
}

function mapDispatchToProps(dispatch: any) {
  return {
    getFeaturesWith: (page: any, keyword: string) =>
      dispatch(getFeatures(page, keyword)),
    increasePageWith: () => dispatch(increasePage(1)),
  };
}

export default connect(mapStateToProps, mapDispatchToProps)(FeatureList);

//featuresSlice.js

import { createSlice } from "@reduxjs/toolkit";
import api from "../api";

const featuresSlice = createSlice({
  name: "features",
  initialState: {
    explore: {
      page: 1,
      features: [],
    },
    favs: [],
  },
  reducers: {
    setExploreFeatures(state, action) {
      const { explore } = state;
      const { payload } = action;
      payload.features.forEach(payloadFeature => {
        const exists = explore.features.find(
          savedFeature => savedFeature.id === payloadFeature.id
        );
        if (!exists) {
          explore.features.push(payloadFeature);
        }
      });
      state.explore.page = payload.page;
    },
  },
});

export const { setExploreFeatures, increasePage, setFavs, setFav } =
  featuresSlice.actions;

export const getFeatures = (page, keyword) => async (dispatch, getState) => {
  const {
    usersReducer: { token },
  } = getState();

  try {
    const {
      data: { results },
    } = await api.features(token, page, keyword);
    dispatch(
      setExploreFeatures({
        features: results,
        page: 1,
      })
    );
  } catch (e) {
    console.warn(e);
  }
};

export default featuresSlice.reducer;

when I submit keyword, it works as I expected at backend.
[07/Feb/2022 01:55:23] "GET /api/v1/features/?search=abcd HTTP/1.1" 200 3615

And I see only lists that match the query in redux-debugger.
But it doesn't re-render the page, which means I only see whole lists.

Is there what I can do for updating state?

Body:
The issue is that your reducer is mutating the state, which should never be the case within the Redux ideology.

Instead, you should be creating a new state object (with your changes applied) and return it from the reducer. And if you don't want to modify the state - just return the original one kept intact. All your reducers should be like that.

reducers: {
    setExploreFeatures(state, action) {
      const { payload } = action;
      const newFeatures = payload.features.filter(x => !state.explore.features.some(y => y.id === x.id));
      return { ...state, explore: { ...state.explore, features: [...state.explore.features, ...newFeatures], page: payload.page } };
    },
  },

along these lines ^^


问:
Windows font issue and how to debug for customer
Recently had a customer send in a ticket complaining that their font has changed (within the week or so).  The font on the site has not changed in probably a decade.  What I suspect is that perhaps a recent windows up that times in line with the change is effecting the font he sees, or, more likely, a setting changed on his end.

the font we use

```<code>font-family: "Helvetica Neue",Helvetica,Arial,sans-serif
</code>```

It is my understanding that Helvetica Neue will likely get replaced by something else on windows since, just from googling, I find that font is not included in windows.

My question is, is there any way I can help debug this on his end to figure out exactly what is going on?  It does make the site difficult to read for this user and I would like to fix it, and also know for sure what I am talking about. I usually try very hard not to just reply with, "looks good on my machine".  Inspecting it shows the same font family as what I posted above.

None of the font options in that css appear to be what is showing.

The one distinguishing trait I can see in the font is the letters de overlap or touch.

This is for web content, the browsers mentioned where most recent Chrome, which I also tested on (verified exact same version numbers) and did not have the issue, and Edge which I do not have.

答:
I have no reputable source to provide, but this exact situation has happened to me before. It was caused by me installing a faulty font of a similar name.

It was hell to read most websites and I had to get a chrome extension to change everything to Arial to be readable. Ask them if they're having this problem on other websites as well then tell them to delete the "Helvetica Neue" font file on their computer (Mine was named Helvatica Neue56878 if it helps). This solved the problem for me.

Body:
Recently had a customer send in a ticket complaining that their font has changed (within the week or so).  The font on the site has not changed in probably a decade.  What I suspect is that perhaps a recent windows up that times in line with the change is effecting the font he sees, or, more likely, a setting changed on his end.

the font we use

font-family: "Helvetica Neue",Helvetica,Arial,sans-serif

It is my understanding that Helvetica Neue will likely get replaced by something else on windows since, just from googling, I find that font is not included in windows.

My question is, is there any way I can help debug this on his end to figure out exactly what is going on?  It does make the site difficult to read for this user and I would like to fix it, and also know for sure what I am talking about. I usually try very hard not to just reply with, "looks good on my machine".  Inspecting it shows the same font family as what I posted above.

None of the font options in that css appear to be what is showing.

The one distinguishing trait I can see in the font is the letters de overlap or touch.

This is for web content, the browsers mentioned where most recent Chrome, which I also tested on (verified exact same version numbers) and did not have the issue, and Edge which I do not have.


问:
How do I store user input as a variable and display it within echo for a custom wordpress plugin?
I'm relatively new to php and WordPress development and can't get the following code to work:

```<code><?php
function addContent(){
    echo '<input type="text" name="firstname">';
    echo '<input type="submit" value="Submit">';

    $firstname = $_GET["firstname"];

    echo "<p> You wrote: $firstname </p>";
}

add_shortcode('addContent', 'addContent');
?>
</code>```

If I manually assign a string to $firstname it shows fine, so the problem is in assigning the variable, which I'm having trouble with.

Thanks

答:
To send input data you must add <form> ...inputs...</form> so your code should looks like this

```<code><?php
add_shortcode('addContent', 'addContent');

function addContent(){
    echo '<form method="GET">'; // printing form tag
    echo '<input type="text" name="firstname">';
    echo '<input type="submit" name="send_btn" value="Submit">';
    echo '</form>';

    if (isset($_GET['send_btn'])) { // checking is form was submitted  then accessing to value
        $firstname = $_GET['firstname'];

        echo "<p> You wrote: $firstname </p>";
    }

}

</code>```

Body:
I'm relatively new to php and WordPress development and can't get the following code to work:

<?php
function addContent(){
    echo '<input type="text" name="firstname">';
    echo '<input type="submit" value="Submit">';

    $firstname = $_GET["firstname"];

    echo "
 You wrote: $firstname
";
}

add_shortcode('addContent', 'addContent');
?>

If I manually assign a string to $firstname it shows fine, so the problem is in assigning the variable, which I'm having trouble with.

Thanks

Body:
To send input data you must add <form> ...inputs...</form> so your code should looks like this

<?php
add_shortcode('addContent', 'addContent');

function addContent(){
    echo '<form method="GET">'; // printing form tag
    echo '<input type="text" name="firstname">';
    echo '<input type="submit" name="send_btn" value="Submit">';
    echo '</form>';

    if (isset($_GET['send_btn'])) { // checking is form was submitted  then accessing to value
        $firstname = $_GET['firstname'];

        echo "<p> You wrote: $firstname </p>";
    }

}


问:
Implementing a Read More link in React.js
I am trying to implement a Read More link, which expands to show
more text after a click.  I am trying to do this the React way.

```<code><!--html-->
<div class="initialText">
  Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy text ever since the 1500s, when an unknown printer took a galley of type and scrambled it to make a type specimen book.
  <a>
    Read More
  </a>
</div>

<div class="fullText">
  Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy text ever since the 1500s, when an unknown printer took a galley of type and scrambled it to make a type specimen book. It has survived not only five centuries, but also the leap into electronic typesetting, remaining essentially unchanged. It was popularised in the 1960s with the release of Letraset sheets containing Lorem Ipsum passages, and more recently with desktop publishing software like Aldus PageMaker including versions of Lorem Ipsum.
</div>
</code>```

```<code>/* JavaScript */
/* @jsx DOM */
var component = React.createClass({

  getInitialState: function() {
    return {
      expanded: false
    };
  },

  expandedText: function() {
    this.setState({
      expanded: true
    });
  },

  render: function() {
    return (
      <div>

      </div>
    );
  }

});
</code>```

I am new to React.  I am not sure how to handle everything inside the render method.  How can I implement this functionality in pure React?

答:
That's the complete solution in React Js:

Call Component:

<TeamCard description="Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy text ever since the 1500s." limit={150}/>```
</div>
</div>
</p>
<p>Component:</p>
<p><div class="snippet" data-lang="js" data-hide="false" data-console="true" data-babel="false">
<div class="snippet-code">
<pre class="snippet-code-js lang-js prettyprint-override"><code>const TeamCard = ({ description, limit }) => {
  const [showAll, setShowAll] = useState(false);
  return (
    <p className="text-center text-white text-base pt-3 font-normal pb-5">
      {description.length > limit ? (
        <div>
          {showAll ? (
            <div>
              {description}
              <br />
              <button
                onClick={() => setShowAll(false)}
                className="text-primary"
              >
                Read Less
              </button>
            </div>
          ) : (
            <div>
              {description.substring(0, limit).concat("...")}
              <br />
              <button onClick={() => setShowAll(true)} className="text-primary">
                Read More
              </button>
            </div>
          )}
        </div>
      ) : (
        description
      )}
    </p>
  );
};
export default TeamCard;</code>```

Body:
I am trying to implement a Read More link, which expands to show
more text after a click.  I am trying to do this the React way.

  Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy text ever since the 1500s, when an unknown printer took a galley of type and scrambled it to make a type specimen book.

    Read More

  Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy text ever since the 1500s, when an unknown printer took a galley of type and scrambled it to make a type specimen book. It has survived not only five centuries, but also the leap into electronic typesetting, remaining essentially unchanged. It was popularised in the 1960s with the release of Letraset sheets containing Lorem Ipsum passages, and more recently with desktop publishing software like Aldus PageMaker including versions of Lorem Ipsum.

/* JavaScript */
/* @jsx DOM */
var component = React.createClass({

  getInitialState: function() {
    return {
      expanded: false
    };
  },

  expandedText: function() {
    this.setState({
      expanded: true
    });
  },

  render: function() {
    return (

    );
  }

});

I am new to React.  I am not sure how to handle everything inside the render method.  How can I implement this functionality in pure React?

Body:
That's the complete solution in React Js:

Call Component:

<TeamCard description="Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy text ever since the 1500s." limit={150}/>

Component:

const TeamCard = ({ description, limit }) => {
  const [showAll, setShowAll] = useState(false);
  return (
    <p className="text-center text-white text-base pt-3 font-normal pb-5">
      {description.length > limit ? (
        <div>
          {showAll ? (
            <div>
              {description}
              <br />
              <button
                onClick={() => setShowAll(false)}
                className="text-primary"
              >
                Read Less
              </button>
            </div>
          ) : (
            <div>
              {description.substring(0, limit).concat("...")}
              <br />
              <button onClick={() => setShowAll(true)} className="text-primary">
                Read More
              </button>
            </div>
          )}
        </div>
      ) : (
        description
      )}
    </p>
  );
};
export default TeamCard;


问:
Complex HTML/CSS Layout
I've been a web developer for really quite some time now, but this is something I've never really done, and my progress so far has used some very complex methods.  The following is the layout I want, there are rules to this layout though, as I will explain in a moment.

```<code>    ----------------------------------------------------
    |                      HEADER                      |
    ----------------------------------------------------
     ________   _______________________________________
    |        | |                                       |
    |        | |                                       |
    |        | |                                       |
    |        | |                                       |
    |        | |                                       |
    |        | |                                       |
    |        | |                                       |
    |        | |                                       |
    |________| |                                       |
               |                                       |
               |                                       |
               |                                       |
               |                                       |
               |                                       |
               |_______________________________________|
</code>```

The sidebar and the header are static and so is the body, however you can scroll the body.  Using the CSS property overflow:scroll on a container for the body section is not an option.  The reason being is that it's an ASP.NET application which utilises the maintainpositiononscrollback, meaning that the browser scrollbar - not a contain scrollbar - is what is used to record the x and y position.  I have achieved this but using some complex layout techniques which just aren't pretty.

So my HTML looks like the following:

```<code><div id="header">
  <ul>
    <li><a href="/">Home</a></li>
    <li><a href="#">Telephone</a></li>
    <li><a href="#">My Department</a></li>
    <li><a href="#">My Profile</a></li>
  </ul>
</div>

<div id="sidebar-container">
  <div id="sidebar">
    <!-- sidebar code using ul's and li's -->
  </div>
</div>

<div id="content-top-border"></div>
<div id="content-left-border"></div>

<div id="content-wrap">
  <div id="content">
    <asp:ContentPlaceHolder ID="MainContent" runat="server" />
  </div>
</div>

<div id="content-right-border"></div>
<div id="content-bottom-border"></div>
<div id="bottom-block"></div>
</code>```

So I use all of these divs for a specific reason, so I can absolutely position borders on the content area.  The rest of the divs actually block off content by changing the background colour of them so you can't see the content when you scroll.

My CSS file is far too large to be posting on here, but here's the general idea:

```<code>div#header {
    height:30px;
    background-color:#d7d7d7;
    position:fixed;
    top:0;
    left:0;
    right:0;
}

div#sidebar {
    position:fixed;
    top:30px;
    left:0;
    overflow:auto;
    bottom:4px;
    width:148px;
    background-color:#d7d7d7;
}

div#sidebar ul {
    border:1px solid #9a9a9a;
    border-bottom:0;
    font-family:Tahoma;
    margin:0 0 0 4px;
    padding:0;
    list-style-type:none;
    font-size:0.75em;
    width:137px;
    clear:both;
    background-color:white;
}
</code>```

So as you can see from the sidebar, I set a fixed width, make it position:fixed and then set a background.  The containing
 has a fixed width thus causing a block.  When you horizontally scroll the page, the content will be hidden when it hits the sidebar.  This creates an effect much like any client-side application.

```<code>#content-wrap {
    position: absolute;
    top: 30px;
    bottom: 15px;
    left: 147px;
    right:0;
    background-color:#d7d7d7;
    z-index:-998;
}

#content {
    z-index:-995;
    position:absolute;
    top:0;
    bottom:0;
    right:0;
    left:0;
    padding:10px 10px 20px 10px;
    background-color:#FFF;
    color:#292929;
}
</code>```

So you can see that the #content has absolute positioning keeping it in that content box.  Anything outside of the content box is never displayed even if you scroll horizontally or vertically.

Are there better ways of doing this?  It seems a heck-of-a-lot of HTML/CSS to be doing something really quite simple.

EDIT: This layout isn't a problem whatsoever.  That part isn't even difficult.  What's difficult is maintaining simple HTML/CSS with a complex layout, and by complex, I don't mean something I'm unable to achieve.  I have achieved the desired effect already, I'm just looking for a better way of doing it.

EDIT: This is based on business requirements, I don't have a choice :)

答: I'd do this with position:fixed and the appropriate margins and width/heights

OwnerDisplayName: Kezzer

ContentLicense: CC BY-SA 2.5

Title: Complex HTML/CSS Layout

Body:
I've been a web developer for really quite some time now, but this is something I've never really done, and my progress so far has used some very complex methods.  The following is the layout I want, there are rules to this layout though, as I will explain in a moment.

    ----------------------------------------------------
    |                      HEADER                      |
    ----------------------------------------------------
     ________   _______________________________________
    |        | |                                       |
    |        | |                                       |
    |        | |                                       |
    |        | |                                       |
    |        | |                                       |
    |        | |                                       |
    |        | |                                       |
    |        | |                                       |
    |________| |                                       |
               |                                       |
               |                                       |
               |                                       |
               |                                       |
               |                                       |
               |_______________________________________|

The sidebar and the header are static and so is the body, however you can scroll the body.  Using the CSS property overflow:scroll on a container for the body section is not an option.  The reason being is that it's an ASP.NET application which utilises the maintainpositiononscrollback, meaning that the browser scrollbar - not a contain scrollbar - is what is used to record the x and y position.  I have achieved this but using some complex layout techniques which just aren't pretty.

So my HTML looks like the following:

Home

Telephone

My Department

My Profile

    <asp:ContentPlaceHolder ID="MainContent" runat="server" />

So I use all of these divs for a specific reason, so I can absolutely position borders on the content area.  The rest of the divs actually block off content by changing the background colour of them so you can't see the content when you scroll.

My CSS file is far too large to be posting on here, but here's the general idea:

div#header {
    height:30px;
    background-color:#d7d7d7;
    position:fixed;
    top:0;
    left:0;
    right:0;
}

div#sidebar {
    position:fixed;
    top:30px;
    left:0;
    overflow:auto;
    bottom:4px;
    width:148px;
    background-color:#d7d7d7;
}

div#sidebar ul {
    border:1px solid #9a9a9a;
    border-bottom:0;
    font-family:Tahoma;
    margin:0 0 0 4px;
    padding:0;
    list-style-type:none;
    font-size:0.75em;
    width:137px;
    clear:both;
    background-color:white;
}

So as you can see from the sidebar, I set a fixed width, make it position:fixed and then set a background.  The containing
 has a fixed width thus causing a block.  When you horizontally scroll the page, the content will be hidden when it hits the sidebar.  This creates an effect much like any client-side application.

#content-wrap {
    position: absolute;
    top: 30px;
    bottom: 15px;
    left: 147px;
    right:0;
    background-color:#d7d7d7;
    z-index:-998;
}

#content {
    z-index:-995;
    position:absolute;
    top:0;
    bottom:0;
    right:0;
    left:0;
    padding:10px 10px 20px 10px;
    background-color:#FFF;
    color:#292929;
}

So you can see that the #content has absolute positioning keeping it in that content box.  Anything outside of the content box is never displayed even if you scroll horizontally or vertically.

Are there better ways of doing this?  It seems a heck-of-a-lot of HTML/CSS to be doing something really quite simple.

EDIT: This layout isn't a problem whatsoever.  That part isn't even difficult.  What's difficult is maintaining simple HTML/CSS with a complex layout, and by complex, I don't mean something I'm unable to achieve.  I have achieved the desired effect already, I'm just looking for a better way of doing it.

EDIT: This is based on business requirements, I don't have a choice :)

LastEditorDisplayName: Kezzer

OwnerDisplayName: annakata


问:
EF Code First With Two DbContexts
This should be a simple one involving EF Code first but I can't wrap my head around the documentation and all the examples I am finding are from older versions.  I am working with the latest (4.1).

Anyway I have some models like:

```<code>public class Foo
{
    public int ID { get; set; }
    public Bar Bar { get; set; }
}

public class Bar
{
    public int ID { get; set; }
    public string Value { get; set; }
}
</code>```

I used some scaffolding with Asp.Net MVC to create my controllers/repositories and when I create a 'Foo' object, it also creates a 'Bar' object even though I set the 'Bar' property from something stored in the database.

```<code>public class FooViewModel
{
    public int ID { get; set; }
    public int BarID { get; set; }
}

public ActionResult Create(FooViewModel foo)
{
    var entity = new Foo()
        {
            ID = foo.ID,
            Bar = _barRepository.Find(foo.BarID)
        };
    _fooRepository.InsertOrUpdate(entity);
    _fooRepository.Save();
    // more stuff
}
</code>```

How can I use fluent syntax for EF in order to stop it from creating a new 'Bar' row in the database?

Update

Here is the generated repository code:

```<code>    public void InsertOrUpdate(Foo foo)
    {
        if (foo.ID == default(int)) {
            // New entity
            context.Foo.Add(foo);
        } else {
            // Existing entity
            context.Foo(foo).State = EntityState.Modified;
        }
    }

    public void Save()
    {
        context.SaveChanges();
    }
</code>```

答: your _fooRepository and _barRepository need to share same DB context instance. If the are using two instances the Bar will be in added state.

Title: EF Code First With Two DbContexts

Body:
This should be a simple one involving EF Code first but I can't wrap my head around the documentation and all the examples I am finding are from older versions.  I am working with the latest (4.1).

Anyway I have some models like:

public class Foo
{
    public int ID { get; set; }
    public Bar Bar { get; set; }
}

public class Bar
{
    public int ID { get; set; }
    public string Value { get; set; }
}

I used some scaffolding with Asp.Net MVC to create my controllers/repositories and when I create a 'Foo' object, it also creates a 'Bar' object even though I set the 'Bar' property from something stored in the database.

public class FooViewModel
{
    public int ID { get; set; }
    public int BarID { get; set; }
}

public ActionResult Create(FooViewModel foo)
{
    var entity = new Foo()
        {
            ID = foo.ID,
            Bar = _barRepository.Find(foo.BarID)
        };
    _fooRepository.InsertOrUpdate(entity);
    _fooRepository.Save();
    // more stuff
}

How can I use fluent syntax for EF in order to stop it from creating a new 'Bar' row in the database?

Update

Here is the generated repository code:

    public void InsertOrUpdate(Foo foo)
    {
        if (foo.ID == default(int)) {
            // New entity
            context.Foo.Add(foo);
        } else {
            // Existing entity
            context.Foo(foo).State = EntityState.Modified;
        }
    }

    public void Save()
    {
        context.SaveChanges();
    }