问:
Errors while trying to use collect() method in pyspark. (Windows 10)
So I've read a dozens of tutorials on how to set up pyspark.
I've set all enviremental variables like HADOOP_HOME, SPARK_HOME e.t.c.
I've downloaded winutils and put it to %SPARK_HOME%/bin.
I've  checked that the version of pyspark is the same as spark, that I have downloaded from official site (3.2.1).
I am using Java JDK 8.
I've tried different versions of Java, Spark/Pyspark but everytime I use collect method on rdd I'm getting a tons of errors.

This is my sample program:

```<code>from pyspark.sql import SparkSession

ss = SparkSession.builder.master('local').appName('name').getOrCreate()

rd = ss.sparkContext.parallelize([1, 2, 3, 4, 5])
rd1 = rd.map(lambda x: x** 2).collect()
print(rd1)
</code>```

And this is what I am getting. (I am sorry I have no idea how to put everything under spoiler)
Any help would be very appreciated!

```<code>Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
22/02/05 21:03:53 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
22/02/05 21:04:06 WARN ProcfsMetricsGetter: Exception when trying to compute pagesize, as a result reporting of ProcessTree metrics is stopped
22/02/05 21:04:06 ERROR Executor: Exception in task 0.0 in stage 0.0 (TID 0)
org.apache.spark.SparkException: Python worker failed to connect back.
    at org.apache.spark.api.python.PythonWorkerFactory.createSimpleWorker(PythonWorkerFactory.scala:188)
    at org.apache.spark.api.python.PythonWorkerFactory.create(PythonWorkerFactory.scala:108)
    at org.apache.spark.SparkEnv.createPythonWorker(SparkEnv.scala:121)
    at org.apache.spark.api.python.BasePythonRunner.compute(PythonRunner.scala:162)
    at org.apache.spark.api.python.PythonRDD.compute(PythonRDD.scala:65)
    at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
    at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
    at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
    at org.apache.spark.scheduler.Task.run(Task.scala:131)
    at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:506)
    at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1462)
    at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:509)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketTimeoutException: Accept timed out
    at java.net.DualStackPlainSocketImpl.waitForNewConnection(Native Method)
    at java.net.DualStackPlainSocketImpl.socketAccept(DualStackPlainSocketImpl.java:131)
    at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:535)
    at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:189)
    at java.net.ServerSocket.implAccept(ServerSocket.java:545)
    at java.net.ServerSocket.accept(ServerSocket.java:513)
    at org.apache.spark.api.python.PythonWorkerFactory.createSimpleWorker(PythonWorkerFactory.scala:175)
    ... 14 more
22/02/05 21:04:06 WARN TaskSetManager: Lost task 0.0 in stage 0.0 (TID 0) (WIN-CH4BAQ3PTMC executor driver): org.apache.spark.SparkException: Python worker failed to connect back.
    at org.apache.spark.api.python.PythonWorkerFactory.createSimpleWorker(PythonWorkerFactory.scala:188)
    at org.apache.spark.api.python.PythonWorkerFactory.create(PythonWorkerFactory.scala:108)
    at org.apache.spark.SparkEnv.createPythonWorker(SparkEnv.scala:121)
    at org.apache.spark.api.python.BasePythonRunner.compute(PythonRunner.scala:162)
    at org.apache.spark.api.python.PythonRDD.compute(PythonRDD.scala:65)
    at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
    at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
    at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
    at org.apache.spark.scheduler.Task.run(Task.scala:131)
    at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:506)
    at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1462)
    at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:509)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketTimeoutException: Accept timed out
    at java.net.DualStackPlainSocketImpl.waitForNewConnection(Native Method)
    at java.net.DualStackPlainSocketImpl.socketAccept(DualStackPlainSocketImpl.java:131)
    at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:535)
    at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:189)
    at java.net.ServerSocket.implAccept(ServerSocket.java:545)
    at java.net.ServerSocket.accept(ServerSocket.java:513)
    at org.apache.spark.api.python.PythonWorkerFactory.createSimpleWorker(PythonWorkerFactory.scala:175)
    ... 14 more

22/02/05 21:04:06 ERROR TaskSetManager: Task 0 in stage 0.0 failed 1 times; aborting job
Traceback (most recent call last):
  File "C:\Users\user\PycharmProjects\pythonProject\spark.py", line 6, in <module>
    print(rd.map(lambda x: x ** 2).collect())
  File "C:\Users\user\PycharmProjects\pythonProject\venv\lib\site-packages\pyspark\rdd.py", line 950, in collect
    sock_info = self.ctx._jvm.PythonRDD.collectAndServe(self._jrdd.rdd())
  File "C:\Users\user\PycharmProjects\pythonProject\venv\lib\site-packages\py4j\java_gateway.py", line 1321, in __call__
    return_value = get_return_value(
  File "C:\Users\user\PycharmProjects\pythonProject\venv\lib\site-packages\pyspark\sql\utils.py", line 111, in deco
    return f(*a, **kw)
  File "C:\Users\user\PycharmProjects\pythonProject\venv\lib\site-packages\py4j\protocol.py", line 326, in get_return_value
    raise Py4JJavaError(
py4j.protocol.Py4JJavaError: An error occurred while calling z:org.apache.spark.api.python.PythonRDD.collectAndServe.
: org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 0.0 failed 1 times, most recent failure: Lost task 0.0 in stage 0.0 (TID 0) (WIN-CH4BAQ3PTMC executor driver): org.apache.spark.SparkException: Python worker failed to connect back.
    at org.apache.spark.api.python.PythonWorkerFactory.createSimpleWorker(PythonWorkerFactory.scala:188)
    at org.apache.spark.api.python.PythonWorkerFactory.create(PythonWorkerFactory.scala:108)
    at org.apache.spark.SparkEnv.createPythonWorker(SparkEnv.scala:121)
    at org.apache.spark.api.python.BasePythonRunner.compute(PythonRunner.scala:162)
    at org.apache.spark.api.python.PythonRDD.compute(PythonRDD.scala:65)
    at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
    at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
    at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
    at org.apache.spark.scheduler.Task.run(Task.scala:131)
    at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:506)
    at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1462)
    at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:509)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketTimeoutException: Accept timed out
    at java.net.DualStackPlainSocketImpl.waitForNewConnection(Native Method)
    at java.net.DualStackPlainSocketImpl.socketAccept(DualStackPlainSocketImpl.java:131)
    at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:535)
    at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:189)
    at java.net.ServerSocket.implAccept(ServerSocket.java:545)
    at java.net.ServerSocket.accept(ServerSocket.java:513)
    at org.apache.spark.api.python.PythonWorkerFactory.createSimpleWorker(PythonWorkerFactory.scala:175)
    ... 14 more

Driver stacktrace:
    at org.apache.spark.scheduler.DAGScheduler.failJobAndIndependentStages(DAGScheduler.scala:2454)
    at org.apache.spark.scheduler.DAGScheduler.$anonfun$abortStage$2(DAGScheduler.scala:2403)
    at org.apache.spark.scheduler.DAGScheduler.$anonfun$abortStage$2$adapted(DAGScheduler.scala:2402)
    at scala.collection.immutable.List.foreach(List.scala:333)
    at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:2402)
    at org.apache.spark.scheduler.DAGScheduler.$anonfun$handleTaskSetFailed$1(DAGScheduler.scala:1160)
    at org.apache.spark.scheduler.DAGScheduler.$anonfun$handleTaskSetFailed$1$adapted(DAGScheduler.scala:1160)
    at scala.Option.foreach(Option.scala:437)
    at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:1160)
    at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:2642)
    at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2584)
    at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2573)
    at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:49)
    at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:938)
    at org.apache.spark.SparkContext.runJob(SparkContext.scala:2214)
    at org.apache.spark.SparkContext.runJob(SparkContext.scala:2235)
    at org.apache.spark.SparkContext.runJob(SparkContext.scala:2254)
    at org.apache.spark.SparkContext.runJob(SparkContext.scala:2279)
    at org.apache.spark.rdd.RDD.$anonfun$collect$1(RDD.scala:1030)
    at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
    at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
    at org.apache.spark.rdd.RDD.withScope(RDD.scala:414)
    at org.apache.spark.rdd.RDD.collect(RDD.scala:1029)
    at org.apache.spark.api.python.PythonRDD$.collectAndServe(PythonRDD.scala:180)
    at org.apache.spark.api.python.PythonRDD.collectAndServe(PythonRDD.scala)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
    at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
    at py4j.Gateway.invoke(Gateway.java:282)
    at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
    at py4j.commands.CallCommand.execute(CallCommand.java:79)
    at py4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:182)
    at py4j.ClientServerConnection.run(ClientServerConnection.java:106)
    at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.spark.SparkException: Python worker failed to connect back.
    at org.apache.spark.api.python.PythonWorkerFactory.createSimpleWorker(PythonWorkerFactory.scala:188)
    at org.apache.spark.api.python.PythonWorkerFactory.create(PythonWorkerFactory.scala:108)
    at org.apache.spark.SparkEnv.createPythonWorker(SparkEnv.scala:121)
    at org.apache.spark.api.python.BasePythonRunner.compute(PythonRunner.scala:162)
    at org.apache.spark.api.python.PythonRDD.compute(PythonRDD.scala:65)
    at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
    at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
    at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
    at org.apache.spark.scheduler.Task.run(Task.scala:131)
    at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:506)
    at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1462)
    at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:509)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    ... 1 more
Caused by: java.net.SocketTimeoutException: Accept timed out
    at java.net.DualStackPlainSocketImpl.waitForNewConnection(Native Method)
    at java.net.DualStackPlainSocketImpl.socketAccept(DualStackPlainSocketImpl.java:131)
    at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:535)
    at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:189)
    at java.net.ServerSocket.implAccept(ServerSocket.java:545)
    at java.net.ServerSocket.accept(ServerSocket.java:513)
    at org.apache.spark.api.python.PythonWorkerFactory.createSimpleWorker(PythonWorkerFactory.scala:175)
    ... 14 more
</code>```

答:
BIG thanks to @blackbishop for the tip!

Python worker failed to connect back

All I did is just added

```<code>import findspark
findspark.init()
</code>```

Everything works now!

Body:
So I've read a dozens of tutorials on how to set up pyspark.
I've set all enviremental variables like HADOOP_HOME, SPARK_HOME e.t.c.
I've downloaded winutils and put it to %SPARK_HOME%/bin.
I've  checked that the version of pyspark is the same as spark, that I have downloaded from official site (3.2.1).
I am using Java JDK 8.
I've tried different versions of Java, Spark/Pyspark but everytime I use collect method on rdd I'm getting a tons of errors.

This is my sample program:

from pyspark.sql import SparkSession

ss = SparkSession.builder.master('local').appName('name').getOrCreate()

rd = ss.sparkContext.parallelize([1, 2, 3, 4, 5])
rd1 = rd.map(lambda x: x** 2).collect()
print(rd1)

And this is what I am getting. (I am sorry I have no idea how to put everything under spoiler)
Any help would be very appreciated!

Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
22/02/05 21:03:53 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
22/02/05 21:04:06 WARN ProcfsMetricsGetter: Exception when trying to compute pagesize, as a result reporting of ProcessTree metrics is stopped
22/02/05 21:04:06 ERROR Executor: Exception in task 0.0 in stage 0.0 (TID 0)
org.apache.spark.SparkException: Python worker failed to connect back.
    at org.apache.spark.api.python.PythonWorkerFactory.createSimpleWorker(PythonWorkerFactory.scala:188)
    at org.apache.spark.api.python.PythonWorkerFactory.create(PythonWorkerFactory.scala:108)
    at org.apache.spark.SparkEnv.createPythonWorker(SparkEnv.scala:121)
    at org.apache.spark.api.python.BasePythonRunner.compute(PythonRunner.scala:162)
    at org.apache.spark.api.python.PythonRDD.compute(PythonRDD.scala:65)
    at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
    at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
    at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
    at org.apache.spark.scheduler.Task.run(Task.scala:131)
    at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:506)
    at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1462)
    at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:509)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketTimeoutException: Accept timed out
    at java.net.DualStackPlainSocketImpl.waitForNewConnection(Native Method)
    at java.net.DualStackPlainSocketImpl.socketAccept(DualStackPlainSocketImpl.java:131)
    at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:535)
    at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:189)
    at java.net.ServerSocket.implAccept(ServerSocket.java:545)
    at java.net.ServerSocket.accept(ServerSocket.java:513)
    at org.apache.spark.api.python.PythonWorkerFactory.createSimpleWorker(PythonWorkerFactory.scala:175)
    ... 14 more
22/02/05 21:04:06 WARN TaskSetManager: Lost task 0.0 in stage 0.0 (TID 0) (WIN-CH4BAQ3PTMC executor driver): org.apache.spark.SparkException: Python worker failed to connect back.
    at org.apache.spark.api.python.PythonWorkerFactory.createSimpleWorker(PythonWorkerFactory.scala:188)
    at org.apache.spark.api.python.PythonWorkerFactory.create(PythonWorkerFactory.scala:108)
    at org.apache.spark.SparkEnv.createPythonWorker(SparkEnv.scala:121)
    at org.apache.spark.api.python.BasePythonRunner.compute(PythonRunner.scala:162)
    at org.apache.spark.api.python.PythonRDD.compute(PythonRDD.scala:65)
    at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
    at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
    at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
    at org.apache.spark.scheduler.Task.run(Task.scala:131)
    at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:506)
    at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1462)
    at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:509)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketTimeoutException: Accept timed out
    at java.net.DualStackPlainSocketImpl.waitForNewConnection(Native Method)
    at java.net.DualStackPlainSocketImpl.socketAccept(DualStackPlainSocketImpl.java:131)
    at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:535)
    at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:189)
    at java.net.ServerSocket.implAccept(ServerSocket.java:545)
    at java.net.ServerSocket.accept(ServerSocket.java:513)
    at org.apache.spark.api.python.PythonWorkerFactory.createSimpleWorker(PythonWorkerFactory.scala:175)
    ... 14 more

22/02/05 21:04:06 ERROR TaskSetManager: Task 0 in stage 0.0 failed 1 times; aborting job
Traceback (most recent call last):
  File "C:\Users\user\PycharmProjects\pythonProject\spark.py", line 6, in <module>
    print(rd.map(lambda x: x ** 2).collect())
  File "C:\Users\user\PycharmProjects\pythonProject\venv\lib\site-packages\pyspark\rdd.py", line 950, in collect
    sock_info = self.ctx._jvm.PythonRDD.collectAndServe(self._jrdd.rdd())
  File "C:\Users\user\PycharmProjects\pythonProject\venv\lib\site-packages\py4j\java_gateway.py", line 1321, in __call__
    return_value = get_return_value(
  File "C:\Users\user\PycharmProjects\pythonProject\venv\lib\site-packages\pyspark\sql\utils.py", line 111, in deco
    return f(*a, **kw)
  File "C:\Users\user\PycharmProjects\pythonProject\venv\lib\site-packages\py4j\protocol.py", line 326, in get_return_value
    raise Py4JJavaError(
py4j.protocol.Py4JJavaError: An error occurred while calling z:org.apache.spark.api.python.PythonRDD.collectAndServe.
: org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 0.0 failed 1 times, most recent failure: Lost task 0.0 in stage 0.0 (TID 0) (WIN-CH4BAQ3PTMC executor driver): org.apache.spark.SparkException: Python worker failed to connect back.
    at org.apache.spark.api.python.PythonWorkerFactory.createSimpleWorker(PythonWorkerFactory.scala:188)
    at org.apache.spark.api.python.PythonWorkerFactory.create(PythonWorkerFactory.scala:108)
    at org.apache.spark.SparkEnv.createPythonWorker(SparkEnv.scala:121)
    at org.apache.spark.api.python.BasePythonRunner.compute(PythonRunner.scala:162)
    at org.apache.spark.api.python.PythonRDD.compute(PythonRDD.scala:65)
    at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
    at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
    at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
    at org.apache.spark.scheduler.Task.run(Task.scala:131)
    at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:506)
    at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1462)
    at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:509)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketTimeoutException: Accept timed out
    at java.net.DualStackPlainSocketImpl.waitForNewConnection(Native Method)
    at java.net.DualStackPlainSocketImpl.socketAccept(DualStackPlainSocketImpl.java:131)
    at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:535)
    at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:189)
    at java.net.ServerSocket.implAccept(ServerSocket.java:545)
    at java.net.ServerSocket.accept(ServerSocket.java:513)
    at org.apache.spark.api.python.PythonWorkerFactory.createSimpleWorker(PythonWorkerFactory.scala:175)
    ... 14 more

Driver stacktrace:
    at org.apache.spark.scheduler.DAGScheduler.failJobAndIndependentStages(DAGScheduler.scala:2454)
    at org.apache.spark.scheduler.DAGScheduler.$anonfun$abortStage$2(DAGScheduler.scala:2403)
    at org.apache.spark.scheduler.DAGScheduler.$anonfun$abortStage$2$adapted(DAGScheduler.scala:2402)
    at scala.collection.immutable.List.foreach(List.scala:333)
    at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:2402)
    at org.apache.spark.scheduler.DAGScheduler.$anonfun$handleTaskSetFailed$1(DAGScheduler.scala:1160)
    at org.apache.spark.scheduler.DAGScheduler.$anonfun$handleTaskSetFailed$1$adapted(DAGScheduler.scala:1160)
    at scala.Option.foreach(Option.scala:437)
    at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:1160)
    at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:2642)
    at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2584)
    at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2573)
    at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:49)
    at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:938)
    at org.apache.spark.SparkContext.runJob(SparkContext.scala:2214)
    at org.apache.spark.SparkContext.runJob(SparkContext.scala:2235)
    at org.apache.spark.SparkContext.runJob(SparkContext.scala:2254)
    at org.apache.spark.SparkContext.runJob(SparkContext.scala:2279)
    at org.apache.spark.rdd.RDD.$anonfun$collect$1(RDD.scala:1030)
    at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
    at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
    at org.apache.spark.rdd.RDD.withScope(RDD.scala:414)
    at org.apache.spark.rdd.RDD.collect(RDD.scala:1029)
    at org.apache.spark.api.python.PythonRDD$.collectAndServe(PythonRDD.scala:180)
    at org.apache.spark.api.python.PythonRDD.collectAndServe(PythonRDD.scala)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
    at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
    at py4j.Gateway.invoke(Gateway.java:282)
    at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
    at py4j.commands.CallCommand.execute(CallCommand.java:79)
    at py4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:182)
    at py4j.ClientServerConnection.run(ClientServerConnection.java:106)
    at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.spark.SparkException: Python worker failed to connect back.
    at org.apache.spark.api.python.PythonWorkerFactory.createSimpleWorker(PythonWorkerFactory.scala:188)
    at org.apache.spark.api.python.PythonWorkerFactory.create(PythonWorkerFactory.scala:108)
    at org.apache.spark.SparkEnv.createPythonWorker(SparkEnv.scala:121)
    at org.apache.spark.api.python.BasePythonRunner.compute(PythonRunner.scala:162)
    at org.apache.spark.api.python.PythonRDD.compute(PythonRDD.scala:65)
    at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
    at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
    at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
    at org.apache.spark.scheduler.Task.run(Task.scala:131)
    at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:506)
    at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1462)
    at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:509)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    ... 1 more
Caused by: java.net.SocketTimeoutException: Accept timed out
    at java.net.DualStackPlainSocketImpl.waitForNewConnection(Native Method)
    at java.net.DualStackPlainSocketImpl.socketAccept(DualStackPlainSocketImpl.java:131)
    at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:535)
    at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:189)
    at java.net.ServerSocket.implAccept(ServerSocket.java:545)
    at java.net.ServerSocket.accept(ServerSocket.java:513)
    at org.apache.spark.api.python.PythonWorkerFactory.createSimpleWorker(PythonWorkerFactory.scala:175)
    ... 14 more

Body:
BIG thanks to @blackbishop for the tip!

Python worker failed to connect back

All I did is just added

import findspark
findspark.init()

Everything works now!


来源: stackexchange

问:
Issue with unique occurences in R vector
I need help please : I have a list "reves" of vectors, and one of them is composed of names :

```<code>reves$personnes
 [1] "rebelle, professeur, gypsie"
 [2] ""
 [3] "corinne, roxane, pdl "
 [4] "fabrice, melissa, bernadette, franck, corinne, elizabeth, tom, roxane"
 [5] "didier, bernadette, franck, elizabeth, roxane, autres"
 [6] "autres"
 [7] "elizabeth, sebastien_houssiere"
 [8] "elizabeth, corinne"
 [9] "genevieve, barbara, camille, famille"
[10] "gypsie, inconnue"
</code>```

At the end I would like to calculate the percentages at which each name appears. So first, I split each line according to "," and I add the names to a new vector :

```<code># Creating vector of characters
new_vec <- c()
for (i in c(1:nrow(reves))){
x <- reves$personnes[i]
y <- strsplit(x, split=",")[[1]]
new_vec <- c(new_vec, y[1:length(y)])
}
</code>```

It seems to work since new_vec is chr [1:32] :

```<code>> new_vec
 [1] "rebelle"              " professeur"          " gypsie"
 [4] NA                     "corinne"              " roxane"
 [7] " pdl "                "fabrice"              " melissa"
[10] " bernadette"          " franck"              " corinne"
[13] " elizabeth"           " tom"                 " roxane"
[16] "didier"               " bernadette"          " franck"
[19] " elizabeth"           " roxane"              " autres"
[22] "autres"               "elizabeth"            " sebastien_houssiere"
[25] "elizabeth"            " corinne"             "genevieve"
[28] " barbara"             " camille"             " famille"
[31] "gypsie"               " inconnue"
</code>```

Using new_vec, I planned to use table(new_vec) to get the appearance rate of each name. However, same names are not counted as unique occurrences. As you can see :

```<code>unique(new_vec)
 [1] "rebelle"              " professeur"          " gypsie"
 [4] NA                     "corinne"              " roxane"
 [7] " pdl "                "fabrice"              " melissa"
[10] " bernadette"          " franck"              " corinne"
[13] " elizabeth"           " tom"                 "didier"
[16] " autres"              "autres"               "elizabeth"
[19] " sebastien_houssiere" "genevieve"            " barbara"
[22] " camille"             " famille"             "gypsie"
[25] " inconnue"
</code>```

and here, we clearly see that, for example, "corinne" appears with a score of 2 in the 1st column and with a score of 1 in the second column :

```<code>> table(new_vec)
new_vec
              autres              barbara           bernadette              camille
                   1                    1                    2                    1
             corinne            elizabeth              famille               franck
                   2                    2                    1                    2
              gypsie             inconnue              melissa                 pdl
                   1                    1                    1                    1
          professeur               roxane  sebastien_houssiere                  tom
                   1                    3                    1                    1
              autres              corinne               didier            elizabeth
                   1                    1                    1                    2
             fabrice            genevieve               gypsie              rebelle
                   1                    1                    1                    1
</code>```

Please, how could I get this new_vec with the correct numbers of occurrences so that I can perform my calculations?

Thanks for your help :)

答:
You do not need a loop so you code can be simplified as follows. First provide reproducible data:</p>
```<code>dput(personnes)
c("rebelle, professeur, gypsie", "", "corinne, roxane, pdl ",
"fabrice, melissa, bernadette, franck, corinne, elizabeth, tom, roxane",
"didier, bernadette, franck, elizabeth, roxane, autres", "autres",
"elizabeth, sebastien_houssiere", "elizabeth, corinne", "genevieve, barbara, camille, famille",
"gypsie, inconnue")

new_vec <- unlist(strsplit(personnes, ", "))
new_vec <- trimws(new_vec)  # Remove space at the end of "pdl "
sort(unique(new_vec))
# [1] "autres"              "barbara"             "bernadette"          "camille"             "corinne"             "didier"              "elizabeth"
# [8] "fabrice"             "famille"             "franck"              "genevieve"           "gypsie"              "inconnue"            "melissa"
# [15] "pdl"                 "professeur"          "rebelle"             "roxane"              "sebastien_houssiere" "tom"
</code>```

ContentLicense: CC BY-SA 4.0

Body:
I need help please : I have a list "reves" of vectors, and one of them is composed of names :

reves$personnes
 [1] "rebelle, professeur, gypsie"
 [2] ""
 [3] "corinne, roxane, pdl "
 [4] "fabrice, melissa, bernadette, franck, corinne, elizabeth, tom, roxane"
 [5] "didier, bernadette, franck, elizabeth, roxane, autres"
 [6] "autres"
 [7] "elizabeth, sebastien_houssiere"
 [8] "elizabeth, corinne"
 [9] "genevieve, barbara, camille, famille"
[10] "gypsie, inconnue"

At the end I would like to calculate the percentages at which each name appears. So first, I split each line according to "," and I add the names to a new vector :

# Creating vector of characters
new_vec <- c()
for (i in c(1:nrow(reves))){
x <- reves$personnes[i]
y <- strsplit(x, split=",")[[1]]
new_vec <- c(new_vec, y[1:length(y)])
}

It seems to work since new_vec is chr [1:32] :

> new_vec
 [1] "rebelle"              " professeur"          " gypsie"
 [4] NA                     "corinne"              " roxane"
 [7] " pdl "                "fabrice"              " melissa"
[10] " bernadette"          " franck"              " corinne"
[13] " elizabeth"           " tom"                 " roxane"
[16] "didier"               " bernadette"          " franck"
[19] " elizabeth"           " roxane"              " autres"
[22] "autres"               "elizabeth"            " sebastien_houssiere"
[25] "elizabeth"            " corinne"             "genevieve"
[28] " barbara"             " camille"             " famille"
[31] "gypsie"               " inconnue"

Using new_vec, I planned to use table(new_vec) to get the appearance rate of each name. However, same names are not counted as unique occurrences. As you can see :

unique(new_vec)
 [1] "rebelle"              " professeur"          " gypsie"
 [4] NA                     "corinne"              " roxane"
 [7] " pdl "                "fabrice"              " melissa"
[10] " bernadette"          " franck"              " corinne"
[13] " elizabeth"           " tom"                 "didier"
[16] " autres"              "autres"               "elizabeth"
[19] " sebastien_houssiere" "genevieve"            " barbara"
[22] " camille"             " famille"             "gypsie"
[25] " inconnue"

and here, we clearly see that, for example, "corinne" appears with a score of 2 in the 1st column and with a score of 1 in the second column :

> table(new_vec)
new_vec
              autres              barbara           bernadette              camille
                   1                    1                    2                    1
             corinne            elizabeth              famille               franck
                   2                    2                    1                    2
              gypsie             inconnue              melissa                 pdl
                   1                    1                    1                    1
          professeur               roxane  sebastien_houssiere                  tom
                   1                    3                    1                    1
              autres              corinne               didier            elizabeth
                   1                    1                    1                    2
             fabrice            genevieve               gypsie              rebelle
                   1                    1                    1                    1

Please, how could I get this new_vec with the correct numbers of occurrences so that I can perform my calculations?

Thanks for your help :)

Body:
You do not need a loop so you code can be simplified as follows. First provide reproducible data:

dput(personnes)
c("rebelle, professeur, gypsie", "", "corinne, roxane, pdl ",
"fabrice, melissa, bernadette, franck, corinne, elizabeth, tom, roxane",
"didier, bernadette, franck, elizabeth, roxane, autres", "autres",
"elizabeth, sebastien_houssiere", "elizabeth, corinne", "genevieve, barbara, camille, famille",
"gypsie, inconnue")

new_vec <- unlist(strsplit(personnes, ", "))
new_vec <- trimws(new_vec)  # Remove space at the end of "pdl "
sort(unique(new_vec))
# [1] "autres"              "barbara"             "bernadette"          "camille"             "corinne"             "didier"              "elizabeth"
# [8] "fabrice"             "famille"             "franck"              "genevieve"           "gypsie"              "inconnue"            "melissa"
# [15] "pdl"                 "professeur"          "rebelle"             "roxane"              "sebastien_houssiere" "tom"


问:
Excel - sumifs - condition when cell contains string
I need little help with excel. I use SUMIF func. Here is my formula.

```<code>=SUMIFS(J:J;B:B;O4;G:G;"=ISTEXT(Intercall)")
</code>```

I would like make a sum from values in J column, where:

values in column B are same as O4

and where values in column G contain specific string

for example value in column G is Intercall - Taiwan -> thi value contains string Intercall

答:
I'm not 100% sure what you're asking, but maybe this will help.

SUMIFS accepts wildcards in criteria. So you can use:

```<code>=SUMIFS(J:J;B:B;O4;G:G;="*Intercall*")
</code>```

ContentLicense: CC BY-SA 3.0

Body:
I need little help with excel. I use SUMIF func. Here is my formula.

=SUMIFS(J:J;B:B;O4;G:G;"=ISTEXT(Intercall)")

I would like make a sum from values in J column, where:

values in column B are same as O4

and where values in column G contain specific string

for example value in column G is Intercall - Taiwan -> thi value contains string Intercall

Body:
I'm not 100% sure what you're asking, but maybe this will help.

SUMIFS accepts wildcards in criteria. So you can use:

=SUMIFS(J:J;B:B;O4;G:G;="*Intercall*")


问:
atoi function in C
Question in C : given a number in character array , find the biggest two digit number (group of two digits should be checked from left to right) with the condition that the number cannot have same two digits . eg. input: 64998434 , output: 84 (not 99).

Solution : atoi expects string not character array .
therefore just increase the array length by 1 and initialize it with terminating character as written below

```<code>#include <stdio.h>
#include <string.h>
#include <stdlib.h>

int main()
{
    char no[] = "64998434";
    int big = 0, temp, j = 0, length;

    length = strlen(no);
    //char str[2] = "";  // not working
    char str[3] = "\0";  // working

    if (length <= 0 || length == 1)
    {
        printf("Invalid input");
        return 0;
    }
    else if (length == 2)
    {
        printf("%d", atoi(no));
        return 0;
    }

    //loop to get two digits from input array and store in another array to convert into
    //integer and than save and check for bigger number

    for (int i = 0; i < length; i += 2)
    {
        str[j] = no[i];
        str[j + 1] = no[i + 1];
        if (str[0] != str[1])
        {
            temp = atoi(str);
            if (temp > big)
            {
                big = temp;
            }
        }
        j = 0;
    }

    if (big != 0)
    {
        printf("%d\n", big);
        return 0;
    }
}
</code>```

答:
Seems like a single pass should do it.

```<code>// Return -1 on error
// Otherwise return the value of the biggest pair
int biggest_pair(const char *s) {
  int biggest = -1;
  while (s[0]) {
    // This and next character a digit?
    if (!(s[0] >= '0' && s[0] <= '9' && s[1] >= '0' && s[1] <= '9')) {
      return -1;
    }
    int test = (s[0] - '0')*10 + s[1] - '0';
    // Are digits different and this is larger than the current best?
    if (s[0] != s[1] && test > biggest) {
      biggest = test;
    }
    s += 2;
  }
  return biggest;
}
</code>```

Code could use isdigit().

```<code>int biggest_pair(const char *s) {
  const char *us = (const char *) s;
  int biggest = -1;
  while (us[0]) {
    // Check this and next character as digits?
    if (!(isdigit(us[0]) && isdigit(us[1])) {
      return -1;
    }
    int test = (us[0] - '0')*10 + us[1] - '0';
    // Are digits different and this is larger than the current best?
    if (us[0] != us[1] && test > biggest) {
      biggest = test;
    }
    us += 2;
  }
  return biggest;
}
</code>```

This 2nd version also properly distinguishes +0 from -9 on those rare non-2's complement machines.

Title: atoi function in C

Body:
Question in C : given a number in character array , find the biggest two digit number (group of two digits should be checked from left to right) with the condition that the number cannot have same two digits . eg. input: 64998434 , output: 84 (not 99).

Solution : atoi expects string not character array .
therefore just increase the array length by 1 and initialize it with terminating character as written below

#include <stdio.h>
#include <string.h>
#include <stdlib.h>

int main()
{
    char no[] = "64998434";
    int big = 0, temp, j = 0, length;

    length = strlen(no);
    //char str[2] = "";  // not working
    char str[3] = "\0";  // working

    if (length <= 0 || length == 1)
    {
        printf("Invalid input");
        return 0;
    }
    else if (length == 2)
    {
        printf("%d", atoi(no));
        return 0;
    }

    //loop to get two digits from input array and store in another array to convert into
    //integer and than save and check for bigger number

    for (int i = 0; i < length; i += 2)
    {
        str[j] = no[i];
        str[j + 1] = no[i + 1];
        if (str[0] != str[1])
        {
            temp = atoi(str);
            if (temp > big)
            {
                big = temp;
            }
        }
        j = 0;
    }

    if (big != 0)
    {
        printf("%d\n", big);
        return 0;
    }
}

Body:
Seems like a single pass should do it.

// Return -1 on error
// Otherwise return the value of the biggest pair
int biggest_pair(const char *s) {
  int biggest = -1;
  while (s[0]) {
    // This and next character a digit?
    if (!(s[0] >= '0' && s[0] <= '9' && s[1] >= '0' && s[1] <= '9')) {
      return -1;
    }
    int test = (s[0] - '0')*10 + s[1] - '0';
    // Are digits different and this is larger than the current best?
    if (s[0] != s[1] && test > biggest) {
      biggest = test;
    }
    s += 2;
  }
  return biggest;
}

Code could use isdigit().

int biggest_pair(const char *s) {
  const char *us = (const char *) s;
  int biggest = -1;
  while (us[0]) {
    // Check this and next character as digits?
    if (!(isdigit(us[0]) && isdigit(us[1])) {
      return -1;
    }
    int test = (us[0] - '0')*10 + us[1] - '0';
    // Are digits different and this is larger than the current best?
    if (us[0] != us[1] && test > biggest) {
      biggest = test;
    }
    us += 2;
  }
  return biggest;
}

This 2nd version also properly distinguishes +0 from -9 on those rare non-2's complement machines.


问:
cant get the correct value while using GEngine->AddOnScreenDebugMessage() Unreal engine 2.27
I am trying to add a lean function to my c++ project, after calculating the YawDelta I want to displayed to my screen using the GEngine->AddOnScreenDebugMessage()

the problem it's always 0 in the viewport screen when pressing play

here is my c++ code of the Lean function which is called in UpdateAnimationProperties (Tick)

```<code>void UShooterAnimInstance::Lean(float DeltaTime)
{
    if (ShooterCharacter == nullptr) return;
    CharacterYawLastFrame = CharacterYaw;
    CharacterYaw = ShooterCharacter->GetActorRotation().Yaw;

    const float Target{ (CharacterYaw - CharacterYawLastFrame) / DeltaTime };

    const float Interp{ FMath::FInterpTo(YawDelta, Target, DeltaTime, 6.f) };
    YawDelta = FMath::Clamp(Interp, -90.f, 90.f);

    if (GEngine) GEngine->AddOnScreenDebugMessage(5,
        -1,
        FColor::Cyan,
        FString::Printf(TEXT("YawDelta : %f"),
            YawDelta));

}
</code>```

PS: if I use UE_LOG, the output log shows the variable 2 times, one with the correct value and the other one is always 0

Please help

答: Have you tried calling the AddOnScreenDebugMessage with YawDelta in the AnimInstance's NativeUpdateAnimation ? Try it there after Super::NativeUpdateAnimation(DeltaSeconds)and see what result you get.

Body:
I am trying to add a lean function to my c++ project, after calculating the YawDelta I want to displayed to my screen using the GEngine->AddOnScreenDebugMessage()

the problem it's always 0 in the viewport screen when pressing play

here is my c++ code of the Lean function which is called in UpdateAnimationProperties (Tick)

void UShooterAnimInstance::Lean(float DeltaTime)
{
    if (ShooterCharacter == nullptr) return;
    CharacterYawLastFrame = CharacterYaw;
    CharacterYaw = ShooterCharacter->GetActorRotation().Yaw;

    const float Target{ (CharacterYaw - CharacterYawLastFrame) / DeltaTime };

    const float Interp{ FMath::FInterpTo(YawDelta, Target, DeltaTime, 6.f) };
    YawDelta = FMath::Clamp(Interp, -90.f, 90.f);

    if (GEngine) GEngine->AddOnScreenDebugMessage(5,
        -1,
        FColor::Cyan,
        FString::Printf(TEXT("YawDelta : %f"),
            YawDelta));

}

PS: if I use UE_LOG, the output log shows the variable 2 times, one with the correct value and the other one is always 0

Please help


问:
text and icon in a antd button are not aligned vertically
my issue is that my text and icon in a antd button are not aligned vertically in my react app.

Here is the code:

```<code>             <Button type="primary" size="large" block icon={<PlusCircleOutlined />} onClick={()=>this.showModalSharePicture()}>
          Text
          </Button>
</code>```

The icon is is little bit low compare to the text witch is well centered in the button.
How can solve this issue that I have in all my buttons?

Here is the rendered html:

```<code><button type="button" class="ant-btn ant-btn-primary ant-btn-lg ant-btn-block"><span role="img" aria-label="plus-circle" class="anticon anticon-plus-circle"><svg viewBox="64 64 896 896" focusable="false" data-icon="plus-circle" width="1em" height="1em" fill="currentColor" aria-hidden="true"><path d="M696 480H544V328c0-4.4-3.6-8-8-8h-48c-4.4 0-8 3.6-8 8v152H328c-4.4 0-8 3.6-8 8v48c0 4.4 3.6 8 8 8h152v152c0 4.4 3.6 8 8 8h48c4.4 0 8-3.6 8-8V544h152c4.4 0 8-3.6 8-8v-48c0-4.4-3.6-8-8-8z"></path><path d="M512 64C264.6 64 64 264.6 64 512s200.6 448 448 448 448-200.6 448-448S759.4 64 512 64zm0 820c-205.4 0-372-166.6-372-372s166.6-372 372-372 372 166.6 372 372-166.6 372-372 372z"></path></svg></span><span>Text</span></button>
</code>```

答:
One possible way to do it is by using the following style:

button,
span {
  display: inline-flex;
  align-items: center;
}
```

You can also use their class names ant-btn, anticon to style them regards your preferences and needs.

Body:
my issue is that my text and icon in a antd button are not aligned vertically in my react app.

Here is the code:

             <Button type="primary" size="large" block icon={<PlusCircleOutlined />} onClick={()=>this.showModalSharePicture()}>
          Text
          </Button>

The icon is is little bit low compare to the text witch is well centered in the button.
How can solve this issue that I have in all my buttons?

Here is the rendered html:

<button type="button" class="ant-btn ant-btn-primary ant-btn-lg ant-btn-block"><svg viewBox="64 64 896 896" focusable="false" data-icon="plus-circle" width="1em" height="1em" fill="currentColor" aria-hidden="true"><path d="M696 480H544V328c0-4.4-3.6-8-8-8h-48c-4.4 0-8 3.6-8 8v152H328c-4.4 0-8 3.6-8 8v48c0 4.4 3.6 8 8 8h152v152c0 4.4 3.6 8 8 8h48c4.4 0 8-3.6 8-8V544h152c4.4 0 8-3.6 8-8v-48c0-4.4-3.6-8-8-8z"></path><path d="M512 64C264.6 64 64 264.6 64 512s200.6 448 448 448 448-200.6 448-448S759.4 64 512 64zm0 820c-205.4 0-372-166.6-372-372s166.6-372 372-372 372 166.6 372 372-166.6 372-372 372z"></path></svg>Text</button>

Body:
One possible way to do it is by using the following style:

button,
span {
  display: inline-flex;
  align-items: center;
}

You can also use their class names ant-btn, anticon to style them regards your preferences and needs.


问:
Send Images with Hikari Python
I made a function to send an image on invocation but instead it ends
up sending 'attachment://img.jpg'

here's the function :

```<code>@bot.command
@lightbulb.command('img', 'Test command')
@lightbulb.implements(lightbulb.SlashCommand)
async def imgsnd(ctx):
    filename = 'img.jpg'
    with open(filename, "rb") as fh:
        f = hikari.File('/Users/admin/Desktop/Bot/'+filename)
    await ctx.respond(f)

</code>```

答:
Sending attachments as a slash command initial response wasnt supported until 2.0.0.dev106. Consider upgrading and the issue will be solved.

Additionally, even tho this wasn't part of the question, just a pointer. That open is not necessary and, on top of that, blocking. Instead, you can reduce your code to this:

@bot.command
@lightbulb.command('img', 'Test command')
@lightbulb.implements(lightbulb.SlashCommand)
async def imgsnd(ctx):
    f = hikari.File('/Users/admin/Desktop/Bot/img.jpg')
    await ctx.respond(f)
```

Title: Send Images with Hikari Python

Body:
I made a function to send an image on invocation but instead it ends
up sending 'attachment://img.jpg'

here's the function :

@bot.command
@lightbulb.command('img', 'Test command')
@lightbulb.implements(lightbulb.SlashCommand)
async def imgsnd(ctx):
    filename = 'img.jpg'
    with open(filename, "rb") as fh:
        f = hikari.File('/Users/admin/Desktop/Bot/'+filename)
    await ctx.respond(f)


问:
How to ensure malicious users won't change cookie value?
I'm developing a login page for a very small site, and for the "remember me button",
I use the user-id which then I encrypt before placing it in the cookie, and when i want to check if he already has a cookie, i uncrypt the value and connect with the user-id given.

But I'm sure that's not secured enough and people will just have to set a cookie with a random value with encrypting and this will make the job, isn't it ?

As solution to this could be to generate a random token, put it in cookie and in database. Then, if you have the token, you can connect.
If user get stolen this token, this is not the page problem isn't it ?

But I don't know how to process differently..
Anyone has a solution ?

答:
To simply answer the asked question: You can't.
Cookies are stored on the users computer and with enough access rights and/or knowledge the user will be able to delete or modify any cookies your website set.

Encryption is taking information and make into non-sense so no one can access the information. If you need encryption is up to you. But I think this kind of session management is implemented already in a lot of ways. One of the simpler would be some kind of "dynamic API Token"-implementation - Storing some kind of Hash-like String in the cookie and in DB. If they match -> login, if not -> logout. (Symfony example: https://symfony.com/doc/current/security/custom_authenticator.html )

Another one would be JWT (JSON Web Tokens), these are indeed encrypted because they send information back and forth.

If you want to implement something like this yourself I would suggest to look at documentation about these two to start.


问:
In React Native app passing navigation to class component puts it in another navigation object
In my React Native app, when I pass navigation to a class component, it puts it in another navigation object and now I have to add one more navigation to call the navigate() method.

So, instead of this.props.navigation.navigate("somewhere"),
I have to use this.props.navigation.navigation.navigate("somewhere")

Here's how I pass navigation from a functional to a class component:

```<code>const ParentComponent = ({ navigation }) => {
   return(
      <View>
          <ChildComponent navigation={navigation} />
      </View>
   );
}

export default ParentComponent;
</code>```

And in the child component which happens to be a class component with access to Redux store, I recieve navigation like this:

```<code>class ChildComponent extends Component {

   render() {
      return(
         <View>
            <TouchableOpacity onPress={() => this.props.navigation.navigation.navigate("Route123")}>
                <Text>Go to Route 123</Text>
            </TouchableOpacity>
         </View>
      );
   }
}

function mapStateToProps(state, navigation) {
   myAccount: state.account.myAccount,
   navigation: navigation
}

export default connect(mapStateToProps)(ChildComponent);
</code>```

Notice that in the ChildComponent, I had to add one more navigation to access the navigate function i.e. this.props.navigation.navigation.navigate("Route123");

I think I'm making a fairly fundamental mistake somewhere but not sure where.

The version of React Navigation in my app is as follows and the React Native version is 0.66.3:

```<code>"@react-navigation/drawer": "^6.1.8",
"@react-navigation/material-bottom-tabs": "^6.0.9",
"@react-navigation/native": "^6.0.6",
"@react-navigation/native-stack": "^6.2.5",
</code>```

答:
As @Hagai Harari stated, you should not put navigation as the second parameter of mapStateToProps function. As described here, second parameter of mapStateProps takes ownProps which corresponds to the component props.

You can use withNavigation. But when I see your dependencies, you are using react-navigation@^6 which does not have HOC called as withNavigation.

You can choose to use this.props.navigation after removing unnecessary navigation param from mapStateToProps.

What I would suggest is to use functional component instead of class components, then you will be able to use useNavigation which react-navigation@^6 exposes. In the end, you will obtain navigation object without repetition of passing it down to children components.

Body:
In my React Native app, when I pass navigation to a class component, it puts it in another navigation object and now I have to add one more navigation to call the navigate() method.

So, instead of this.props.navigation.navigate("somewhere"),
I have to use this.props.navigation.navigation.navigate("somewhere")

Here's how I pass navigation from a functional to a class component:

const ParentComponent = ({ navigation }) => {
   return(
      <View>
          <ChildComponent navigation={navigation} />
      </View>
   );
}

export default ParentComponent;

And in the child component which happens to be a class component with access to Redux store, I recieve navigation like this:

class ChildComponent extends Component {

   render() {
      return(
         <View>
            <TouchableOpacity onPress={() => this.props.navigation.navigation.navigate("Route123")}>
                <Text>Go to Route 123</Text>
            </TouchableOpacity>
         </View>
      );
   }
}

function mapStateToProps(state, navigation) {
   myAccount: state.account.myAccount,
   navigation: navigation
}

export default connect(mapStateToProps)(ChildComponent);

Notice that in the ChildComponent, I had to add one more navigation to access the navigate function i.e. this.props.navigation.navigation.navigate("Route123");

I think I'm making a fairly fundamental mistake somewhere but not sure where.

The version of React Navigation in my app is as follows and the React Native version is 0.66.3:

"@react-navigation/drawer": "^6.1.8",
"@react-navigation/material-bottom-tabs": "^6.0.9",
"@react-navigation/native": "^6.0.6",
"@react-navigation/native-stack": "^6.2.5",


问:
How to return a collection of files from a given directory in Elixir?
In Ruby, I can use

```<code>target_files = Dir["/some/dir/path/*.rb"]
#=> ["/some/dir/path/foo.rb", "/some/dir/path/bar.rb", "/some/dir/path/baz.rb"]
</code>```

which will return an array of all of the matching files in a directory. How can I do something similar in Elixir?

答:
Path.wildcard/1 can handle recursive folders with two stars '**' (it works similar to Dir.glob in Ruby). E.g.

Path.wildcard("../data/??/**/*.yml")

Body:
In Ruby, I can use

target_files = Dir["/some/dir/path/*.rb"]
#=> ["/some/dir/path/foo.rb", "/some/dir/path/bar.rb", "/some/dir/path/baz.rb"]

which will return an array of all of the matching files in a directory. How can I do something similar in Elixir?


问:
Why using the count function in neo4j with graphql returns two values called low and high?
If I execute the following cypher in Neo4j browser returns the expected values

```<code>MATCH (n:Document)
RETURN {
    year: n.year ,
    countdocs : COUNT(n)
}
</code>```

Result:

```<code>{"countdocs":3,"year":"2018"}
</code>```

But If I execute the same cypher in neo4j-graphql

```<code>type Query {
    totalActivityOverTime: [JSONObject] @cypher(statement: """
       MATCH (n:Document)
        RETURN {
         year: n.year ,
         countdocs : COUNT(n)
       }
    """)
}
</code>```

returns :

```<code>  {
    "countdocs": {
      "low": 3,
      "high": 0
    },
    "year": "2018"
  },
</code>```

What means the values low and high?

Thanks!

答:
Thanks to @Sbunzini and @stdob-- I found the solution:

Schema:

```<code>type Activity{
  year: String
  countdocs: Int
}

type Query {
    totalActivityOverTime: [Activity] @cypher(statement: """
       MATCH (n:Document)
        RETURN {
         year: n.year ,
         countdocs : COUNT(n)
       }
    """)
}
</code>```

GraphQL:

```<code>{
  totalActivityOverTime{
    year
    countdocs
  }
}
</code>```

Thanks!

Body:
If I execute the following cypher in Neo4j browser returns the expected values

MATCH (n:Document)
RETURN {
    year: n.year ,
    countdocs : COUNT(n)
}

Result:

{"countdocs":3,"year":"2018"}

But If I execute the same cypher in neo4j-graphql

type Query {
    totalActivityOverTime: [JSONObject] @cypher(statement: """
       MATCH (n:Document)
        RETURN {
         year: n.year ,
         countdocs : COUNT(n)
       }
    """)
}

returns :

  {
    "countdocs": {
      "low": 3,
      "high": 0
    },
    "year": "2018"
  },

What means the values low and high?

Thanks!

Body:
Thanks to @Sbunzini and @stdob-- I found the solution:

Schema:

type Activity{
  year: String
  countdocs: Int
}

type Query {
    totalActivityOverTime: [Activity] @cypher(statement: """
       MATCH (n:Document)
        RETURN {
         year: n.year ,
         countdocs : COUNT(n)
       }
    """)
}

GraphQL:

{
  totalActivityOverTime{
    year
    countdocs
  }
}

Thanks!


问:
file.copy: overcoming Windows long path/filename limitation
I have a table in R containing many files that I need copied to a destination folder. The files are spread out over dozens of folders, each several sub-folders down. I have successfully used the following code to find all of the files and their locations:

(fastq_files <- list.files(Illumina_output, ".fastq.gz", recursive = TRUE, include.dirs = TRUE) %>% as_tibble)

After appending the full path, I have a tibble that looks something like this:

full_path

Q:/IlluminaOutput/2019/091119 AB NGS/Data/Intensities/BaseCalls/19-15897-HLA-091119-AB-NGS_S14_L001_R1_001.fastq.gz

Q:/IlluminaOutput/2019/091119 AB NGS/Data/Intensities/BaseCalls/19-15236-HLA-091119-AB-NGS_S14_L001_R2_001.fastq.gz

Q:/IlluminaOutput/2018/062818AB NGS/Data/Intensities/BaseCalls/18-06875-HLA-062818-NGS_S11_L001_R1_001.fastq.gz

Using the file.copy function gives an error that the file name is too long, a known issue in Windows (I am using RStudio on Windows 10).

I found that if I set the working directory directory to the file location, I am able to copy files. Starting with a table like this:

file
path

19-14889-HLA-091119-AB-NGS_S14_L001_R1_001.fastq.gz
Q:/IlluminaOutput/2019/091119 AB NGS/Data/Intensities/BaseCalls/

19-14889-HLA-091119-AB-NGS_S14_L001_R2_001.fastq.gz
Q:/IlluminaOutput/2019/091119 AB NGS/Data/Intensities/BaseCalls/

18-09772-HLA-062818-NGS_S11_L001_R1_001.fastq.gz
Q:/IlluminaOutput/2018/062818AB NGS/Data/Intensities/BaseCalls/

18-09772-HLA-062818-NGS_S11_L001_R2_001.fastq.gz
Q:/IlluminaOutput/2018/062818AB NGS/Data/Intensities/BaseCalls/

I used the following code to sucsessfully copy the first file:

```<code>(dir <- as.character(as.vector(file_and_path[1,2])))
setwd(dir)
(file <- as.character(as.vector(file_and_path[1,1])))
(file.copy(file, Trusight_output) %>% as.tibble)
</code>```

I got this to work, but I don't know how to apply these steps to every column in my table. I think i probably have to use the lapply function, but I'm not sure how to construct it.

答:
This should do the trick, assuming that file_and_path$file and  file_and_path$path are both character vectors and that Trusight_output is an absolute path:

f <- function(file, from, to) {
    cwd <- setwd(from)
    on.exit(setwd(cwd))
    file.copy(file, to)
}
Map(f, file = file_and_path$file, from = file_and_path$path, to = Trusight_output)
```

We use Map here rather than lapply because we are applying a function of more than one argument. FWIW, operations like this are often better suited for PowerShell.

Body:
I have a table in R containing many files that I need copied to a destination folder. The files are spread out over dozens of folders, each several sub-folders down. I have successfully used the following code to find all of the files and their locations:

(fastq_files <- list.files(Illumina_output, ".fastq.gz", recursive = TRUE, include.dirs = TRUE) %>% as_tibble)

After appending the full path, I have a tibble that looks something like this:

full_path

Q:/IlluminaOutput/2019/091119 AB NGS/Data/Intensities/BaseCalls/19-15897-HLA-091119-AB-NGS_S14_L001_R1_001.fastq.gz

Q:/IlluminaOutput/2019/091119 AB NGS/Data/Intensities/BaseCalls/19-15236-HLA-091119-AB-NGS_S14_L001_R2_001.fastq.gz

Q:/IlluminaOutput/2018/062818AB NGS/Data/Intensities/BaseCalls/18-06875-HLA-062818-NGS_S11_L001_R1_001.fastq.gz

Using the file.copy function gives an error that the file name is too long, a known issue in Windows (I am using RStudio on Windows 10).

I found that if I set the working directory directory to the file location, I am able to copy files. Starting with a table like this:

file
path

19-14889-HLA-091119-AB-NGS_S14_L001_R1_001.fastq.gz
Q:/IlluminaOutput/2019/091119 AB NGS/Data/Intensities/BaseCalls/

19-14889-HLA-091119-AB-NGS_S14_L001_R2_001.fastq.gz
Q:/IlluminaOutput/2019/091119 AB NGS/Data/Intensities/BaseCalls/

18-09772-HLA-062818-NGS_S11_L001_R1_001.fastq.gz
Q:/IlluminaOutput/2018/062818AB NGS/Data/Intensities/BaseCalls/

18-09772-HLA-062818-NGS_S11_L001_R2_001.fastq.gz
Q:/IlluminaOutput/2018/062818AB NGS/Data/Intensities/BaseCalls/

I used the following code to sucsessfully copy the first file:

(dir <- as.character(as.vector(file_and_path[1,2])))
setwd(dir)
(file <- as.character(as.vector(file_and_path[1,1])))
(file.copy(file, Trusight_output) %>% as.tibble)

I got this to work, but I don't know how to apply these steps to every column in my table. I think i probably have to use the lapply function, but I'm not sure how to construct it.

Body:
This should do the trick, assuming that file_and_path$file and  file_and_path$path are both character vectors and that Trusight_output is an absolute path:

f <- function(file, from, to) {
    cwd <- setwd(from)
    on.exit(setwd(cwd))
    file.copy(file, to)
}
Map(f, file = file_and_path$file, from = file_and_path$path, to = Trusight_output)

We use Map here rather than lapply because we are applying a function of more than one argument. FWIW, operations like this are often better suited for PowerShell.


问:
join same table with multiple columns and get the latest record with user data
i have two tables named users and message, I want to join both tables and get the last message from each user's conversation with user name.

I want last message by timestamp between two users whether it is from_id or to_id, and similarly, want last message from every user from which I had any conversation.

messages table having columns from_id, to_id, messages.

Below is my code.

```<code>$chats = DB::table('users')
    ->leftJoin('messages', function($query)
    {
        $query->on('users.id', '=', 'messages.to_id')
        ->whereRaw('messages.id IN (select MAX(messages.id) from messages join users on users.id = messages.to_id  group by users.id)');
    })
    ->select('users.id as user_id', 'users.name', DB::raw('CONCAT("https://www.interwebs.co.in/puzzle/attach/", users.avatar) AS image') , 'users.mobile', 'messages.id', 'messages.from_id', 'messages.to_id', 'messages.body as message', 'messages.attachment', 'messages.seen as seen_count', 'messages.created_at')
    ->where('messages.from_id', $request->user_id)
    ->get();

</code>```

My messages table is

```<code>| from_id  | to_id     | message  | created_at          |
| ---------| --------- | ---------| ------------------  |
| 1        | 2         | abc      | 2022-02-04 10:55:34 |
| 2        | 1         | cdef     | 2022-02-05 10:56:34 |
| 1        | 3         | defg     | 2022-02-06 10:57:34 |
| 1        | 3         | hijk     | 2022-02-07 10:58:34 |
| 1        | 2         | lmop     | 2022-02-08 10:59:34 |

</code>```

I have tried this but it is giving me the last message of each conversation of current logged in user and not others.

答:
I would ideally use sql to get the data. First you need to get the Max of created_at grouped by from_id and to_id, then join the results to the same table on the three fields from_id, to_id and created_at. You can also attach the users table to both from_id and to_id as aliases (from_user/to_user)...

This first query gives from_id, to_id, last_message_created_at:

```<code>SELECT
    from_id,
    to_id,
    max(created_at) last_message_created_at
FROM messages
GROUP BY from_id, to_id
</code>```

Join the results to the original messages table as follows. The final SQL would be something like this.

```<code>SELECT
    from_user.id,
    to_user.id,
    msgs.message,
    msgs.created_at
    # add any additional field from msgs/from_user/to_user aliases as required
FROM messages msgs
JOIN (             #join the first query above here
    SELECT
        from_id,
        to_id,
        max(created_at) last_message_created_at
    FROM messages
    GROUP BY from_id, to_id
) last_msgs ON
    (msgs.from_id = last_msgs.from_id
    AND msgs.to_id = last_msgs.to_id
    AND msgs.created_at = last_msgs.last_message_created_at)

JOIN users from_user on from_user.id = msgs.from_id
JOIN users to_user on to_user.id = msgs.to_id
</code>```

Use Laravel function DB::select to fetch results of the query into array. Put the above sql string into a variable $sql

```<code>$results_array = DB::select($sql)
</code>```

Read about DB::select here. You can also use parameterised queries.

And here are some examples running raw sql queries in Laravel

Body:
i have two tables named users and message, I want to join both tables and get the last message from each user's conversation with user name.

I want last message by timestamp between two users whether it is from_id or to_id, and similarly, want last message from every user from which I had any conversation.

messages table having columns from_id, to_id, messages.

Below is my code.

$chats = DB::table('users')
    ->leftJoin('messages', function($query)
    {
        $query->on('users.id', '=', 'messages.to_id')
        ->whereRaw('messages.id IN (select MAX(messages.id) from messages join users on users.id = messages.to_id  group by users.id)');
    })
    ->select('users.id as user_id', 'users.name', DB::raw('CONCAT("https://www.interwebs.co.in/puzzle/attach/", users.avatar) AS image') , 'users.mobile', 'messages.id', 'messages.from_id', 'messages.to_id', 'messages.body as message', 'messages.attachment', 'messages.seen as seen_count', 'messages.created_at')
    ->where('messages.from_id', $request->user_id)
    ->get();

My messages table is

| from_id  | to_id     | message  | created_at          |
| ---------| --------- | ---------| ------------------  |
| 1        | 2         | abc      | 2022-02-04 10:55:34 |
| 2        | 1         | cdef     | 2022-02-05 10:56:34 |
| 1        | 3         | defg     | 2022-02-06 10:57:34 |
| 1        | 3         | hijk     | 2022-02-07 10:58:34 |
| 1        | 2         | lmop     | 2022-02-08 10:59:34 |

I have tried this but it is giving me the last message of each conversation of current logged in user and not others.

Body:
I would ideally use sql to get the data. First you need to get the Max of created_at grouped by from_id and to_id, then join the results to the same table on the three fields from_id, to_id and created_at. You can also attach the users table to both from_id and to_id as aliases (from_user/to_user)...

This first query gives from_id, to_id, last_message_created_at:

SELECT
    from_id,
    to_id,
    max(created_at) last_message_created_at
FROM messages
GROUP BY from_id, to_id

Join the results to the original messages table as follows. The final SQL would be something like this.

SELECT
    from_user.id,
    to_user.id,
    msgs.message,
    msgs.created_at
    # add any additional field from msgs/from_user/to_user aliases as required
FROM messages msgs
JOIN (             #join the first query above here
    SELECT
        from_id,
        to_id,
        max(created_at) last_message_created_at
    FROM messages
    GROUP BY from_id, to_id
) last_msgs ON
    (msgs.from_id = last_msgs.from_id
    AND msgs.to_id = last_msgs.to_id
    AND msgs.created_at = last_msgs.last_message_created_at)

JOIN users from_user on from_user.id = msgs.from_id
JOIN users to_user on to_user.id = msgs.to_id

Use Laravel function DB::select to fetch results of the query into array. Put the above sql string into a variable $sql

$results_array = DB::select($sql)

Read about DB::select here. You can also use parameterised queries.

And here are some examples running raw sql queries in Laravel


问:
Win7 API to programmatically add a group to a security policy?
I have found the API to add a group programmatically, the next logical step is to add this group to a security policy, such as "Create Global Objects".

What is the Win7 API to programmatically add a group to a security policy?

答:
How To Manage User Privileges Programmatically in Windows NT

To look up the SID for a group, use LsaLookupNames


问:
Why is the parse_number function saying my character vector is not a character?
I'm using R to pull out numbers from strings of ids. In the past, I've used readr's parse_number() function, but recently, I'm getting a bizarre error where it's saying that my character column is not character:

```<code>library(dplyr)
library(readr)

test <- tibble(id_numbers = c("ICKLSD5", "DFTEM4", "DPDFE45"))

test %>%
  parse_number(id_numbers)

Error in parse_vector(x, col_number(), na = na, locale = locale, trim_ws = trim_ws) :
  is.character(x) is not TRUE

</code>```

But, the column is clearly a character vector:

```<code>
#Yields true
is.character(test$id_numbers)
</code>```

And, even wrapping the column in as.character still gives the same error:

```<code>test %>%
  parse_number(as.character(id_numbers))
</code>```

I know there's other workarounds with other functions, but I really want to use parse_number(). Can anyone explain why I'm getting this error and how I can fix it?

答:
You can't pipe your data frame directly into parse_number. You would need to pipe into a mutate:

```<code>test %>% mutate(id_numbers=parse_number(id_numbers))
</code>```

assuming you want to change the data.frame itself. Because

```<code>test %>% parse_number(id_numbers)
</code>```

is the same as

```<code>parse_number(test, id_numbers)
</code>```

and test is not a character vector, it is a data.frame, hence you get the error.

Body:
I'm using R to pull out numbers from strings of ids. In the past, I've used readr's parse_number() function, but recently, I'm getting a bizarre error where it's saying that my character column is not character:

library(dplyr)
library(readr)

test <- tibble(id_numbers = c("ICKLSD5", "DFTEM4", "DPDFE45"))

test %>%
  parse_number(id_numbers)

Error in parse_vector(x, col_number(), na = na, locale = locale, trim_ws = trim_ws) :
  is.character(x) is not TRUE

But, the column is clearly a character vector:

#Yields true
is.character(test$id_numbers)

And, even wrapping the column in as.character still gives the same error:

test %>%
  parse_number(as.character(id_numbers))

I know there's other workarounds with other functions, but I really want to use parse_number(). Can anyone explain why I'm getting this error and how I can fix it?

Body:
You can't pipe your data frame directly into parse_number. You would need to pipe into a mutate:

test %>% mutate(id_numbers=parse_number(id_numbers))

assuming you want to change the data.frame itself. Because

test %>% parse_number(id_numbers)

is the same as

parse_number(test, id_numbers)

and test is not a character vector, it is a data.frame, hence you get the error.


问:
Get the first row of each group of unique values in another column
I want to extract from a dataframe only the first row of each value other than one of the columns (pandas) for example:

```<code>df
   col_A col_B
0      1     x
1      2    xx
2      3    xx
3      4     y
4      5     y
</code>```

to

```<code>df1
  col_A col_B
0      1     x
1      2    xx
2      4     y
</code>```

答:
Use groupby + first:

```<code>firsts = df.groupby('col_B', as_index=False).first()
</code>```

Output:

```<code>>>> firsts
  col_B  col_A
0     x      1
1    xx      2
2     y      4
</code>```

If the order of the columns is important:

```<code>firsts = df.loc[df.groupby('col_B', as_index=False).first().index]
</code>```

Output:

```<code>>>> firsts
   col_A col_B
0      1     x
1      2    xx
2      3    xx
</code>```

Body:
I want to extract from a dataframe only the first row of each value other than one of the columns (pandas) for example:

df
   col_A col_B
0      1     x
1      2    xx
2      3    xx
3      4     y
4      5     y

to

df1
  col_A col_B
0      1     x
1      2    xx
2      4     y

Body:
Use groupby + first:

firsts = df.groupby('col_B', as_index=False).first()

Output:

>>> firsts
  col_B  col_A
0     x      1
1    xx      2
2     y      4

If the order of the columns is important:

firsts = df.loc[df.groupby('col_B', as_index=False).first().index]

Output:

>>> firsts
   col_A col_B
0      1     x
1      2    xx
2      3    xx


答:
```<code>db.collection.aggregate([
{
  $project: {
    n: {
      n1: "$n1",
      n2: "$n2"
    }
  }
},
{
  $project: {
    lookupFields: {
      $objectToArray: "$n"
    }
  }
},
{
  $addFields: {
    lookupFields: {
      $map: {
        input: "$lookupFields",
        as: "lf",
        in: {
          n: "$$lf.k",
          v: "$$lf.v"
        }
      }
    }
  }
 }
])
</code>```

explained:

project the n1&n2 fields inside object n

project the object n as new field lookupFileds as array.

map to rename the array object keys k,v to n,v as requested

playground

Title: Transform objects into key/value array mongodb

Body:
I have this data:

[
  {
    "_id": "0001",
    "n1": "0001",
    "n2": 1234
  },
  {
    "_id": "0002",
    "n1": "0002",
    "n2": 9876
  }
]

What's the best way to transform it into this, using aggregation framework?:

[
  {
    "_id": "0001",
    "lookupFields": [
      {
        "n": "n1",
        "v": "0001"
      },
      {
        "n": "n2",
        "v": 1234
      }
    ]
  },
  {
    "_id": "0002",
    "lookupFields": [
      {
        "n": "n1",
        "v": "0002"
      },
      {
        "n": "n2",
        "v": 9876
      }
    ]
  }
]

Thanks for your precious help

Body:
db.collection.aggregate([
{
  $project: {
    n: {
      n1: "$n1",
      n2: "$n2"
    }
  }
},
{
  $project: {
    lookupFields: {
      $objectToArray: "$n"
    }
  }
},
{
  $addFields: {
    lookupFields: {
      $map: {
        input: "$lookupFields",
        as: "lf",
        in: {
          n: "$$lf.k",
          v: "$$lf.v"
        }
      }
    }
  }
 }
])

explained:

project the n1&n2 fields inside object n

project the object n as new field lookupFileds as array.

map to rename the array object keys k,v to n,v as requested

playground


问:
Understanding capture by reference in C++ lamdba functions
I thought I understood how capture by reference works in C++ until I faced this situation:

auto inrcrementer = []() {
    int counter = 0;
    return [&counter]() {
        return counter++;
    };
};

int main() {
    auto inc = inrcrementer();
    cout << inc() << ", " << inc() << ", " << inc() << ", " << endl;

    return 0;
}
```
<p>I was expecting this to return <code>0, 1, 2</code> but it returns <code>0, 32765, 32765</code>. Why?</p>
<p>Moreover, if I change it to this:</p>
<pre class="lang-cpp prettyprint-override"><code>auto inrcrementer = []() {
    int counter = 0;
    return [counter]() mutable {
        return counter++;
    };
};

int main() {
    auto inc = inrcrementer();
    cout << inc() << ", " << inc() << ", " << inc() << ", " << endl;

    return 0;
}
</code>```

It is fixed and it returns as expected. What is the difference between the two?

答:
The problem is that in your first example, the lambda stores a reference to, rather than a copy of, counter.  But counter goes out of scope when incrementer returns so that reference is left dangling.  Then, when you invoke the lambda returned by incrementer, that reference is invalid.

What this usually means in practise is that it is pointing at an area of the stack that has been / is being used for something else.  That's why you get weird results.

In your second example, the lambda has its own copy of counter, and that remains in scope for as long as the lambda remains in existence.  Since you have declared the lambda mutable, it can do whatever it likes with its (private) copy of counter without anybody else messing with it.

So capture by reference can be dangerous.  Use with care!  A reference is just a pointer really, under the skin, but because it doesn't look like one it's easy to slip up.

Body:
I thought I understood how capture by reference works in C++ until I faced this situation:

auto inrcrementer = []() {
    int counter = 0;
    return [&counter]() {
        return counter++;
    };
};

int main() {
    auto inc = inrcrementer();
    cout << inc() << ", " << inc() << ", " << inc() << ", " << endl;

    return 0;
}

I was expecting this to return 0, 1, 2 but it returns 0, 32765, 32765. Why?

Moreover, if I change it to this:

auto inrcrementer = []() {
    int counter = 0;
    return [counter]() mutable {
        return counter++;
    };
};

int main() {
    auto inc = inrcrementer();
    cout << inc() << ", " << inc() << ", " << inc() << ", " << endl;

    return 0;
}

It is fixed and it returns as expected. What is the difference between the two?


问:
Extract function error in oracle invalid extract source
I was trying to extract minute from a datetime field . But i am getting  the below issue . Please share any thoughts.</p>
```<code>code :-
select extract( minute from UPD_Action_STRT_Holi_update ) from my table

Error
ORA-30076: invalid extract field for extract source

Sample Data :-
UPD_Action_STRT_Holi_update
9/3/2021 12:39:27 PM
``
Extract source is datetime field
</code>```

答:
Oracle does not have a DATETIME data type; it has either a DATE or a TIMESTAMP and they both have year, month, day, hour, minute and second components; TIMESTAMP can, optionally, have fractional seconds and time zone components.

You get that error if your column has the DATE data type.

The EXTRACT function documentation states that:

you can extract only YEAR, MONTH, and DAY from a DATE value.

If you want to use EXTRACT to get the MINUTE value then cast the DATE to a TIMESTAMP:

select extract( minute from CAST(UPD_Action_STRT_Holi_update AS TIMESTAMP) )
from   my_table
```

db<>fiddle here

Body:
I was trying to extract minute from a datetime field . But i am getting  the below issue . Please share any thoughts.

code :-
select extract( minute from UPD_Action_STRT_Holi_update ) from my table

Error
ORA-30076: invalid extract field for extract source

Sample Data :-
UPD_Action_STRT_Holi_update
9/3/2021 12:39:27 PM
``
Extract source is datetime field

Body:
Oracle does not have a DATETIME data type; it has either a DATE or a TIMESTAMP and they both have year, month, day, hour, minute and second components; TIMESTAMP can, optionally, have fractional seconds and time zone components.

You get that error if your column has the DATE data type.

The EXTRACT function documentation states that:

you can extract only YEAR, MONTH, and DAY from a DATE value.

If you want to use EXTRACT to get the MINUTE value then cast the DATE to a TIMESTAMP:

select extract( minute from CAST(UPD_Action_STRT_Holi_update AS TIMESTAMP) )
from   my_table

db<>fiddle here


答:
Figured it out. Thank you your comment helped:</p>
```<code>for key, value in racoondict.items():
    try:
        for value in racoondict[key]:
            if float(value).is_integer():
                racoondict[key][racoondict[key].index(value)] = int(value)
            else:
                racoondict[key][racoondict[key].index(value)] = float(value)
    except:
        pass
</code>```

Title: Why can I change the type of dictionary values inside the loop but its failing when I check

Body:
I have a dictionary of mixed value types: floats, int and string. When I first open and read the values are are all strings and I need to change them to the appropriate type. I made a for/try loop that seemed to work, the print statements inside the loop show that the value have changed. But when I double checked I saw they were all still strings? I cant seem to figure out whats going wrong.

Data:

>>> racoondict
{'Year': ['2008', '2008', '2008', '2008', '2008', '2008', '2008', '2008', '2008', '2008', '2008', '2008', '2008', '2008'], 'Day': ['6/19/2008', '6/19/2008', '6/19/2008', '6/19/2008', '6/19/2008', '6/19/2008', '6/19/2008', '6/19/2008', '6/19/2008', '6/19/2008', '6/19/2008', '6/19/2008', '6/19/2008', '6/19/2008'], 'Time': ['6:00 AM', '7:00 AM', '8:00 AM', '8:00 AM', '9:00 AM', '9:00 AM', '10:00 AM', '11:00 AM', '11:00 AM', '12:00 PM', '1:00 PM', '2:00 PM', '3:00 PM', '4:00 PM'], 'George #': ['6', '6', '6', '6', '6', '6', '6', '6', '6', '6', '6', '6', '6', '6'], ' X': ['591071.374756626', '591112.752918467', '591159.738875885', '591177.048458523', '591184.067274814', '591192.036321619', '591210.13894318', '591257.350537335', '591307.574400613', '591303.184361127', '591243.186822335', '591183.256748391', '591123.432415996', '591121.339525596'], ' Y': ['4504664.29938361', '4504620.85001397', '4504583.53621216', '4504546.70960308', '4504531.77688002', '4504547.28129659', '4504582.5013916', '4504619.52929159', '4504641.73534564', '4504639.79432702', '4504639.25087614', '4504636.35497609', '4504631.76702955', '4504571.80354225'], ' Asleep': ['False', 'False', 'False', 'False', 'False', 'False', 'False', 'False', 'False', 'False', 'False', 'False', 'False', 'False'], 'Behavior Mode': ['SafeSearchModifier', 'RiskySearchModifier', 'RiskyForageModifier', 'RiskyForageModifier', 'RiskyForageModifier', 'RiskyForageModifier', 'RiskyForageModifier', 'RiskyForageModifier', 'RiskyForageModifier', 'RiskyForageModifier', 'RiskyForageModifier', 'RiskyForageModifier', 'RiskyForageModifier', 'RiskyForageModifier'], 'Energy Level': ['1000', '887.8', '777.8', '704.1', '687.6', '770.6', '698', '588', '486.8', '478', '368', '258', '148', '37.9999999999999'], 'Risk': ['0.00192', '0.002', '0.00134', '0.000165', '0.00017', '0.00132', '0.002', '0.00184', '0.00016', '0.002', '0.002', '0.002', '0.002', '0.00078'], 'ProbFoodCap': ['0.045', '0.1', '0.067', '0.33', '0.34', '0.066', '0.1', '0.092', '0.008', '0.1', '0.1', '0.1', '0.1', '0.039'], 'MVL': ['0.715', '0.65', '0.65', '0.3', '0.3', '0.65', '0.65', '0.65', '0.65', '0.65', '0.65', '0.65', '0.65', '0.65'], 'MSL': ['60', '60', '40.2', '16.5', '17', '39.6', '60', '55.2', '4.8', '60', '60', '60', '60', '23.4'], 'PercptionDist': ['50.5', '50', '50', '50', '50', '50', '50', '50', '50', '50', '50', '50', '50', '50'], 'Percent Step': ['1', '1', '0.67', '0.33', '0.34', '0.66', '1', '0.92', '0.08', '1', '1', '1', '1', '0.39']}

Loop:

for key, value in racoondict.items():
    try:
        for value in racoondict[key]:
            value = float(value)
            if value.is_integer():
                value = int(value)
            print(type(value), value)
    except ValueError:
        print(type(value), value)
        pass

Output:

<class 'int'> 2008
<class 'int'> 2008
<class 'int'> 2008
<class 'int'> 2008
<class 'int'> 2008
<class 'int'> 2008
<class 'int'> 2008
<class 'int'> 2008
<class 'int'> 2008
<class 'int'> 2008
<class 'int'> 2008
<class 'int'> 2008
<class 'int'> 2008
<class 'str'> 6/19/2008
<class 'str'> 6:00 AM
<class 'int'> 6
<class 'int'> 6
<class 'int'> 6
<class 'int'> 6
<class 'int'> 6
<class 'int'> 6
<class 'int'> 6
<class 'int'> 6
<class 'int'> 6
<class 'int'> 6
<class 'int'> 6
<class 'int'> 6
<class 'int'> 6
<class 'int'> 6
<class 'float'> 591071.374756626
<class 'float'> 591112.752918467
<class 'float'> 591159.738875885
<class 'float'> 591177.048458523
<class 'float'> 591184.067274814
<class 'float'> 591192.036321619
<class 'float'> 591210.13894318
<class 'float'> 591257.350537335
<class 'float'> 591307.574400613
<class 'float'> 591303.184361127
etc

Check:

for key, value in racoondict.items():
    for value in racoondict[key]:
        print (type(value),value)

<class 'str'> 2008
<class 'str'> 2008
<class 'str'> 2008
<class 'str'> 2008
<class 'str'> 2008
<class 'str'> 2008
<class 'str'> 2008
<class 'str'> 2008
<class 'str'> 2008
<class 'str'> 2008
<class 'str'> 2008
<class 'str'> 2008
<class 'str'> 2008
<class 'str'> 2008
<class 'str'> 6/19/2008
<class 'str'> 6/19/2008
<class 'str'> 6/19/2008
<class 'str'> 6/19/2008
<class 'str'> 6/19/2008
<class 'str'> 6/19/2008
<class 'str'> 6/19/2008
<class 'str'> 6/19/2008
<class 'str'> 6/19/2008
<class 'str'> 6/19/2008
<class 'str'> 6/19/2008
<class 'str'> 6/19/2008
<class 'str'> 6/19/2008
<class 'str'> 6/19/2008
<class 'str'> 6:00 AM
<class 'str'> 7:00 AM
<class 'str'> 8:00 AM
<class 'str'> 8:00 AM
<class 'str'> 9:00 AM
<class 'str'> 9:00 AM
<class 'str'> 10:00 AM
<class 'str'> 11:00 AM
<class 'str'> 11:00 AM
<class 'str'> 12:00 PM
<class 'str'> 1:00 PM
<class 'str'> 2:00 PM
<class 'str'> 3:00 PM
<class 'str'> 4:00 PM
<class 'str'> 6
<class 'str'> 6
<class 'str'> 6
<class 'str'> 6
<class 'str'> 6
<class 'str'> 6
<class 'str'> 6
<class 'str'> 6
<class 'str'> 6
<class 'str'> 6
<class 'str'> 6
<class 'str'> 6
<class 'str'> 6
<class 'str'> 6
<class 'str'> 591071.374756626
<class 'str'> 591112.752918467
<class 'str'> 591159.738875885
<class 'str'> 591177.048458523
<class 'str'> 591184.067274814
<class 'str'> 591192.036321619
<class 'str'> 591210.13894318
<class 'str'> 591257.350537335
<class 'str'> 591307.574400613
<class 'str'> 591303.184361127
etc

Body:
Figured it out. Thank you your comment helped:

for key, value in racoondict.items():
    try:
        for value in racoondict[key]:
            if float(value).is_integer():
                racoondict[key][racoondict[key].index(value)] = int(value)
            else:
                racoondict[key][racoondict[key].index(value)] = float(value)
    except:
        pass


问:
What are the underlying git merge processes within the staging area?
Git does the merge magic, and then lets the user resolve real conflicts, which is as it should be. I'm looking for a low level description of the how and why of the basic git merge and how it uses the staging area.

I've just read the Git Parable, and the comment on here that

Even taking into account the fact that its is "parable" and not recount of the history of Git (which you can find in some detail on Git Wiki, by the way), one point stays: it is IMVHO bad practice to explain staging area in the terms of splitting changes into more than one commit and/or comitting with dity tree, i.e. with some changes uncomitted. Staging area main strength (besides being explicit version of other SCMs implicit to-be-added area) is dealing with CONFLICTED MERGE, and that is how it should be explained, I think.

The git merge man page identifies the stage 1/2/3 elements of the merge, but obviously doesn't go into details of whys and wherefores.

Can folk advise on any articles on how and why git manages to achieve the results others don't (over and above the Linus V Bram detailed in Wincent's blog), i.e. the alleged Trivial part?

Most web articles assume that merges 'just happen', and I haven't found anything that explains the issues (e.g. the need for small commits, the value of a common commit, etc).

答:
Most every VCS employs the basic concept of a three-way merge.  This compares two branches with a common ancestor of each, so if a line of code is different between the two branches, you know which branch changed it.  If they both changed it, you have a merge conflict that must be resolved by a human.

There are a few cases where it is difficult to determine a suitable common ancestor.  A lot of research went into different algorithms for this, many involving the tracking of additional metadata with the commits.

Linus' essential innovation was the tracking of trees rather than files.  That's sort of a subtle distinction.  To illustrate with the example from Wincent's blog, consider a file foo in branch A.  You branch off to make branch B.  In branch A foo is renamed to bar.  In branch B, it is deleted.  You then attempt to merge.

If you are tracking files, it goes like this:

Before branching, version 1 of file foo is created.

After the next commit, branch A points to version 2 of foo, which is a deleted file, and version 1 of new file bar.

After the next commit, branch B points to version 2.1 of foo, which is a deleted file.

When you merge, version 2 and 2.1 of foo are compared and found to be identical.  No merge conflict there.  Branch B doesn't even have a file called bar, so no conflict there either.  You end up with the merge algorithm silently accepting branch A's rename, even though there was a real conflict between foo being deleted and it being renamed.

If you are tracking trees, it goes like this:

Before branching, a blob with hash dcb8bd7a97ab39f4c156a1a96d4b10720a39fb81 is created.  A tree is created with an entry containing a label foo pointing to the hash.

After the next commit, branch A points to a tree with an entry containing a label bar pointing to the same hash.

After the next commit, branch B points to an empty tree.

When you merge, the trees are compared, with B showing a deletion and A showing a rename of the blob dcb8bd7a97ab39f4c156a1a96d4b10720a39fb81.  Human is asked which one he prefers.

You can mitigate the effect somewhat with a file-tracking VCS by adding metadata for renames, but git's way uses its normal standard data structure.  Also, the metadata way has difficulties with complex merges where there are many possible choices for the common ancestor.  You could put a billion possible paths between the common ancestor and the two branch heads, and git will still see a blob with the same hash and be able to detect a rename and a delete.  It's also difficult to preserve metadata when accepting changes in a patch via email, for instance.

It gets a little trickier with a renamed file that changes at the same time, but by tracking the trees, git has all the information it needs.  It sees blob dcb8bd7a97ab39f4c156a1a96d4b10720a39fb81 gone from both branches, but it also sees a new tree entry pointing to a new blob, and can compare the two.  If a significant portion of the file matches, it's considered a rename.  Obviously this breaks down if you make a ton of changes in a renamed file, but at some point no merge algorithm is going to be able to help you.

See this email from Linus for more insight about his philosophy on this topic.