问:
Spark-submit issue loading classes
I'm using HDP 2.6. I downloaded newest version of Spark (2.2.1) and using spark-submit I'm trying to run my jar (build with same version of Spark as assembly). However, I'm getting error:

```<code>Class org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider not found
</code>```

my $HADOOP_CONF_DIR is /etc/hadoop/conf which is link to /usr/hdp/current/hadoop-client/conf
my yarn-site.xml in yarn.application.classpath contains entry: /usr/hdp/current/hadoop-yarn-client/* This directory contains jar hadoop-yarn-common.jar which contains class org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider . That is why don't understand what is going.

I did this checks base on suggestions in:
link 1
link 2
Bellow full stacktrace if usefull:

```<code>[root@omm101 bin]# pwd
/opt/spark_2.2.1/spark-2.2.1-bin-hadoop2.7/bin
[root@omm101 bin]# echo $HADOOP_CONF_DIR
/etc/hadoop/conf
[root@omm101 bin]# ./spark-submit --class net.atos.ooredooom.reportengine.trareport.TraReport --master yarn --deploy-mode client --num-executors 18 --executor-cores 5 --executor-memory 15g --driver-memory 1g /root/jars/report-compute-engine.jar
18/01/25 17:22:50 INFO spark.SparkContext: Running Spark version 2.2.1
18/01/25 17:22:50 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
18/01/25 17:22:51 INFO spark.SparkContext: Submitted application: TRA-Report
18/01/25 17:22:51 INFO spark.SecurityManager: Changing view acls to: root
18/01/25 17:22:51 INFO spark.SecurityManager: Changing modify acls to: root
18/01/25 17:22:51 INFO spark.SecurityManager: Changing view acls groups to:
18/01/25 17:22:51 INFO spark.SecurityManager: Changing modify acls groups to:
18/01/25 17:22:51 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users  with view permissions: Set(root); groups with view permissions: Set(); users  with modify permissions: Set(root); groups with modify permissions: Set()
18/01/25 17:22:51 INFO util.Utils: Successfully started service 'sparkDriver' on port 41613.
18/01/25 17:22:51 INFO spark.SparkEnv: Registering MapOutputTracker
18/01/25 17:22:51 INFO spark.SparkEnv: Registering BlockManagerMaster
18/01/25 17:22:51 INFO storage.BlockManagerMasterEndpoint: Using org.apache.spark.storage.DefaultTopologyMapper for getting topology information
18/01/25 17:22:51 INFO storage.BlockManagerMasterEndpoint: BlockManagerMasterEndpoint up
18/01/25 17:22:51 INFO storage.DiskBlockManager: Created local directory at /tmp/blockmgr-27408430-08bf-4252-b320-e68e6d103154
18/01/25 17:22:51 INFO memory.MemoryStore: MemoryStore started with capacity 366.3 MB
18/01/25 17:22:51 INFO spark.SparkEnv: Registering OutputCommitCoordinator
18/01/25 17:22:51 INFO util.log: Logging initialized @1743ms
18/01/25 17:22:51 INFO server.Server: jetty-9.3.z-SNAPSHOT
18/01/25 17:22:51 INFO server.Server: Started @1814ms
18/01/25 17:22:51 INFO server.AbstractConnector: Started ServerConnector@c835d12{HTTP/1.1,[http/1.1]}{0.0.0.0:4040}
18/01/25 17:22:51 INFO util.Utils: Successfully started service 'SparkUI' on port 4040.
18/01/25 17:22:51 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@2a2bb0eb{/jobs,null,AVAILABLE,@Spark}
18/01/25 17:22:51 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@58783f6c{/jobs/json,null,AVAILABLE,@Spark}
18/01/25 17:22:51 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@512d92b{/jobs/job,null,AVAILABLE,@Spark}
18/01/25 17:22:51 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1bc53649{/jobs/job/json,null,AVAILABLE,@Spark}
18/01/25 17:22:51 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@47d93e0d{/stages,null,AVAILABLE,@Spark}
18/01/25 17:22:51 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@751e664e{/stages/json,null,AVAILABLE,@Spark}
18/01/25 17:22:51 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@182b435b{/stages/stage,null,AVAILABLE,@Spark}
18/01/25 17:22:51 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@3704122f{/stages/stage/json,null,AVAILABLE,@Spark}
18/01/25 17:22:51 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@60afd40d{/stages/pool,null,AVAILABLE,@Spark}
18/01/25 17:22:51 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@3f2049b6{/stages/pool/json,null,AVAILABLE,@Spark}
18/01/25 17:22:51 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@ea27e34{/storage,null,AVAILABLE,@Spark}
18/01/25 17:22:51 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@e72dba7{/storage/json,null,AVAILABLE,@Spark}
18/01/25 17:22:51 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1dfd5f51{/storage/rdd,null,AVAILABLE,@Spark}
18/01/25 17:22:51 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@24855019{/storage/rdd/json,null,AVAILABLE,@Spark}
18/01/25 17:22:51 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@4d4d8fcf{/environment,null,AVAILABLE,@Spark}
18/01/25 17:22:51 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@6f0628de{/environment/json,null,AVAILABLE,@Spark}
18/01/25 17:22:51 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1e392345{/executors,null,AVAILABLE,@Spark}
18/01/25 17:22:51 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@4ced35ed{/executors/json,null,AVAILABLE,@Spark}
18/01/25 17:22:51 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@7bd69e82{/executors/threadDump,null,AVAILABLE,@Spark}
18/01/25 17:22:51 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@51b01960{/executors/threadDump/json,null,AVAILABLE,@Spark}
18/01/25 17:22:51 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@27dc79f7{/static,null,AVAILABLE,@Spark}
18/01/25 17:22:51 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@7674a051{/,null,AVAILABLE,@Spark}
18/01/25 17:22:51 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@6754ef00{/api,null,AVAILABLE,@Spark}
18/01/25 17:22:51 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@3301500b{/jobs/job/kill,null,AVAILABLE,@Spark}
18/01/25 17:22:51 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@15deb1dc{/stages/stage/kill,null,AVAILABLE,@Spark}
18/01/25 17:22:51 INFO ui.SparkUI: Bound SparkUI to 0.0.0.0, and started at http://10.4.110.24:4040
18/01/25 17:22:51 INFO spark.SparkContext: Added JAR file:/root/jars/report-compute-engine.jar at spark://10.4.110.24:41613/jars/report-compute-engine.jar with timestamp 1516886571716
18/01/25 17:22:52 WARN shortcircuit.DomainSocketFactory: The short-circuit local reads feature cannot be used because libhadoop cannot be loaded.
18/01/25 17:22:52 INFO impl.TimelineClientImpl: Timeline service address: http://omm103.in.nawras.com.om:8188/ws/v1/timeline/
18/01/25 17:22:52 INFO service.AbstractService: Service org.apache.hadoop.yarn.client.api.impl.YarnClientImpl failed in state STARTED; cause: java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider not found
java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider not found
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2227)
        at org.apache.hadoop.yarn.client.RMProxy.createRMFailoverProxyProvider(RMProxy.java:161)
        at org.apache.hadoop.yarn.client.RMProxy.createRMProxy(RMProxy.java:94)
        at org.apache.hadoop.yarn.client.ClientRMProxy.createRMProxy(ClientRMProxy.java:72)
        at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceStart(YarnClientImpl.java:187)
        at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
        at org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:153)
        at org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.start(YarnClientSchedulerBackend.scala:56)
        at org.apache.spark.scheduler.TaskSchedulerImpl.start(TaskSchedulerImpl.scala:173)
        at org.apache.spark.SparkContext.<init>(SparkContext.scala:509)
        at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2516)
        at org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:918)
        at org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:910)
        at scala.Option.getOrElse(Option.scala:121)
        at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:910)
        at net.atos.ooredooom.reportengine.trareport.TraReport$.main(TraReport.scala:14)
        at net.atos.ooredooom.reportengine.trareport.TraReport.main(TraReport.scala)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:497)
        at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:775)
        at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:180)
        at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:205)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:119)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider not found
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2195)
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2219)
        ... 25 more
Caused by: java.lang.ClassNotFoundException: Class org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider not found
        at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2101)
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2193)
        ... 26 more
18/01/25 17:22:52 ERROR spark.SparkContext: Error initializing SparkContext.
java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider not found
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2227)
        at org.apache.hadoop.yarn.client.RMProxy.createRMFailoverProxyProvider(RMProxy.java:161)
        at org.apache.hadoop.yarn.client.RMProxy.createRMProxy(RMProxy.java:94)
        at org.apache.hadoop.yarn.client.ClientRMProxy.createRMProxy(ClientRMProxy.java:72)
        at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceStart(YarnClientImpl.java:187)
        at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
        at org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:153)
        at org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.start(YarnClientSchedulerBackend.scala:56)
        at org.apache.spark.scheduler.TaskSchedulerImpl.start(TaskSchedulerImpl.scala:173)
        at org.apache.spark.SparkContext.<init>(SparkContext.scala:509)
        at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2516)
        at org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:918)
        at org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:910)
        at scala.Option.getOrElse(Option.scala:121)
        at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:910)
        at net.atos.ooredooom.reportengine.trareport.TraReport$.main(TraReport.scala:14)
        at net.atos.ooredooom.reportengine.trareport.TraReport.main(TraReport.scala)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:497)
        at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:775)
        at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:180)
        at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:205)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:119)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider not found
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2195)
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2219)
        ... 25 more
Caused by: java.lang.ClassNotFoundException: Class org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider not found
        at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2101)
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2193)
        ... 26 more
18/01/25 17:22:52 INFO server.AbstractConnector: Stopped Spark@c835d12{HTTP/1.1,[http/1.1]}{0.0.0.0:4040}
18/01/25 17:22:52 INFO ui.SparkUI: Stopped Spark web UI at http://10.4.110.24:4040
18/01/25 17:22:52 WARN cluster.YarnSchedulerBackend$YarnSchedulerEndpoint: Attempted to request executors before the AM has registered!
18/01/25 17:22:52 INFO cluster.YarnClientSchedulerBackend: Stopped
18/01/25 17:22:52 INFO spark.MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
18/01/25 17:22:52 INFO memory.MemoryStore: MemoryStore cleared
18/01/25 17:22:52 INFO storage.BlockManager: BlockManager stopped
18/01/25 17:22:52 INFO storage.BlockManagerMaster: BlockManagerMaster stopped
18/01/25 17:22:52 WARN metrics.MetricsSystem: Stopping a MetricsSystem that is not running
18/01/25 17:22:52 INFO scheduler.OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
18/01/25 17:22:52 INFO spark.SparkContext: Successfully stopped SparkContext
Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider not found
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2227)
        at org.apache.hadoop.yarn.client.RMProxy.createRMFailoverProxyProvider(RMProxy.java:161)
        at org.apache.hadoop.yarn.client.RMProxy.createRMProxy(RMProxy.java:94)
        at org.apache.hadoop.yarn.client.ClientRMProxy.createRMProxy(ClientRMProxy.java:72)
        at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceStart(YarnClientImpl.java:187)
        at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
        at org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:153)
        at org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.start(YarnClientSchedulerBackend.scala:56)
        at org.apache.spark.scheduler.TaskSchedulerImpl.start(TaskSchedulerImpl.scala:173)
        at org.apache.spark.SparkContext.<init>(SparkContext.scala:509)
        at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2516)
        at org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:918)
        at org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:910)
        at scala.Option.getOrElse(Option.scala:121)
        at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:910)
        at net.atos.ooredooom.reportengine.trareport.TraReport$.main(TraReport.scala:14)
        at net.atos.ooredooom.reportengine.trareport.TraReport.main(TraReport.scala)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:497)
        at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:775)
        at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:180)
        at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:205)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:119)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider not found
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2195)
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2219)
        ... 25 more
Caused by: java.lang.ClassNotFoundException: Class org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider not found
        at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2101)
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2193)
        ... 26 more
18/01/25 17:22:52 INFO util.ShutdownHookManager: Shutdown hook called
18/01/25 17:22:52 INFO util.ShutdownHookManager: Deleting directory /tmp/spark-8ea65e11-e6d7-480c-874b-e35071fa6d7f
</code>```

Any support/hint would be greatly appreciated.

EDIT:
Due to lack of better ideas, to spark-submit I added --jars /usr/hdp/current/hadoop-yarn-client/hadoop-yarn-common-2.7.3.2.6.1.0-129.jar and this caused:IllegalAccessError: tried to access method org.apache.hadoop.yarn.clientRMFailoverProxyProvider.getProxyInternal()

when this didn't worked I tried to put hadoop-yarn-common-2.7.3.2.6.1.0-129.jar into .../<spark_dir>/jars
And I got the same result.

So basically the question is why spark is not using hdp jars. And should it use that (because I see this IllegalAccessError when I force using this lib). In Spark 2.2.1 there is a jar hadoop-yarn-common-2.7.3.jar but this jar doesn't contain RequestHedgingRMFailoverProxyProvider (maybe HDP specific?)

答:
HDP default config uses org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider for the implementation. however, Spark does not pack with this impl. but the other one called ConfiguredRMFailoverProxyProvider. both of them implement interface RMFailoverProxyProvider. (refer to this doc: https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-common/apidocs/org/apache/hadoop/yarn/client/RMFailoverProxyProvider.html)

so in order to fix this, just follow this instruction and change the config on HDP. https://community.hortonworks.com/content/supportkb/178800/errorclass-orgapachehadoopyarnclientrequesthedging.html

Title: Spark-submit issue loading classes

Body:
I'm using HDP 2.6. I downloaded newest version of Spark (2.2.1) and using spark-submit I'm trying to run my jar (build with same version of Spark as assembly). However, I'm getting error:

Class org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider not found

my $HADOOP_CONF_DIR is /etc/hadoop/conf which is link to /usr/hdp/current/hadoop-client/conf
my yarn-site.xml in yarn.application.classpath contains entry: /usr/hdp/current/hadoop-yarn-client/* This directory contains jar hadoop-yarn-common.jar which contains class org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider . That is why don't understand what is going.

I did this checks base on suggestions in:
link 1
link 2
Bellow full stacktrace if usefull:

[root@omm101 bin]# pwd
/opt/spark_2.2.1/spark-2.2.1-bin-hadoop2.7/bin
[root@omm101 bin]# echo $HADOOP_CONF_DIR
/etc/hadoop/conf
[root@omm101 bin]# ./spark-submit --class net.atos.ooredooom.reportengine.trareport.TraReport --master yarn --deploy-mode client --num-executors 18 --executor-cores 5 --executor-memory 15g --driver-memory 1g /root/jars/report-compute-engine.jar
18/01/25 17:22:50 INFO spark.SparkContext: Running Spark version 2.2.1
18/01/25 17:22:50 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
18/01/25 17:22:51 INFO spark.SparkContext: Submitted application: TRA-Report
18/01/25 17:22:51 INFO spark.SecurityManager: Changing view acls to: root
18/01/25 17:22:51 INFO spark.SecurityManager: Changing modify acls to: root
18/01/25 17:22:51 INFO spark.SecurityManager: Changing view acls groups to:
18/01/25 17:22:51 INFO spark.SecurityManager: Changing modify acls groups to:
18/01/25 17:22:51 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users  with view permissions: Set(root); groups with view permissions: Set(); users  with modify permissions: Set(root); groups with modify permissions: Set()
18/01/25 17:22:51 INFO util.Utils: Successfully started service 'sparkDriver' on port 41613.
18/01/25 17:22:51 INFO spark.SparkEnv: Registering MapOutputTracker
18/01/25 17:22:51 INFO spark.SparkEnv: Registering BlockManagerMaster
18/01/25 17:22:51 INFO storage.BlockManagerMasterEndpoint: Using org.apache.spark.storage.DefaultTopologyMapper for getting topology information
18/01/25 17:22:51 INFO storage.BlockManagerMasterEndpoint: BlockManagerMasterEndpoint up
18/01/25 17:22:51 INFO storage.DiskBlockManager: Created local directory at /tmp/blockmgr-27408430-08bf-4252-b320-e68e6d103154
18/01/25 17:22:51 INFO memory.MemoryStore: MemoryStore started with capacity 366.3 MB
18/01/25 17:22:51 INFO spark.SparkEnv: Registering OutputCommitCoordinator
18/01/25 17:22:51 INFO util.log: Logging initialized @1743ms
18/01/25 17:22:51 INFO server.Server: jetty-9.3.z-SNAPSHOT
18/01/25 17:22:51 INFO server.Server: Started @1814ms
18/01/25 17:22:51 INFO server.AbstractConnector: Started ServerConnector@c835d12{HTTP/1.1,[http/1.1]}{0.0.0.0:4040}
18/01/25 17:22:51 INFO util.Utils: Successfully started service 'SparkUI' on port 4040.
18/01/25 17:22:51 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@2a2bb0eb{/jobs,null,AVAILABLE,@Spark}
18/01/25 17:22:51 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@58783f6c{/jobs/json,null,AVAILABLE,@Spark}
18/01/25 17:22:51 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@512d92b{/jobs/job,null,AVAILABLE,@Spark}
18/01/25 17:22:51 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1bc53649{/jobs/job/json,null,AVAILABLE,@Spark}
18/01/25 17:22:51 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@47d93e0d{/stages,null,AVAILABLE,@Spark}
18/01/25 17:22:51 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@751e664e{/stages/json,null,AVAILABLE,@Spark}
18/01/25 17:22:51 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@182b435b{/stages/stage,null,AVAILABLE,@Spark}
18/01/25 17:22:51 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@3704122f{/stages/stage/json,null,AVAILABLE,@Spark}
18/01/25 17:22:51 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@60afd40d{/stages/pool,null,AVAILABLE,@Spark}
18/01/25 17:22:51 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@3f2049b6{/stages/pool/json,null,AVAILABLE,@Spark}
18/01/25 17:22:51 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@ea27e34{/storage,null,AVAILABLE,@Spark}
18/01/25 17:22:51 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@e72dba7{/storage/json,null,AVAILABLE,@Spark}
18/01/25 17:22:51 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1dfd5f51{/storage/rdd,null,AVAILABLE,@Spark}
18/01/25 17:22:51 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@24855019{/storage/rdd/json,null,AVAILABLE,@Spark}
18/01/25 17:22:51 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@4d4d8fcf{/environment,null,AVAILABLE,@Spark}
18/01/25 17:22:51 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@6f0628de{/environment/json,null,AVAILABLE,@Spark}
18/01/25 17:22:51 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1e392345{/executors,null,AVAILABLE,@Spark}
18/01/25 17:22:51 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@4ced35ed{/executors/json,null,AVAILABLE,@Spark}
18/01/25 17:22:51 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@7bd69e82{/executors/threadDump,null,AVAILABLE,@Spark}
18/01/25 17:22:51 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@51b01960{/executors/threadDump/json,null,AVAILABLE,@Spark}
18/01/25 17:22:51 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@27dc79f7{/static,null,AVAILABLE,@Spark}
18/01/25 17:22:51 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@7674a051{/,null,AVAILABLE,@Spark}
18/01/25 17:22:51 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@6754ef00{/api,null,AVAILABLE,@Spark}
18/01/25 17:22:51 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@3301500b{/jobs/job/kill,null,AVAILABLE,@Spark}
18/01/25 17:22:51 INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@15deb1dc{/stages/stage/kill,null,AVAILABLE,@Spark}
18/01/25 17:22:51 INFO ui.SparkUI: Bound SparkUI to 0.0.0.0, and started at http://10.4.110.24:4040
18/01/25 17:22:51 INFO spark.SparkContext: Added JAR file:/root/jars/report-compute-engine.jar at spark://10.4.110.24:41613/jars/report-compute-engine.jar with timestamp 1516886571716
18/01/25 17:22:52 WARN shortcircuit.DomainSocketFactory: The short-circuit local reads feature cannot be used because libhadoop cannot be loaded.
18/01/25 17:22:52 INFO impl.TimelineClientImpl: Timeline service address: http://omm103.in.nawras.com.om:8188/ws/v1/timeline/
18/01/25 17:22:52 INFO service.AbstractService: Service org.apache.hadoop.yarn.client.api.impl.YarnClientImpl failed in state STARTED; cause: java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider not found
java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider not found
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2227)
        at org.apache.hadoop.yarn.client.RMProxy.createRMFailoverProxyProvider(RMProxy.java:161)
        at org.apache.hadoop.yarn.client.RMProxy.createRMProxy(RMProxy.java:94)
        at org.apache.hadoop.yarn.client.ClientRMProxy.createRMProxy(ClientRMProxy.java:72)
        at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceStart(YarnClientImpl.java:187)
        at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
        at org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:153)
        at org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.start(YarnClientSchedulerBackend.scala:56)
        at org.apache.spark.scheduler.TaskSchedulerImpl.start(TaskSchedulerImpl.scala:173)
        at org.apache.spark.SparkContext.<init>(SparkContext.scala:509)
        at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2516)
        at org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:918)
        at org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:910)
        at scala.Option.getOrElse(Option.scala:121)
        at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:910)
        at net.atos.ooredooom.reportengine.trareport.TraReport$.main(TraReport.scala:14)
        at net.atos.ooredooom.reportengine.trareport.TraReport.main(TraReport.scala)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:497)
        at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:775)
        at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:180)
        at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:205)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:119)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider not found
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2195)
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2219)
        ... 25 more
Caused by: java.lang.ClassNotFoundException: Class org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider not found
        at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2101)
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2193)
        ... 26 more
18/01/25 17:22:52 ERROR spark.SparkContext: Error initializing SparkContext.
java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider not found
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2227)
        at org.apache.hadoop.yarn.client.RMProxy.createRMFailoverProxyProvider(RMProxy.java:161)
        at org.apache.hadoop.yarn.client.RMProxy.createRMProxy(RMProxy.java:94)
        at org.apache.hadoop.yarn.client.ClientRMProxy.createRMProxy(ClientRMProxy.java:72)
        at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceStart(YarnClientImpl.java:187)
        at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
        at org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:153)
        at org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.start(YarnClientSchedulerBackend.scala:56)
        at org.apache.spark.scheduler.TaskSchedulerImpl.start(TaskSchedulerImpl.scala:173)
        at org.apache.spark.SparkContext.<init>(SparkContext.scala:509)
        at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2516)
        at org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:918)
        at org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:910)
        at scala.Option.getOrElse(Option.scala:121)
        at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:910)
        at net.atos.ooredooom.reportengine.trareport.TraReport$.main(TraReport.scala:14)
        at net.atos.ooredooom.reportengine.trareport.TraReport.main(TraReport.scala)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:497)
        at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:775)
        at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:180)
        at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:205)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:119)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider not found
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2195)
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2219)
        ... 25 more
Caused by: java.lang.ClassNotFoundException: Class org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider not found
        at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2101)
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2193)
        ... 26 more
18/01/25 17:22:52 INFO server.AbstractConnector: Stopped Spark@c835d12{HTTP/1.1,[http/1.1]}{0.0.0.0:4040}
18/01/25 17:22:52 INFO ui.SparkUI: Stopped Spark web UI at http://10.4.110.24:4040
18/01/25 17:22:52 WARN cluster.YarnSchedulerBackend$YarnSchedulerEndpoint: Attempted to request executors before the AM has registered!
18/01/25 17:22:52 INFO cluster.YarnClientSchedulerBackend: Stopped
18/01/25 17:22:52 INFO spark.MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
18/01/25 17:22:52 INFO memory.MemoryStore: MemoryStore cleared
18/01/25 17:22:52 INFO storage.BlockManager: BlockManager stopped
18/01/25 17:22:52 INFO storage.BlockManagerMaster: BlockManagerMaster stopped
18/01/25 17:22:52 WARN metrics.MetricsSystem: Stopping a MetricsSystem that is not running
18/01/25 17:22:52 INFO scheduler.OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
18/01/25 17:22:52 INFO spark.SparkContext: Successfully stopped SparkContext
Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider not found
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2227)
        at org.apache.hadoop.yarn.client.RMProxy.createRMFailoverProxyProvider(RMProxy.java:161)
        at org.apache.hadoop.yarn.client.RMProxy.createRMProxy(RMProxy.java:94)
        at org.apache.hadoop.yarn.client.ClientRMProxy.createRMProxy(ClientRMProxy.java:72)
        at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceStart(YarnClientImpl.java:187)
        at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
        at org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:153)
        at org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.start(YarnClientSchedulerBackend.scala:56)
        at org.apache.spark.scheduler.TaskSchedulerImpl.start(TaskSchedulerImpl.scala:173)
        at org.apache.spark.SparkContext.<init>(SparkContext.scala:509)
        at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2516)
        at org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:918)
        at org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:910)
        at scala.Option.getOrElse(Option.scala:121)
        at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:910)
        at net.atos.ooredooom.reportengine.trareport.TraReport$.main(TraReport.scala:14)
        at net.atos.ooredooom.reportengine.trareport.TraReport.main(TraReport.scala)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:497)
        at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:775)
        at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:180)
        at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:205)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:119)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider not found
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2195)
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2219)
        ... 25 more
Caused by: java.lang.ClassNotFoundException: Class org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider not found
        at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2101)
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2193)
        ... 26 more
18/01/25 17:22:52 INFO util.ShutdownHookManager: Shutdown hook called
18/01/25 17:22:52 INFO util.ShutdownHookManager: Deleting directory /tmp/spark-8ea65e11-e6d7-480c-874b-e35071fa6d7f

Any support/hint would be greatly appreciated.

EDIT:
Due to lack of better ideas, to spark-submit I added --jars /usr/hdp/current/hadoop-yarn-client/hadoop-yarn-common-2.7.3.2.6.1.0-129.jar and this caused:IllegalAccessError: tried to access method org.apache.hadoop.yarn.clientRMFailoverProxyProvider.getProxyInternal()

when this didn't worked I tried to put hadoop-yarn-common-2.7.3.2.6.1.0-129.jar into .../<spark_dir>/jars
And I got the same result.

So basically the question is why spark is not using hdp jars. And should it use that (because I see this IllegalAccessError when I force using this lib). In Spark 2.2.1 there is a jar hadoop-yarn-common-2.7.3.jar but this jar doesn't contain RequestHedgingRMFailoverProxyProvider (maybe HDP specific?)


来源: stackexchange

问:
Spring - Thymeleaf: Exception processing template
I'm following the a book Spring in Action 5th ed., but I believe that is presents a bug.

This is the GitHub of the book. I'm arrived on chap 3 tacos-jdbc
source of code

When submit my order a sudden error appears:

and in this way on the terminal:

2019-05-25 16:58:18.164 ERROR 11777 --- [nio-8080-exec-7] org.thymeleaf.TemplateEngine             :
    [THYMELEAF][http-nio-8080-exec-7] Exception processing template
    "orderForm": An error happened during template parsing (template:
    "class path resource [templates/orderForm.html]")

org.thymeleaf.exceptions.TemplateInputException: An error happened
  during template parsing (template: "class path resource
  [templates/orderForm.html]")

OrderController:

```<code>@Controller
@RequestMapping("/orders")
@SessionAttributes("order")
public class OrderController {

    private OrderRepository orderRepo;

    public OrderController(OrderRepository orderRepo) {
        this.orderRepo = orderRepo;
    }

    @GetMapping("/current")
    public String orderForm() {
        return "orderForm";
    }

    @PostMapping
    public String processOrder(@Valid Order order, Errors errors,
                               SessionStatus sessionStatus) {
        if (errors.hasErrors()) {
            return "orderForm";
        }

        orderRepo.save(order);
        sessionStatus.setComplete();

        return "redirect:/";
    }

}
</code>```

and orderForm:

<!DOCTYPE html>

    <title>Taco Cloud</title>
    <link rel="stylesheet" th:href="@{/styles.css}" />

<form method="POST" th:action="@{/orders}" th:object="${order}">

Order your taco creations!

    Design another taco

        Please correct the problems below and resubmit.

Deliver my taco masterpieces to...

    Name:
    <input type="text" th:field="*{name}"/>

    Street address:
    <input type="text" th:field="*{street}"/>

    City:
    <input type="text" th:field="*{city}"/>

    State:
    <input type="text" th:field="*{state}"/>

    Zip code:
    <input type="text" th:field="*{zip}"/>

    Credit Card #:
    <input type="text" th:field="*{ccNumber}"/>
    CC Num Error

    Expiration:
    <input type="text" th:field="*{ccExpiration}"/>

    CVV:
    <input type="text" th:field="*{ccCVV}"/>

    <input type="submit" value="Submit order"/>
</form>

```

答:
I think you should handle the display of the validation errors in all the input in the orderForm.html as mentioned in the book as following: (Source Git)

```<code><!-- tag::allButValidation[] -->
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml"
      xmlns:th="http://www.thymeleaf.org">
  <head>
    <title>Taco Cloud</title>
    <link rel="stylesheet" th:href="@{/styles.css}" />
  </head>

  <body>

    <form method="POST" th:action="@{/orders}" th:object="${order}">
      <h1>Order your taco creations!</h1>

      <img th:src="@{/images/TacoCloud.png}"/>
      <a th:href="@{/design}" id="another">Design another taco</a><br/>

      <div th:if="${#fields.hasErrors()}">
        <span class="validationError">
        Please correct the problems below and resubmit.
        </span>
      </div>

      <h3>Deliver my taco masterpieces to...</h3>
      <label for="name">Name: </label>
      <input type="text" th:field="*{name}"/>
<!-- end::allButValidation[] -->
      <span class="validationError"
            th:if="${#fields.hasErrors('name')}"
            th:errors="*{name}">Name Error</span>
<!-- tag::allButValidation[] -->
      <br/>

      <label for="street">Street address: </label>
      <input type="text" th:field="*{street}"/>
<!-- end::allButValidation[] -->
      <span class="validationError"
            th:if="${#fields.hasErrors('street')}"
            th:errors="*{street}">Street Error</span>
<!-- tag::allButValidation[] -->
      <br/>

      <label for="city">City: </label>
      <input type="text" th:field="*{city}"/>
<!-- end::allButValidation[] -->
      <span class="validationError"
            th:if="${#fields.hasErrors('city')}"
            th:errors="*{city}">City Error</span>
<!-- tag::allButValidation[] -->
      <br/>

      <label for="state">State: </label>
      <input type="text" th:field="*{state}"/>
<!-- end::allButValidation[] -->
      <span class="validationError"
            th:if="${#fields.hasErrors('state')}"
            th:errors="*{state}">State Error</span>
<!-- tag::allButValidation[] -->
      <br/>

      <label for="zip">Zip code: </label>
      <input type="text" th:field="*{zip}"/>
<!-- end::allButValidation[] -->
      <span class="validationError"
            th:if="${#fields.hasErrors('zip')}"
            th:errors="*{zip}">Zip Error</span>
<!-- tag::allButValidation[] -->
      <br/>

      <h3>Here's how I'll pay...</h3>
<!-- tag::validatedField[] -->
      <label for="ccNumber">Credit Card #: </label>
      <input type="text" th:field="*{ccNumber}"/>
<!-- end::allButValidation[] -->
      <span class="validationError"
            th:if="${#fields.hasErrors('ccNumber')}"
            th:errors="*{ccNumber}">CC Num Error</span>
<!-- tag::allButValidation[] -->
<!-- end::validatedField[] -->
      <br/>

      <label for="ccExpiration">Expiration: </label>
      <input type="text" th:field="*{ccExpiration}"/>
<!-- end::allButValidation[] -->
      <span class="validationError"
            th:if="${#fields.hasErrors('ccExpiration')}"
            th:errors="*{ccExpiration}">CC Num Error</span>
<!-- tag::allButValidation[] -->
      <br/>

      <label for="ccCVV">CVV: </label>
      <input type="text" th:field="*{ccCVV}"/>
<!-- end::allButValidation[] -->
      <span class="validationError"
            th:if="${#fields.hasErrors('ccCVV')}"
            th:errors="*{ccCVV}">CC Num Error</span>
<!-- tag::allButValidation[] -->
      <br/>

      <input type="submit" value="Submit order"/>
    </form>

  </body>
</html>
<!-- end::allButValidation[] -->
</code>```

I think you are not inserting the right information in the form according to validation rules added in the beans as explained in the chapter. By displaying the validation error, you will know exactly which input is not correctly inserted when you submit the order.

After investigating your code. The name of attributes in the Order.java are not the same as in the view page orderForm.html

For example, in orderForm the attribute is name

```<code>  <h3>Deliver my taco masterpieces to...</h3>
    <label for="name">Name: </label>
    <input type="text" th:field="*{name}"/>
</code>```

Whereas in Order.java is deliveryName.

```<code> @NotBlank(message="Delivery name is required")
    private String deliveryName;
</code>```

The solution is to use the same name of the attribute in the Order.java and orderForm.html page.

ContentLicense: CC BY-SA 4.0

Body:
I'm following the a book Spring in Action 5th ed., but I believe that is presents a bug.

This is the GitHub of the book. I'm arrived on chap 3 tacos-jdbc
source of code

When submit my order a sudden error appears:

and in this way on the terminal:

2019-05-25 16:58:18.164 ERROR 11777 --- [nio-8080-exec-7] org.thymeleaf.TemplateEngine             :
    [THYMELEAF][http-nio-8080-exec-7] Exception processing template
    "orderForm": An error happened during template parsing (template:
    "class path resource [templates/orderForm.html]")

org.thymeleaf.exceptions.TemplateInputException: An error happened
  during template parsing (template: "class path resource
  [templates/orderForm.html]")

OrderController:

@Controller
@RequestMapping("/orders")
@SessionAttributes("order")
public class OrderController {

    private OrderRepository orderRepo;

    public OrderController(OrderRepository orderRepo) {
        this.orderRepo = orderRepo;
    }

    @GetMapping("/current")
    public String orderForm() {
        return "orderForm";
    }

    @PostMapping
    public String processOrder(@Valid Order order, Errors errors,
                               SessionStatus sessionStatus) {
        if (errors.hasErrors()) {
            return "orderForm";
        }

        orderRepo.save(order);
        sessionStatus.setComplete();

        return "redirect:/";
    }

}

and orderForm:

<!DOCTYPE html>

    <title>Taco Cloud</title>
    <link rel="stylesheet" th:href="@{/styles.css}" />

<form method="POST" th:action="@{/orders}" th:object="${order}">

Order your taco creations!

    Design another taco

        Please correct the problems below and resubmit.

Deliver my taco masterpieces to...

    Name:
    <input type="text" th:field="*{name}"/>

    Street address:
    <input type="text" th:field="*{street}"/>

    City:
    <input type="text" th:field="*{city}"/>

    State:
    <input type="text" th:field="*{state}"/>

    Zip code:
    <input type="text" th:field="*{zip}"/>

    Credit Card #:
    <input type="text" th:field="*{ccNumber}"/>
    CC Num Error

    Expiration:
    <input type="text" th:field="*{ccExpiration}"/>

    CVV:
    <input type="text" th:field="*{ccCVV}"/>

    <input type="submit" value="Submit order"/>
</form>

Body:
I think you should handle the display of the validation errors in all the input in the orderForm.html as mentioned in the book as following: (Source Git)

<!-- tag::allButValidation[] -->
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml"
      xmlns:th="http://www.thymeleaf.org">
  <head>
    <title>Taco Cloud</title>
    <link rel="stylesheet" th:href="@{/styles.css}" />
  </head>

  <body>

    <form method="POST" th:action="@{/orders}" th:object="${order}">
      <h1>Order your taco creations!</h1>

      <img th:src="@{/images/TacoCloud.png}"/>
      <a th:href="@{/design}" id="another">Design another taco</a><br/>

      <div th:if="${#fields.hasErrors()}">
        <span class="validationError">
        Please correct the problems below and resubmit.
        </span>
      </div>

      <h3>Deliver my taco masterpieces to...</h3>
      <label for="name">Name: </label>
      <input type="text" th:field="*{name}"/>
<!-- end::allButValidation[] -->
      <span class="validationError"
            th:if="${#fields.hasErrors('name')}"
            th:errors="*{name}">Name Error</span>
<!-- tag::allButValidation[] -->
      <br/>

      <label for="street">Street address: </label>
      <input type="text" th:field="*{street}"/>
<!-- end::allButValidation[] -->
      <span class="validationError"
            th:if="${#fields.hasErrors('street')}"
            th:errors="*{street}">Street Error</span>
<!-- tag::allButValidation[] -->
      <br/>

      <label for="city">City: </label>
      <input type="text" th:field="*{city}"/>
<!-- end::allButValidation[] -->
      <span class="validationError"
            th:if="${#fields.hasErrors('city')}"
            th:errors="*{city}">City Error</span>
<!-- tag::allButValidation[] -->
      <br/>

      <label for="state">State: </label>
      <input type="text" th:field="*{state}"/>
<!-- end::allButValidation[] -->
      <span class="validationError"
            th:if="${#fields.hasErrors('state')}"
            th:errors="*{state}">State Error</span>
<!-- tag::allButValidation[] -->
      <br/>

      <label for="zip">Zip code: </label>
      <input type="text" th:field="*{zip}"/>
<!-- end::allButValidation[] -->
      <span class="validationError"
            th:if="${#fields.hasErrors('zip')}"
            th:errors="*{zip}">Zip Error</span>
<!-- tag::allButValidation[] -->
      <br/>

      <h3>Here's how I'll pay...</h3>
<!-- tag::validatedField[] -->
      <label for="ccNumber">Credit Card #: </label>
      <input type="text" th:field="*{ccNumber}"/>
<!-- end::allButValidation[] -->
      <span class="validationError"
            th:if="${#fields.hasErrors('ccNumber')}"
            th:errors="*{ccNumber}">CC Num Error</span>
<!-- tag::allButValidation[] -->
<!-- end::validatedField[] -->
      <br/>

      <label for="ccExpiration">Expiration: </label>
      <input type="text" th:field="*{ccExpiration}"/>
<!-- end::allButValidation[] -->
      <span class="validationError"
            th:if="${#fields.hasErrors('ccExpiration')}"
            th:errors="*{ccExpiration}">CC Num Error</span>
<!-- tag::allButValidation[] -->
      <br/>

      <label for="ccCVV">CVV: </label>
      <input type="text" th:field="*{ccCVV}"/>
<!-- end::allButValidation[] -->
      <span class="validationError"
            th:if="${#fields.hasErrors('ccCVV')}"
            th:errors="*{ccCVV}">CC Num Error</span>
<!-- tag::allButValidation[] -->
      <br/>

      <input type="submit" value="Submit order"/>
    </form>

  </body>
</html>
<!-- end::allButValidation[] -->

I think you are not inserting the right information in the form according to validation rules added in the beans as explained in the chapter. By displaying the validation error, you will know exactly which input is not correctly inserted when you submit the order.

After investigating your code. The name of attributes in the Order.java are not the same as in the view page orderForm.html

For example, in orderForm the attribute is name

  <h3>Deliver my taco masterpieces to...</h3>
    <label for="name">Name: </label>
    <input type="text" th:field="*{name}"/>

Whereas in Order.java is deliveryName.

 @NotBlank(message="Delivery name is required")
    private String deliveryName;

The solution is to use the same name of the attribute in the Order.java and orderForm.html page.


问:
React: can I use a prop named "index"
You can see from the code blocks below that inside Description.js I had to pass down a prop named "index", because handleChange is a function that must be called with two parameters handleChange(newValue,index).

The handleChange function updates the state, which in this case, is an array if length 3 (one value for each input).

And since the index is created by the Description.js component, I had to pass that down as props.

It works as intended.

QUESTION

Is this a bad practice? Is index a reserved word somehow (of React, Javascript or HTML)?

Is there a better approach to this?

App.js

```<code>import React, { useState, useCallback } from "react";
import ReactDOM from "react-dom";
import Description from "./Description";

function App() {
  console.log("Rendering App...");
  const [inputValues, setInputValues] = useState(["", "", ""]);

  const handleChange = useCallback((newValue, index) => {
    setInputValues(prevState => {
      const aux = Array.from(prevState);
      aux[index] = newValue;
      return aux;
    });
  }, []);

  return <Description values={inputValues} handleChange={handleChange} />;
}
</code>```

Description.js

```<code>import React from "react";
import TextInput from "./TextInput";

function Description(props) {
  console.log("Rendering Description...");
  const inputItems = props.values.map((item, index) => (
    <div key={index}>
      <div>Input {index + 1}</div>
      <TextInput value={item} handleChange={props.handleChange} index={index} />
    </div>
  ));

  return <React.Fragment>{inputItems}</React.Fragment>;
}
</code>```

TextInput.js

```<code>import React from "react";

const TextInput = React.memo(function TextInput(props) {
  console.log("Rendering TextInput..." + props.index);
  return (
    <input
      type="text"
      value={props.value}
      onChange={event => props.handleChange(event.target.value, props.index)}
    />
  );
});

</code>```

答: No, index is not a reserved prop name. Actually the only reserved names that I know of are ref and key

Title: React: can I use a prop named "index"

Body:
You can see from the code blocks below that inside Description.js I had to pass down a prop named "index", because handleChange is a function that must be called with two parameters handleChange(newValue,index).

The handleChange function updates the state, which in this case, is an array if length 3 (one value for each input).

And since the index is created by the Description.js component, I had to pass that down as props.

It works as intended.

QUESTION

Is this a bad practice? Is index a reserved word somehow (of React, Javascript or HTML)?

Is there a better approach to this?

App.js

import React, { useState, useCallback } from "react";
import ReactDOM from "react-dom";
import Description from "./Description";

function App() {
  console.log("Rendering App...");
  const [inputValues, setInputValues] = useState(["", "", ""]);

  const handleChange = useCallback((newValue, index) => {
    setInputValues(prevState => {
      const aux = Array.from(prevState);
      aux[index] = newValue;
      return aux;
    });
  }, []);

  return <Description values={inputValues} handleChange={handleChange} />;
}

Description.js

import React from "react";
import TextInput from "./TextInput";

function Description(props) {
  console.log("Rendering Description...");
  const inputItems = props.values.map((item, index) => (

Input {index + 1}

      <TextInput value={item} handleChange={props.handleChange} index={index} />

  ));

  return <React.Fragment>{inputItems}</React.Fragment>;
}

TextInput.js

import React from "react";

const TextInput = React.memo(function TextInput(props) {
  console.log("Rendering TextInput..." + props.index);
  return (
    <input
      type="text"
      value={props.value}
      onChange={event => props.handleChange(event.target.value, props.index)}
    />
  );
});


问:
json_to_record with embedded encoded json in PostgreSQL >= 10
With PostgreSQL 9.6, it's possible to convert JSON embedded as a string within JSON as follows:

```<code># select * from json_to_record('{"key":"{\"embedded\": 42}"}') as x(key json);
       key
------------------
 {"embedded": 42}
(1 row)
</code>```

This fails with PostgreSQL 10 and above:

```<code># select * from json_to_record('{"key":"{\"embedded\": 42}"}') as x(key json);
ERROR:  invalid input syntax for type json
DETAIL:  Token "embedded" is invalid.
CONTEXT:  JSON data, line 1: "{"embedded...
</code>```

The PostgreSQL 10 release notes mention changes to populate_json and related functions (including, presumable json_to_record), apparently to deal with non-quoted embedded JSON:

With this change, array-type fields in the destination SQL type are properly converted from JSON arrays, and composite-type fields are properly converted from JSON objects. Previously, such cases would fail because the text representation of the JSON value would be fed to array_in() or record_in(), and its syntax would not match what those input functions expect.

It seems the error when used with quoted JSON is an accidental side-effect of this change. Is there a way achieve the 9.6 behavior with the newer versions?

答:
You can first get the column as text and then cast it to json.

```<code>SELECT x.key::json
       FROM json_to_record('{"key":"{\"embedded\": 42}"}') x(key text);
</code>```

db<>fiddle

Body:
With PostgreSQL 9.6, it's possible to convert JSON embedded as a string within JSON as follows:

# select * from json_to_record('{"key":"{\"embedded\": 42}"}') as x(key json);
       key
------------------
 {"embedded": 42}
(1 row)

This fails with PostgreSQL 10 and above:

# select * from json_to_record('{"key":"{\"embedded\": 42}"}') as x(key json);
ERROR:  invalid input syntax for type json
DETAIL:  Token "embedded" is invalid.
CONTEXT:  JSON data, line 1: "{"embedded...

The PostgreSQL 10 release notes mention changes to populate_json and related functions (including, presumable json_to_record), apparently to deal with non-quoted embedded JSON:

With this change, array-type fields in the destination SQL type are properly converted from JSON arrays, and composite-type fields are properly converted from JSON objects. Previously, such cases would fail because the text representation of the JSON value would be fed to array_in() or record_in(), and its syntax would not match what those input functions expect.

It seems the error when used with quoted JSON is an accidental side-effect of this change. Is there a way achieve the 9.6 behavior with the newer versions?

Body:
You can first get the column as text and then cast it to json.

SELECT x.key::json
       FROM json_to_record('{"key":"{\"embedded\": 42}"}') x(key text);

db<>fiddle


问:
Python Selenium: Unable to locate the element by Xpath
I am trying to click a save button via selenium, however, I am getting the error that it is unable to locate the element.

this is the html part of the website

```<code>from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

driver = webdriver.Chrome()
driver.get("xxx")

WebDriverWait(driver,10).until(EC.presence_of_element_located(driver.find_element_by_id("DivFlashViewerMain_SavePdfButtonIcon")))
driver.find_element_by_xpath('//*[@id="DivFlashViewerMain_SavePdfButtonIcon"]').click()
</code>```

This is the error I am getting:

NoSuchElementException: no such element: Unable to locate element:
  {"method":"id","selector":"DivFlashViewerMain_SavePdfButtonIcon"}

  (Session info: chrome=74.0.3729.169)   (Driver info:
  chromedriver=74.0.3729.6
  (255758eccf3d244491b8a1317aa76e1ce10d57e9-refs/branch-heads/3729@{#29}),platform=Windows
  NT 10.0.17763 x86_64)

答:
You should use element_to_be_clickable not just presence_of_element_located.

It should look something like this:

```<code>from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

driver = webdriver.Chrome()
driver.get("xxx")

button = WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.ID, "DivFlashViewerMain_SavePdfButtonIcon")))
button.click()
</code>```

Hope this helps!

Body:
I am trying to click a save button via selenium, however, I am getting the error that it is unable to locate the element.

this is the html part of the website

from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

driver = webdriver.Chrome()
driver.get("xxx")

WebDriverWait(driver,10).until(EC.presence_of_element_located(driver.find_element_by_id("DivFlashViewerMain_SavePdfButtonIcon")))
driver.find_element_by_xpath('//*[@id="DivFlashViewerMain_SavePdfButtonIcon"]').click()

This is the error I am getting:

NoSuchElementException: no such element: Unable to locate element:
  {"method":"id","selector":"DivFlashViewerMain_SavePdfButtonIcon"}

  (Session info: chrome=74.0.3729.169)   (Driver info:
  chromedriver=74.0.3729.6
  (255758eccf3d244491b8a1317aa76e1ce10d57e9-refs/branch-heads/3729@{#29}),platform=Windows
  NT 10.0.17763 x86_64)

Body:
You should use element_to_be_clickable not just presence_of_element_located.

It should look something like this:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

driver = webdriver.Chrome()
driver.get("xxx")

button = WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.ID, "DivFlashViewerMain_SavePdfButtonIcon")))
button.click()

Hope this helps!


问:
GCP Object download metric
Is there a metric that can be used to determine how many times an object has been downloaded from a GCP bucket?</p>

答:
You can use the Audit logs to track storage activity from authenticated users.

For getting how many times an object has been viewed, you have to enable the DATA_READ access logs.

Then, you can search on the Logs Viewer using this advanced filter:

```<code>resource.type="gcs_bucket"
resource.labels.bucket_name=[BUCKET_NAME]
logName="projects/[project_id]/logs/cloudaudit.googleapis.com%2Fdata_access"
protoPayload.resourceName="projects/_/buckets/[bucket_name]/objects/[object_name]"
</code>```

Note that this only works if the object has not public access.

Title: GCP Object download metric

Body:
You can use the Audit logs to track storage activity from authenticated users.

For getting how many times an object has been viewed, you have to enable the DATA_READ access logs.

Then, you can search on the Logs Viewer using this advanced filter:

resource.type="gcs_bucket"
resource.labels.bucket_name=[BUCKET_NAME]
logName="projects/[project_id]/logs/cloudaudit.googleapis.com%2Fdata_access"
protoPayload.resourceName="projects/_/buckets/[bucket_name]/objects/[object_name]"

Note that this only works if the object has not public access.


问:
Perl SSH connection to execute telnet
I tried the following to access a router via a central admin server as "ssh hop" server

```<code>
#!/usr/bin/perl -X

use strict;
use Net::OpenSSH;
use Net::Telnet;

my $lhost = "linuxserver";
my $luser = "linuxuser";
my $lpass = "linuxpassword";

my $chost = "routername";
my $cpass = "Routerpassword";

my $prompt = '/(?:Password: |[>])/m';
my @commands = ("show users\r");

my $ssh = Net::OpenSSH->new($lhost,
'user' => $luser,
'password' => $lpass,
'master_opts' => [ '-t' ],
#'async' => 1 # if enabled then password cannot be set here
);
my ($pty, $err, $pid) = $ssh->open2pty("telnet $chost");

my $t = new Net::Telnet(
-telnetmode => 0,
-fhopen => $pty,
-prompt => $prompt,
-cmd_remove_mode => 1,
-output_record_separator => "\r",
#-dump_log => "debug.log",
);

my $end = 0;

while (!$end) {
  my ($pre, $post) = $t->waitfor($prompt);
  if ($post =~ /Password: /m) {
    # send password
    $t->print("$cpass");
  }
  elsif ($post =~ /[>#]/ && @commands) {
    my $cmd = shift(@commands);
    if ($cmd !~ /[\r\n]/) {
      $t->print($cmd);
    }
    else {
      print $t->cmd($cmd);
    }
  }
  else {
    $end = 1;
    $t->cmd("exit");
  }
}
#close $pty;
$t->close();
</code>```

Unfortunately I always get the following error:
read error: Input/output error at test.pl line 71

Can somebody help me please or is there a better solution only to test if a telnet connection via the "hop" server is possible or not?

The connection looks like:
workstation --ssh-> server --telnet-> router

Thanks in advance.

答:
Getting Net::Telnet to work over Net::OpenSSH sometimes is not as easy as it should be and it requires some experimentation to get to the right combination of flags and calls that make it work.

For instance, instead of telneting to the target host, use netcat to open a raw connection (or Net::OpenSSH support for TCP forwarding if tunnels are allowed on the proxy).

Expect + Net::OpenSSH may be a better option.

ContentLicense: CC BY-SA 3.0

Title: Perl SSH connection to execute telnet

Body:
I tried the following to access a router via a central admin server as "ssh hop" server

#!/usr/bin/perl -X

use strict;
use Net::OpenSSH;
use Net::Telnet;

my $lhost = "linuxserver";
my $luser = "linuxuser";
my $lpass = "linuxpassword";

my $chost = "routername";
my $cpass = "Routerpassword";

my $prompt = '/(?:Password: |[>])/m';
my @commands = ("show users\r");

my $ssh = Net::OpenSSH->new($lhost,
'user' => $luser,
'password' => $lpass,
'master_opts' => [ '-t' ],
#'async' => 1 # if enabled then password cannot be set here
);
my ($pty, $err, $pid) = $ssh->open2pty("telnet $chost");

my $t = new Net::Telnet(
-telnetmode => 0,
-fhopen => $pty,
-prompt => $prompt,
-cmd_remove_mode => 1,
-output_record_separator => "\r",
#-dump_log => "debug.log",
);

my $end = 0;

while (!$end) {
  my ($pre, $post) = $t->waitfor($prompt);
  if ($post =~ /Password: /m) {
    # send password
    $t->print("$cpass");
  }
  elsif ($post =~ /[>#]/ && @commands) {
    my $cmd = shift(@commands);
    if ($cmd !~ /[\r\n]/) {
      $t->print($cmd);
    }
    else {
      print $t->cmd($cmd);
    }
  }
  else {
    $end = 1;
    $t->cmd("exit");
  }
}
#close $pty;
$t->close();

Unfortunately I always get the following error:
read error: Input/output error at test.pl line 71

Can somebody help me please or is there a better solution only to test if a telnet connection via the "hop" server is possible or not?

The connection looks like:
workstation --ssh-> server --telnet-> router

Thanks in advance.


问:
Send data from javascript to view and return modified data
Now I'm just testing how it works so I use a trivial example. I have a variable called text in js. I want to send it to view, do some manipulations with it and return it to js. I managed to send text via ajax POST request but I'm having troubles understanding how I can return the updated value. Here's my script

```<code>    function ajaxTest(){
        var text = "text to send";
        $.ajax({
            url:'test/',
            type:'POST',
            data:{text : text, csrfmiddlewaretoken: '{{ csrf_token }}'},
            success:function () {
                console.log("Ajax test success")
            },

            error:function () {
                console.log("Ajax test failure")
            }
        })
    }
</code>```

My urlpatterns:

```<code>from .views import *
urlpatterns=[
    path('test/', testingAjax),
]
</code>```

And I have view that get this text

```<code>def testingAjax(request):
    if request.method == 'POST':
        x = request.POST.get('text')
        textToReturn = x + " text to return"
        return HttpResponse #And here trouble begins
    elif request.method == 'GET':
        status = "Nuh uh"
        return HttpResponse(status)
</code>```

What should I put in HttpResponse and how can I catch this data in javascript now (for console log it for example)?

答:
You should be able to simply return HttpResponse(textToReturn).

However, you will likely want to look into JsonResponse and return a data structure for your javascript to do something with.

```<code>from django.http import JsonResponse, HttpResponse

def testingAjax(request):
if request.method == 'POST':
    x = request.POST.get('text')
    textToReturn = x + " text to return"
    return JsonResponse({'result':textToReturn,})
elif request.method == 'GET':
    status = "Nuh uh"
    return HttpResponse(status)
</code>```

For the javascript to log the data that comes back in your first example would be like:

```<code>function ajaxTest(){
    var text = "text to send";
    $.ajax({
        url:'test/',
        type:'POST',
        data:{text : text, csrfmiddlewaretoken: '{{ csrf_token }}'},
        success:function (data) {
            console.log("Ajax test success");
            console.log(data);
        },

        error:function () {
            console.log("Ajax test failure")
        }
    })
}
</code>```

Body:
Now I'm just testing how it works so I use a trivial example. I have a variable called text in js. I want to send it to view, do some manipulations with it and return it to js. I managed to send text via ajax POST request but I'm having troubles understanding how I can return the updated value. Here's my script

    function ajaxTest(){
        var text = "text to send";
        $.ajax({
            url:'test/',
            type:'POST',
            data:{text : text, csrfmiddlewaretoken: '{{ csrf_token }}'},
            success:function () {
                console.log("Ajax test success")
            },

            error:function () {
                console.log("Ajax test failure")
            }
        })
    }

My urlpatterns:

from .views import *
urlpatterns=[
    path('test/', testingAjax),
]

And I have view that get this text

def testingAjax(request):
    if request.method == 'POST':
        x = request.POST.get('text')
        textToReturn = x + " text to return"
        return HttpResponse #And here trouble begins
    elif request.method == 'GET':
        status = "Nuh uh"
        return HttpResponse(status)

What should I put in HttpResponse and how can I catch this data in javascript now (for console log it for example)?

Body:
You should be able to simply return HttpResponse(textToReturn).

However, you will likely want to look into JsonResponse and return a data structure for your javascript to do something with.

from django.http import JsonResponse, HttpResponse

def testingAjax(request):
if request.method == 'POST':
    x = request.POST.get('text')
    textToReturn = x + " text to return"
    return JsonResponse({'result':textToReturn,})
elif request.method == 'GET':
    status = "Nuh uh"
    return HttpResponse(status)

For the javascript to log the data that comes back in your first example would be like:

function ajaxTest(){
    var text = "text to send";
    $.ajax({
        url:'test/',
        type:'POST',
        data:{text : text, csrfmiddlewaretoken: '{{ csrf_token }}'},
        success:function (data) {
            console.log("Ajax test success");
            console.log(data);
        },

        error:function () {
            console.log("Ajax test failure")
        }
    })
}


问:
Font-Awesome doesnt work in NativeScript although i followed every step
The Icons I included in Nativescript dont work.
I followed every Step in this tutorial but the Icons look like in the picture.

I tried using different versions of Font-Awesome.
The Desktop and the web-version. I know that the Web-Version is the right one but you never know.

I Included Font-Awesome like this.

```<code>.far {
    font-family: 'Font Awesome 5 Free', fa-regular-400;
    font-weight: 400;
}

.fab {
    font-family: 'Font Awesome 5 Brands', fa-brands-400;
    font-weight: 400;
}

.fas {
    font-family: 'Font Awesome 5 Free', fa-solid-900;
    font-weight: 900;
}
</code>```

And i used it like so:

```<code><Label class="far" style="font-size: 50em; color: blue" text=""></Label>
</code>```

答:
For custom fonts, you need to add the font file in a folder src/fonts.
Based on your picture, it's Android you are using, and on that platform, your CSS font-family must match the font file name, ie. FontAwesome if your font file is named FontAwesome.ttf.
On iOS the font family must match the name in the font file - check out https://www.nativescript.org/blog/using-custom-fonts-in-a-nativescript-app
for more details.

Edit: Maybe you are just missing quotes around the font (file) names,

ie.

```<code>.far {
    font-family: 'Font Awesome 5 Free', 'fa-regular-400';
}

.fab {
    font-family: 'Font Awesome 5 Brands', 'fa-brands-400';
}

.fas {
    font-family: 'Font Awesome 5 Free', 'fa-solid-900';
}
</code>```

I've created a Playground:
https://play.nativescript.org/?template=play-ng&id=yUV2G5

and the three icons are displayed correctly on iOS.

Body:
The Icons I included in Nativescript dont work.
I followed every Step in this tutorial but the Icons look like in the picture.

I tried using different versions of Font-Awesome.
The Desktop and the web-version. I know that the Web-Version is the right one but you never know.

I Included Font-Awesome like this.

.far {
    font-family: 'Font Awesome 5 Free', fa-regular-400;
    font-weight: 400;
}

.fab {
    font-family: 'Font Awesome 5 Brands', fa-brands-400;
    font-weight: 400;
}

.fas {
    font-family: 'Font Awesome 5 Free', fa-solid-900;
    font-weight: 900;
}

And i used it like so:

Body:
For custom fonts, you need to add the font file in a folder src/fonts.
Based on your picture, it's Android you are using, and on that platform, your CSS font-family must match the font file name, ie. FontAwesome if your font file is named FontAwesome.ttf.
On iOS the font family must match the name in the font file - check out https://www.nativescript.org/blog/using-custom-fonts-in-a-nativescript-app
for more details.

Edit: Maybe you are just missing quotes around the font (file) names,

ie.

.far {
    font-family: 'Font Awesome 5 Free', 'fa-regular-400';
}

.fab {
    font-family: 'Font Awesome 5 Brands', 'fa-brands-400';
}

.fas {
    font-family: 'Font Awesome 5 Free', 'fa-solid-900';
}

I've created a Playground:
https://play.nativescript.org/?template=play-ng&id=yUV2G5

and the three icons are displayed correctly on iOS.


问:
Use ItemChange event handler to update an item property
I have a list of contacts in Outlook updated through other program. I don't touch the list myself. Every time a contact is changed, I want to update company name through VBA.

```<code>Public WithEvents objNewContact As Items
Public Sub Initialize_handler()
    Set objNewContact = Application.GetNamespace("MAPI").GetDefaultFolder(olFolderContacts).Items
End Sub

Public Sub objNewContact_ItemChange(ByVal Item As Object)
    Item.CompanyName = "NewCompanyName"
    MsgBox "Company name changed to " & Item.CompanyName
End Sub
</code>```

If I edit the contact myself through Outlook, it works. If edited through other source, it shows the MsgBox telling me that the company name has been changed, but the change is not saved.

If I add "Item.Save" it creates an infinite loop.

```<code>Public Sub objNewContact_ItemChange(ByVal Item As Object)
    Item.CompanyName = "NewCompanyName"
    MsgBox "Company name changed to " & Item.CompanyName
    Item.Save
End Sub
</code>```

答:
You could wrap the inner behavior of your change method in a If statement.
This way it will stop the infinite loop at the 2nd iteration as the Item won't be updated anymore :

```<code>Public Sub objNewContact_ItemChange(ByVal Item As Object)
    If Item.CompanyName != "NewCompanyName" Then
        Item.CompanyName = "NewCompanyName"
        MsgBox "Company name changed to " & Item.CompanyName
        Item.Save
    End If
End Sub
</code>```

Body:
I have a list of contacts in Outlook updated through other program. I don't touch the list myself. Every time a contact is changed, I want to update company name through VBA.

Public WithEvents objNewContact As Items
Public Sub Initialize_handler()
    Set objNewContact = Application.GetNamespace("MAPI").GetDefaultFolder(olFolderContacts).Items
End Sub

Public Sub objNewContact_ItemChange(ByVal Item As Object)
    Item.CompanyName = "NewCompanyName"
    MsgBox "Company name changed to " & Item.CompanyName
End Sub

If I edit the contact myself through Outlook, it works. If edited through other source, it shows the MsgBox telling me that the company name has been changed, but the change is not saved.

If I add "Item.Save" it creates an infinite loop.

Public Sub objNewContact_ItemChange(ByVal Item As Object)
    Item.CompanyName = "NewCompanyName"
    MsgBox "Company name changed to " & Item.CompanyName
    Item.Save
End Sub

Body:
You could wrap the inner behavior of your change method in a If statement.
This way it will stop the infinite loop at the 2nd iteration as the Item won't be updated anymore :

Public Sub objNewContact_ItemChange(ByVal Item As Object)
    If Item.CompanyName != "NewCompanyName" Then
        Item.CompanyName = "NewCompanyName"
        MsgBox "Company name changed to " & Item.CompanyName
        Item.Save
    End If
End Sub


问:
C# Writeline that responds with name
I haven't used variables on C# yet so I don't know how I would go about having the next Console.Writeline respond saying, "Hello X, welcome!". Adding the users input to the X.</p>

```<code>Console.WriteLine("What is your name?");
String answer = Console.ReadLine();

if (answer == string)
    Console.WriteLine("Okay , welcome!");
}
</code>```

答:
```<code>public static void Main()
{
    Console.WriteLine("Do you want to play?");
    String answer = Console.ReadLine();

    Console.WriteLine(string.Format( @"Hello {0}, welcome", answer));
}
</code>```

Title: C# Writeline that responds with name

Body:
I haven't used variables on C# yet so I don't know how I would go about having the next Console.Writeline respond saying, "Hello X, welcome!". Adding the users input to the X.

Console.WriteLine("What is your name?");
String answer = Console.ReadLine();

if (answer == string)
    Console.WriteLine("Okay , welcome!");
}


问:
"class, interface, or enum expected" error when trying to create package
I'm trying to declare a package in a file as follows:

```<code>import java.util.*;

package rtg;

public class Generate
{
    // ...
}
</code>```

But I'm getting an error when I try to compile this:

```<code>Generate.java:3: class, interface, or enum expected package rtg;
</code>```

Why am I getting this error?

答:
it should be

```<code>package rtg;
import java.util.*;
public class Generate{
}
</code>```

In java you first define package then imports and then class. See wiki here: Java_package and Oracle's tutorial here: Java Packages

Edit

Now to call Genereate class from a class in same folder that is rtg folder:

```<code>package rtg;
public class GUI{
     Generate gen = new Generate();
}
</code>```

Make sure all words are spelled correctly.

Body:
I'm trying to declare a package in a file as follows:

import java.util.*;

package rtg;

public class Generate
{
    // ...
}

But I'm getting an error when I try to compile this:

Generate.java:3: class, interface, or enum expected package rtg;

Why am I getting this error?

Body:
it should be

package rtg;
import java.util.*;
public class Generate{
}

In java you first define package then imports and then class. See wiki here: Java_package and Oracle's tutorial here: Java Packages

Edit

Now to call Genereate class from a class in same folder that is rtg folder:

package rtg;
public class GUI{
     Generate gen = new Generate();
}

Make sure all words are spelled correctly.


问:
fit_transform() takes 2 positional arguments but 3 were given with LabelBinarizer
I am totally new to Machine Learning and I have been working with unsupervised learning technique.

Image shows my sample Data(After all Cleaning) Screenshot :
Sample Data

I have this two Pipline built to Clean the Data:

```<code>num_attribs = list(housing_num)
cat_attribs = ["ocean_proximity"]

print(type(num_attribs))

num_pipeline = Pipeline([
    ('selector', DataFrameSelector(num_attribs)),
    ('imputer', Imputer(strategy="median")),
    ('attribs_adder', CombinedAttributesAdder()),
    ('std_scaler', StandardScaler()),
])

cat_pipeline = Pipeline([
    ('selector', DataFrameSelector(cat_attribs)),
    ('label_binarizer', LabelBinarizer())
])
</code>```

Then I did the union of this two pipelines and the code for the same is shown below :

```<code>from sklearn.pipeline import FeatureUnion

full_pipeline = FeatureUnion(transformer_list=[
        ("num_pipeline", num_pipeline),
        ("cat_pipeline", cat_pipeline),
    ])
</code>```

Now I am trying to do fit_transform on the Data But Its showing Me the Error.

Code for Transformation:

```<code>housing_prepared = full_pipeline.fit_transform(housing)
housing_prepared
</code>```

Error message:

fit_transform() takes 2 positional arguments but 3 were given

答:
Simply, what you can do is define following class just before your pipeline:

```<code>class NewLabelBinarizer(LabelBinarizer):
    def fit(self, X, y=None):
        return super(NewLabelBinarizer, self).fit(X)
    def transform(self, X, y=None):
        return super(NewLabelBinarizer, self).transform(X)
    def fit_transform(self, X, y=None):
        return super(NewLabelBinarizer, self).fit(X).transform(X)
</code>```

Then the rest of the code is like the one has mentioned in the book with a tiny modification in cat_pipeline before pipeline concatenation - follow as:

```<code>cat_pipeline = Pipeline([
    ("selector", DataFrameSelector(cat_attribs)),
    ("label_binarizer", NewLabelBinarizer())])
</code>```

You DONE!

Body:
I am totally new to Machine Learning and I have been working with unsupervised learning technique.

Image shows my sample Data(After all Cleaning) Screenshot :
Sample Data

I have this two Pipline built to Clean the Data:

num_attribs = list(housing_num)
cat_attribs = ["ocean_proximity"]

print(type(num_attribs))

num_pipeline = Pipeline([
    ('selector', DataFrameSelector(num_attribs)),
    ('imputer', Imputer(strategy="median")),
    ('attribs_adder', CombinedAttributesAdder()),
    ('std_scaler', StandardScaler()),
])

cat_pipeline = Pipeline([
    ('selector', DataFrameSelector(cat_attribs)),
    ('label_binarizer', LabelBinarizer())
])

Then I did the union of this two pipelines and the code for the same is shown below :

from sklearn.pipeline import FeatureUnion

full_pipeline = FeatureUnion(transformer_list=[
        ("num_pipeline", num_pipeline),
        ("cat_pipeline", cat_pipeline),
    ])

Now I am trying to do fit_transform on the Data But Its showing Me the Error.

Code for Transformation:

housing_prepared = full_pipeline.fit_transform(housing)
housing_prepared

Error message:

fit_transform() takes 2 positional arguments but 3 were given

Body:
Simply, what you can do is define following class just before your pipeline:

class NewLabelBinarizer(LabelBinarizer):
    def fit(self, X, y=None):
        return super(NewLabelBinarizer, self).fit(X)
    def transform(self, X, y=None):
        return super(NewLabelBinarizer, self).transform(X)
    def fit_transform(self, X, y=None):
        return super(NewLabelBinarizer, self).fit(X).transform(X)

Then the rest of the code is like the one has mentioned in the book with a tiny modification in cat_pipeline before pipeline concatenation - follow as:

cat_pipeline = Pipeline([
    ("selector", DataFrameSelector(cat_attribs)),
    ("label_binarizer", NewLabelBinarizer())])

You DONE!


问:
How to enumurate values from a list and then sum in a function?
I'm trying to create a function that uses the value and the index of the value, starting from 1 to put the values in a function and sums the whole list.

In other words I'm trying to get the present value of a list of cashflows that are not the same, therefor have to calculate the present value of each cash flow and sum them.

This is what I have tried.

```<code>cash = [10,10,10,10, 10, 10, 11, 10]
DiscoutRate = 0.12
    for (period, value) in enumerate(cash, start=1):
        PV = value/((1+DiscountRate)**period)
        print(PV)
</code>```

But I see that the results are not correct, any thoughts?

答:
Note that this solution uses a Generator Expression.</p>

```<code>>>> sum(value/((1 + DiscountRate) ** period) for period, value in enumerate(cash, start=1))
50.1287
</code>```

Body:
I'm trying to create a function that uses the value and the index of the value, starting from 1 to put the values in a function and sums the whole list.

In other words I'm trying to get the present value of a list of cashflows that are not the same, therefor have to calculate the present value of each cash flow and sum them.

This is what I have tried.

cash = [10,10,10,10, 10, 10, 11, 10]
DiscoutRate = 0.12
    for (period, value) in enumerate(cash, start=1):
        PV = value/((1+DiscountRate)**period)
        print(PV)

But I see that the results are not correct, any thoughts?

Body:
Note that this solution uses a Generator Expression.

>>> sum(value/((1 + DiscountRate) ** period) for period, value in enumerate(cash, start=1))
50.1287


问:
Unkeyed Container JSON Swift Decodable
I am trying to decode some JSON. Here is an example of the JSON:

```<code> [
    {
        "type": "departure",
        "status": "landed",
        "departure": {
            "iataCode": "JFK",
            "icaoCode": "KJFK",
            "scheduledTime": "2017-12-11T01:06:00.000",
            "estimatedRunway": "2017-12-11T02:07:00.000",
            "actualRunway": "2017-12-11T02:07:00.000" },
        "arrival": {
            "iataCode": "CVG",
            "icaoCode": "KCVG",
            "estimatedRunway": "2017-12-11T03:38:00.000",
            "actualRunway": "2017-12-11T03:38:00.000"
        },
        "airline": {
            "name": "Atlas Air",
            "iataCode": "5Y",
            "icaoCode": "GTI"
        },
        "flight": {
            "number": "302",
            "iataNumber": "5Y302",
            "icaoNumber": "GTI302"
        }
    },
    {

        //Same keys as above.
    },
    //Etc.
]
</code>```

It begins as an unkeyed array.  This is then follewed by JSON containers that are also unkeyed.  I am having trouble breaking it apart using this code:

```<code>struct Dataset: Decodable {

    var data: [FlightData]

    init(from decoder: Decoder) throws {
        var container = try decoder.unkeyedContainer()
        print(container)
        data = [try container.decode(FlightData.self)]
    }

struct FlightData: Decodable {

    var type: String //Arrival or Departure
    var status: String //Flight Status

    var departure: Departure
    var arrival: Arrival
    var airline: Airline
    var flight: Flight

    struct Departure: Decodable {

        var iataCode: String
        var icaoCode: String
        var terminal: String
        var gate: String
        var scheduledTime: String
        var estimatedTime: String
        var actualTime: String
        var estimatedRunway: String
        var actualRunway: String

    }

    struct Arrival: Decodable {

        var iataCode: String
        var icaoCode: String
        var terminal: String
        var gate: String
        var baggage: String
        var scheduledTime: String
        var estimatedTime: String
        var actualTime: String
        var estimatedRunway: String
        var actualRunway: String

    }

    struct Airline: Decodable {

        var name: String
        var iataCode: String
        var icaoCode: String

    }

    struct Flight: Decodable {

        var number: String
        var iataNumber: String
        var icaoNumber: String

    }

}
}
</code>```

Im new to JSON and Swift Decodable so I am a bit confused what I am doing wrong?
Does anyone know how I can fix my issue?
Right now I am getting the warning that it is expecting an array but it is finding a dictionary. Therefore, I think I have successfully gotten past the first unkeyed container but I cant get into the rest of it.

答:
Remove your init method and then do</p>

```<code>let decoder = JSONDecoder()

do {
    data = try decoder.decode([FlightData].self, from: data)
} catch {
    print(error)
}
</code>```

Title: Unkeyed Container JSON Swift Decodable

Body:
I am trying to decode some JSON. Here is an example of the JSON:

 [
    {
        "type": "departure",
        "status": "landed",
        "departure": {
            "iataCode": "JFK",
            "icaoCode": "KJFK",
            "scheduledTime": "2017-12-11T01:06:00.000",
            "estimatedRunway": "2017-12-11T02:07:00.000",
            "actualRunway": "2017-12-11T02:07:00.000" },
        "arrival": {
            "iataCode": "CVG",
            "icaoCode": "KCVG",
            "estimatedRunway": "2017-12-11T03:38:00.000",
            "actualRunway": "2017-12-11T03:38:00.000"
        },
        "airline": {
            "name": "Atlas Air",
            "iataCode": "5Y",
            "icaoCode": "GTI"
        },
        "flight": {
            "number": "302",
            "iataNumber": "5Y302",
            "icaoNumber": "GTI302"
        }
    },
    {

        //Same keys as above.
    },
    //Etc.
]

It begins as an unkeyed array.  This is then follewed by JSON containers that are also unkeyed.  I am having trouble breaking it apart using this code:

struct Dataset: Decodable {

    var data: [FlightData]

    init(from decoder: Decoder) throws {
        var container = try decoder.unkeyedContainer()
        print(container)
        data = [try container.decode(FlightData.self)]
    }

struct FlightData: Decodable {

    var type: String //Arrival or Departure
    var status: String //Flight Status

    var departure: Departure
    var arrival: Arrival
    var airline: Airline
    var flight: Flight

    struct Departure: Decodable {

        var iataCode: String
        var icaoCode: String
        var terminal: String
        var gate: String
        var scheduledTime: String
        var estimatedTime: String
        var actualTime: String
        var estimatedRunway: String
        var actualRunway: String

    }

    struct Arrival: Decodable {

        var iataCode: String
        var icaoCode: String
        var terminal: String
        var gate: String
        var baggage: String
        var scheduledTime: String
        var estimatedTime: String
        var actualTime: String
        var estimatedRunway: String
        var actualRunway: String

    }

    struct Airline: Decodable {

        var name: String
        var iataCode: String
        var icaoCode: String

    }

    struct Flight: Decodable {

        var number: String
        var iataNumber: String
        var icaoNumber: String

    }

}
}

Im new to JSON and Swift Decodable so I am a bit confused what I am doing wrong?
Does anyone know how I can fix my issue?
Right now I am getting the warning that it is expecting an array but it is finding a dictionary. Therefore, I think I have successfully gotten past the first unkeyed container but I cant get into the rest of it.

Body:
Remove your init method and then do

let decoder = JSONDecoder()

do {
    data = try decoder.decode([FlightData].self, from: data)
} catch {
    print(error)
}


问:
Sending data from JavaScript to C++, byte values over 127 are replaced with "replacement character"
I'm trying to send buffers of data from a JavaScript application to a C++ application via ZeroMQ (C++ and npm zeromq). If the buffer contains a byte that is > 127, then the byte of data is replaced with 3 bytes:

\xEF\xBF\xBD

Research shows me that this the UTF-8 value for the Unicode replacement character (U+FFFD).

This doesn't seem to happen anywhere in JavaScript before calling the ZeroMQ send function and the data is already corrupted by the time I can receive it on the C++ side via a ZeroMQ recv call.

I've tried changing the buffer type on the JavaScript side to UintArray type to see if that changed anything, but to no luck. I'm not sure what else to try.

Is there any way to solve this issue without working in the npm zeromq source files (I'm not sure how to do that or if I want to).

Update * Code Example of message building *

```<code>var data = new ProtoData.Data1();
data.setTemp(128); // type is int32
var payload = data.serializeBinary();
var size = payload.length + 4; // 16 bits
var head1 = 4; // 8 bits
var head2 = 4; // 8 bits
var payload_buf = Buffer.from(payload);
// create the header
var header = Buffer.allocUnsafe(4);
header.writeUInt16LE(size, 0);
header.writeUInt8(head1, 2);
header.writeUInt8(head2, 3);
var msg = Buffer.concat([header, payload_buf]);
zmqPubSock.sock.send(msg);
</code>```

答:
I think you nailed the diagnosis. It looks like an open issue with zeromq.js, and rightly so—a messaging library that doesn't do binary!?!

The issue reporter suggests zeromq-ng instead.

Another alternative is to use Base64.

Body:
I'm trying to send buffers of data from a JavaScript application to a C++ application via ZeroMQ (C++ and npm zeromq). If the buffer contains a byte that is > 127, then the byte of data is replaced with 3 bytes:

\xEF\xBF\xBD

Research shows me that this the UTF-8 value for the Unicode replacement character (U+FFFD).

This doesn't seem to happen anywhere in JavaScript before calling the ZeroMQ send function and the data is already corrupted by the time I can receive it on the C++ side via a ZeroMQ recv call.

I've tried changing the buffer type on the JavaScript side to UintArray type to see if that changed anything, but to no luck. I'm not sure what else to try.

Is there any way to solve this issue without working in the npm zeromq source files (I'm not sure how to do that or if I want to).

Update * Code Example of message building *

var data = new ProtoData.Data1();
data.setTemp(128); // type is int32
var payload = data.serializeBinary();
var size = payload.length + 4; // 16 bits
var head1 = 4; // 8 bits
var head2 = 4; // 8 bits
var payload_buf = Buffer.from(payload);
// create the header
var header = Buffer.allocUnsafe(4);
header.writeUInt16LE(size, 0);
header.writeUInt8(head1, 2);
header.writeUInt8(head2, 3);
var msg = Buffer.concat([header, payload_buf]);
zmqPubSock.sock.send(msg);


问:
Ada: Array length modification when adding anew element
I try to create a dynamic array package. I have a generic package with a generic type and then I have declared in .ads an array (Positive indexes) of elements Elem_Type. I was curious if it is possible to create a procedure Append(Vector, Item) where to modify Vector's length, allowing him to take one more element. Something like when you have a buffer[100] in C and allocate him 1 more element to make it 101 then add something on position 101.</p>

答:
In Ada the general way to do this is to create a new and bigger block, then copy the contents to it and free the old block.  I often suggest allocating new blocks in multiples of some large chunk size so you don't have to reallocate much.  So like if you have to allocate a new array, instead of adding just one extra element, add 500 (or some number that makes sense).

Note that even in C, the realloc() function often does exactly this.  It isn't common in non trivial projects for the realloc() function to be able to actually just add another element.  It often times has to create an entire new array and do the copy, just like I am suggesting for Ada.

Some other options in Ada include making your own storage pool that makes reallocating a lot easier and faster, but you have to come up with the algorithm on how to do that or find someone elses.  You can also do bindings to the C functions, but you cannot mix malloc/realloc/free with Ada's new/unchecked_deallocation