问:
Getting error while importing data from Mysql to hdfs using hadoop?
I am trying to import data from Mysql to hdfs using hadoop, and also trying to create the table and databse is hive.I am trying with below command

```<code>sqoop import --connect jdbc:mysql://localhost/Mobile --username root --password 12345678 --table Accesories --target-dir /user/harsh/Mobile1 --fields-terminated-by "," --hive-import --create-hive-table --hive-table mob.cust
</code>```

Mobile is my database and Accessories is my table which I am trying to import.Eveeything run successfully but getting error while trying to load data in hive.
But I am getting below error:

```<code>17/10/17 03:51:30 INFO mapreduce.Job:  map 0% reduce 0%
17/10/17 03:52:46 INFO mapreduce.Job:  map 40% reduce 0%
17/10/17 03:52:48 INFO mapreduce.Job:  map 60% reduce 0%
17/10/17 03:52:50 INFO mapreduce.Job:  map 100% reduce 0%
17/10/17 03:53:13 INFO mapreduce.Job: Job job_1508188554902_0003 completed successfully
17/10/17 03:53:16 INFO mapreduce.Job: Counters: 30
    File System Counters
        FILE: Number of bytes read=0
        FILE: Number of bytes written=664170
        FILE: Number of read operations=0
        FILE: Number of large read operations=0
        FILE: Number of write operations=0
        HDFS: Number of bytes read=521
        HDFS: Number of bytes written=88
        HDFS: Number of read operations=20
        HDFS: Number of large read operations=0
        HDFS: Number of write operations=10
    Job Counters
        Launched map tasks=5
        Other local map tasks=5
        Total time spent by all maps in occupied slots (ms)=398814
        Total time spent by all reduces in occupied slots (ms)=0
        Total time spent by all map tasks (ms)=398814
        Total vcore-seconds taken by all map tasks=398814
        Total megabyte-seconds taken by all map tasks=408385536
    Map-Reduce Framework
        Map input records=5
        Map output records=5
        Input split bytes=521
        Spilled Records=0
        Failed Shuffles=0
        Merged Map outputs=0
        GC time elapsed (ms)=8631
        CPU time spent (ms)=11240
        Physical memory (bytes) snapshot=768512000
        Virtual memory (bytes) snapshot=9839693824
        Total committed heap usage (bytes)=448790528
    File Input Format Counters
        Bytes Read=0
    File Output Format Counters
        Bytes Written=88
17/10/17 03:53:16 INFO mapreduce.ImportJobBase: Transferred 88 bytes in 144.4472 seconds (0.6092 bytes/sec)
17/10/17 03:53:16 INFO mapreduce.ImportJobBase: Retrieved 5 records.
Tue Oct 17 03:53:17 IST 2017 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
17/10/17 03:53:17 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `Accesories` AS t LIMIT 1
17/10/17 03:53:18 INFO hive.HiveImport: Loading uploaded data into Hive
17/10/17 03:55:10 INFO hive.HiveImport: SLF4J: Class path contains multiple SLF4J bindings.
17/10/17 03:55:10 INFO hive.HiveImport: SLF4J: Found binding in [jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
17/10/17 03:55:10 INFO hive.HiveImport: SLF4J: Found binding in [jar:file:/usr/local/hbase/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
17/10/17 03:55:10 INFO hive.HiveImport: SLF4J: Found binding in [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
17/10/17 03:55:10 INFO hive.HiveImport: SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
17/10/17 03:55:13 INFO hive.HiveImport:
17/10/17 03:55:13 INFO hive.HiveImport: Logging initialized using configuration in jar:file:/usr/local/hive/lib/hive-common-2.1.0.jar!/hive-log4j2.properties Async: true
17/10/17 03:55:26 INFO hive.HiveImport: Exception in thread "main" java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:578)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:518)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:705)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:641)
17/10/17 03:55:26 INFO hive.HiveImport:     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
17/10/17 03:55:26 INFO hive.HiveImport:     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
17/10/17 03:55:26 INFO hive.HiveImport:     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
17/10/17 03:55:26 INFO hive.HiveImport:     at java.lang.reflect.Method.invoke(Method.java:498)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
17/10/17 03:55:26 INFO hive.HiveImport: Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:226)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.ql.metadata.Hive.<init>(Hive.java:366)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.ql.metadata.Hive.create(Hive.java:310)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.ql.metadata.Hive.getInternal(Hive.java:290)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:266)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:545)
17/10/17 03:55:26 INFO hive.HiveImport:     ... 8 more
17/10/17 03:55:26 INFO hive.HiveImport: Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1627)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:80)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:130)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:101)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3317)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3356)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3336)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3590)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:236)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:221)
17/10/17 03:55:26 INFO hive.HiveImport:     ... 13 more
17/10/17 03:55:26 INFO hive.HiveImport: Caused by: java.lang.reflect.InvocationTargetException
17/10/17 03:55:26 INFO hive.HiveImport:     at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
17/10/17 03:55:26 INFO hive.HiveImport:     at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
17/10/17 03:55:26 INFO hive.HiveImport:     at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
17/10/17 03:55:26 INFO hive.HiveImport:     at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1625)
17/10/17 03:55:26 INFO hive.HiveImport:     ... 22 more
17/10/17 03:55:26 INFO hive.HiveImport: Caused by: javax.jdo.JDODataStoreException: Exception thrown obtaining schema column information from datastore
17/10/17 03:55:26 INFO hive.HiveImport: NestedThrowables:
17/10/17 03:55:26 INFO hive.HiveImport: java.sql.SQLException: Column name pattern can not be NULL or empty.
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:543)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:720)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:740)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.metastore.ObjectStore.setMetaStoreSchemaVersion(ObjectStore.java:7763)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.metastore.ObjectStore.checkSchema(ObjectStore.java:7657)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.metastore.ObjectStore.verifySchema(ObjectStore.java:7632)
17/10/17 03:55:26 INFO hive.HiveImport:     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
17/10/17 03:55:26 INFO hive.HiveImport:     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
17/10/17 03:55:26 INFO hive.HiveImport:     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
17/10/17 03:55:26 INFO hive.HiveImport:     at java.lang.reflect.Method.invoke(Method.java:498)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:101)
17/10/17 03:55:26 INFO hive.HiveImport:     at com.sun.proxy.$Proxy21.verifySchema(Unknown Source)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:547)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:612)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:398)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:78)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:84)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:6396)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:236)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:70)
17/10/17 03:55:26 INFO hive.HiveImport:     ... 27 more
17/10/17 03:55:26 INFO hive.HiveImport: Caused by: java.sql.SQLException: Column name pattern can not be NULL or empty.
17/10/17 03:55:26 INFO hive.HiveImport:     at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:545)
17/10/17 03:55:26 INFO hive.HiveImport:     at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:513)
17/10/17 03:55:26 INFO hive.HiveImport:     at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:505)
17/10/17 03:55:26 INFO hive.HiveImport:     at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:479)
17/10/17 03:55:26 INFO hive.HiveImport:     at com.mysql.cj.jdbc.DatabaseMetaData.getColumns(DatabaseMetaData.java:2074)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.store.rdbms.adapter.BaseDatastoreAdapter.getColumns(BaseDatastoreAdapter.java:1575)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.store.rdbms.schema.RDBMSSchemaHandler.refreshTableData(RDBMSSchemaHandler.java:1103)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.store.rdbms.schema.RDBMSSchemaHandler.getRDBMSTableInfoForTable(RDBMSSchemaHandler.java:1015)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.store.rdbms.schema.RDBMSSchemaHandler.getRDBMSTableInfoForTable(RDBMSSchemaHandler.java:965)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.store.rdbms.schema.RDBMSSchemaHandler.getSchemaData(RDBMSSchemaHandler.java:338)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.store.rdbms.RDBMSStoreManager.getColumnInfoForTable(RDBMSStoreManager.java:2392)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.store.rdbms.table.TableImpl.initializeColumnInfoFromDatastore(TableImpl.java:324)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.store.rdbms.RDBMSStoreManager$ClassAdder.performTablesValidation(RDBMSStoreManager.java:3401)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.store.rdbms.RDBMSStoreManager$ClassAdder.run(RDBMSStoreManager.java:2877)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.store.rdbms.AbstractSchemaTransaction.execute(AbstractSchemaTransaction.java:119)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.store.rdbms.RDBMSStoreManager.manageClasses(RDBMSStoreManager.java:1608)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.store.rdbms.RDBMSStoreManager.getDatastoreClass(RDBMSStoreManager.java:671)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.store.rdbms.RDBMSStoreManager.getPropertiesForGenerator(RDBMSStoreManager.java:2069)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.store.AbstractStoreManager.getStrategyValue(AbstractStoreManager.java:1271)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.ExecutionContextImpl.newObjectId(ExecutionContextImpl.java:3759)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.state.StateManagerImpl.setIdentity(StateManagerImpl.java:2267)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.state.StateManagerImpl.initialiseForPersistentNew(StateManagerImpl.java:484)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.state.StateManagerImpl.initialiseForPersistentNew(StateManagerImpl.java:120)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.state.ObjectProviderFactoryImpl.newForPersistentNew(ObjectProviderFactoryImpl.java:218)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.ExecutionContextImpl.persistObjectInternal(ExecutionContextImpl.java:2078)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.ExecutionContextImpl.persistObjectWork(ExecutionContextImpl.java:1922)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.ExecutionContextImpl.persistObject(ExecutionContextImpl.java:1777)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.ExecutionContextThreadedImpl.persistObject(ExecutionContextThreadedImpl.java:217)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:715)
17/10/17 03:55:26 INFO hive.HiveImport:     ... 45 more
17/10/17 03:55:26 ERROR tool.ImportTool: Encountered IOException running import job: java.io.IOException: Hive exited with status 1
    at org.apache.sqoop.hive.HiveImport.executeExternalHiveScript(HiveImport.java:389)
    at org.apache.sqoop.hive.HiveImport.executeScript(HiveImport.java:339)
    at org.apache.sqoop.hive.HiveImport.importTable(HiveImport.java:240)
    at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:514)
    at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
    at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
    at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
    at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
</code>```

Mysql version :  5.7.19-0ubuntu0.17.04.1
Hive version : version 2
Hadoop version : 2.7.1

答: What I did is I made sure I copied hive-site.xml to sqoop conf folder mine looks like this /opt/sqoop/conf/. Once I did that it worked!!!.

Body:
I am trying to import data from Mysql to hdfs using hadoop, and also trying to create the table and databse is hive.I am trying with below command

sqoop import --connect jdbc:mysql://localhost/Mobile --username root --password 12345678 --table Accesories --target-dir /user/harsh/Mobile1 --fields-terminated-by "," --hive-import --create-hive-table --hive-table mob.cust

Mobile is my database and Accessories is my table which I am trying to import.Eveeything run successfully but getting error while trying to load data in hive.
But I am getting below error:

17/10/17 03:51:30 INFO mapreduce.Job:  map 0% reduce 0%
17/10/17 03:52:46 INFO mapreduce.Job:  map 40% reduce 0%
17/10/17 03:52:48 INFO mapreduce.Job:  map 60% reduce 0%
17/10/17 03:52:50 INFO mapreduce.Job:  map 100% reduce 0%
17/10/17 03:53:13 INFO mapreduce.Job: Job job_1508188554902_0003 completed successfully
17/10/17 03:53:16 INFO mapreduce.Job: Counters: 30
    File System Counters
        FILE: Number of bytes read=0
        FILE: Number of bytes written=664170
        FILE: Number of read operations=0
        FILE: Number of large read operations=0
        FILE: Number of write operations=0
        HDFS: Number of bytes read=521
        HDFS: Number of bytes written=88
        HDFS: Number of read operations=20
        HDFS: Number of large read operations=0
        HDFS: Number of write operations=10
    Job Counters
        Launched map tasks=5
        Other local map tasks=5
        Total time spent by all maps in occupied slots (ms)=398814
        Total time spent by all reduces in occupied slots (ms)=0
        Total time spent by all map tasks (ms)=398814
        Total vcore-seconds taken by all map tasks=398814
        Total megabyte-seconds taken by all map tasks=408385536
    Map-Reduce Framework
        Map input records=5
        Map output records=5
        Input split bytes=521
        Spilled Records=0
        Failed Shuffles=0
        Merged Map outputs=0
        GC time elapsed (ms)=8631
        CPU time spent (ms)=11240
        Physical memory (bytes) snapshot=768512000
        Virtual memory (bytes) snapshot=9839693824
        Total committed heap usage (bytes)=448790528
    File Input Format Counters
        Bytes Read=0
    File Output Format Counters
        Bytes Written=88
17/10/17 03:53:16 INFO mapreduce.ImportJobBase: Transferred 88 bytes in 144.4472 seconds (0.6092 bytes/sec)
17/10/17 03:53:16 INFO mapreduce.ImportJobBase: Retrieved 5 records.
Tue Oct 17 03:53:17 IST 2017 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
17/10/17 03:53:17 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `Accesories` AS t LIMIT 1
17/10/17 03:53:18 INFO hive.HiveImport: Loading uploaded data into Hive
17/10/17 03:55:10 INFO hive.HiveImport: SLF4J: Class path contains multiple SLF4J bindings.
17/10/17 03:55:10 INFO hive.HiveImport: SLF4J: Found binding in [jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
17/10/17 03:55:10 INFO hive.HiveImport: SLF4J: Found binding in [jar:file:/usr/local/hbase/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
17/10/17 03:55:10 INFO hive.HiveImport: SLF4J: Found binding in [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
17/10/17 03:55:10 INFO hive.HiveImport: SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
17/10/17 03:55:13 INFO hive.HiveImport:
17/10/17 03:55:13 INFO hive.HiveImport: Logging initialized using configuration in jar:file:/usr/local/hive/lib/hive-common-2.1.0.jar!/hive-log4j2.properties Async: true
17/10/17 03:55:26 INFO hive.HiveImport: Exception in thread "main" java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:578)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:518)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:705)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:641)
17/10/17 03:55:26 INFO hive.HiveImport:     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
17/10/17 03:55:26 INFO hive.HiveImport:     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
17/10/17 03:55:26 INFO hive.HiveImport:     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
17/10/17 03:55:26 INFO hive.HiveImport:     at java.lang.reflect.Method.invoke(Method.java:498)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
17/10/17 03:55:26 INFO hive.HiveImport: Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:226)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.ql.metadata.Hive.<init>(Hive.java:366)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.ql.metadata.Hive.create(Hive.java:310)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.ql.metadata.Hive.getInternal(Hive.java:290)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:266)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:545)
17/10/17 03:55:26 INFO hive.HiveImport:     ... 8 more
17/10/17 03:55:26 INFO hive.HiveImport: Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1627)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:80)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:130)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:101)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3317)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3356)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3336)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3590)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:236)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:221)
17/10/17 03:55:26 INFO hive.HiveImport:     ... 13 more
17/10/17 03:55:26 INFO hive.HiveImport: Caused by: java.lang.reflect.InvocationTargetException
17/10/17 03:55:26 INFO hive.HiveImport:     at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
17/10/17 03:55:26 INFO hive.HiveImport:     at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
17/10/17 03:55:26 INFO hive.HiveImport:     at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
17/10/17 03:55:26 INFO hive.HiveImport:     at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1625)
17/10/17 03:55:26 INFO hive.HiveImport:     ... 22 more
17/10/17 03:55:26 INFO hive.HiveImport: Caused by: javax.jdo.JDODataStoreException: Exception thrown obtaining schema column information from datastore
17/10/17 03:55:26 INFO hive.HiveImport: NestedThrowables:
17/10/17 03:55:26 INFO hive.HiveImport: java.sql.SQLException: Column name pattern can not be NULL or empty.
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:543)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:720)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:740)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.metastore.ObjectStore.setMetaStoreSchemaVersion(ObjectStore.java:7763)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.metastore.ObjectStore.checkSchema(ObjectStore.java:7657)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.metastore.ObjectStore.verifySchema(ObjectStore.java:7632)
17/10/17 03:55:26 INFO hive.HiveImport:     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
17/10/17 03:55:26 INFO hive.HiveImport:     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
17/10/17 03:55:26 INFO hive.HiveImport:     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
17/10/17 03:55:26 INFO hive.HiveImport:     at java.lang.reflect.Method.invoke(Method.java:498)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:101)
17/10/17 03:55:26 INFO hive.HiveImport:     at com.sun.proxy.$Proxy21.verifySchema(Unknown Source)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:547)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:612)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:398)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:78)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:84)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:6396)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:236)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:70)
17/10/17 03:55:26 INFO hive.HiveImport:     ... 27 more
17/10/17 03:55:26 INFO hive.HiveImport: Caused by: java.sql.SQLException: Column name pattern can not be NULL or empty.
17/10/17 03:55:26 INFO hive.HiveImport:     at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:545)
17/10/17 03:55:26 INFO hive.HiveImport:     at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:513)
17/10/17 03:55:26 INFO hive.HiveImport:     at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:505)
17/10/17 03:55:26 INFO hive.HiveImport:     at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:479)
17/10/17 03:55:26 INFO hive.HiveImport:     at com.mysql.cj.jdbc.DatabaseMetaData.getColumns(DatabaseMetaData.java:2074)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.store.rdbms.adapter.BaseDatastoreAdapter.getColumns(BaseDatastoreAdapter.java:1575)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.store.rdbms.schema.RDBMSSchemaHandler.refreshTableData(RDBMSSchemaHandler.java:1103)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.store.rdbms.schema.RDBMSSchemaHandler.getRDBMSTableInfoForTable(RDBMSSchemaHandler.java:1015)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.store.rdbms.schema.RDBMSSchemaHandler.getRDBMSTableInfoForTable(RDBMSSchemaHandler.java:965)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.store.rdbms.schema.RDBMSSchemaHandler.getSchemaData(RDBMSSchemaHandler.java:338)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.store.rdbms.RDBMSStoreManager.getColumnInfoForTable(RDBMSStoreManager.java:2392)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.store.rdbms.table.TableImpl.initializeColumnInfoFromDatastore(TableImpl.java:324)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.store.rdbms.RDBMSStoreManager$ClassAdder.performTablesValidation(RDBMSStoreManager.java:3401)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.store.rdbms.RDBMSStoreManager$ClassAdder.run(RDBMSStoreManager.java:2877)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.store.rdbms.AbstractSchemaTransaction.execute(AbstractSchemaTransaction.java:119)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.store.rdbms.RDBMSStoreManager.manageClasses(RDBMSStoreManager.java:1608)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.store.rdbms.RDBMSStoreManager.getDatastoreClass(RDBMSStoreManager.java:671)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.store.rdbms.RDBMSStoreManager.getPropertiesForGenerator(RDBMSStoreManager.java:2069)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.store.AbstractStoreManager.getStrategyValue(AbstractStoreManager.java:1271)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.ExecutionContextImpl.newObjectId(ExecutionContextImpl.java:3759)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.state.StateManagerImpl.setIdentity(StateManagerImpl.java:2267)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.state.StateManagerImpl.initialiseForPersistentNew(StateManagerImpl.java:484)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.state.StateManagerImpl.initialiseForPersistentNew(StateManagerImpl.java:120)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.state.ObjectProviderFactoryImpl.newForPersistentNew(ObjectProviderFactoryImpl.java:218)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.ExecutionContextImpl.persistObjectInternal(ExecutionContextImpl.java:2078)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.ExecutionContextImpl.persistObjectWork(ExecutionContextImpl.java:1922)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.ExecutionContextImpl.persistObject(ExecutionContextImpl.java:1777)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.ExecutionContextThreadedImpl.persistObject(ExecutionContextThreadedImpl.java:217)
17/10/17 03:55:26 INFO hive.HiveImport:     at org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:715)
17/10/17 03:55:26 INFO hive.HiveImport:     ... 45 more
17/10/17 03:55:26 ERROR tool.ImportTool: Encountered IOException running import job: java.io.IOException: Hive exited with status 1
    at org.apache.sqoop.hive.HiveImport.executeExternalHiveScript(HiveImport.java:389)
    at org.apache.sqoop.hive.HiveImport.executeScript(HiveImport.java:339)
    at org.apache.sqoop.hive.HiveImport.importTable(HiveImport.java:240)
    at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:514)
    at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
    at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
    at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
    at org.apache.sqoop.Sqoop.main(Sqoop.java:236)

Mysql version :  5.7.19-0ubuntu0.17.04.1
Hive version : version 2
Hadoop version : 2.7.1


来源: stackexchange

问:
Using AWK to substitute
First time attempting to tinker with AWK and use it to take input from a file like the following:

data.csv

James,Jones,30,Mr,Main St

Melissa,Greene,200,Mrs,Wall St

Robert,Krupp,410,Mr,Random St

and process it into a LaTeX Template

data.tex

```<code>\newcommand{\customertitle}{XYZ} %ggf. \
\newcommand{\customerName}{Max Sample} % Name \
\newcommand{\customerStreet}{Str} % Street \
\newcommand{\customerZIP}{12345} % ZIP
</code>```

First I tried to replace the customer name this way

```<code>awk 'BEGIN{FS=","}{print $1 " " $2 }' data.csv | xargs -I{} sed "s/Max Sample/{}/" data.tex > names
</code>```

which gave me a merged file.. and therefore I subsequently attempted to render the file as single .tex files by inserting a keyword "#TEST" at the end of the original file, so I could use it as a Record Separator to get me back to single files with the following command:

```<code>awk 'BEGIN {FS=RS="#TEST"} {i=1}{ while (i <= NF) {print $i >>"final"NR".tex"; i++}}' names
</code>```

Even though that worked for this one field, for multiple fields it doesn't seem to be a proper solution as is though. (title, street, zip code)

That's why I'm now attempting to get it working with the gsub action in AWK.

Tried those different approaches. Based on what I could find regarding it thus far, that's what I came up with:

```<code>awk 'BEGIN {FS=","}NR==FNR{a[FNR]=$4;next}{gsub ("XYZ",a[FNR]);print}' data.csv data.tex
</code>```

which replaces XYZ with  nothing

```<code>awk 'BEGIN {FS=","}NR==FNR{a[FNR]=$4;next}RS="#TEST"{for (i in a) {gsub("XYZ",i);print}}' data.csv data.tex \
</code>```

which counts four times to 7

Tried those also with the merged file, ie. the "names" output from the first command and didn't get it to work.

What am I missing? Can the gsub command not replace a string with an array? Is a loop required?

I'm stuck, hope someone can help out here

答:
I hope it is your case.
Create a file csv_to_latex.awk and put this code:

```<code>BEGIN{
    FS=","
    while(getline < latex > 0) {

        lax_array[$0]
    }
}

{
    name = $1" "$2
    zip = $3
    status = $4
    street = $5

    for (lax_key in lax_array)
    {
        if (lax_key ~ /XYZ/)
        {
            gsub("{XYZ}", "{"status"}", lax_key)
            print lax_key
        }
        else if (lax_key ~ /Max Sample/)
        {
            gsub("{Max Sample}", "{"name"}", lax_key)
            print lax_key
        }
        else if (lax_key ~ /Str/)
        {
            gsub("{Str}", "{"street"}", lax_key)
            print lax_key
        }
        else if (lax_key ~ /12345/)
        {
            gsub("{12345}", "{"zip"}", lax_key)
            print lax_key
        }
    }
}

</code>```

To execute this code, use in terminal:

```<code>awk -v latex="data.tex" -f csv_to_latex.awk data.csv
</code>```

The output:

```<code>\newcommand{\customertitle}{Mr} %ggf. \
\newcommand{\customerName}{James Jones} % Name \
\newcommand{\customerStreet}{Main St} % Street \
\newcommand{\customerZIP}{30} % ZIP
\newcommand{\customertitle}{Mrs} %ggf. \
\newcommand{\customerName}{Melissa Greene} % Name \
\newcommand{\customerStreet}{Wall St} % Street \
\newcommand{\customerZIP}{200} % ZIP
\newcommand{\customertitle}{Mr} %ggf. \
\newcommand{\customerName}{Robert Krupp} % Name \
\newcommand{\customerStreet}{Random St} % Street \
\newcommand{\customerZIP}{410} % ZIP

</code>```

ContentLicense: CC BY-SA 4.0

Title: Using AWK to substitute

Body:
First time attempting to tinker with AWK and use it to take input from a file like the following:

data.csv

James,Jones,30,Mr,Main St

Melissa,Greene,200,Mrs,Wall St

Robert,Krupp,410,Mr,Random St

and process it into a LaTeX Template

data.tex

\newcommand{\customertitle}{XYZ} %ggf. \
\newcommand{\customerName}{Max Sample} % Name \
\newcommand{\customerStreet}{Str} % Street \
\newcommand{\customerZIP}{12345} % ZIP

First I tried to replace the customer name this way

awk 'BEGIN{FS=","}{print $1 " " $2 }' data.csv | xargs -I{} sed "s/Max Sample/{}/" data.tex > names

which gave me a merged file.. and therefore I subsequently attempted to render the file as single .tex files by inserting a keyword "#TEST" at the end of the original file, so I could use it as a Record Separator to get me back to single files with the following command:

awk 'BEGIN {FS=RS="#TEST"} {i=1}{ while (i <= NF) {print $i >>"final"NR".tex"; i++}}' names

Even though that worked for this one field, for multiple fields it doesn't seem to be a proper solution as is though. (title, street, zip code)

That's why I'm now attempting to get it working with the gsub action in AWK.

Tried those different approaches. Based on what I could find regarding it thus far, that's what I came up with:

awk 'BEGIN {FS=","}NR==FNR{a[FNR]=$4;next}{gsub ("XYZ",a[FNR]);print}' data.csv data.tex

which replaces XYZ with  nothing

awk 'BEGIN {FS=","}NR==FNR{a[FNR]=$4;next}RS="#TEST"{for (i in a) {gsub("XYZ",i);print}}' data.csv data.tex \

which counts four times to 7

Tried those also with the merged file, ie. the "names" output from the first command and didn't get it to work.

What am I missing? Can the gsub command not replace a string with an array? Is a loop required?

I'm stuck, hope someone can help out here

Body:
I hope it is your case.
Create a file csv_to_latex.awk and put this code:

BEGIN{
    FS=","
    while(getline < latex > 0) {

        lax_array[$0]
    }
}

{
    name = $1" "$2
    zip = $3
    status = $4
    street = $5

    for (lax_key in lax_array)
    {
        if (lax_key ~ /XYZ/)
        {
            gsub("{XYZ}", "{"status"}", lax_key)
            print lax_key
        }
        else if (lax_key ~ /Max Sample/)
        {
            gsub("{Max Sample}", "{"name"}", lax_key)
            print lax_key
        }
        else if (lax_key ~ /Str/)
        {
            gsub("{Str}", "{"street"}", lax_key)
            print lax_key
        }
        else if (lax_key ~ /12345/)
        {
            gsub("{12345}", "{"zip"}", lax_key)
            print lax_key
        }
    }
}

To execute this code, use in terminal:

awk -v latex="data.tex" -f csv_to_latex.awk data.csv

The output:

\newcommand{\customertitle}{Mr} %ggf. \
\newcommand{\customerName}{James Jones} % Name \
\newcommand{\customerStreet}{Main St} % Street \
\newcommand{\customerZIP}{30} % ZIP
\newcommand{\customertitle}{Mrs} %ggf. \
\newcommand{\customerName}{Melissa Greene} % Name \
\newcommand{\customerStreet}{Wall St} % Street \
\newcommand{\customerZIP}{200} % ZIP
\newcommand{\customertitle}{Mr} %ggf. \
\newcommand{\customerName}{Robert Krupp} % Name \
\newcommand{\customerStreet}{Random St} % Street \
\newcommand{\customerZIP}{410} % ZIP


问:
C++ Visual 2019 error const char incompatible
I get this error :

```<code>Das Argument vom Typ ""const char *"" ist mit dem Parameter vom Typ ""char *"" inkompatibel.
</code>```

The argument from type const char is with the parameter from type char incompatible.

```<code>Animal::Animal(const char* k )
{
    next = 0;
    kind = new char[strlen(k) + 1];
    strcpy(kind,k);
}
</code>```

答:
This error occurs if the kind data member is defined as type const char *:

```<code>  error: invalid conversion from 'const char*' to 'char*' [-fpermissive]
</code>```

Defining the kind data member of type char * solves the problem:

#include <iostream>
#include <cstring>

using namespace std;

class Animal
{
    public:
        Animal(const char *k)
        {
            next = 0;
            kind = new char[strlen(k) + 1];
            strcpy(kind, k);
        }

        const char* getKind() const
        {
            return kind;
        }
    private:
        int next;
        char *kind; /* The kind data member is defined as type "const *". */
};

int main()
{
    const char *k = "Cat";
    Animal animal(k);
    cout << "Kind: " << animal.getKind() << endl;

    return 0;
}
```
<p>Result:</p>
```Kind: Cat
```

Body:
I get this error :

Das Argument vom Typ ""const char *"" ist mit dem Parameter vom Typ ""char *"" inkompatibel.

The argument from type const char is with the parameter from type char incompatible.

Animal::Animal(const char* k )
{
    next = 0;
    kind = new char[strlen(k) + 1];
    strcpy(kind,k);
}

Body:
This error occurs if the kind data member is defined as type const char *:

  error: invalid conversion from 'const char*' to 'char*' [-fpermissive]

Defining the kind data member of type char * solves the problem:

#include <iostream>
#include <cstring>

using namespace std;

class Animal
{
    public:
        Animal(const char *k)
        {
            next = 0;
            kind = new char[strlen(k) + 1];
            strcpy(kind, k);
        }

        const char* getKind() const
        {
            return kind;
        }
    private:
        int next;
        char *kind; /* The kind data member is defined as type "const *". */
};

int main()
{
    const char *k = "Cat";
    Animal animal(k);
    cout << "Kind: " << animal.getKind() << endl;

    return 0;
}

Result:

Kind: Cat


问:
Quarkus @ApplicationScoped working in dev run, but not when running the built jar
I have an abstract class, which has an implementation with @ApplicationScoped.

This works without an issue in quarkusDev, but when running the built app, it complains of a function that I have implemented in the abstract class.

java.lang.VerifyError: class me.l3n.[...].GoodReadsCrawlerService_ClientProxy overrides final method me.l3n.bot.[...].CrawlerService .parseHtml(Ljava/lang/String;Lkotlin/coroutines/Continuation;)Ljava/lang/Object;

```<code>abstract class CrawlerService {
    protected suspend infix fun parseHtml(url: String): Document {
        /* ... */
    }
}

@ApplicationScoped
class GoodReadsCrawlerService: CrawlerService() {
    /* implements other abstract functions of base class */
}
</code>```

Note that I'm not even overriding that parseHtml() function.

But if I change the GoodReadsCrawlerService to Singleton(), OR add the open keyword to parseHtml, it works just fine in any case.

The actual example can be found in my repo.

I'd assume the ClientProxy is treating the abstract class as an interface, so implements all functions for proxying, assuming they're all abstract.

But why doesn't this happen in quarkusDev?

答: This was a small issue on Quarkus itself, fixed in 2.7.1-final.</p>

Body:
I have an abstract class, which has an implementation with @ApplicationScoped.

This works without an issue in quarkusDev, but when running the built app, it complains of a function that I have implemented in the abstract class.

java.lang.VerifyError: class me.l3n.[...].GoodReadsCrawlerService_ClientProxy overrides final method me.l3n.bot.[...].CrawlerService .parseHtml(Ljava/lang/String;Lkotlin/coroutines/Continuation;)Ljava/lang/Object;

abstract class CrawlerService {
    protected suspend infix fun parseHtml(url: String): Document {
        /* ... */
    }
}

@ApplicationScoped
class GoodReadsCrawlerService: CrawlerService() {
    /* implements other abstract functions of base class */
}

Note that I'm not even overriding that parseHtml() function.

But if I change the GoodReadsCrawlerService to Singleton(), OR add the open keyword to parseHtml, it works just fine in any case.

The actual example can be found in my repo.

I'd assume the ClientProxy is treating the abstract class as an interface, so implements all functions for proxying, assuming they're all abstract.

But why doesn't this happen in quarkusDev?


问:
Convert data on pre-post repeated measures from long to wide by filtering data to get time point as value
I have a 14K row table of 370 liver transplant patients with transplant date and various repeated lab tests done before and after the procedure. I want to get pre-transplant, immediate post-transplant, and 3/6/12/18/24/36 month lab results.

ID
Transp Date
Lab Units
Lab Type
Tme
Lab Val

0000001
2011-01-11

VCA IgG Index
0
6487.0

0000001
2011-01-11

VCA IgM Index
0
11230.0

0000002
2011-01-03
Copies/mL
CMV Quant PCR
3
100.0

0000002
2011-01-03
Copies/mL
EBV Quant PCR
3
683.0.

I did round(datediff) of transplant date and lab test date to get the month timepoint (Tme). My client wants the final table to have one record and all data values per row. Headers something like this:

ID|TrnsplDate|LabType1|Units1|PreVal|Val0|Val3|Val6|Val12|Val18|Val24|Val36|LabType2|Units2|PreVal|Val0|Val3|Val6|Val12|Val18|Val24|Val36|LabType3|Units3|PreVal|Val0|Val3|Val6|Val12|Val18|Val24|Val36|LabType4|Units4|PreVal|Val0|Val3|Val6|Val12|Val18|Val24|Val36

Can anyone knowledgeable in R guide me on where to start? I use Rstudio. Thanks in advance.

答:
Try this, which will put everything for the same ID on one line, then you can adjust column names and order as needed using colnames(df) and indexing (i.e., something like colorder <- c(2,3,5,1,7,12,...); df[, colorder].

```<code>### Set up data
library(lubridate)
df <- data.frame(ID = rep(sprintf("SID%s",seq(1:2)),2),
                 transdate = seq(mdy("01/01/2000"), mdy("01/4/2000"),1),
                 labunits = c(NA, NA, rep("Copies/mL",2)),
                 labtype = c(rep("VCA IgG Index",2),"CMV Quant PCR", "EBV Quant PCR"),
                 time = c(0,0,2,2),
                 labval = sample(100:2000, 4))
# Transform
df2 <- tidyr::pivot_wider(df, names_from = labtype, values_from = -ID)

# ----------------------------
# Edit: separate by lab type
df_bylab <- split(df, df$labtype)

# output each lab type to CSV
for(i in 1:length(df)){
  write.csv(df[i], paste0(names(df)[i], ".csv"))
}

</code>```

Based on the limited data provided, I am not sure if some columns could be collapsed (for instance, it seems like you only have one date per SID, so all the date columns could be collapsed to one column.

I would also like to point out this is not an ideal structure for the data, so perhaps your best bet is to try to convince your client otherwise!

Body:
Try this, which will put everything for the same ID on one line, then you can adjust column names and order as needed using colnames(df) and indexing (i.e., something like colorder <- c(2,3,5,1,7,12,...); df[, colorder].

### Set up data
library(lubridate)
df <- data.frame(ID = rep(sprintf("SID%s",seq(1:2)),2),
                 transdate = seq(mdy("01/01/2000"), mdy("01/4/2000"),1),
                 labunits = c(NA, NA, rep("Copies/mL",2)),
                 labtype = c(rep("VCA IgG Index",2),"CMV Quant PCR", "EBV Quant PCR"),
                 time = c(0,0,2,2),
                 labval = sample(100:2000, 4))
# Transform
df2 <- tidyr::pivot_wider(df, names_from = labtype, values_from = -ID)

# ----------------------------
# Edit: separate by lab type
df_bylab <- split(df, df$labtype)

# output each lab type to CSV
for(i in 1:length(df)){
  write.csv(df[i], paste0(names(df)[i], ".csv"))
}

Based on the limited data provided, I am not sure if some columns could be collapsed (for instance, it seems like you only have one date per SID, so all the date columns could be collapsed to one column.

I would also like to point out this is not an ideal structure for the data, so perhaps your best bet is to try to convince your client otherwise!


问:
Input Email and Checkbox validation to Enable Submit button using javascript
I have the following javascript code for a form in which I validate email with certain patterns and a checkbox with I agree with T&C label. If any of them failed then I disable the submit button. It is all working but I am using too many functions which are causing some conflicts in which if one condition gets true then submit button is available to click.

I am not able to verify both conditions at the same time because both are of different types. There are some answers on Stack overflow but they are of the same type of fields like 2 inputs, 3 checkboxes but not different fields.

Here's the code:

```<code>    window.onload = function ninjaInit() {
  validatEmail();
    validatCheckbox();
  jQuery('#nf-field-20').keyup(validatEmail);
    jQuery('#nf-field-13').change(validatCheckbox);
  }

function validatEmail() {
  var email = document.getElementById('nf-field-20');
    //console.log(email.value);
  if (
    /^[a-zA-Z0-9._%+-]+@(?!gmail.com)(?!yahoo.com)(?!hotmail.com)(?!yahoo.co.in)(?!aol.com)(?!live.com)(?!outlook.com)[a-zA-Z0-9_-]+.[a-zA-Z0-9-.]{2,61}$/.test(email.value)) {
    itWorksem();
  } else {
    notWorkingem();
  }
}
function validatCheckbox() {
    if(!jQuery('#nf-field-13').hasClass("nf-checked")){
             console.log('checkbox not checked');
        checkBoGood();
             }else{
             console.log('checkbox checked');
              checkBoBad();
             }
}

function itWorksem() {
  var errMessage = document.getElementById("nf-error-20");
  //jQuery("#nf-field-20-wrap").removeClass("nf-error");
  jQuery("#nf-field-15").removeClass("btn-disable");
  jQuery("#nf-field-20").removeClass("input-error");
  errMessage.textContent = "";
}
function notWorkingem() {
  var errMessage = document.getElementById("nf-error-20");
  //jQuery("#nf-field-20-wrap").addClass("nf-error");
  jQuery("#nf-field-15").addClass("btn-disable");
  jQuery("#nf-field-20").addClass("input-error");
  errMessage.textContent = "You need to provide a business address.";
}
function checkBoGood(){
    var errMsgPp = document.getElementById("nf-error-13");
    jQuery("#nf-field-15").removeClass("btn-disable");
//  errMsgPp.textContent = "";
}
function checkBoBad(){
    var errMsgPp = document.getElementById("nf-error-13");
    jQuery("#nf-field-15").addClass("btn-disable");
}
</code>```

Some things need to be considered here:

I am not able to check checkbox status with standard functions because that checkbox also uses a class to check if it is checked or not by adding "nf-checked" class that is why I have to check if that element has the class name "nf-checked" or not. FYI this ninja form.

I can't use Document.ready and $ because that is not working in WordPress.

All this code I have done myself so please correct me if I am doing some silly mistake. I am very much a beginner in Javascript.

If Html helps then I am adding that too below. I am not adding the whole code because I can target element using IDs.

```<code><!--Email container Start-->
<div id="nf-field-20-container" class="nf-field-container textbox-container  label-hidden ">
    <div class="nf-before-field">
    </div>
    <div class="nf-field">
        <div id="nf-field-20-wrap" class="field-wrap textbox-wrap" data-field-id="20">
            <div class="nf-field-label"><label for="nf-field-20" id="nf-label-field-20" class="">Work email address  </label>
            </div>
            <div class="nf-field-element">
                <input type="text" value="" class="ninja-forms-field nf-element input-error" placeholder="Work email address" id="nf-field-20" name="nf-field-20" aria-invalid="false" aria-describedby="nf-error-20" aria-labelledby="nf-label-field-20">
                </div>
        </div>
    </div>
    <div class="nf-after-field">
        <nf-section>
            <div class="nf-input-limit">
            </div>
            <div id="nf-error-20" class="nf-error-wrap nf-error" role="alert">You need to provide a business address.</div>
        </nf-section>
    </div>
</div>
<!--Email container Ends-->

<!--Checkbox Start-->
<div id="nf-field-13-container" class="nf-field-container checkbox-container  label-right one-half first ">
    <div class="nf-before-field">
    </div>
    <div class="nf-field"><div id="nf-field-13-wrap" class="field-wrap checkbox-wrap" data-field-id="13">
        <div class="nf-field-label"><label for="nf-field-13" id="nf-label-field-13" class=" nf-checked-label">I accept hubsell’s <a href="#">privacy policy</a>.  </label>
        </div>
    <div class="nf-field-element">
        <input id="nf-field-13" name="nf-field-13" aria-describedby="nf-error-13" class="ninja-forms-field nf-checked nf-element" type="checkbox" value="1" checked="" aria-labelledby="nf-label-field-13">
        </div>
    </div>
</div>
<div class="nf-after-field"><nf-section>
    <div class="nf-input-limit"></div>
    <div id="nf-error-13" class="nf-error-wrap nf-error" role="alert"></div>
    </nf-section>
    </div>
</div>
<!--Checkbox Ends-->

<!--Submit Start-->
<input id="nf-field-15" class="ninja-forms-field nf-element" type="button" value="Submit">
<!--Submit Ends-->
</code>```

window.onload = function ninjaInit() {
  validatEmail();
  validatCheckbox();
  jQuery('#nf-field-20').keyup(validatEmail);
  jQuery('#nf-field-13').change(validatCheckbox);
}

function validatEmail() {
  var email = document.getElementById('nf-field-20');
  //console.log(email.value);
  if (
    /^[a-zA-Z0-9._%+-]+@(?!gmail.com)(?!yahoo.com)(?!hotmail.com)(?!yahoo.co.in)(?!aol.com)(?!live.com)(?!outlook.com)[a-zA-Z0-9_-]+.[a-zA-Z0-9-.]{2,61}$/.test(email.value)) {
    itWorksem();
  } else {
    notWorkingem();
  }
}

function validatCheckbox() {
  if (!jQuery('#nf-field-13').hasClass("nf-checked")) {
    console.log('checkbox not checked');
    checkBoGood();
  } else {
    console.log('checkbox checked');
    checkBoBad();
  }
}

function itWorksem() {
  var errMessage = document.getElementById("nf-error-20");
  //jQuery("#nf-field-20-wrap").removeClass("nf-error");
  jQuery("#nf-field-15").removeClass("btn-disable");
  jQuery("#nf-field-20").removeClass("input-error");
  errMessage.textContent = "";
}

function notWorkingem() {
  var errMessage = document.getElementById("nf-error-20");
  //jQuery("#nf-field-20-wrap").addClass("nf-error");
  jQuery("#nf-field-15").addClass("btn-disable");
  jQuery("#nf-field-20").addClass("input-error");
  errMessage.textContent = "You need to provide a business address.";
}

function checkBoGood() {
  var errMsgPp = document.getElementById("nf-error-13");
  jQuery("#nf-field-15").removeClass("btn-disable");
  //  errMsgPp.textContent = "";
}

function checkBoBad() {
  var errMsgPp = document.getElementById("nf-error-13");
  jQuery("#nf-field-15").addClass("btn-disable");
}```
<pre class="snippet-code-html lang-html prettyprint-override"><code><script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.3.1/jquery.min.js"></script>
<!--Email container Start-->
<div id="nf-field-20-container" class="nf-field-container textbox-container  label-hidden ">
  <div class="nf-before-field">
  </div>
  <div class="nf-field">
    <div id="nf-field-20-wrap" class="field-wrap textbox-wrap" data-field-id="20">
      <div class="nf-field-label"><label for="nf-field-20" id="nf-label-field-20" class="">Work email address  </label>
      </div>
      <div class="nf-field-element">
        <input type="text" value="" class="ninja-forms-field nf-element input-error" placeholder="Work email address" id="nf-field-20" name="nf-field-20" aria-invalid="false" aria-describedby="nf-error-20" aria-labelledby="nf-label-field-20">
      </div>
    </div>
  </div>
  <div class="nf-after-field">
    <nf-section>
      <div class="nf-input-limit">
      </div>
      <div id="nf-error-20" class="nf-error-wrap nf-error" role="alert">You need to provide a business address.</div>
    </nf-section>
  </div>
</div>
<!--Email container Ends-->

<!--Checkbox Start-->
<div id="nf-field-13-container" class="nf-field-container checkbox-container  label-right one-half first ">
  <div class="nf-before-field">
  </div>
  <div class="nf-field">
    <div id="nf-field-13-wrap" class="field-wrap checkbox-wrap" data-field-id="13">
      <div class="nf-field-label"><label for="nf-field-13" id="nf-label-field-13" class=" nf-checked-label">I accept hubsell’s privacy policy.  </label>
      </div>
      <div class="nf-field-element">
        <input id="nf-field-13" name="nf-field-13" aria-describedby="nf-error-13" class="ninja-forms-field nf-checked nf-element" type="checkbox" value="1" checked="" aria-labelledby="nf-label-field-13">
      </div>
    </div>
  </div>
  <div class="nf-after-field">
    <nf-section>
      <div class="nf-input-limit"></div>
      <div id="nf-error-13" class="nf-error-wrap nf-error" role="alert"></div>
    </nf-section>
  </div>
</div>
<!--Checkbox Ends-->

<!--Submit Start-->
<input id="nf-field-15" class="ninja-forms-field nf-element" type="button" value="Submit">
<!--Submit Ends--></code>```

You can see the input[type="checkbox"] has that nf-checked class already I mark the default value as checked in ninja forms.

Update: I have added the proper HTML code after ruleboy21 comment.

答:
Replace your JavaScript with this</p>
```<code>jQuery(document).ready(function($){
    // add all the names of the fields you want to validate in this object
    // if the field has a valid default value, give it a value of true else pass false
    let successBag = {
        'nf-field-20': false,
        'nf-field-13': true
    };

    // function to disable the submit button
    function disableForm(key, value){
        if(key) successBag[key] = value;
        if(Object.values(successBag).includes(false)){
            $("#nf-field-15").attr('disabled', 'disabled').addClass("btn-disable");
        }else{
            $("#nf-field-15").removeAttr('disabled').removeClass("btn-disable");
        }
    }
    // disable/enable the form by default
    disableForm(null, null);

    $('#nf-field-20').on('input', function(){
        let errMessage = $("#nf-error-20");

        if(/^([a-zA-Z0-9_.+-]{2,4})+\@(([a-zA-Z0-9-]{2,8})+\.)+([a-zA-Z0-9]{2,4})+$/.test(this.value)){
            $(this).removeClass("input-error");
            errMessage.text("");
            disableForm(this.name, true);
        }else{
            $(this).addClass("input-error");
            errMessage.text("You need to provide a business address.");
            disableForm(this.name, false);
        }
    });
    $('#nf-field-13').on('change', function(){
        let errMessage = $("#nf-error-13");

        if($(this).is(':checked')){
            errMessage.text("");
            disableForm(this.name, true);
        }else{
            errMessage.text("Privacy policy is required.");
            disableForm(this.name, false);
        }
    });
});
</code>```

Body:
I have the following javascript code for a form in which I validate email with certain patterns and a checkbox with I agree with T&C label. If any of them failed then I disable the submit button. It is all working but I am using too many functions which are causing some conflicts in which if one condition gets true then submit button is available to click.

I am not able to verify both conditions at the same time because both are of different types. There are some answers on Stack overflow but they are of the same type of fields like 2 inputs, 3 checkboxes but not different fields.

Here's the code:

    window.onload = function ninjaInit() {
  validatEmail();
    validatCheckbox();
  jQuery('#nf-field-20').keyup(validatEmail);
    jQuery('#nf-field-13').change(validatCheckbox);
  }

function validatEmail() {
  var email = document.getElementById('nf-field-20');
    //console.log(email.value);
  if (
    /^[a-zA-Z0-9._%+-]+@(?!gmail.com)(?!yahoo.com)(?!hotmail.com)(?!yahoo.co.in)(?!aol.com)(?!live.com)(?!outlook.com)[a-zA-Z0-9_-]+.[a-zA-Z0-9-.]{2,61}$/.test(email.value)) {
    itWorksem();
  } else {
    notWorkingem();
  }
}
function validatCheckbox() {
    if(!jQuery('#nf-field-13').hasClass("nf-checked")){
             console.log('checkbox not checked');
        checkBoGood();
             }else{
             console.log('checkbox checked');
              checkBoBad();
             }
}

function itWorksem() {
  var errMessage = document.getElementById("nf-error-20");
  //jQuery("#nf-field-20-wrap").removeClass("nf-error");
  jQuery("#nf-field-15").removeClass("btn-disable");
  jQuery("#nf-field-20").removeClass("input-error");
  errMessage.textContent = "";
}
function notWorkingem() {
  var errMessage = document.getElementById("nf-error-20");
  //jQuery("#nf-field-20-wrap").addClass("nf-error");
  jQuery("#nf-field-15").addClass("btn-disable");
  jQuery("#nf-field-20").addClass("input-error");
  errMessage.textContent = "You need to provide a business address.";
}
function checkBoGood(){
    var errMsgPp = document.getElementById("nf-error-13");
    jQuery("#nf-field-15").removeClass("btn-disable");
//  errMsgPp.textContent = "";
}
function checkBoBad(){
    var errMsgPp = document.getElementById("nf-error-13");
    jQuery("#nf-field-15").addClass("btn-disable");
}

Some things need to be considered here:

I am not able to check checkbox status with standard functions because that checkbox also uses a class to check if it is checked or not by adding "nf-checked" class that is why I have to check if that element has the class name "nf-checked" or not. FYI this ninja form.

I can't use Document.ready and $ because that is not working in WordPress.

All this code I have done myself so please correct me if I am doing some silly mistake. I am very much a beginner in Javascript.

If Html helps then I am adding that too below. I am not adding the whole code because I can target element using IDs.

Work email address

                <input type="text" value="" class="ninja-forms-field nf-element input-error" placeholder="Work email address" id="nf-field-20" name="nf-field-20" aria-invalid="false" aria-describedby="nf-error-20" aria-labelledby="nf-label-field-20">

        <nf-section>

You need to provide a business address.

        </nf-section>

I accept hubsell’s privacy policy.

        <input id="nf-field-13" name="nf-field-13" aria-describedby="nf-error-13" class="ninja-forms-field nf-checked nf-element" type="checkbox" value="1" checked="" aria-labelledby="nf-label-field-13">

<nf-section>

    </nf-section>

<input id="nf-field-15" class="ninja-forms-field nf-element" type="button" value="Submit">

window.onload = function ninjaInit() {
  validatEmail();
  validatCheckbox();
  jQuery('#nf-field-20').keyup(validatEmail);
  jQuery('#nf-field-13').change(validatCheckbox);
}

function validatEmail() {
  var email = document.getElementById('nf-field-20');
  //console.log(email.value);
  if (
    /^[a-zA-Z0-9._%+-]+@(?!gmail.com)(?!yahoo.com)(?!hotmail.com)(?!yahoo.co.in)(?!aol.com)(?!live.com)(?!outlook.com)[a-zA-Z0-9_-]+.[a-zA-Z0-9-.]{2,61}$/.test(email.value)) {
    itWorksem();
  } else {
    notWorkingem();
  }
}

function validatCheckbox() {
  if (!jQuery('#nf-field-13').hasClass("nf-checked")) {
    console.log('checkbox not checked');
    checkBoGood();
  } else {
    console.log('checkbox checked');
    checkBoBad();
  }
}

function itWorksem() {
  var errMessage = document.getElementById("nf-error-20");
  //jQuery("#nf-field-20-wrap").removeClass("nf-error");
  jQuery("#nf-field-15").removeClass("btn-disable");
  jQuery("#nf-field-20").removeClass("input-error");
  errMessage.textContent = "";
}

function notWorkingem() {
  var errMessage = document.getElementById("nf-error-20");
  //jQuery("#nf-field-20-wrap").addClass("nf-error");
  jQuery("#nf-field-15").addClass("btn-disable");
  jQuery("#nf-field-20").addClass("input-error");
  errMessage.textContent = "You need to provide a business address.";
}

function checkBoGood() {
  var errMsgPp = document.getElementById("nf-error-13");
  jQuery("#nf-field-15").removeClass("btn-disable");
  //  errMsgPp.textContent = "";
}

function checkBoBad() {
  var errMsgPp = document.getElementById("nf-error-13");
  jQuery("#nf-field-15").addClass("btn-disable");
}

Work email address

        <input type="text" value="" class="ninja-forms-field nf-element input-error" placeholder="Work email address" id="nf-field-20" name="nf-field-20" aria-invalid="false" aria-describedby="nf-error-20" aria-labelledby="nf-label-field-20">

    <nf-section>

You need to provide a business address.

    </nf-section>

I accept hubsell’s privacy policy.

        <input id="nf-field-13" name="nf-field-13" aria-describedby="nf-error-13" class="ninja-forms-field nf-checked nf-element" type="checkbox" value="1" checked="" aria-labelledby="nf-label-field-13">

    <nf-section>

    </nf-section>

<input id="nf-field-15" class="ninja-forms-field nf-element" type="button" value="Submit">

You can see the input[type="checkbox"] has that nf-checked class already I mark the default value as checked in ninja forms.

Update: I have added the proper HTML code after ruleboy21 comment.

Body:
Replace your JavaScript with this

jQuery(document).ready(function($){
    // add all the names of the fields you want to validate in this object
    // if the field has a valid default value, give it a value of true else pass false
    let successBag = {
        'nf-field-20': false,
        'nf-field-13': true
    };

    // function to disable the submit button
    function disableForm(key, value){
        if(key) successBag[key] = value;
        if(Object.values(successBag).includes(false)){
            $("#nf-field-15").attr('disabled', 'disabled').addClass("btn-disable");
        }else{
            $("#nf-field-15").removeAttr('disabled').removeClass("btn-disable");
        }
    }
    // disable/enable the form by default
    disableForm(null, null);

    $('#nf-field-20').on('input', function(){
        let errMessage = $("#nf-error-20");

        if(/^([a-zA-Z0-9_.+-]{2,4})+\@(([a-zA-Z0-9-]{2,8})+\.)+([a-zA-Z0-9]{2,4})+$/.test(this.value)){
            $(this).removeClass("input-error");
            errMessage.text("");
            disableForm(this.name, true);
        }else{
            $(this).addClass("input-error");
            errMessage.text("You need to provide a business address.");
            disableForm(this.name, false);
        }
    });
    $('#nf-field-13').on('change', function(){
        let errMessage = $("#nf-error-13");

        if($(this).is(':checked')){
            errMessage.text("");
            disableForm(this.name, true);
        }else{
            errMessage.text("Privacy policy is required.");
            disableForm(this.name, false);
        }
    });
});


问:
Scroll a PDF with Selenium python
I'm trying to scroll through a PDF with Selenium but It doesn't appear to be working.

The code I'm working with is a little in depth but here's the crucial bits.

```<code>    driver = webdriver.Firefox()

    driver.maximize_window()
    driver.get("http://scop-qb.org/wp-content/uploads/SCOP-B-Round-2.pdf")

</code>```
```<code>    for rectangle in rectangles:
        if not previous is None:
            driver.execute_script("window.scrollTo({0}, {1})".format(rectangle[0], rectangle[1]))
</code>```

I've been searching for a solution all night, if anyone could help that'd be great.

答:
An alternative would be to chain a bunch of down-arrow keys or spacebars together:</p>
```<code>driver.get("http://scop-qb.org/wp-content/uploads/SCOP-B-Round-2.pdf")
time.sleep(1)
actions = ActionChains(driver)
for _ in range(3): # Adjust for how far down the page you want
    actions.send_keys(Keys.ARROW_DOWN).perform()
</code>```

Title: Scroll a PDF with Selenium python

Body:
I'm trying to scroll through a PDF with Selenium but It doesn't appear to be working.

The code I'm working with is a little in depth but here's the crucial bits.

    driver = webdriver.Firefox()

    driver.maximize_window()
    driver.get("http://scop-qb.org/wp-content/uploads/SCOP-B-Round-2.pdf")

    for rectangle in rectangles:
        if not previous is None:
            driver.execute_script("window.scrollTo({0}, {1})".format(rectangle[0], rectangle[1]))

I've been searching for a solution all night, if anyone could help that'd be great.

Body:
An alternative would be to chain a bunch of down-arrow keys or spacebars together:

driver.get("http://scop-qb.org/wp-content/uploads/SCOP-B-Round-2.pdf")
time.sleep(1)
actions = ActionChains(driver)
for _ in range(3): # Adjust for how far down the page you want
    actions.send_keys(Keys.ARROW_DOWN).perform()


问:
How to set iBeacon TX power byte
I am working on the ESP32 microcontroller and I would like to implement iBeacon advertising feature. I have been reading about the iBeacon. I have learnt about the specific format that iBeacon packet uses:
https://os.mbed.com/blog/entry/BLE-Beacons-URIBeacon-AltBeacons-iBeacon/

From what I understand, iBeacon preset is set and not meant to be modified. I must set a custom UUID, major and minor numbers such as:

```<code>uint8_t Beacon_UUID[16] = {0x00,0x11,0x22,0x33,0x44,0x55,0x66,0x77,0x88,0x99,0xAA,0xBB,0xCC,0xDD,0xEE,0xFF};

uint8_t Beacon_MAJOR[2] = {0x12,0x34};

uint8_t Beacon_MINOR[2] = {0x56,0x78};
</code>```

The only thing that I am confused about is the TX Power byte. What should I set it to?

According to the website that I have referred above:

Blockquote
A scanning application reads the UUID, major number and minor number and references them against a database to get information about the beacon; the beacon itself carries no descriptive information - it requires this external database to be useful. The TX power field is used with the measured signal strength to determine how far away the beacon is from the smart phone. Please note that TxPower must be calibrated on a beacon-by-beacon basis by the user to be accurate.
Blockquote

It mentions what is TxPower and how it should be determined but I still cannot make any sense out of it. Why would I need to measure how far away the beacon is from the smart phone if? That should be done by the iBeacon scanner not the advertiser(me).

答:
When you are making a hardware device transmit iBeacon, it is your responsibility to measure the output power of the transmitter and put the corresponding value into the TxPower byte of the iBeacon transmission.

Why?  Because receiving applications that detect your beacon need to know how strong your transmitter is to estimate distance.  Otherwise there would be no way for the receiving application to tell if a medium signal level like -75 dB is from a nearby weak transmitter or a far away strong transmitter.

The basic procedure is to put a receiver exactly one meter away from your transmitter and measure the RSSI at that distance.  The one meter RSSI is what you put into TxPower byte of the iBeacon advertisement.

The specifics of how to measure this properly can be a bit tricky, because every receiver has a different "specificity" meaning they will read a higher or lower RSSI depending on their antenna gain.  When Apple came out with iBeacon several years ago, they declared the reference receiver an iPhone 4S -- this was the newest phone available at that time.  You would run beacon detection app like AirLocate (not available in the App Store) or my Beacon Locate (available in the App Store).  The basic procedure is to aim the back of the phone at the beacon when it is exactly one meter away and use the app to measure the RSSI.  Many detector apps have a "calibrate" feature which averages RSSI measurements over 30 seconds or so.  For best results when calibrating, do this with both transmitter and receiver at least 3 feet above the ground and minimize metal or dense walls nearby.  Ideally, you would do this outdoors using two plastic tripods (or do the same inside an antenna chamber.)

It is hard to find a reference iPhone 4S these days, and other iPhone models can measure surprisingly different RSSI values.   My tests show that an iPhone SE 2nd edition measures signals very similarly to an iPhone 4S.  But even these models are not made anymore.  If you cannot get one of these,  use the oldest iPhone you can get without a case and take the best measurement you can as described above.  Obviously a ideal measurement requires more effort -- you have to decide how much effort you are willing to put into this.  An ideal measurement is only important if you expect receiving applications to want to get the best distance measurements possible.

Title: How to set iBeacon TX power byte

Body:
I am working on the ESP32 microcontroller and I would like to implement iBeacon advertising feature. I have been reading about the iBeacon. I have learnt about the specific format that iBeacon packet uses:
https://os.mbed.com/blog/entry/BLE-Beacons-URIBeacon-AltBeacons-iBeacon/

From what I understand, iBeacon preset is set and not meant to be modified. I must set a custom UUID, major and minor numbers such as:

uint8_t Beacon_UUID[16] = {0x00,0x11,0x22,0x33,0x44,0x55,0x66,0x77,0x88,0x99,0xAA,0xBB,0xCC,0xDD,0xEE,0xFF};

uint8_t Beacon_MAJOR[2] = {0x12,0x34};

uint8_t Beacon_MINOR[2] = {0x56,0x78};

The only thing that I am confused about is the TX Power byte. What should I set it to?

According to the website that I have referred above:

Blockquote
A scanning application reads the UUID, major number and minor number and references them against a database to get information about the beacon; the beacon itself carries no descriptive information - it requires this external database to be useful. The TX power field is used with the measured signal strength to determine how far away the beacon is from the smart phone. Please note that TxPower must be calibrated on a beacon-by-beacon basis by the user to be accurate.
Blockquote

It mentions what is TxPower and how it should be determined but I still cannot make any sense out of it. Why would I need to measure how far away the beacon is from the smart phone if? That should be done by the iBeacon scanner not the advertiser(me).


问:
Selecting multiple radio buttons (through jquery, javasscript?)
I currently have a form with lets say 5 radio button entries (see below).
What im looking archive is the following:
- Being able to choose multiple radio buttons - lets say 3 and submit the form.

Currently I got it working fine with PHP, SQL, but im only able to choose one radiobutton and submit that.

I figure it would also come in handy being able to deselect a radio button in case you wrongly click one.

My guess is that this can be done through some javascript? Any suggestions? Online examples perhaps?

```<code><form id="pollform" action="poll.php" method="post">
<input id="option-1" type="radio" value="1" name="poll">
<label for="option-1">Select option 1</label>

<input id="option-2" type="radio" value="2" name="poll">
<label for="option-2">Select option 2</label>

<input id="option-3" type="radio" value="3" name="poll">
<label for="option-3">Select option 3</label>

<input id="option-4" type="radio" value="4" name="poll">
<label for="option-4">Select option 4</label>

<input id="option-5" type="radio" value="5" name="poll">
<label for="option-5">Select option 5</label>
</form>
</code>```

答:
As it has same name poll you will not be able to do that as input type radio is specialized in selecting a single value from multiple inputs.

You can use input type checkbox for that and make them as an array:

```<code><form id="pollform" action="poll.php" method="post">
    <input id="option-1" type="checkbox" value="1" name="poll[]">
    <label for="option-1">Select option 1</label>

    <input id="option-2" type="checkbox" value="2" name="poll[]">
    <label for="option-2">Select option 2</label>

    <input id="option-3" type="checkbox" value="3" name="poll[]">
    <label for="option-3">Select option 3</label>

    <input id="option-4" type="checkbox" value="4" name="poll[]">
    <label for="option-4">Select option 4</label>

    <input id="option-5" type="checkbox" value="5" name="poll[]">
    <label for="option-5">Select option 5</label>
</form>
</code>```

LIMIT (with jQuery) the number:

```<code>$("input[type=checkbox][name=poll[]]").click(function() {
    var numberSel = $("input[type=checkbox][name=poll[]]:checked").length >= 3;
    $("input[type=checkbox][name=poll[]]").not(":checked").attr("disabled",numberSel);
});
</code>```

ContentLicense: CC BY-SA 3.0

Body:
I currently have a form with lets say 5 radio button entries (see below).
What im looking archive is the following:
- Being able to choose multiple radio buttons - lets say 3 and submit the form.

Currently I got it working fine with PHP, SQL, but im only able to choose one radiobutton and submit that.

I figure it would also come in handy being able to deselect a radio button in case you wrongly click one.

My guess is that this can be done through some javascript? Any suggestions? Online examples perhaps?

<form id="pollform" action="poll.php" method="post">
<input id="option-1" type="radio" value="1" name="poll">
Select option 1

<input id="option-2" type="radio" value="2" name="poll">
Select option 2

<input id="option-3" type="radio" value="3" name="poll">
Select option 3

<input id="option-4" type="radio" value="4" name="poll">
Select option 4

<input id="option-5" type="radio" value="5" name="poll">
Select option 5
</form>

Body:
As it has same name poll you will not be able to do that as input type radio is specialized in selecting a single value from multiple inputs.

You can use input type checkbox for that and make them as an array:

<form id="pollform" action="poll.php" method="post">
    <input id="option-1" type="checkbox" value="1" name="poll[]">
    <label for="option-1">Select option 1</label>

    <input id="option-2" type="checkbox" value="2" name="poll[]">
    <label for="option-2">Select option 2</label>

    <input id="option-3" type="checkbox" value="3" name="poll[]">
    <label for="option-3">Select option 3</label>

    <input id="option-4" type="checkbox" value="4" name="poll[]">
    <label for="option-4">Select option 4</label>

    <input id="option-5" type="checkbox" value="5" name="poll[]">
    <label for="option-5">Select option 5</label>
</form>

LIMIT (with jQuery) the number:

$("input[type=checkbox][name=poll[]]").click(function() {
    var numberSel = $("input[type=checkbox][name=poll[]]:checked").length >= 3;
    $("input[type=checkbox][name=poll[]]").not(":checked").attr("disabled",numberSel);
});


问:
Run [Run] section before [Files] section
My setup downloads a file called launcher.zip to the {tmp} folder and during and after installation it runs the following scripts:

[Files]
Source: "{tmp}\launchers\Launcher"; DestDir: "{app}"; Flags: onlyifdoesntexist recursesubdirs external;

[Run]
Filename: "{tmp}\7zip\7za.exe"; Parameters: "x ""{tmp}\launchers\launcher.zip"" -o""{tmp}\launchers"" * -r -aoa"; Flags: runhidden runascurrentuser;
```

But this way the [Files] section runs before the [Run] section so it doesn't find the folder extracted to the {tmp} folder.

I could just make the [Run] script extract to the destination folder, but I want to use the [Files] section to set the Flags: onlyifdoesntexist

Because I don't want it to replace files when extracting, unless a specific Component is selected, which in this case is launchers\reinstall (Another script removes the files if Component is selected)

So I need the [Run] section script to be executed before the [Files] section

Is there any way to do this?

答:
Instead of running a [Run] section, I ran a [Codes] section to extract, and added a BeforeInstall to my [Files] section

My answer here:
Extract file during installation

Body:
My setup downloads a file called launcher.zip to the {tmp} folder and during and after installation it runs the following scripts:

[Files]
Source: "{tmp}\launchers\Launcher"; DestDir: "{app}"; Flags: onlyifdoesntexist recursesubdirs external;

[Run]
Filename: "{tmp}\7zip\7za.exe"; Parameters: "x ""{tmp}\launchers\launcher.zip"" -o""{tmp}\launchers"" * -r -aoa"; Flags: runhidden runascurrentuser;

But this way the [Files] section runs before the [Run] section so it doesn't find the folder extracted to the {tmp} folder.

I could just make the [Run] script extract to the destination folder, but I want to use the [Files] section to set the Flags: onlyifdoesntexist

Because I don't want it to replace files when extracting, unless a specific Component is selected, which in this case is launchers\reinstall (Another script removes the files if Component is selected)

So I need the [Run] section script to be executed before the [Files] section

Is there any way to do this?


问:
How to save pyspark dataframe from hive into iginte cache?
I would like to save my pyspark dataframe into ignite cache following the approach from this blog, but some error happened.
Error message is:

```<code>Traceback (most recent call last):
  File "test_ignite.py", line 19, in <module>
    .option("config", configFile) \
  File "/home/hdoop/.local/lib/python3.6/site-packages/pyspark/sql/readwriter.py", line 738, in save
    self._jwrite.save()
  File "/home/hdoop/.local/lib/python3.6/site-packages/py4j/java_gateway.py", line 1322, in __call__
    answer, self.gateway_client, self.target_id, self.name)
  File "/home/hdoop/.local/lib/python3.6/site-packages/pyspark/sql/utils.py", line 111, in deco
    return f(*a, **kw)
  File "/home/hdoop/.local/lib/python3.6/site-packages/py4j/protocol.py", line 328, in get_return_value
    format(target_id, ".", name), value)
py4j.protocol.Py4JJavaError: An error occurred while calling o42.save.
: java.lang.ClassNotFoundException:
Failed to find data source: ignite. Please find packages at
http://spark.apache.org/third-party-projects.html

        at org.apache.spark.sql.errors.QueryExecutionErrors$.failedToFindDataSourceError(QueryExecutionErrors.scala:443)
        at org.apache.spark.sql.execution.datasources.DataSource$.lookupDataSource(DataSource.scala:670)
        at org.apache.spark.sql.execution.datasources.DataSource$.lookupDataSourceV2(DataSource.scala:720)
        at org.apache.spark.sql.DataFrameWriter.lookupV2Provider(DataFrameWriter.scala:852)
        at org.apache.spark.sql.DataFrameWriter.saveInternal(DataFrameWriter.scala:256)
        at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:247)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
        at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
        at py4j.Gateway.invoke(Gateway.java:282)
        at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
        at py4j.commands.CallCommand.execute(CallCommand.java:79)
        at py4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:182)
        at py4j.ClientServerConnection.run(ClientServerConnection.java:106)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: ignite.DefaultSource
        at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
        at org.apache.spark.sql.execution.datasources.DataSource$.$anonfun$lookupDataSource$5(DataSource.scala:656)
        at scala.util.Try$.apply(Try.scala:213)
        at org.apache.spark.sql.execution.datasources.DataSource$.$anonfun$lookupDataSource$4(DataSource.scala:656)
        at scala.util.Failure.orElse(Try.scala:224)
        at org.apache.spark.sql.execution.datasources.DataSource$.lookupDataSource(DataSource.scala:656)
        ... 16 more
</code>```

These are my steps:

I executed a Python script. python3 test_ignite.py

Created a SparkSession and connected to hive.

Tried to get a Pyspark dataframe from hive.

Saved it to the ignite cache. (error occurred)

My test_ignite.py code is:

```<code>import os
import findspark
findspark.init()
from pyspark.sql import SparkSession
from pyignite import Client

spark = SparkSession \
.builder \
.appName("consumer_hive") \
.config("hive.metastore.uris", "thrift://localhost:9083") \
.config("spark.driver.extraClassPath", "/usr/local/ignite/libs/*jar:/usr/local/ignite/libs/optional/ignite-spark/*jar:/usr/local/ignite/libs/ignite-spring/*jar:/usr/local/ignite/libs/ignite-indexing/*jar") \
.enableHiveSupport() \
.getOrCreate()

df = spark.sql("select * from hive_table")

configFile = os.environ['IGNITE_HOME'] + "config/default-config.xml"

df.write.format("ignite") \
            .option("table", "TickData") \
            .option("config", configFile) \
            .save()
</code>```

I have found the same issue from here, but the error still exists after adding the spark.driver.extraClassPath.
Should I set any config or setting because I never modify /ignite/config/default-config.xml

```<code><?xml version="1.0" encoding="UTF-8"?>

<!--
  Licensed to the Apache Software Foundation (ASF) under one or more
  contributor license agreements.  See the NOTICE file distributed with
  this work for additional information regarding copyright ownership.
  The ASF licenses this file to You under the Apache License, Version 2.0
  (the "License"); you may not use this file except in compliance with
  the License.  You may obtain a copy of the License at

       http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License.
-->

<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="
       http://www.springframework.org/schema/beans
       http://www.springframework.org/schema/beans/spring-beans.xsd">
    <!--
        Alter configuration below as needed.
    -->
    <bean id="grid.cfg" class="org.apache.ignite.configuration.IgniteConfiguration"/>
</beans>
</code>```

My Spark config:

spark-defaults.conf

```<code>spark.master spark://pc1-vm1:7077
spark.eventLog.enabled true
spark.eventLog.dir hdfs://pc1-vm1:9000/user/hdoop/log
spark.serializer org.apache.spark.serializer.KryoSerializer
spark.driver.memory 4g
spark.executor.memory 6g
spark.executor.cores 2
spark.dynamicAllocation.maxExecutors 2
spark.ui.enabled true

spark.driver.extraClassPath /usr/local/ignite/libs/*jar:/usr/local/ignite/libs/optional/ignite-spark/*jar:/usr/local/ignite/libs/ignite-spring/*jar:/usr/local/ignite/libs/ignite-indexing/*jar
</code>```

spark-env.sh

```<code>export PYSPARK_PYTHON=python3
export SCALA_HOME=/usr/lib/scala
export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
export HADOOP_HOME=/usr/local/hadoop
export HADOOP_CONF_DIR=/usr/local/hadoop/etc/hadoop
export SPARK_MASTER_HOST=pc1-vm1
export SPARK_WORKER_MEMORY=32G
export SPARK_DAEMON_MEMORY=4G
export SPARK_DIST_CLASSPATH=$(/usr/local/hadoop/bin/hadoop classpath)
export HADOOP_CLASSPATH='hadoop classpath'
export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:$HBASE_CLASSPATH
export HIVE_HOME=/home/hadoop/hive
export SPARK_CLASSPATH=$HIVE_HOME/lib/mysql-connector-java-8.0.22.jar:$SPARK_CLASSPATH
</code>```

These are the version I used:

```<code>- Hadoop v2.10.1
- Spark v2.3.4
- Hive v2.1.1
- Python v3.7.5
- openjdk-8-jre
- Ignite v2.7
</code>```

Could anybody help me, please?

答:
You need ignite JARs on the executor classpath, so spark.driver.extraClassPath should probably be spark.executor.extraClassPath, but you should rather use spark-submit --packages flag to add spark-ignite library

Alternatively, the docs mention that you can modify spark-env.sh to add Ignite libraries to SPARK_CLASSPATH (which you've not done)

https://ignite.apache.org/docs/latest/extensions-and-integrations/ignite-for-spark/installation

Body:
I would like to save my pyspark dataframe into ignite cache following the approach from this blog, but some error happened.
Error message is:

Traceback (most recent call last):
  File "test_ignite.py", line 19, in <module>
    .option("config", configFile) \
  File "/home/hdoop/.local/lib/python3.6/site-packages/pyspark/sql/readwriter.py", line 738, in save
    self._jwrite.save()
  File "/home/hdoop/.local/lib/python3.6/site-packages/py4j/java_gateway.py", line 1322, in __call__
    answer, self.gateway_client, self.target_id, self.name)
  File "/home/hdoop/.local/lib/python3.6/site-packages/pyspark/sql/utils.py", line 111, in deco
    return f(*a, **kw)
  File "/home/hdoop/.local/lib/python3.6/site-packages/py4j/protocol.py", line 328, in get_return_value
    format(target_id, ".", name), value)
py4j.protocol.Py4JJavaError: An error occurred while calling o42.save.
: java.lang.ClassNotFoundException:
Failed to find data source: ignite. Please find packages at
http://spark.apache.org/third-party-projects.html

        at org.apache.spark.sql.errors.QueryExecutionErrors$.failedToFindDataSourceError(QueryExecutionErrors.scala:443)
        at org.apache.spark.sql.execution.datasources.DataSource$.lookupDataSource(DataSource.scala:670)
        at org.apache.spark.sql.execution.datasources.DataSource$.lookupDataSourceV2(DataSource.scala:720)
        at org.apache.spark.sql.DataFrameWriter.lookupV2Provider(DataFrameWriter.scala:852)
        at org.apache.spark.sql.DataFrameWriter.saveInternal(DataFrameWriter.scala:256)
        at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:247)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
        at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
        at py4j.Gateway.invoke(Gateway.java:282)
        at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
        at py4j.commands.CallCommand.execute(CallCommand.java:79)
        at py4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:182)
        at py4j.ClientServerConnection.run(ClientServerConnection.java:106)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: ignite.DefaultSource
        at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
        at org.apache.spark.sql.execution.datasources.DataSource$.$anonfun$lookupDataSource$5(DataSource.scala:656)
        at scala.util.Try$.apply(Try.scala:213)
        at org.apache.spark.sql.execution.datasources.DataSource$.$anonfun$lookupDataSource$4(DataSource.scala:656)
        at scala.util.Failure.orElse(Try.scala:224)
        at org.apache.spark.sql.execution.datasources.DataSource$.lookupDataSource(DataSource.scala:656)
        ... 16 more

These are my steps:

I executed a Python script. python3 test_ignite.py

Created a SparkSession and connected to hive.

Tried to get a Pyspark dataframe from hive.

Saved it to the ignite cache. (error occurred)

My test_ignite.py code is:

import os
import findspark
findspark.init()
from pyspark.sql import SparkSession
from pyignite import Client

spark = SparkSession \
.builder \
.appName("consumer_hive") \
.config("hive.metastore.uris", "thrift://localhost:9083") \
.config("spark.driver.extraClassPath", "/usr/local/ignite/libs/*jar:/usr/local/ignite/libs/optional/ignite-spark/*jar:/usr/local/ignite/libs/ignite-spring/*jar:/usr/local/ignite/libs/ignite-indexing/*jar") \
.enableHiveSupport() \
.getOrCreate()

df = spark.sql("select * from hive_table")

configFile = os.environ['IGNITE_HOME'] + "config/default-config.xml"

df.write.format("ignite") \
            .option("table", "TickData") \
            .option("config", configFile) \
            .save()

I have found the same issue from here, but the error still exists after adding the spark.driver.extraClassPath.
Should I set any config or setting because I never modify /ignite/config/default-config.xml

<?xml version="1.0" encoding="UTF-8"?>

<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="
       http://www.springframework.org/schema/beans
       http://www.springframework.org/schema/beans/spring-beans.xsd">

    <bean id="grid.cfg" class="org.apache.ignite.configuration.IgniteConfiguration"/>
</beans>

My Spark config:

spark-defaults.conf

spark.master spark://pc1-vm1:7077
spark.eventLog.enabled true
spark.eventLog.dir hdfs://pc1-vm1:9000/user/hdoop/log
spark.serializer org.apache.spark.serializer.KryoSerializer
spark.driver.memory 4g
spark.executor.memory 6g
spark.executor.cores 2
spark.dynamicAllocation.maxExecutors 2
spark.ui.enabled true

spark.driver.extraClassPath /usr/local/ignite/libs/*jar:/usr/local/ignite/libs/optional/ignite-spark/*jar:/usr/local/ignite/libs/ignite-spring/*jar:/usr/local/ignite/libs/ignite-indexing/*jar

spark-env.sh

export PYSPARK_PYTHON=python3
export SCALA_HOME=/usr/lib/scala
export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
export HADOOP_HOME=/usr/local/hadoop
export HADOOP_CONF_DIR=/usr/local/hadoop/etc/hadoop
export SPARK_MASTER_HOST=pc1-vm1
export SPARK_WORKER_MEMORY=32G
export SPARK_DAEMON_MEMORY=4G
export SPARK_DIST_CLASSPATH=$(/usr/local/hadoop/bin/hadoop classpath)
export HADOOP_CLASSPATH='hadoop classpath'
export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:$HBASE_CLASSPATH
export HIVE_HOME=/home/hadoop/hive
export SPARK_CLASSPATH=$HIVE_HOME/lib/mysql-connector-java-8.0.22.jar:$SPARK_CLASSPATH

These are the version I used:

- Hadoop v2.10.1
- Spark v2.3.4
- Hive v2.1.1
- Python v3.7.5
- openjdk-8-jre
- Ignite v2.7

Could anybody help me, please?


问:
I need to get dynamic dependent variable value that increases when the other variable increases in PHP
Like this when x is 1000 and y becomes 0.1 and when x is 2000 and y becomes 0.2 like that in php.
Please help me if you are a good at it.you can see in here as an example </p>

答:
if there is a certain ratio between x and y like in your example y=(x/10000). Why you are not just doing simple math in php. $y = $x/10000 where $x is set by you and $y is set by this formulation. If the $y is already set, you can just, re-set it with the same formula which overwrites previous set value. If I understand you properly.

EDIT UPON COMMENT BELOW:
Try this simple function below. Check the demo

```<code>function so_set_y($x){
    return floor($x/1000)/10;
}
</code>```

DEMO

Body:
if there is a certain ratio between x and y like in your example y=(x/10000). Why you are not just doing simple math in php. $y = $x/10000 where $x is set by you and $y is set by this formulation. If the $y is already set, you can just, re-set it with the same formula which overwrites previous set value. If I understand you properly.

EDIT UPON COMMENT BELOW:
Try this simple function below. Check the demo

function so_set_y($x){
    return floor($x/1000)/10;
}

DEMO


问:
How can I write a where clause in SQL to filter a DATETIME column by the time of day?
I have data that is timestamped with a DATETIME column and I want to filter it down to the set of records where the DATETIME is between 9:30am and 5:30pm for any day.  What's the best way to do this?

UPDATE: Changed because I need precision to the minute, not just hour.  Sorry about that.

答:
```<code>Select * from tbl
where ts between
convert(datetime,convert(varchar,ts, 101)) + convert(datetime,'9:30 am')
And Convert(datetime,convert(varchar,ts,101)) + convert(datetime,'5:30 pm')
</code>```

This i believe will use index. I cannot test, i post this with ipod

[EDIT from my computer]

```<code>create function  dbo.DaysTime(@date as datetime, @time datetime)
returns datetime
as
begin

   /* datediff approach on scraping the
   time part of datetime: inspired by Russ Cam */

   return dateadd(d, datediff(d, 0, @d), 0)/*the date's day*/ + @time

end

select * from tbl where ts
between dbo.DaysTime(ts, '9:30 AM') and dbo.DaysTime(ts, '5:30 PM')
</code>```

This will still use an index as we didn't put an expression on the left side of the where's condition.

OwnerDisplayName: Bialecki

ContentLicense: CC BY-SA 2.5

LastEditorDisplayName: Milen A. Radev

Body:
Select * from tbl
where ts between
convert(datetime,convert(varchar,ts, 101)) + convert(datetime,'9:30 am')
And Convert(datetime,convert(varchar,ts,101)) + convert(datetime,'5:30 pm')

This i believe will use index. I cannot test, i post this with ipod

[EDIT from my computer]

create function  dbo.DaysTime(@date as datetime, @time datetime)
returns datetime
as
begin

   /* datediff approach on scraping the
   time part of datetime: inspired by Russ Cam */

   return dateadd(d, datediff(d, 0, @d), 0)/*the date's day*/ + @time

end

select * from tbl where ts
between dbo.DaysTime(ts, '9:30 AM') and dbo.DaysTime(ts, '5:30 PM')

This will still use an index as we didn't put an expression on the left side of the where's condition.

OwnerDisplayName: Michael Buen

LastEditorDisplayName: Michael Buen