问:
Given a groupId and authenticated user, how do I check the user's membership and role (owner vs member) in one network call?
As far as I know, Microsoft Graph lets me query for /groups/{groupid}/members or /groups/{groupid}//owners separately, but not both in one call. Is this possible?

In my context, I know the groupId and the authenticated calling userId. How do I check whether this user belongs to the group and if so, their owner/member role in one network call? Is this possible?

答:
One option to assume would be to expand and include members and owners properties via $expand query option but unfortunately the following query is not supported:

```<code>https://graph.microsoft.com/v1.0/groups/{group-id}?$select=owners,members&$expand=owners,members
</code>```

since max only 1 object is allowed to be expanded per Group resource

JSON batching comes to the rescue here, the following example demonstrates how to retrieve Group members and owners within a single request:

```<code>POST https://graph.microsoft.com/v1.0/$batch
Accept: application/json
Content-Type: application/json

{
  "requests": [
    {
      "id": "1",
      "method": "GET",
      "url": "/groups/{group-id}/members"
    },
    {
      "id": "2",
      "method": "GET",
      "url": "/groups/{group-id}/owners"
    }
  ]
}
</code>```

Body:
One option to assume would be to expand and include members and owners properties via $expand query option but unfortunately the following query is not supported:

https://graph.microsoft.com/v1.0/groups/{group-id}?$select=owners,members&$expand=owners,members

since max only 1 object is allowed to be expanded per Group resource

JSON batching comes to the rescue here, the following example demonstrates how to retrieve Group members and owners within a single request:

POST https://graph.microsoft.com/v1.0/$batch
Accept: application/json
Content-Type: application/json

{
  "requests": [
    {
      "id": "1",
      "method": "GET",
      "url": "/groups/{group-id}/members"
    },
    {
      "id": "2",
      "method": "GET",
      "url": "/groups/{group-id}/owners"
    }
  ]
}


来源: stackexchange

问:
problem in using camera in android
following is the code by which i am capturing images in my app. After capturing the image it saves it in SDcard and shows done and retake when i click the done button i want the captured image to be uploaded to an url.

i am not able to know from where the done and retake button gets appeared pls help me...

```<code>protected static final int TAKE_RECEIPT = 0;
Button b1;
    Intent myIntent;

    @Override
    public void onCreate(Bundle savedInstanceState)
    {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.main);

        b1 = (Button)findViewById(R.id.widget98);

        b1.setOnClickListener(new OnClickListener()
        {
            @Override
            public void onClick(View view)
            {
                takePictureFromCamera();
            }

            private void takePictureFromCamera()
            {
                 Intent intent = new Intent(MediaStore.ACTION_IMAGE_CAPTURE);
 startActivityForResult(intent, TAKE_RECEIPT);
            }
</code>```

答: The "OK"/"Retake" buttons are shown on the so called "post-capture alert" UI which is part of the Camera application (check /packages/apps/Camera/src/com/android/camera/Camera.java if you have the Android source code downloaded), and I'm afraid there is no way to get rid of them.</p>

ContentLicense: CC BY-SA 3.0

Title: problem in using camera in android

Body:
following is the code by which i am capturing images in my app. After capturing the image it saves it in SDcard and shows done and retake when i click the done button i want the captured image to be uploaded to an url.

i am not able to know from where the done and retake button gets appeared pls help me...

protected static final int TAKE_RECEIPT = 0;
Button b1;
    Intent myIntent;

    @Override
    public void onCreate(Bundle savedInstanceState)
    {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.main);

        b1 = (Button)findViewById(R.id.widget98);

        b1.setOnClickListener(new OnClickListener()
        {
            @Override
            public void onClick(View view)
            {
                takePictureFromCamera();
            }

            private void takePictureFromCamera()
            {
                 Intent intent = new Intent(MediaStore.ACTION_IMAGE_CAPTURE);
 startActivityForResult(intent, TAKE_RECEIPT);
            }


问:
Can't compile scala&java project. Fail to find transient dependencies
I have a huge feeling that I'm missing something.
I'm trying to compile Java&Scala project at my work. Luckliy I was able to repuduce my problem on a simple dummy project.
I'm using scala-maven-plugin(https://github.com/davidB/scala-maven-plugin). I'm also base my example on this question

Aiming to solve Scala&Java multi-module project. I have root and two modules m1 and m2.

m1 pom:

```<code><?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>samples.scala-maven-plugin</groupId>
        <artifactId>prj_multi_modules</artifactId>
        <version>0.0.1-SNAPSHOT</version>
        <relativePath>../pom.xml</relativePath>
    </parent>
    <artifactId>m1</artifactId>
    <packaging>jar</packaging>
    <dependencies>
        <dependency>
            <groupId>org.scala-lang</groupId>
            <artifactId>scala-library</artifactId>
        </dependency>
        <dependency>
            <groupId>samples.scala-maven-plugin</groupId>
            <artifactId>m2</artifactId>
            <version>0.0.1-SNAPSHOT</version>
        </dependency>
    </dependencies>
</project>
</code>```

m2 pom:

```<code><?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>samples.scala-maven-plugin</groupId>
        <artifactId>prj_multi_modules</artifactId>
        <version>0.0.1-SNAPSHOT</version>
        <relativePath>../pom.xml</relativePath>
    </parent>
    <artifactId>m2</artifactId>
    <packaging>jar</packaging>

    <properties>
        <cdh.version>cdh5.7.6</cdh.version>
        <build.prop.dir>..</build.prop.dir>
        <scala.version>2.11.8</scala.version>
        <scala.version.major>2.11</scala.version.major>
        <spark.version>2.3.0.cloudera4</spark.version>
        <hadoop.version>2.6.0-${cdh.version}</hadoop.version>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.scala-lang</groupId>
            <artifactId>scala-library</artifactId>
        </dependency>
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-core_${scala.version.major}</artifactId>
            <version>${spark.version}</version>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-sql_${scala.version.major}</artifactId>
            <version>${spark.version}</version>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-mllib_${scala.version.major}</artifactId>
            <version>${spark.version}</version>
            <scope>provided</scope>
        </dependency>
        <dependency> <!-- remove this for spark 2.3? -->
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-hive_${scala.version.major}</artifactId>
            <version>${spark.version}</version>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>com.github.scopt</groupId>
            <artifactId>scopt_${scala.version.major}</artifactId>
            <version>3.3.0</version>
        </dependency>
        <dependency>
            <groupId>com.databricks</groupId>
            <artifactId>spark-csv_${scala.version.major}</artifactId>
            <version>1.5.0</version>
        </dependency>
        <dependency>
            <groupId>org.apache.commons</groupId>
            <artifactId>commons-csv</artifactId>
            <version>1.1</version>
        </dependency>
        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-common</artifactId>
            <version>${hadoop.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-client</artifactId>
            <version>${hadoop.version}</version>
        </dependency>
        <dependency>
            <groupId>com.holdenkarau</groupId>
            <artifactId>spark-testing-base_${scala.version.major}</artifactId>
            <version>1.6.0_0.7.4</version>
            <scope>test</scope>
        </dependency>
    </dependencies>
</project>
</code>```

root pom:

```<code><?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelVersion>

    <repositories>
        <repository>
            <id>cloudera</id>
            <url>https://repository.cloudera.com/artifactory/cloudera-repos/</url>
        </repository>
    </repositories>

  <groupId>samples.scala-maven-plugin</groupId>
  <artifactId>prj_multi_modules</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  <packaging>pom</packaging>

  <properties>
    <cdh.version>cdh5.7.6</cdh.version>
    <build.prop.dir>..</build.prop.dir>
    <scala.version>2.11.8</scala.version>
    <scala.version.major>2.11</scala.version.major>
    <spark.version>2.3.0.cloudera4</spark.version>
    <hadoop.version>2.6.0-${cdh.version}</hadoop.version>
  </properties>

  <dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>org.scala-lang</groupId>
      <artifactId>scala-library</artifactId>
        <version>${scala.version}</version>
    </dependency>
  </dependencies>
  </dependencyManagement>

  <build>
    <pluginManagement>
      <plugins>
        <plugin>
          <groupId>net.alchim31.maven</groupId>
          <artifactId>scala-maven-plugin</artifactId>
          <version>3.2.2</version>
          <executions>
            <execution>
              <goals>
                <goal>compile</goal>
                <goal>testCompile</goal>
              </goals>
            </execution>
          </executions>
        </plugin>
      </plugins>
  </pluginManagement>
  <plugins>
    <plugin>
      <groupId>net.alchim31.maven</groupId>
      <artifactId>scala-maven-plugin</artifactId>
    </plugin>
  </plugins>
</build>

  <modules>
      <module>m1</module>
      <module>m2</module>
  </modules>
</project>
</code>```

My goal is very simple I want m1 project to get all spark dpedencies from m2 module in order to compile a spark code in m1 module.

This is the spark code in m1:

```<code>package p1

import org.apache.spark.sql.SparkSession

class MyClass(spark: SparkSession) {
  import spark.implicits._

  def run(input: String) = {
    val df = spark.read.parquet(input)

    df.show()
  }

}

object MyClass {

  case class Params(input: String = "")

  def main(args: Array[String]): Unit = {

    val parser = new scopt.OptionParser[Params](usageHeader) {
      opt[String]('i', "input") required() action { (s, params) => params.copy(input = s) }
    }

    parser.parse(args, Params()) match {
      case Some(p) =>
        val spark = SparkSession.builder
          .appName("spark2 example")
          .getOrCreate()

        val job = new Spark2Exp(spark)
        job.run(p.input)

        spark.stop()
      case None =>
        System.exit(1)
    }
  }
  val usageHeader: String = "n/a"
}
</code>```

m2 compilation is working smoothly. While compiling m1 all java compilation process is good but when trying to compile scala code it fails not finding spark dependencies as if it doesn't transient between the dependent modules.

In addition if I copy all m2 spark dependencies to m1 pom everything is working.

I can't seem to understand what I'm doing wrong. I am sure it's scala - maven issue since this scenario is pretty straight forward in java.

Adding compilation error:

```<code>"C:\Program Files\Java\jdk1.8.0_151\bin\java" -Dmaven.multiModuleProjectDirectory=C:\dev\scala-maven-plugin "-Dmaven.home=C:\Program Files\JetBrains\IntelliJ IDEA 2017.3\plugins\maven\lib\maven3" "-Dclassworlds.conf=C:\Program Files\JetBrains\IntelliJ IDEA 2017.3\plugins\maven\lib\maven3\bin\m2.conf" "-javaagent:C:\Program Files\JetBrains\IntelliJ IDEA 2017.3\lib\idea_rt.jar=55662:C:\Program Files\JetBrains\IntelliJ IDEA 2017.3\bin" -Dfile.encoding=UTF-8 -classpath "C:\Program Files\JetBrains\IntelliJ IDEA 2017.3\plugins\maven\lib\maven3\boot\plexus-classworlds-2.5.2.jar" org.codehaus.classworlds.Launcher -Didea.version=2017.3 clean install
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] prj_multi_modules
[INFO] m2
[INFO] m1
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building prj_multi_modules 0.0.1-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ prj_multi_modules ---
[INFO]
[INFO] --- scala-maven-plugin:3.2.2:compile (default) @ prj_multi_modules ---
[INFO] No sources to compile
[INFO]
[INFO] --- scala-maven-plugin:3.2.2:testCompile (default) @ prj_multi_modules ---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-install-plugin:2.4:install (default-install) @ prj_multi_modules ---
[INFO] Installing C:\dev\scala-maven-plugin\samples\prj_multi_modules\pom.xml to C:\Users\raphael.peretz\.m2\repository\samples\scala-maven-plugin\prj_multi_modules\0.0.1-SNAPSHOT\prj_multi_modules-0.0.1-SNAPSHOT.pom
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building m2 0.0.1-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ m2 ---
[INFO] Deleting C:\dev\scala-maven-plugin\samples\prj_multi_modules\m2\target
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ m2 ---
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory C:\dev\scala-maven-plugin\samples\prj_multi_modules\m2\src\main\resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ m2 ---
[INFO] No sources to compile
[INFO]
[INFO] --- scala-maven-plugin:3.2.2:compile (default) @ m2 ---
[WARNING]  Expected all dependencies to require Scala version: 2.11.8
[WARNING]  samples.scala-maven-plugin:m2:0.0.1-SNAPSHOT requires scala version: 2.11.8
[WARNING]  com.twitter:chill_2.11:0.8.0 requires scala version: 2.11.8
[WARNING]  org.apache.spark:spark-core_2.11:2.3.0.cloudera4 requires scala version: 2.11.8
[WARNING]  org.json4s:json4s-jackson_2.11:3.2.11 requires scala version: 2.11.8
[WARNING]  org.json4s:json4s-core_2.11:3.2.11 requires scala version: 2.11.8
[WARNING]  org.json4s:json4s-ast_2.11:3.2.11 requires scala version: 2.11.8
[WARNING]  org.json4s:json4s-core_2.11:3.2.11 requires scala version: 2.11.0
[WARNING] Multiple versions of scala libraries detected!
[INFO] No sources to compile
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ m2 ---
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory C:\dev\scala-maven-plugin\samples\prj_multi_modules\m2\src\test\resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ m2 ---
[INFO] No sources to compile
[INFO]
[INFO] --- scala-maven-plugin:3.2.2:testCompile (default) @ m2 ---
[WARNING]  Expected all dependencies to require Scala version: 2.11.8
[WARNING]  samples.scala-maven-plugin:m2:0.0.1-SNAPSHOT requires scala version: 2.11.8
[WARNING]  com.twitter:chill_2.11:0.8.0 requires scala version: 2.11.8
[WARNING]  org.apache.spark:spark-core_2.11:2.3.0.cloudera4 requires scala version: 2.11.8
[WARNING]  org.json4s:json4s-jackson_2.11:3.2.11 requires scala version: 2.11.8
[WARNING]  org.json4s:json4s-core_2.11:3.2.11 requires scala version: 2.11.8
[WARNING]  org.json4s:json4s-ast_2.11:3.2.11 requires scala version: 2.11.8
[WARNING]  org.json4s:json4s-core_2.11:3.2.11 requires scala version: 2.11.0
[WARNING] Multiple versions of scala libraries detected!
[INFO] No sources to compile
[INFO]
[INFO] --- maven-surefire-plugin:2.12.4:test (default-test) @ m2 ---
[INFO] No tests to run.
[INFO]
[INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ m2 ---
[WARNING] JAR will be empty - no content was marked for inclusion!
[INFO] Building jar: C:\dev\scala-maven-plugin\samples\prj_multi_modules\m2\target\m2-0.0.1-SNAPSHOT.jar
[INFO]
[INFO] --- maven-install-plugin:2.4:install (default-install) @ m2 ---
[INFO] Installing C:\dev\scala-maven-plugin\samples\prj_multi_modules\m2\target\m2-0.0.1-SNAPSHOT.jar to C:\Users\raphael.peretz\.m2\repository\samples\scala-maven-plugin\m2\0.0.1-SNAPSHOT\m2-0.0.1-SNAPSHOT.jar
[INFO] Installing C:\dev\scala-maven-plugin\samples\prj_multi_modules\m2\pom.xml to C:\Users\raphael.peretz\.m2\repository\samples\scala-maven-plugin\m2\0.0.1-SNAPSHOT\m2-0.0.1-SNAPSHOT.pom
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building m1 0.0.1-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ m1 ---
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ m1 ---
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory C:\dev\scala-maven-plugin\samples\prj_multi_modules\m1\src\main\resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ m1 ---
[INFO] No sources to compile
[INFO]
[INFO] --- scala-maven-plugin:3.2.2:compile (default) @ m1 ---
[INFO] C:\dev\scala-maven-plugin\samples\prj_multi_modules\m1\src\main\scala:-1: info: compiling
[INFO] Compiling 1 source files to C:\dev\scala-maven-plugin\samples\prj_multi_modules\m1\target\classes at 1558957820236
[ERROR] C:\dev\scala-maven-plugin\samples\prj_multi_modules\m1\src\main\scala\p1\MyClass.scala:3: error: object spark is not a member of package org.apache
[ERROR] import org.apache.spark.sql.SparkSession
[ERROR]                   ^
[ERROR] C:\dev\scala-maven-plugin\samples\prj_multi_modules\m1\src\main\scala\p1\MyClass.scala:5: error: not found: type SparkSession
[ERROR] class MyClass(spark: SparkSession) {
[ERROR]                      ^
[ERROR] C:\dev\scala-maven-plugin\samples\prj_multi_modules\m1\src\main\scala\p1\MyClass.scala:29: error: not found: value SparkSession
[ERROR]         val spark = SparkSession.builder
[ERROR]                     ^
[ERROR] C:\dev\scala-maven-plugin\samples\prj_multi_modules\m1\src\main\scala\p1\MyClass.scala:33: error: not found: type Spark2Exp
[ERROR]         val job = new Spark2Exp(spark)
[ERROR]                       ^
[ERROR] four errors found
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] prj_multi_modules .................................. SUCCESS [  1.435 s]
[INFO] m2 ................................................. SUCCESS [ 22.342 s]
[INFO] m1 ................................................. FAILURE [  3.497 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 27.379 s
[INFO] Finished at: 2019-05-27T14:50:23+03:00
[INFO] Final Memory: 52M/927M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile (default) on project m1: wrap: org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :m1

Process finished with exit code 1
</code>```

答:
Since you need spark in both m1 and m2, why not delcare the spark dependency in root pom file?

Of course, you can remove the duplicate one in m2 afterwards.

ContentLicense: CC BY-SA 4.0

Body:
I have a huge feeling that I'm missing something.
I'm trying to compile Java&Scala project at my work. Luckliy I was able to repuduce my problem on a simple dummy project.
I'm using scala-maven-plugin(https://github.com/davidB/scala-maven-plugin). I'm also base my example on this question

Aiming to solve Scala&Java multi-module project. I have root and two modules m1 and m2.

m1 pom:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>samples.scala-maven-plugin</groupId>
        <artifactId>prj_multi_modules</artifactId>
        <version>0.0.1-SNAPSHOT</version>
        <relativePath>../pom.xml</relativePath>
    </parent>
    <artifactId>m1</artifactId>
    <packaging>jar</packaging>
    <dependencies>
        <dependency>
            <groupId>org.scala-lang</groupId>
            <artifactId>scala-library</artifactId>
        </dependency>
        <dependency>
            <groupId>samples.scala-maven-plugin</groupId>
            <artifactId>m2</artifactId>
            <version>0.0.1-SNAPSHOT</version>
        </dependency>
    </dependencies>
</project>

m2 pom:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>samples.scala-maven-plugin</groupId>
        <artifactId>prj_multi_modules</artifactId>
        <version>0.0.1-SNAPSHOT</version>
        <relativePath>../pom.xml</relativePath>
    </parent>
    <artifactId>m2</artifactId>
    <packaging>jar</packaging>

    <properties>
        <cdh.version>cdh5.7.6</cdh.version>
        <build.prop.dir>..</build.prop.dir>
        <scala.version>2.11.8</scala.version>
        <scala.version.major>2.11</scala.version.major>
        <spark.version>2.3.0.cloudera4</spark.version>
        <hadoop.version>2.6.0-${cdh.version}</hadoop.version>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.scala-lang</groupId>
            <artifactId>scala-library</artifactId>
        </dependency>
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-core_${scala.version.major}</artifactId>
            <version>${spark.version}</version>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-sql_${scala.version.major}</artifactId>
            <version>${spark.version}</version>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-mllib_${scala.version.major}</artifactId>
            <version>${spark.version}</version>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-hive_${scala.version.major}</artifactId>
            <version>${spark.version}</version>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>com.github.scopt</groupId>
            <artifactId>scopt_${scala.version.major}</artifactId>
            <version>3.3.0</version>
        </dependency>
        <dependency>
            <groupId>com.databricks</groupId>
            <artifactId>spark-csv_${scala.version.major}</artifactId>
            <version>1.5.0</version>
        </dependency>
        <dependency>
            <groupId>org.apache.commons</groupId>
            <artifactId>commons-csv</artifactId>
            <version>1.1</version>
        </dependency>
        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-common</artifactId>
            <version>${hadoop.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-client</artifactId>
            <version>${hadoop.version}</version>
        </dependency>
        <dependency>
            <groupId>com.holdenkarau</groupId>
            <artifactId>spark-testing-base_${scala.version.major}</artifactId>
            <version>1.6.0_0.7.4</version>
            <scope>test</scope>
        </dependency>
    </dependencies>
</project>

root pom:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelVersion>

    <repositories>
        <repository>
            <id>cloudera</id>
            <url>https://repository.cloudera.com/artifactory/cloudera-repos/</url>
        </repository>
    </repositories>

  <groupId>samples.scala-maven-plugin</groupId>
  <artifactId>prj_multi_modules</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  <packaging>pom</packaging>

  <properties>
    <cdh.version>cdh5.7.6</cdh.version>
    <build.prop.dir>..</build.prop.dir>
    <scala.version>2.11.8</scala.version>
    <scala.version.major>2.11</scala.version.major>
    <spark.version>2.3.0.cloudera4</spark.version>
    <hadoop.version>2.6.0-${cdh.version}</hadoop.version>
  </properties>

  <dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>org.scala-lang</groupId>
      <artifactId>scala-library</artifactId>
        <version>${scala.version}</version>
    </dependency>
  </dependencies>
  </dependencyManagement>

  <build>
    <pluginManagement>
      <plugins>
        <plugin>
          <groupId>net.alchim31.maven</groupId>
          <artifactId>scala-maven-plugin</artifactId>
          <version>3.2.2</version>
          <executions>
            <execution>
              <goals>
                <goal>compile</goal>
                <goal>testCompile</goal>
              </goals>
            </execution>
          </executions>
        </plugin>
      </plugins>
  </pluginManagement>
  <plugins>
    <plugin>
      <groupId>net.alchim31.maven</groupId>
      <artifactId>scala-maven-plugin</artifactId>
    </plugin>
  </plugins>
</build>

  <modules>
      <module>m1</module>
      <module>m2</module>
  </modules>
</project>

My goal is very simple I want m1 project to get all spark dpedencies from m2 module in order to compile a spark code in m1 module.

This is the spark code in m1:

package p1

import org.apache.spark.sql.SparkSession

class MyClass(spark: SparkSession) {
  import spark.implicits._

  def run(input: String) = {
    val df = spark.read.parquet(input)

    df.show()
  }

}

object MyClass {

  case class Params(input: String = "")

  def main(args: Array[String]): Unit = {

    val parser = new scopt.OptionParser[Params](usageHeader) {
      opt[String]('i', "input") required() action { (s, params) => params.copy(input = s) }
    }

    parser.parse(args, Params()) match {
      case Some(p) =>
        val spark = SparkSession.builder
          .appName("spark2 example")
          .getOrCreate()

        val job = new Spark2Exp(spark)
        job.run(p.input)

        spark.stop()
      case None =>
        System.exit(1)
    }
  }
  val usageHeader: String = "n/a"
}

m2 compilation is working smoothly. While compiling m1 all java compilation process is good but when trying to compile scala code it fails not finding spark dependencies as if it doesn't transient between the dependent modules.

In addition if I copy all m2 spark dependencies to m1 pom everything is working.

I can't seem to understand what I'm doing wrong. I am sure it's scala - maven issue since this scenario is pretty straight forward in java.

Adding compilation error:

"C:\Program Files\Java\jdk1.8.0_151\bin\java" -Dmaven.multiModuleProjectDirectory=C:\dev\scala-maven-plugin "-Dmaven.home=C:\Program Files\JetBrains\IntelliJ IDEA 2017.3\plugins\maven\lib\maven3" "-Dclassworlds.conf=C:\Program Files\JetBrains\IntelliJ IDEA 2017.3\plugins\maven\lib\maven3\bin\m2.conf" "-javaagent:C:\Program Files\JetBrains\IntelliJ IDEA 2017.3\lib\idea_rt.jar=55662:C:\Program Files\JetBrains\IntelliJ IDEA 2017.3\bin" -Dfile.encoding=UTF-8 -classpath "C:\Program Files\JetBrains\IntelliJ IDEA 2017.3\plugins\maven\lib\maven3\boot\plexus-classworlds-2.5.2.jar" org.codehaus.classworlds.Launcher -Didea.version=2017.3 clean install
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] prj_multi_modules
[INFO] m2
[INFO] m1
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building prj_multi_modules 0.0.1-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ prj_multi_modules ---
[INFO]
[INFO] --- scala-maven-plugin:3.2.2:compile (default) @ prj_multi_modules ---
[INFO] No sources to compile
[INFO]
[INFO] --- scala-maven-plugin:3.2.2:testCompile (default) @ prj_multi_modules ---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-install-plugin:2.4:install (default-install) @ prj_multi_modules ---
[INFO] Installing C:\dev\scala-maven-plugin\samples\prj_multi_modules\pom.xml to C:\Users\raphael.peretz\.m2\repository\samples\scala-maven-plugin\prj_multi_modules\0.0.1-SNAPSHOT\prj_multi_modules-0.0.1-SNAPSHOT.pom
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building m2 0.0.1-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ m2 ---
[INFO] Deleting C:\dev\scala-maven-plugin\samples\prj_multi_modules\m2\target
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ m2 ---
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory C:\dev\scala-maven-plugin\samples\prj_multi_modules\m2\src\main\resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ m2 ---
[INFO] No sources to compile
[INFO]
[INFO] --- scala-maven-plugin:3.2.2:compile (default) @ m2 ---
[WARNING]  Expected all dependencies to require Scala version: 2.11.8
[WARNING]  samples.scala-maven-plugin:m2:0.0.1-SNAPSHOT requires scala version: 2.11.8
[WARNING]  com.twitter:chill_2.11:0.8.0 requires scala version: 2.11.8
[WARNING]  org.apache.spark:spark-core_2.11:2.3.0.cloudera4 requires scala version: 2.11.8
[WARNING]  org.json4s:json4s-jackson_2.11:3.2.11 requires scala version: 2.11.8
[WARNING]  org.json4s:json4s-core_2.11:3.2.11 requires scala version: 2.11.8
[WARNING]  org.json4s:json4s-ast_2.11:3.2.11 requires scala version: 2.11.8
[WARNING]  org.json4s:json4s-core_2.11:3.2.11 requires scala version: 2.11.0
[WARNING] Multiple versions of scala libraries detected!
[INFO] No sources to compile
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ m2 ---
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory C:\dev\scala-maven-plugin\samples\prj_multi_modules\m2\src\test\resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ m2 ---
[INFO] No sources to compile
[INFO]
[INFO] --- scala-maven-plugin:3.2.2:testCompile (default) @ m2 ---
[WARNING]  Expected all dependencies to require Scala version: 2.11.8
[WARNING]  samples.scala-maven-plugin:m2:0.0.1-SNAPSHOT requires scala version: 2.11.8
[WARNING]  com.twitter:chill_2.11:0.8.0 requires scala version: 2.11.8
[WARNING]  org.apache.spark:spark-core_2.11:2.3.0.cloudera4 requires scala version: 2.11.8
[WARNING]  org.json4s:json4s-jackson_2.11:3.2.11 requires scala version: 2.11.8
[WARNING]  org.json4s:json4s-core_2.11:3.2.11 requires scala version: 2.11.8
[WARNING]  org.json4s:json4s-ast_2.11:3.2.11 requires scala version: 2.11.8
[WARNING]  org.json4s:json4s-core_2.11:3.2.11 requires scala version: 2.11.0
[WARNING] Multiple versions of scala libraries detected!
[INFO] No sources to compile
[INFO]
[INFO] --- maven-surefire-plugin:2.12.4:test (default-test) @ m2 ---
[INFO] No tests to run.
[INFO]
[INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ m2 ---
[WARNING] JAR will be empty - no content was marked for inclusion!
[INFO] Building jar: C:\dev\scala-maven-plugin\samples\prj_multi_modules\m2\target\m2-0.0.1-SNAPSHOT.jar
[INFO]
[INFO] --- maven-install-plugin:2.4:install (default-install) @ m2 ---
[INFO] Installing C:\dev\scala-maven-plugin\samples\prj_multi_modules\m2\target\m2-0.0.1-SNAPSHOT.jar to C:\Users\raphael.peretz\.m2\repository\samples\scala-maven-plugin\m2\0.0.1-SNAPSHOT\m2-0.0.1-SNAPSHOT.jar
[INFO] Installing C:\dev\scala-maven-plugin\samples\prj_multi_modules\m2\pom.xml to C:\Users\raphael.peretz\.m2\repository\samples\scala-maven-plugin\m2\0.0.1-SNAPSHOT\m2-0.0.1-SNAPSHOT.pom
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building m1 0.0.1-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ m1 ---
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ m1 ---
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory C:\dev\scala-maven-plugin\samples\prj_multi_modules\m1\src\main\resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ m1 ---
[INFO] No sources to compile
[INFO]
[INFO] --- scala-maven-plugin:3.2.2:compile (default) @ m1 ---
[INFO] C:\dev\scala-maven-plugin\samples\prj_multi_modules\m1\src\main\scala:-1: info: compiling
[INFO] Compiling 1 source files to C:\dev\scala-maven-plugin\samples\prj_multi_modules\m1\target\classes at 1558957820236
[ERROR] C:\dev\scala-maven-plugin\samples\prj_multi_modules\m1\src\main\scala\p1\MyClass.scala:3: error: object spark is not a member of package org.apache
[ERROR] import org.apache.spark.sql.SparkSession
[ERROR]                   ^
[ERROR] C:\dev\scala-maven-plugin\samples\prj_multi_modules\m1\src\main\scala\p1\MyClass.scala:5: error: not found: type SparkSession
[ERROR] class MyClass(spark: SparkSession) {
[ERROR]                      ^
[ERROR] C:\dev\scala-maven-plugin\samples\prj_multi_modules\m1\src\main\scala\p1\MyClass.scala:29: error: not found: value SparkSession
[ERROR]         val spark = SparkSession.builder
[ERROR]                     ^
[ERROR] C:\dev\scala-maven-plugin\samples\prj_multi_modules\m1\src\main\scala\p1\MyClass.scala:33: error: not found: type Spark2Exp
[ERROR]         val job = new Spark2Exp(spark)
[ERROR]                       ^
[ERROR] four errors found
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] prj_multi_modules .................................. SUCCESS [  1.435 s]
[INFO] m2 ................................................. SUCCESS [ 22.342 s]
[INFO] m1 ................................................. FAILURE [  3.497 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 27.379 s
[INFO] Finished at: 2019-05-27T14:50:23+03:00
[INFO] Final Memory: 52M/927M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile (default) on project m1: wrap: org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :m1

Process finished with exit code 1


问:
Is it possible to use a SPI flash chip with a arduino (Atmega328p)
Is it possible to use an SPI flash chip with a minimum sector erase size of 4KB with an Atmega 328p (2KB of internal RAM)?

The problem I'm seeing: in order for data to be written to the flash, a page must be erased. If you want to update data inside a sector you first need to read all the pages inside the sector, save them off the chip, erase the sector and write the pages in order back with your changes.

But a sector of 4KB can't be saved inside a 2KB RAM, so what solutions do other people use for this situation? Because I have seen Arduinos be used in this setup but couldn't determine what their solution was.

(Arduino is just an example of a small microcontroller in this question)

答:
You can simply use a dual-redundant page scheme.  Given two 4kb pages organised thus:

```<code> Page A           Page B
+------------+   +------------+
| Sequence A |   | Sequence B |
!~Sequence A |   !~Sequence B |
+------------+   +------------+
|            |   |            |
|            |   |            |
| Data A     |   |  Data B    |
|            |   |            |
|            |   |            |
|            |   |            |
|            |   |            |
|            |   |            |
|            |   |            |
|            |   |            |
+------------+   +------------+
</code>```

Initially you write Data A and set Sequence A to zero its inverse ~Sequence A.  On start-up you inspect each page's sequence number and set the page with the highest valid sequence number (valid when sequence ^ ~sequnece == 0xff)  as the current read page, and the other as the current write page.

When you update the data, you erase the current write page, write the data area, then set the sequence number to the sequence number of the read page plus 1 (modulo 256), and set its inverse.  Then swap the current read/write pages.  When updating you can copy the data from one page to the other without buffering the entire page, and modify just the part you intend to change.

If there is a power failure or reset during a write, the page sequence number will not be valid as it is written last and both the sequence and inverse sequence write must complete, so the partially written data will be invalidated and the previous valid page will be selected on start-up.

When selecting the current active page, you of course have to deal with the wrap-around (i.e. when the sequence numbers are 255 and zero, zero is the newer).

This scheme is suitable for data that will be read often and written seldom.  The nature and frequency of your NV read/write cycles will dictate a suitable mechanism since different solutions would be appropriate for sequential logging or non-volatile event counters.

Body:
You can simply use a dual-redundant page scheme.  Given two 4kb pages organised thus:

 Page A           Page B
+------------+   +------------+
| Sequence A |   | Sequence B |
!~Sequence A |   !~Sequence B |
+------------+   +------------+
|            |   |            |
|            |   |            |
| Data A     |   |  Data B    |
|            |   |            |
|            |   |            |
|            |   |            |
|            |   |            |
|            |   |            |
|            |   |            |
|            |   |            |
+------------+   +------------+

Initially you write Data A and set Sequence A to zero its inverse ~Sequence A.  On start-up you inspect each page's sequence number and set the page with the highest valid sequence number (valid when sequence ^ ~sequnece == 0xff)  as the current read page, and the other as the current write page.

When you update the data, you erase the current write page, write the data area, then set the sequence number to the sequence number of the read page plus 1 (modulo 256), and set its inverse.  Then swap the current read/write pages.  When updating you can copy the data from one page to the other without buffering the entire page, and modify just the part you intend to change.

If there is a power failure or reset during a write, the page sequence number will not be valid as it is written last and both the sequence and inverse sequence write must complete, so the partially written data will be invalidated and the previous valid page will be selected on start-up.

When selecting the current active page, you of course have to deal with the wrap-around (i.e. when the sequence numbers are 255 and zero, zero is the newer).

This scheme is suitable for data that will be read often and written seldom.  The nature and frequency of your NV read/write cycles will dictate a suitable mechanism since different solutions would be appropriate for sequential logging or non-volatile event counters.


问:
How to extract x-atmosphere-tracking-id from a string?
I am trying to extract X-Atmosphere-tracking-id from the response body.
I got the response in the following way:

```<code>40|41be6f15-2e19-4e1d-aaff-d4b92ae47e48|0||
</code>```

Now I want to extract only:

```<code>41be6f15-2e19-4e1d-aaff-d4b92ae47e48
</code>```

which is dynamically generated on every execution.

Also, It would be great if someone shares tips and tricks as well as reference videos/sites of load testing using JMeter for Vaading application.

答:
I suggest using simple awk script, parsing fields by | and extracting the 2nd field.

```<code>echo "40|41be6f15-2e19-4e1d-aaff-d4b92ae47e48|0||" | awk -F "|" '{print $2}'
</code>```

Output

```<code>41be6f15-2e19-4e1d-aaff-d4b92ae47e48
</code>```

Body:
I am trying to extract X-Atmosphere-tracking-id from the response body.
I got the response in the following way:

40|41be6f15-2e19-4e1d-aaff-d4b92ae47e48|0||

Now I want to extract only:

41be6f15-2e19-4e1d-aaff-d4b92ae47e48

which is dynamically generated on every execution.

Also, It would be great if someone shares tips and tricks as well as reference videos/sites of load testing using JMeter for Vaading application.

Body:
I suggest using simple awk script, parsing fields by | and extracting the 2nd field.

echo "40|41be6f15-2e19-4e1d-aaff-d4b92ae47e48|0||" | awk -F "|" '{print $2}'

Output

41be6f15-2e19-4e1d-aaff-d4b92ae47e48


问:
How to sum the data and transfer it to Firebase?
I have a field in Firebase: balance. What I need is that after a person enters a sum 100, the field changes the value to 100. Then when the person enters a sum of 50, the field value becomes 150.

How do I write processing logic on the client? Summation of data. I think it is over-easy, but I need your help!

```<code>    mDatabaseUsers.addValueEventListener(new ValueEventListener() {
        @Override
        public void onDataChange(DataSnapshot dataSnapshot) {
            String user_id = mAuth.getCurrentUser().getUid();

            String balance = dataSnapshot.child(user_id).child("Balance").getValue(String.class);

            mCountPayment.setText(balance + " ₽");
        }

        @Override
        public void onCancelled(DatabaseError databaseError) {

        }
    });

    mPaymentButton.setOnClickListener(new View.OnClickListener() {
        @Override
        public void onClick(View v) {
            paymentIntent();
        }
    });
}

private void paymentIntent() {

    final String user_id = mAuth.getCurrentUser().getUid();

    final String count = mPaymentList.getText().toString().trim();

    if (!TextUtils.isEmpty(count)) {

        mDatabaseUsers.addValueEventListener(new ValueEventListener() {
            @Override
            public void onDataChange(DataSnapshot dataSnapshot) {
                mDatabaseUsers.child(user_id).child("Balance").setValue(count);
            }

            @Override
            public void onCancelled(DatabaseError databaseError) {

            }
        });
            }
        }
</code>```

答:
To write a value to the realtime database based on an existing value, you will need to use a transaction.

The basic code for your transaction will look like this:

```<code>DatabaseReference balanceRef = FirebaseDatabase.getInstance().getReference("/Users/"+ user_id +"/Balance");
Long delta = 50l;
balanceRef.runTransaction(new Transaction.Handler() {
    @Override
    public Transaction.Result doTransaction(MutableData mutableData) {
        Long balance = mutableData.getValue(Long.class);
        if (balance == null) {
            mutableData.setValue(delta);
        }
        else {
            Long amount = mutableData.getValue(Long.class);
            amount = amount + delta;
            mutableData.setValue(amount);
        }
        return Transaction.success(mutableData);
    }

    @Override
    public void onComplete(DatabaseError databaseError, boolean b,
                           DataSnapshot dataSnapshot) {
        // Transaction completed
        Log.d(TAG, "balanceTransaction:onComplete:" + databaseError);
    }
});
</code>```

Body:
I have a field in Firebase: balance. What I need is that after a person enters a sum 100, the field changes the value to 100. Then when the person enters a sum of 50, the field value becomes 150.

How do I write processing logic on the client? Summation of data. I think it is over-easy, but I need your help!

    mDatabaseUsers.addValueEventListener(new ValueEventListener() {
        @Override
        public void onDataChange(DataSnapshot dataSnapshot) {
            String user_id = mAuth.getCurrentUser().getUid();

            String balance = dataSnapshot.child(user_id).child("Balance").getValue(String.class);

            mCountPayment.setText(balance + " ₽");
        }

        @Override
        public void onCancelled(DatabaseError databaseError) {

        }
    });

    mPaymentButton.setOnClickListener(new View.OnClickListener() {
        @Override
        public void onClick(View v) {
            paymentIntent();
        }
    });
}

private void paymentIntent() {

    final String user_id = mAuth.getCurrentUser().getUid();

    final String count = mPaymentList.getText().toString().trim();

    if (!TextUtils.isEmpty(count)) {

        mDatabaseUsers.addValueEventListener(new ValueEventListener() {
            @Override
            public void onDataChange(DataSnapshot dataSnapshot) {
                mDatabaseUsers.child(user_id).child("Balance").setValue(count);
            }

            @Override
            public void onCancelled(DatabaseError databaseError) {

            }
        });
            }
        }

Body:
To write a value to the realtime database based on an existing value, you will need to use a transaction.

The basic code for your transaction will look like this:

DatabaseReference balanceRef = FirebaseDatabase.getInstance().getReference("/Users/"+ user_id +"/Balance");
Long delta = 50l;
balanceRef.runTransaction(new Transaction.Handler() {
    @Override
    public Transaction.Result doTransaction(MutableData mutableData) {
        Long balance = mutableData.getValue(Long.class);
        if (balance == null) {
            mutableData.setValue(delta);
        }
        else {
            Long amount = mutableData.getValue(Long.class);
            amount = amount + delta;
            mutableData.setValue(amount);
        }
        return Transaction.success(mutableData);
    }

    @Override
    public void onComplete(DatabaseError databaseError, boolean b,
                           DataSnapshot dataSnapshot) {
        // Transaction completed
        Log.d(TAG, "balanceTransaction:onComplete:" + databaseError);
    }
});


问:
where is the instancemethod decorator?
In my code I have a method that returns an instance of a class, like this:

```<code>class MyClass:
  def fun( self, *args ): # the method
    return Props( self, *args )

class Props: # the returned object
  def __init__( self, parent, *args ):
    self.parent = parent
    self.args = args
</code>```

To keep things organized I am considering to place Props inside MyClass. I would then like to bypass fun and directly make the class an instance method of MyClass, like this:

```<code>class MyClass:
  @instancemethod # does not exist!
  class fun:
    def __init__( self, parent, *args ):
      self.parent = parent
      self.args = args
</code>```

Note the comment - the instancemethod decorator does not exist.

Is there a way to do this, i.e. to turn a callable object into an instance method? If I change @instancemethod to @classmethod the construction works, except then of course parent is the class, not the instance. I am surprised that I cannot find anything about what seems to be the opposite operation.

Curious to have this cleared up!

Edit:

It seems that my question was not clear. What I have is a member function, fun, that returns not a single value or tuple but an object full of data. This data is generated based on the contents of the MyClass object and function arguments. My initial code does exactly what I want. The second code is how I would prefer to write it.

Moreover I noticed that the decorator I am looking for is simply the following:

```<code>def instancemethod( cls ):
  def f( *args ):
    return cls( *args )
  return f
</code>```

It is, of course, identical to the 'fun' method that I aimed to bypass. Note also that the trivial 'return cls' is not identical, even though it might seem like that at first sight.

With this decorator, my second class definition is valid and produces the desired result, namely, a.fun() returns an object that is (potentially) initialized based on the data in a:

```<code>a = MyClass()
p = a.fun(1,2,3)

print a        # <__main__.MyClass instance at 0xb775b84c>
print p.parent # <__main__.MyClass instance at 0xb775b84c>
print p.args   # (1, 2, 3)
</code>```

This still leaves me with the question if the instancemethod defined here is not available as a python builtin, because it seems an omission next to classmethod and staticmethod. But, if not, I suppose I can live with this construction.

答:
You don't need an instancemethod decorator, this could should work:

```<code>class MyClass:
  class fun:
    def __init__( self, parent, *args ):
      self.parent = parent
      self.args = args

m = MyClass()
f = m.fun(None, 1, 2, 3)
print f.args
</code>```

You could also change your first example to

```<code>class Props:
    def __init__( self, parent, *args ):
        self.parent = parent
        self.args = args

class MyClass:
    fun = Props

m = MyClass()
f = m.fun(None, 1, 2, 3)
print f.args
</code>```

Title: where is the instancemethod decorator?

Body:
In my code I have a method that returns an instance of a class, like this:

class MyClass:
  def fun( self, *args ): # the method
    return Props( self, *args )

class Props: # the returned object
  def __init__( self, parent, *args ):
    self.parent = parent
    self.args = args

To keep things organized I am considering to place Props inside MyClass. I would then like to bypass fun and directly make the class an instance method of MyClass, like this:

class MyClass:
  @instancemethod # does not exist!
  class fun:
    def __init__( self, parent, *args ):
      self.parent = parent
      self.args = args

Note the comment - the instancemethod decorator does not exist.

Is there a way to do this, i.e. to turn a callable object into an instance method? If I change @instancemethod to @classmethod the construction works, except then of course parent is the class, not the instance. I am surprised that I cannot find anything about what seems to be the opposite operation.

Curious to have this cleared up!

Edit:

It seems that my question was not clear. What I have is a member function, fun, that returns not a single value or tuple but an object full of data. This data is generated based on the contents of the MyClass object and function arguments. My initial code does exactly what I want. The second code is how I would prefer to write it.

Moreover I noticed that the decorator I am looking for is simply the following:

def instancemethod( cls ):
  def f( *args ):
    return cls( *args )
  return f

It is, of course, identical to the 'fun' method that I aimed to bypass. Note also that the trivial 'return cls' is not identical, even though it might seem like that at first sight.

With this decorator, my second class definition is valid and produces the desired result, namely, a.fun() returns an object that is (potentially) initialized based on the data in a:

a = MyClass()
p = a.fun(1,2,3)

print a        # <__main__.MyClass instance at 0xb775b84c>
print p.parent # <__main__.MyClass instance at 0xb775b84c>
print p.args   # (1, 2, 3)

This still leaves me with the question if the instancemethod defined here is not available as a python builtin, because it seems an omission next to classmethod and staticmethod. But, if not, I suppose I can live with this construction.

Body:
You don't need an instancemethod decorator, this could should work:

class MyClass:
  class fun:
    def __init__( self, parent, *args ):
      self.parent = parent
      self.args = args

m = MyClass()
f = m.fun(None, 1, 2, 3)
print f.args

You could also change your first example to

class Props:
    def __init__( self, parent, *args ):
        self.parent = parent
        self.args = args

class MyClass:
    fun = Props

m = MyClass()
f = m.fun(None, 1, 2, 3)
print f.args


问:
How do I test a method defined within a functional component, that interacts with DOM elements and has no arguments
I have been having trouble getting 100% test coverage on one of my buttons (A React functional components.) Basically when it is clicked, it executes some code and then also calls another method from within this onClick called resetButtons. This method will find all the buttons like it in the app and remove a class. This is a preemptive behavior so that only one button at a time can be active.

So far I have tested the click using .simulate, passing in a mocked domElement. And then test that the domElement.classList.add method is called with 'active'.

Obviously this being a DOM centered operation, I am finding it very difficult to test the resetButtons method that lies within the component. especially considering it doesn't have any methods.

I have tried defining the resetButtons method outside of the component and then exported it so the jest test could import it. However I have been unable to test the method as it seems to want it to be a spy or mock, and not the method itself. (Matcher error: received value must be a mock or spy function
)

Here is the react Functional Component:

```<code>import React from 'react';
import PropTypes from 'prop-types';
import classes from './MainButton.module.scss';

const MainButton = (props) => {
  const resetButtons = () => {
    const elements = document.getElementsByClassName('mainButton');
    for (let i = 0; i < elements.length; i += 1) {
      elements[i].classList.remove('active');
    }
  };

  const handleClick = (event) => {
    if (!event.target.classList.contains('active')) {
      resetButtons();
      event.target.classList.add('active');
      props.setVisualState(props.className.split('-')[0]);
    }
  };

  return (
    <button
      onClick={handleClick}
      type="button"
      className={`${classes.mainButton} ${props.className}`}
    >
      {props.children}
    </button>
  );
};

MainButton.propTypes = {
  children: PropTypes.node,
  className: PropTypes.string,
  setVisualState: PropTypes.func.isRequired,
};

MainButton.defaultProps = {
  children: 'Button',
  className: '',
};

export default MainButton;

</code>```

Here is the Test

```<code>import React from 'react';
import { shallow } from 'enzyme';
import MainButton from './MainButton';

describe('MainButton', () => {
  const domElement = { classList: { contains: jest.fn(), remove: jest.fn(), add: jest.fn() } };
  const setVisualStateMock = jest.fn();
  const mainButton = shallow(<MainButton setVisualState={setVisualStateMock} />);

  it(' is rendered properly', () => {
    expect(mainButton).toMatchSnapshot();
  });

  describe('when clicked', () => {
    beforeEach(() => {
      mainButton.find('button').simulate('click', { target: domElement });
    });

    it('it runs `classlist.add` to assign `active` class', () => {
      expect(domElement.classList.add).toHaveBeenCalledWith('active');
    });

    it('it runs set visual state to update `Allergen` container `state`', () => {
      expect(setVisualStateMock).toHaveBeenCalled();
    });
  });
});
</code>```

Currently the coverage report is reporting 92% coverage, but the branch is at 50 and the line that is causing the trouble is on line 9 (the elements[i].classList.remove('active'); line.

I know at 90% I should probably just move on but this is something I want to be able to figure out. Feel like getting head around this will make me a better tested.

Hope you guys can help!

答:
Fumbling around in the DOM yourself is an anti-pattern. That's React's job. Instead of manipulating the dom with target.classList.add you should have a state property that holds the status which of your inputs is currently active. Then, while rendering you can say className={isActiveInput ? "active": null}.

Because the state is not specific to your MainButton component you would lift the state up. If you have the state somewhere in the parent you don't have to crudely search for DOM elements by classname and manipulate the dom yourself.

Simply put, the rule of React is: You define how things are supposed to look like, React takes care that your definition becomes reality in the dom. If you manipulate the DOM yourself - you're doing it wrong.

When all of this is done, you will have no problem at all with tests, because all you have to do is provide the proper state and props, which is easy, and check that your callback is triggered onClick.

EDIT: Advanced version would be to use Context, but I'd go with state lifting first.

Body:
I have been having trouble getting 100% test coverage on one of my buttons (A React functional components.) Basically when it is clicked, it executes some code and then also calls another method from within this onClick called resetButtons. This method will find all the buttons like it in the app and remove a class. This is a preemptive behavior so that only one button at a time can be active.

So far I have tested the click using .simulate, passing in a mocked domElement. And then test that the domElement.classList.add method is called with 'active'.

Obviously this being a DOM centered operation, I am finding it very difficult to test the resetButtons method that lies within the component. especially considering it doesn't have any methods.

I have tried defining the resetButtons method outside of the component and then exported it so the jest test could import it. However I have been unable to test the method as it seems to want it to be a spy or mock, and not the method itself. (Matcher error: received value must be a mock or spy function
)

Here is the react Functional Component:

import React from 'react';
import PropTypes from 'prop-types';
import classes from './MainButton.module.scss';

const MainButton = (props) => {
  const resetButtons = () => {
    const elements = document.getElementsByClassName('mainButton');
    for (let i = 0; i < elements.length; i += 1) {
      elements[i].classList.remove('active');
    }
  };

  const handleClick = (event) => {
    if (!event.target.classList.contains('active')) {
      resetButtons();
      event.target.classList.add('active');
      props.setVisualState(props.className.split('-')[0]);
    }
  };

  return (
    <button
      onClick={handleClick}
      type="button"
      className={`${classes.mainButton} ${props.className}`}
    >
      {props.children}
    </button>
  );
};

MainButton.propTypes = {
  children: PropTypes.node,
  className: PropTypes.string,
  setVisualState: PropTypes.func.isRequired,
};

MainButton.defaultProps = {
  children: 'Button',
  className: '',
};

export default MainButton;

Here is the Test

import React from 'react';
import { shallow } from 'enzyme';
import MainButton from './MainButton';

describe('MainButton', () => {
  const domElement = { classList: { contains: jest.fn(), remove: jest.fn(), add: jest.fn() } };
  const setVisualStateMock = jest.fn();
  const mainButton = shallow(<MainButton setVisualState={setVisualStateMock} />);

  it(' is rendered properly', () => {
    expect(mainButton).toMatchSnapshot();
  });

  describe('when clicked', () => {
    beforeEach(() => {
      mainButton.find('button').simulate('click', { target: domElement });
    });

    it('it runs `classlist.add` to assign `active` class', () => {
      expect(domElement.classList.add).toHaveBeenCalledWith('active');
    });

    it('it runs set visual state to update `Allergen` container `state`', () => {
      expect(setVisualStateMock).toHaveBeenCalled();
    });
  });
});

Currently the coverage report is reporting 92% coverage, but the branch is at 50 and the line that is causing the trouble is on line 9 (the elements[i].classList.remove('active'); line.

I know at 90% I should probably just move on but this is something I want to be able to figure out. Feel like getting head around this will make me a better tested.

Hope you guys can help!


问:
Create one object instance per class in hierarchy
I was asked this question in an interview.

There are 3 classes A, B extends A & C extends B. We have to design these classes conforming to these constraints

Client can instantiate only one instance of A, one instance of B & one instance of C using their default constructor with new keyword.

Trying to create another instance of any of these class will result in an exception.

The designer of the class have to enforce the above 2 rules, so that client will experience the above rules implicitly (i.e. it should not be the responsibility of client to conform to above rules).

I suggested an approach using an static Map<Class, Object>. So for e.g. when somebody called new B() it would check if map.contains(B.class). If yes then throw exception & if not then save instance in map & let the object be created.

But the next question was how would I enforce this approach on each class? As per my approach each constructor would have to carefully populate the map otherwise it will break the constraint.

How would I solve this problem?

答:
You could create set of classes that were created then in constructor of class A check if this.getClass() was created already or not. It not then add it, otherwise throw an exception.
Below simple example:

```<code>static class A {
    private static Set< Class< ? > > createdClasses = new HashSet<>();
    {
        if( !createdClasses.add( this.getClass() ) ) {
            throw new RuntimeException( "Cannot create 2nd class" );
        }
    }
}
static class B extends A {}
static class C extends B {}
</code>```

Note: All classes that extend A will have this behaviour!

Body:
You could create set of classes that were created then in constructor of class A check if this.getClass() was created already or not. It not then add it, otherwise throw an exception.
Below simple example:

static class A {
    private static Set< Class< ? > > createdClasses = new HashSet<>();
    {
        if( !createdClasses.add( this.getClass() ) ) {
            throw new RuntimeException( "Cannot create 2nd class" );
        }
    }
}
static class B extends A {}
static class C extends B {}

Note: All classes that extend A will have this behaviour!


答:
Static state may work like so:</p>

```<code>import java.util.HashSet;
import java.util.Set;

class Aa {
    private static Set<Class> set = new HashSet();

    static void validateAndAdd(Class c) {
        if (set.contains(c) ) {
            throw new IllegalStateException("...");
        }
        set.add(c);
        System.out.println(set);
    }

    public Aa() {
        validateAndAdd(getClass());
    }
}

class Bb extends Aa {
}

class Cc extends Bb {
}

public class Main {
    public static void main(String[] args) throws Exception {
        new Cc(); // [class Cc]
        new Aa(); // [class Cc, class Aa]
        new Bb(); // [class Bb, class Cc, class Aa]
        new Bb(); // IllegalStateException: ...
    }
}
</code>```

Title: Create one object instance per class in hierarchy

Body:
I was asked this question in an interview.

There are 3 classes A, B extends A & C extends B. We have to design these classes conforming to these constraints

Client can instantiate only one instance of A, one instance of B & one instance of C using their default constructor with new keyword.

Trying to create another instance of any of these class will result in an exception.

The designer of the class have to enforce the above 2 rules, so that client will experience the above rules implicitly (i.e. it should not be the responsibility of client to conform to above rules).

I suggested an approach using an static Map<Class, Object>. So for e.g. when somebody called new B() it would check if map.contains(B.class). If yes then throw exception & if not then save instance in map & let the object be created.

But the next question was how would I enforce this approach on each class? As per my approach each constructor would have to carefully populate the map otherwise it will break the constraint.

How would I solve this problem?

Body:
Static state may work like so:

import java.util.HashSet;
import java.util.Set;

class Aa {
    private static Set<Class> set = new HashSet();

    static void validateAndAdd(Class c) {
        if (set.contains(c) ) {
            throw new IllegalStateException("...");
        }
        set.add(c);
        System.out.println(set);
    }

    public Aa() {
        validateAndAdd(getClass());
    }
}

class Bb extends Aa {
}

class Cc extends Bb {
}

public class Main {
    public static void main(String[] args) throws Exception {
        new Cc(); // [class Cc]
        new Aa(); // [class Cc, class Aa]
        new Bb(); // [class Bb, class Cc, class Aa]
        new Bb(); // IllegalStateException: ...
    }
}


问:
In what cases [NSDocument fileURL] can be nil?
I'm creating the document with:

```<code>[NSDocumentController openDocumentWithContentsOfURL:display:completionHandler:]
</code>```

When I'm working with the instance of NSDocument and sometimes I'm getting crashes because document.fileURL == nil. What cases in NSDocument lifecycle can cause this?

答: document.fileURL is nil until you save the document for the first time.

Body:
I'm creating the document with:

[NSDocumentController openDocumentWithContentsOfURL:display:completionHandler:]

When I'm working with the instance of NSDocument and sometimes I'm getting crashes because document.fileURL == nil. What cases in NSDocument lifecycle can cause this?


问:
Protocol Extension where Self is ClassA OR ClassB
I have a protocol for which I want to provide default functionality in the case that the conforming class is either ClassA OR ClassB. Is it possible to use || in a protocol extension where clause? I've tried the following but it doesn't work:

```<code>extension Protocol where Self: ClassA || Self: ClassB {
    func method() {
        // implementation if conforming class is ClassA OR ClassB
    }
}
</code>```

答:
No, you cannot use || (or something equivalent) in the where clause of a protocol extension.

If the extension method needs certain functionality which is present in both ClassA and ClassB then you can define that in a protocol  (to which both classes conform), and constrain the extension to that protocol.

Body:
I have a protocol for which I want to provide default functionality in the case that the conforming class is either ClassA OR ClassB. Is it possible to use || in a protocol extension where clause? I've tried the following but it doesn't work:

extension Protocol where Self: ClassA || Self: ClassB {
    func method() {
        // implementation if conforming class is ClassA OR ClassB
    }
}


问:
Process large csv file and limit goroutines
I'm trying to find the best efficient way to read a csv file (~1M row).
Each row contain a HTTP link to an image which I need to download.

This is my current code using worker pools:

```<code>func worker(queue chan []string, worknumber int, done, ks chan bool) {
    for true {
        select {
        case url := <-queue:
            fmt.Println("doing work!", url, "worknumber", worknumber)
            processData(url) // HTTP download
            done <- true
        case <-ks:
            fmt.Println("worker halted, number", worknumber)
            return
        }
    }
}

func main() {
    start := time.Now()
    flag.Parse()
    fmt.Print(strings.Join(flag.Args(), "\n"))
    if *filename == "REQUIRED" {
        return
    }

    csvfile, err := os.Open(*filename)
    if err != nil {
        fmt.Println(err)
        return
    }
    count, _ := lineCounter(csvfile)
    fmt.Printf("Total count: %d\n", count)
    csvfile.Seek(0, 0)

    defer csvfile.Close()

    //bar := pb.StartNew(count)
    bar := progressbar.NewOptions(count)
    bar.RenderBlank()

    reader := csv.NewReader(csvfile)

    //channel for terminating the workers
    killsignal := make(chan bool)

    //queue of jobs
    q := make(chan []string)
    // done channel takes the result of the job
    done := make(chan bool)

    numberOfWorkers := *numChannels
    for i := 0; i < numberOfWorkers; i++ {
        go worker(q, i, done, killsignal)
    }

    i := 0
    for {
        record, err := reader.Read()
        if err == io.EOF {
            break
        } else if err != nil {
            fmt.Println(err)
            return
        }
        i++

        go func(r []string, i int) {
            q <- r
            bar.Add(1)
        }(record, i)
    }

    // a deadlock occurs if c >= numberOfJobs
    for c := 0; c < count; c++ {
        <-done
    }

    fmt.Println("finished")

    // cleaning workers
    close(killsignal)
    time.Sleep(2 * time.Second)

    fmt.Printf("\n%2fs", time.Since(start).Seconds())
}
</code>```

My issue here is that it opens a lot of goroutines, use all the memory and crash.

What would be the best way to limit it?

答:
I striped out the progress bar as i did not want to bother about it, but overall this is closer to what you are looking for.

It does not genuinely handle errors, they simply fail in a fatal state.

I have added context and cancellation support.

You might want to check for
https://godoc.org/golang.org/x/sync/errgroup#Group.Go

As a general recommentation, you need to learn the golang patterns and their usage.

It is obvious you have not worked that enough, or that you are in process of learning.

Its not the fastest program at all, but it does the job.

This is only a draft to get you back on a better direction.

```<code>package main

import (
    "context"
    "encoding/csv"
    "flag"
    "fmt"
    "io"
    "log"
    "os"
    "os/signal"
    "sync"
    "time"
)

func worker(ctx context.Context, dst chan string, src chan []string) {
    for {
        select {
        case url, ok := <-src: // you must check for readable state of the channel.
            if !ok {
                return
            }
            dst <- fmt.Sprintf("out of %v", url) // do somethingg useful.
        case <-ctx.Done(): // if the context is cancelled, quit.
            return
        }
    }
}

func main() {

    // create a context
    ctx, cancel := context.WithCancel(context.Background())
    defer cancel()
    // that cancels at ctrl+C
    go onSignal(os.Interrupt, cancel)

    // parse command line arguments
    var filename string
    var numberOfWorkers int
    flag.StringVar(&filename, "filename", "", "src file")
    flag.IntVar(&numberOfWorkers, "c", 2, "concurrent workers")
    flag.Parse()

    // check arguments
    if filename == "" {
        log.Fatal("filename required")
    }

    start := time.Now()

    csvfile, err := os.Open(filename)
    if err != nil {
        log.Fatal(err)
    }
    defer csvfile.Close()

    reader := csv.NewReader(csvfile)

    // create the pair of input/output channels for the controller=>workers com.
    src := make(chan []string)
    out := make(chan string)

    // use a waitgroup to manage synchronization
    var wg sync.WaitGroup

    // declare the workers
    for i := 0; i < numberOfWorkers; i++ {
        wg.Add(1)
        go func() {
            defer wg.Done()
            worker(ctx, out, src)
        }()
    }

    // read the csv and write it to src
    go func() {
        for {
            record, err := reader.Read()
            if err == io.EOF {
                break
            } else if err != nil {
                log.Fatal(err)
            }
            src <- record // you might select on ctx.Done().
        }
        close(src) // close src to signal workers that no more job are incoming.
    }()

    // wait for worker group to finish and close out
    go func() {
        wg.Wait() // wait for writers to quit.
        close(out) // when you close(out) it breaks the below loop.
    }()

    // drain the output
    for res := range out {
        fmt.Println(res)
    }

    fmt.Printf("\n%2fs", time.Since(start).Seconds())
}

func onSignal(s os.Signal, h func()) {
    c := make(chan os.Signal, 1)
    signal.Notify(c, s)
    <-c
    h()
}
</code>```

Body:
I'm trying to find the best efficient way to read a csv file (~1M row).
Each row contain a HTTP link to an image which I need to download.

This is my current code using worker pools:

func worker(queue chan []string, worknumber int, done, ks chan bool) {
    for true {
        select {
        case url := <-queue:
            fmt.Println("doing work!", url, "worknumber", worknumber)
            processData(url) // HTTP download
            done <- true
        case <-ks:
            fmt.Println("worker halted, number", worknumber)
            return
        }
    }
}

func main() {
    start := time.Now()
    flag.Parse()
    fmt.Print(strings.Join(flag.Args(), "\n"))
    if *filename == "REQUIRED" {
        return
    }

    csvfile, err := os.Open(*filename)
    if err != nil {
        fmt.Println(err)
        return
    }
    count, _ := lineCounter(csvfile)
    fmt.Printf("Total count: %d\n", count)
    csvfile.Seek(0, 0)

    defer csvfile.Close()

    //bar := pb.StartNew(count)
    bar := progressbar.NewOptions(count)
    bar.RenderBlank()

    reader := csv.NewReader(csvfile)

    //channel for terminating the workers
    killsignal := make(chan bool)

    //queue of jobs
    q := make(chan []string)
    // done channel takes the result of the job
    done := make(chan bool)

    numberOfWorkers := *numChannels
    for i := 0; i < numberOfWorkers; i++ {
        go worker(q, i, done, killsignal)
    }

    i := 0
    for {
        record, err := reader.Read()
        if err == io.EOF {
            break
        } else if err != nil {
            fmt.Println(err)
            return
        }
        i++

        go func(r []string, i int) {
            q <- r
            bar.Add(1)
        }(record, i)
    }

    // a deadlock occurs if c >= numberOfJobs
    for c := 0; c < count; c++ {
        <-done
    }

    fmt.Println("finished")

    // cleaning workers
    close(killsignal)
    time.Sleep(2 * time.Second)

    fmt.Printf("\n%2fs", time.Since(start).Seconds())
}

My issue here is that it opens a lot of goroutines, use all the memory and crash.

What would be the best way to limit it?

Body:
I striped out the progress bar as i did not want to bother about it, but overall this is closer to what you are looking for.

It does not genuinely handle errors, they simply fail in a fatal state.

I have added context and cancellation support.

You might want to check for
https://godoc.org/golang.org/x/sync/errgroup#Group.Go

As a general recommentation, you need to learn the golang patterns and their usage.

It is obvious you have not worked that enough, or that you are in process of learning.

Its not the fastest program at all, but it does the job.

This is only a draft to get you back on a better direction.

package main

import (
    "context"
    "encoding/csv"
    "flag"
    "fmt"
    "io"
    "log"
    "os"
    "os/signal"
    "sync"
    "time"
)

func worker(ctx context.Context, dst chan string, src chan []string) {
    for {
        select {
        case url, ok := <-src: // you must check for readable state of the channel.
            if !ok {
                return
            }
            dst <- fmt.Sprintf("out of %v", url) // do somethingg useful.
        case <-ctx.Done(): // if the context is cancelled, quit.
            return
        }
    }
}

func main() {

    // create a context
    ctx, cancel := context.WithCancel(context.Background())
    defer cancel()
    // that cancels at ctrl+C
    go onSignal(os.Interrupt, cancel)

    // parse command line arguments
    var filename string
    var numberOfWorkers int
    flag.StringVar(&filename, "filename", "", "src file")
    flag.IntVar(&numberOfWorkers, "c", 2, "concurrent workers")
    flag.Parse()

    // check arguments
    if filename == "" {
        log.Fatal("filename required")
    }

    start := time.Now()

    csvfile, err := os.Open(filename)
    if err != nil {
        log.Fatal(err)
    }
    defer csvfile.Close()

    reader := csv.NewReader(csvfile)

    // create the pair of input/output channels for the controller=>workers com.
    src := make(chan []string)
    out := make(chan string)

    // use a waitgroup to manage synchronization
    var wg sync.WaitGroup

    // declare the workers
    for i := 0; i < numberOfWorkers; i++ {
        wg.Add(1)
        go func() {
            defer wg.Done()
            worker(ctx, out, src)
        }()
    }

    // read the csv and write it to src
    go func() {
        for {
            record, err := reader.Read()
            if err == io.EOF {
                break
            } else if err != nil {
                log.Fatal(err)
            }
            src <- record // you might select on ctx.Done().
        }
        close(src) // close src to signal workers that no more job are incoming.
    }()

    // wait for worker group to finish and close out
    go func() {
        wg.Wait() // wait for writers to quit.
        close(out) // when you close(out) it breaks the below loop.
    }()

    // drain the output
    for res := range out {
        fmt.Println(res)
    }

    fmt.Printf("\n%2fs", time.Since(start).Seconds())
}

func onSignal(s os.Signal, h func()) {
    c := make(chan os.Signal, 1)
    signal.Notify(c, s)
    <-c
    h()
}


问:
Web Crawling Any Pages using Java
I am very new to this web crawling. I am using crawler4j to crawl the websites. I am collecting the required information by crawling these sites. My problem here is I was unable to crawl the content. Because result of crawling is JavaScript code. But i can get DOM HTML on web browser' inspect. How can i get actual DOM HTML by using java.

Here is getting DOM by using Jsoup or Crawler4j.

```<code><!doctype html>
<html class="no-js" lang="en">
 <head>
  <title>title</title>
 </head>
 <body class="component mouse-active body--navigation-fixed" data-js="_mouseActive, _fixed, _component">
  <script type="text/javascript">
        var utag_data = {
            page_name: "personel:rootpage", // Page name
            channel: "personel:rootpage2", // s.channel (Section)
            page_type: "personel:rootpage2", // s.channel (Section)
            login_status: "not-logged-in", // s.channel (Section)
        }
    </script>
  <script type="text/javascript">
    some javascript code
</script>
  <form id="postForm" action="https://testurl.com/index.php" method="post">
  </form>
  <script type="text/javascript">
        some javascript code
</script>
 </body>
</html>
</code>```

i expected that result;

```<code><html lang="tr" class="gccc_com_tr">
<head>
</head>
<body class="page--home" data-gr-c-s-loaded="true">
<div class="header">
    <div class="nav-top">
    <div class="container">
        <ul class="menu menu-left">
            <li class="active"> <a href="https://www.vodafone.com.tr/" accesskey="B">Bireysel</a> </li>
            <li> <a href="https://www.vodafone.com.tr/VodafoneBusiness/index.php" accesskey="K">Kurumsal</a> </li>
            <li> <a href="https://www.vodafone.com.tr/en/roam-with-vodafone.php">Visiting Turkey</a> </li>
        </ul>
        <ul class="menu menu-right">
            <li> <a href="http://www.vodafone.com.tr/auto-login.php?pageId=InvoicePayment&paymentFlag=true" accesskey="A">Fatura Öde</a> </li>
            <li> <a href="http://www.vodafone.com.tr/auto-login.php?pageId=TopupPayment&paymentFlag=true" accesskey="L">TL Yükle</a> </li>
        </ul>
    </div>
    </div>
    <nav class="navbar">
        <div class="container">
            <div id="navbar">
                <div class="nav navbar-logo" itemscope="" itemtype="https://schema.org/Organization">
                    <a href="/" class="logo" itemprop="url">
                        <span class="icon-vodafone">
                            <span class="path1"></span>
                        </span>
                        <span style="display: none;">
                            Vodafone
                        </span>
                    </a>
                </div>
                <ul class="nav navbar-nav list-nav-main">
                    <div class="">
                        <li class="hide--lg hide--md"></li>
                    </div>
                </ul>
                </ul>
            </div>
        </div>
    </nav>
</div>
<footer class="row">
    <div class="container top">
    </div>
</footer>
<script type="text/javascript" src="/assets/v2/js/script.build.app.js" defer="" async=""></script>
</body>
</html>
</code>```

答:
As described in the related #49 or #197 on the project page of crawler4j , Javascript execution / rendering of content using ajax / javascript is not possible with crawler4j at the moment as it does not include a javascript engine for that purpose.

However, you could by-pass this behaviour by adding a combination of Selenium and/or CasperJS and/or PhantomJS in front of crawler4j.

Title: Web Crawling Any Pages using Java

Body:
I am very new to this web crawling. I am using crawler4j to crawl the websites. I am collecting the required information by crawling these sites. My problem here is I was unable to crawl the content. Because result of crawling is JavaScript code. But i can get DOM HTML on web browser' inspect. How can i get actual DOM HTML by using java.

Here is getting DOM by using Jsoup or Crawler4j.

<!doctype html>

  <title>title</title>

  <form id="postForm" action="https://testurl.com/index.php" method="post">
  </form>

i expected that result;

 Bireysel

 Kurumsal

 Visiting Turkey

 Fatura Öde

 TL Yükle

                            Vodafone


问:
I want to give 'edit-link i.e edit' if ans column of table from database is null
I am retrieving answer from ans column by using jdbc and showing it into a jsp page in a form of a table, if ans column is null or empty then I have to give a user edit link, but if ans column has a value then I'm going to show answer but the problem is if-else condition is not working. Please help me to solve this, My code is given below:

```<code><div class="table-responsive">
    <table class="table center-aligned-table">
        <thead>
        <tr class="text-primary">
            <th>Question No</th>
            <th>Question Name</th>
            <th>Answer:</th>
            <th> </th>
            <th> </th>

            <th></th>
            <th></th>
            <th></th>
        </tr>
        </thead>
    <tbody>
        <%
            try{
               int oopa=1;
               //String nme=(String)session.getAttribute("vname7");
               DbCon cc=new DbCon();
               Connection onn=cc.fun();
               Statement stt=onn.createStatement();

               ResultSet r=stt.executeQuery("select ques,ans from postquestion;");

                while(r.next()){
        %>

        <tr class="">
            <td><center><%=oopa++%></center></td>
            <td><%=r.getString(1)%></td>
            <%
                if(r.getString(2)==null)
                {
            %>

            <td><a href="">Edit</a></td>

            <%
                }
                else
                {
            %>
            <td><%=r.getString(2)%></td>

            <%
            }
            %>

            <td> </td>

        </tr>

            <%

            }
}
            catch(Exception vjin){
                System.out.println("I am vjin: "+vjin);
                vjin.printStackTrace();
            }

            %>

    </tbody>
    </table>
</div>

</code>```

答:
You can check for null values using wasNull() method of ResultSet.For example like below :

```<code>        <%
         //getting value of column in "a"
            String a =  r.getString(2);
          //checking if resultset return null i.e a is null then do below:
            if(r.wasNull())
                {
            %>

            <td><a href="">Edit</a></td>

            <%
                }
                else
                {
            %>
            <td><%=r.getString(2)%></td>

            <%
            }
            %>
</code>```

Body:
I am retrieving answer from ans column by using jdbc and showing it into a jsp page in a form of a table, if ans column is null or empty then I have to give a user edit link, but if ans column has a value then I'm going to show answer but the problem is if-else condition is not working. Please help me to solve this, My code is given below:

            Question No
            Question Name
            Answer:

        <%
            try{
               int oopa=1;
               //String nme=(String)session.getAttribute("vname7");
               DbCon cc=new DbCon();
               Connection onn=cc.fun();
               Statement stt=onn.createStatement();

               ResultSet r=stt.executeQuery("select ques,ans from postquestion;");

                while(r.next()){
        %>

            <%=oopa++%>
            <%=r.getString(1)%>
            <%
                if(r.getString(2)==null)
                {
            %>

            Edit

            <%
                }
                else
                {
            %>
            <%=r.getString(2)%>

            <%
            }
            %>

            <%

            }
}
            catch(Exception vjin){
                System.out.println("I am vjin: "+vjin);
                vjin.printStackTrace();
            }

            %>

Body:
You can check for null values using wasNull() method of ResultSet.For example like below :

        <%
         //getting value of column in "a"
            String a =  r.getString(2);
          //checking if resultset return null i.e a is null then do below:
            if(r.wasNull())
                {
            %>

            <td><a href="">Edit</a></td>

            <%
                }
                else
                {
            %>
            <td><%=r.getString(2)%></td>

            <%
            }
            %>


问:
Help creating exam grading program in Python
I am trying to create a program that reads multiple choice answers from a txt file and compares them to a set answer key.  This is what I have so far but the problem is that that when I run it, the answer key gets stuck on a single letter throughout the life of the program.  I have put a print statement right after the for answerKey line and it prints out correctly, but when it compares the "exam" answers to the answer key it gets stuck and always thinks "A" should be the correct answer. Which is weird because it is the 3rd entry in my sample answer key.

Here's the code:

```<code>answerKey = open("answerkey.txt" , 'r')
studentExam = open("studentexam.txt" , 'r')
index = 0
numCorrect = 0
for line in answerKey:
    answer = line.split()
for line in studentExam:
    studentAnswer = line.split()
    if studentAnswer != answer:
        print("You got question number", index + 1, "wrong\nThe correct answer was" ,answer , "but you answered", studentAnswer)
        index += 1
    else:
        numCorrect += 1
        index += 1
grade = int((numCorrect / 20) * 100)
print("The number of correctly answered questions:" , numCorrect)
print("The number of incorrectly answered questions:" , 20 - numCorrect)
print("Your grade is" ,grade ,"%")
if grade <= 75:
    print("You have not passed")
else:
    print("Congrats! You passed!")
</code>```

Thanks for any help you can give me!

答:
The problem is that you're not nesting the loops properly.

This loop runs first, and ends up setting answer to the last line of the answerKey.

```<code>for line in answerKey:
    answer = line.split()
</code>```

The for line in studentExam: loop runs afterwards, but answer doesn't change in this loop and will stay the same.

The solution is combining the loops using zip:

```<code>for answerLine, studentLine in zip(answerKey, studentExam):
    answer = answerLine.split()
    studentAnswer = studentLine.split()
</code>```

Also, remember to close the files when you're done with them:

```<code>answerKey.close()
studentExam.close()
</code>```

Body:
I am trying to create a program that reads multiple choice answers from a txt file and compares them to a set answer key.  This is what I have so far but the problem is that that when I run it, the answer key gets stuck on a single letter throughout the life of the program.  I have put a print statement right after the for answerKey line and it prints out correctly, but when it compares the "exam" answers to the answer key it gets stuck and always thinks "A" should be the correct answer. Which is weird because it is the 3rd entry in my sample answer key.

Here's the code:

answerKey = open("answerkey.txt" , 'r')
studentExam = open("studentexam.txt" , 'r')
index = 0
numCorrect = 0
for line in answerKey:
    answer = line.split()
for line in studentExam:
    studentAnswer = line.split()
    if studentAnswer != answer:
        print("You got question number", index + 1, "wrong\nThe correct answer was" ,answer , "but you answered", studentAnswer)
        index += 1
    else:
        numCorrect += 1
        index += 1
grade = int((numCorrect / 20) * 100)
print("The number of correctly answered questions:" , numCorrect)
print("The number of incorrectly answered questions:" , 20 - numCorrect)
print("Your grade is" ,grade ,"%")
if grade <= 75:
    print("You have not passed")
else:
    print("Congrats! You passed!")

Thanks for any help you can give me!

Body:
The problem is that you're not nesting the loops properly.

This loop runs first, and ends up setting answer to the last line of the answerKey.

for line in answerKey:
    answer = line.split()

The for line in studentExam: loop runs afterwards, but answer doesn't change in this loop and will stay the same.

The solution is combining the loops using zip:

for answerLine, studentLine in zip(answerKey, studentExam):
    answer = answerLine.split()
    studentAnswer = studentLine.split()

Also, remember to close the files when you're done with them:

answerKey.close()
studentExam.close()


问:
How to logout with GET request in SpringBoot WebFlux
How do i configure securityWebFilterChain(ServerHttpSecurity http)  so that my application logs out on GET /logout?

I have SpringBoot 2 Spring 5 and WebFlux

I tried:

```<code>  http
    .logout()
      .requiresLogout(ServerWebExchangeMatchers.pathMatchers(HttpMethod.GET, "/logout"))
      .logoutSuccessHandler(logoutSuccessHandler("/after-life"))

</code>```

Problem is, a LogoutPageGeneratingWebFilter sits earlier than the LogoutWebFilter in the emitted SecurityWebFilterChain. In that there is a hardcoded .pathMatchers(HttpMethod.GET, "/logout") - which causes my application to always emit a html page on a GET request.

I found no way to suppress the automatic logout page generation :(

答:
As mentioned in the documentation,

The default is that Spring Security will generate a log in page at "/login" and a log out page at "/logout". If this is customized:
  The default log in & log out page are no longer provided
  The application must render a log in page at the provided URL
  The application must render an authentication error page at the provided URL + "?error"
  Authentication will occur for POST to the provided URL

Custom configuration to have default login and without default logout.

    @Bean
    public SecurityWebFilterChain securityWebFilterChain(ServerHttpSecurity httpSecurity){

        LoginPageGeneratingWebFilter loginpage= new LoginPageGeneratingWebFilter();
        loginpage.setFormLoginEnabled(true);
        return httpSecurity
                .addFilterAt(loginpage, SecurityWebFiltersOrder.LOGIN_PAGE_GENERATING)
                .authorizeExchange()
                    .pathMatchers("/home").authenticated()
                        .and().formLogin()
                            .loginPage("/login")
                        .and()
                        .logout()
                        .logoutUrl("/logout").requiresLogout(ServerWebExchangeMatchers.pathMatchers(HttpMethod.GET, "/logout"))
                        .and()

                .build();

    }
```

Body:
How do i configure securityWebFilterChain(ServerHttpSecurity http)  so that my application logs out on GET /logout?

I have SpringBoot 2 Spring 5 and WebFlux

I tried:

  http
    .logout()
      .requiresLogout(ServerWebExchangeMatchers.pathMatchers(HttpMethod.GET, "/logout"))
      .logoutSuccessHandler(logoutSuccessHandler("/after-life"))

Problem is, a LogoutPageGeneratingWebFilter sits earlier than the LogoutWebFilter in the emitted SecurityWebFilterChain. In that there is a hardcoded .pathMatchers(HttpMethod.GET, "/logout") - which causes my application to always emit a html page on a GET request.

I found no way to suppress the automatic logout page generation :(


问:
drupal forum function hook to validate a list of words before posting
how can I implement a hook to validate the words that I'm posting ?
it seems that the forum lacks that feature: prohibited words
so I want to implement one even if I have to make my own module
I just need to know what function to hook</p>

答:
Have you looked at the existing modules?

A quick search finds Wordfilter and Phonetic Wordfilter. I would suggest you try these out, even if they don't do exactly what you need their code will probably help point you in the right direction.

+++ EDIT +++

If you must do it when they post then use hook_nodeapi

If you want to remove word automatically then run the 'presave' operation to alter the body before saving. Something along the lines of;

```<code>function mymodule_nodeapi(&$node, $op, $a3 = NULL, $a4 = NULL){
    if($op == 'presave' && $node->type == 'forum'){
        $node->body = preg_replace('#\b(word1|word2|word3)\b#i', '*removed*', $node->body);
    }
}
</code>```

Or if you wanted to prevent the user from posting until they had removed any banned words then you use the 'validate' operation. Something like;

```<code>function mymodule_nodeapi(&$node, $op, $a3 = NULL, $a4 = NULL){
    if($op == 'validate' && $node->type == 'forum'){
        if(preg_match('#\b(word1|word2|word3)\b#i', $node->body)){
            form_set_error('body', 'You have used restricted words');
        }
    }
}
</code>```

Body:
Have you looked at the existing modules?

A quick search finds Wordfilter and Phonetic Wordfilter. I would suggest you try these out, even if they don't do exactly what you need their code will probably help point you in the right direction.

+++ EDIT +++

If you must do it when they post then use hook_nodeapi

If you want to remove word automatically then run the 'presave' operation to alter the body before saving. Something along the lines of;

function mymodule_nodeapi(&$node, $op, $a3 = NULL, $a4 = NULL){
    if($op == 'presave' && $node->type == 'forum'){
        $node->body = preg_replace('#\b(word1|word2|word3)\b#i', '*removed*', $node->body);
    }
}

Or if you wanted to prevent the user from posting until they had removed any banned words then you use the 'validate' operation. Something like;

function mymodule_nodeapi(&$node, $op, $a3 = NULL, $a4 = NULL){
    if($op == 'validate' && $node->type == 'forum'){
        if(preg_match('#\b(word1|word2|word3)\b#i', $node->body)){
            form_set_error('body', 'You have used restricted words');
        }
    }
}