问:
Hidden test cases not passing for Google Foobar Challenge Doomsday Fuel
I'm working my way through the Google Foobar challenge and am now at the level 3 challenge Doomsday Fuel. The instructions are as follows:

Doomsday Fuel

Making fuel for the LAMBCHOP's reactor core is a tricky process because of the exotic matter involved. It starts as raw ore, then during processing, begins randomly changing between forms, eventually reaching a stable form. There may be multiple stable forms that a sample could ultimately reach, not all of which are useful as fuel.

Commander Lambda has tasked you to help the scientists increase fuel creation efficiency by predicting the end state of a given ore sample. You have carefully studied the different structures that the ore can take and which transitions it undergoes. It appears that, while random, the probability of each structure transforming is fixed. That is, each time the ore is in 1 state, it has the same probabilities of entering the next state (which might be the same state).  You have recorded the observed transitions in a matrix. The others in the lab have hypothesized more exotic forms that the ore can become, but you haven't seen all of them.

Write a function solution(m) that takes an array of array of nonnegative ints representing how many times that state has gone to the next state and return an array of ints for each terminal state giving the exact probabilities of each terminal state, represented as the numerator for each state, then the denominator for all of them at the end and in simplest form. The matrix is at most 10 by 10. It is guaranteed that no matter which state the ore is in, there is a path from that state to a terminal state. That is, the processing will always eventually end in a stable state. The ore starts in state 0. The denominator will fit within a signed 32-bit integer during the calculation, as long as the fraction is simplified regularly.

```<code>>For example, consider the matrix m:
[
  [0,1,0,0,0,1],  # s0, the initial state, goes to s1 and s5 with equal probability
  [4,0,0,3,2,0],  # s1 can become s0, s3, or s4, but with different probabilities
  [0,0,0,0,0,0],  # s2 is terminal, and unreachable (never observed in practice)
  [0,0,0,0,0,0],  # s3 is terminal
  [0,0,0,0,0,0],  # s4 is terminal
  [0,0,0,0,0,0],  # s5 is terminal
]
So, we can consider different paths to terminal states, such as:
s0 -> s1 -> s3
s0 -> s1 -> s0 -> s1 -> s0 -> s1 -> s4
s0 -> s1 -> s0 -> s5
Tracing the probabilities of each, we find that
s2 has probability 0
s3 has probability 3/14
s4 has probability 1/7
s5 has probability 9/14
So, putting that together, and making a common denominator, gives an answer in the form of
[s2.numerator, s3.numerator, s4.numerator, s5.numerator, denominator] which is
[0, 3, 2, 9, 14].
</code>```

Languages

To provide a Java solution, edit Solution.java
  To provide a Python solution, edit solution.py

```<code>Test cases
==========
>Your code should pass the following test cases.
Note that it may also be run against hidden test cases not shown here.

>-- Java cases --
Input:
Solution.solution({{0, 2, 1, 0, 0}, {0, 0, 0, 3, 4}, {0, 0, 0, 0, 0}, {0, 0, 0, 0,0}, {0, 0, 0, 0, 0}})
Output:
    [7, 6, 8, 21]

>Input:
Solution.solution({{0, 1, 0, 0, 0, 1}, {4, 0, 0, 3, 2, 0}, {0, 0, 0, 0, 0, 0}, {0, 0, 0, 0, 0, 0}, {0, 0, 0, 0, 0, 0}, {0, 0, 0, 0, 0, 0}})
Output:
    [0, 3, 2, 9, 14]

>-- Python cases --
Input:
solution.solution([[0, 2, 1, 0, 0], [0, 0, 0, 3, 4], [0, 0, 0, 0, 0], [0, 0, 0, 0,0], [0, 0, 0, 0, 0]])
Output:
    [7, 6, 8, 21]

>Input:
solution.solution([[0, 1, 0, 0, 0, 1], [4, 0, 0, 3, 2, 0], [0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0]])
Output:
    [0, 3, 2, 9, 14]

>Use verify [file] to test your solution and see how it does. When you are finished editing your code, use submit [file] to submit your answer. If your solution passes the test cases, it will be removed from your home folder.

I have written the following code to solve it:
</code>```

```<code>import java.util.ArrayList;
public class Solution {
    public static int[] solution(int[][] m) {
        double[][] mDouble = toDouble(m);
        //TODO: change the double back into an int
        // GOAL ONE: find Q matrix :
        // 1:seperate the input into two 2d arrays
        ArrayList<double[]> ters = new ArrayList<double[]>();
        ArrayList<double[]> nonTers = new ArrayList<double[]>();
        for(int i = 0; i < mDouble.length; i++){
            boolean isTerminal = true;
            int sum = 0;
            for(int j = 0; j < mDouble[0].length; j++){
                sum += mDouble[i][j];
                if(mDouble[i][j] != 0){
                    isTerminal = false;
                }
            }

            if(isTerminal){
                ters.add(mDouble[i]);
            }else{
                for(int j = 0; j < mDouble[0].length; j++){
                    mDouble[i][j] = mDouble[i][j]/sum;
                }
                nonTers.add(mDouble[i]);
            }
        }
        double[][] terminalStates = new double[ters.size()][m.length];
        double[][] nonTerminalStates = new double[nonTers.size()][m.length];

        for(int i = 0; i < ters.size(); i++){
            terminalStates[i] = ters.get(i);
        }
        for(int i = 0; i < nonTers.size(); i++){
            nonTerminalStates[i] = nonTers.get(i);
        }
        // 2: Plug into a function that will create the 2d array
        double[][] QMatrix = getQMatrix(nonTerminalStates);
        // GOAL TWO: find I matrix
        double[][] IMatrix = makeIMatrix(QMatrix.length);
        //GOAL 3: Find F matrix
        //1: subtract the q matrix from the I matrix
        double[][] AMatrix = SubtractMatrices(IMatrix, QMatrix);
        //2: find the inverse TODO WRITE FUNCTION
        double[][] FMatrix = invert(AMatrix);
        //GOAL 4: multiply by R Matrix
        //1: find r Matrx
        double[][] RMatrix = getRMatrix(nonTerminalStates, terminalStates.length);
        //2: use multiply function to get FR Matrix
        double[][] FRMatrix = multiplyMatrices(FMatrix, RMatrix);
        //GOAL 5: find answer array
        //1: get the one dimensional answer
        double[] unsimplifiedAns = FRMatrix[0];
        //2: get fractions for the answers
        int[] ans = fractionAns(unsimplifiedAns);
        return ans;
    }
    public static int[] fractionAns(double[] uAns){
        int[] ans = new int[uAns.length + 1];
        int[] denominators = new int[uAns.length];
        int[] numerators = new int[uAns.length];
        for(int i = 0; i < uAns.length; i++){
            denominators[i] = (int)(convertDecimalToFraction(uAns[i])[1]);
            numerators[i] = (int)(convertDecimalToFraction(uAns[i])[0]);
        }
        int lcm = (int) lcm_of_array_elements(denominators);
        for(int i = 0; i < uAns.length; i++){
            ans[i] = numerators[i]*(lcm/convertDecimalToFraction(uAns[i])[1]);
        }
        ans[ans.length-1] = lcm;
        return ans;
    }

    static private int[] convertDecimalToFraction(double x){
        double tolerance = 1.0E-10;
        double h1=1; double h2=0;
        double k1=0; double k2=1;
        double b = x;
        do {
            double a = Math.floor(b);
            double aux = h1; h1 = a*h1+h2; h2 = aux;
            aux = k1; k1 = a*k1+k2; k2 = aux;
            b = 1/(b-a);
        } while (Math.abs(x-h1/k1) > x*tolerance);

        return new int[]{(int)h1, (int)k1};
    }
   public static long lcm_of_array_elements(int[] element_array)
    {
        long lcm_of_array_elements = 1;
        int divisor = 2;

        while (true) {
            int counter = 0;
            boolean divisible = false;

            for (int i = 0; i < element_array.length; i++) {

                // lcm_of_array_elements (n1, n2, ... 0) = 0.
                // For negative number we convert into
                // positive and calculate lcm_of_array_elements.

                if (element_array[i] == 0) {
                    return 0;
                }
                else if (element_array[i] < 0) {
                    element_array[i] = element_array[i] * (-1);
                }
                if (element_array[i] == 1) {
                    counter++;
                }

                // Divide element_array by devisor if complete
                // division i.e. without remainder then replace
                // number with quotient; used for find next factor
                if (element_array[i] % divisor == 0) {
                    divisible = true;
                    element_array[i] = element_array[i] / divisor;
                }
            }

            // If divisor able to completely divide any number
            // from array multiply with lcm_of_array_elements
            // and store into lcm_of_array_elements and continue
            // to same divisor for next factor finding.
            // else increment divisor
            if (divisible) {
                lcm_of_array_elements = lcm_of_array_elements * divisor;
            }
            else {
                divisor++;
            }

            // Check if all element_array is 1 indicate
            // we found all factors and terminate while loop.
            if (counter == element_array.length) {
                return lcm_of_array_elements;
            }
        }
    }
    public static double[][] toDouble(int[][] ma){
        double[][] retArr = new double[ma.length][ma.length];
        for(int i = 0; i < retArr.length; i++){
            for(int j = 0; j < retArr[0].length; j++){
                retArr[i][j] = (ma[i][j]);
            }
        }
        return retArr;
    }
    public static double[][] getRMatrix(double[][] nonTerminals, int terminalLength){
        double[][] retArr = new double[nonTerminals.length][terminalLength];
        for(int i = 0; i < retArr.length; i++){
            for(int j = nonTerminals.length; j < nonTerminals[0].length; j++){
                retArr[i][j-nonTerminals.length] = (nonTerminals[i][j]);
            }
        }
        return retArr;
    }

    public static double[][] multiplyMatrices(double[][] firstMatrix, double[][] secondMatrix){
        int r1 = firstMatrix.length;
        int c1 = firstMatrix[0].length;
        int c2 = secondMatrix[0].length;
        double[][] product = new double[r1][c2];
        for(int i = 0; i < r1; i++) {
            for (int j = 0; j < c2; j++) {
                for (int k = 0; k < c1; k++) {
                    product[i][j] += firstMatrix[i][k] * secondMatrix[k][j];
                }
            }
        }

        return product;
    }
    public static double[][] inverseMatrix(double[][] Amatrix){
        return null;
    }
    public static double[][] SubtractMatrices(double[][] I, double[][] Q){
        double[][] retArr = new double[I.length][I.length];
        for(int i = 0; i < retArr.length; i++){
            for(int j = 0; j < retArr.length; j++){
                retArr[i][j] = I[i][j]-Q[i][j];
            }
        }
        return retArr;
    }
    public static double[][] getQMatrix(double[][] qArr){
        int size = qArr.length;
        double[][] retArr = new double[size][size];
        for(int i = 0; i < size; i++){
            for(int j = 0; j < size; j++){
                retArr[i][j] = qArr[i][j];
            }
        }
        return retArr;
    }
    public static double[][] makeIMatrix(int size){
        double[][] retArr = new double[size][size];
        for(int i = 0; i < size; i++){
            for(int j = 0; j < size; j++){
                if(i == j){
                    retArr[i][j] = 1;
                }else{
                    retArr[i][j] = 0;
                }
            }
        }
        return retArr;
    }
    public static double[][] invert(double a[][])
    {
        int n = a.length;
        double x[][] = new double[n][n];
        double b[][] = new double[n][n];
        int index[] = new int[n];
        for (int i=0; i<n; ++i)
            b[i][i] = 1;

 // Transform the matrix into an upper triangle
        gaussian(a, index);

 // Update the matrix b[i][j] with the ratios stored
        for (int i=0; i<n-1; ++i)
            for (int j=i+1; j<n; ++j)
                for (int k=0; k<n; ++k)
                    b[index[j]][k]
                            -= a[index[j]][i]*b[index[i]][k];

 // Perform backward substitutions
        for (int i=0; i<n; ++i)
        {
            x[n-1][i] = b[index[n-1]][i]/a[index[n-1]][n-1];
            for (int j=n-2; j>=0; --j)
            {
                x[j][i] = b[index[j]][i];
                for (int k=j+1; k<n; ++k)
                {
                    x[j][i] -= a[index[j]][k]*x[k][i];
                }
                x[j][i] /= a[index[j]][j];
            }
        }
        return x;
    }

// Method to carry out the partial-pivoting Gaussian
// elimination.  Here index[] stores pivoting order.

    public static void gaussian(double a[][], int index[])
    {
        int n = index.length;
        double c[] = new double[n];

 // Initialize the index
        for (int i=0; i<n; ++i)
            index[i] = i;

 // Find the rescaling factors, one from each row
        for (int i=0; i<n; ++i)
        {
            double c1 = 0;
            for (int j=0; j<n; ++j)
            {
                double c0 = Math.abs(a[i][j]);
                if (c0 > c1) c1 = c0;
            }
            c[i] = c1;
        }

 // Search the pivoting element from each column
        int k = 0;
        for (int j=0; j<n-1; ++j)
        {
            double pi1 = 0;
            for (int i=j; i<n; ++i)
            {
                double pi0 = Math.abs(a[index[i]][j]);
                pi0 /= c[index[i]];
                if (pi0 > pi1)
                {
                    pi1 = pi0;
                    k = i;
                }
            }

   // Interchange rows according to the pivoting order
            int itmp = index[j];
            index[j] = index[k];
            index[k] = itmp;
            for (int i=j+1; i<n; ++i)
            {
                double pj = a[index[i]][j]/a[index[j]][j];

 // Record pivoting ratios below the diagonal
                a[index[i]][j] = pj;

 // Modify other elements accordingly
                for (int l=j+1; l<n; ++l)
                    a[index[i]][l] -= pj*a[index[j]][l];
            }
        }
    }

}
</code>```

It passes all the test cases but two hidden ones I cannot see.

I've tried all the test cases I possibly could to find the fault in my code but I cannot.

Are there any test cases here where my code fails?

答:
The problem lies in the line

```<code>double[] unsimplifiedAns = FRMatrix[0];
</code>```

The above is true only if state 0 is non-terminating.

Otherwise the output array will be all '0's except the first and last element as '1'.

Body:
I'm working my way through the Google Foobar challenge and am now at the level 3 challenge Doomsday Fuel. The instructions are as follows:

Doomsday Fuel

Making fuel for the LAMBCHOP's reactor core is a tricky process because of the exotic matter involved. It starts as raw ore, then during processing, begins randomly changing between forms, eventually reaching a stable form. There may be multiple stable forms that a sample could ultimately reach, not all of which are useful as fuel.

Commander Lambda has tasked you to help the scientists increase fuel creation efficiency by predicting the end state of a given ore sample. You have carefully studied the different structures that the ore can take and which transitions it undergoes. It appears that, while random, the probability of each structure transforming is fixed. That is, each time the ore is in 1 state, it has the same probabilities of entering the next state (which might be the same state).  You have recorded the observed transitions in a matrix. The others in the lab have hypothesized more exotic forms that the ore can become, but you haven't seen all of them.

Write a function solution(m) that takes an array of array of nonnegative ints representing how many times that state has gone to the next state and return an array of ints for each terminal state giving the exact probabilities of each terminal state, represented as the numerator for each state, then the denominator for all of them at the end and in simplest form. The matrix is at most 10 by 10. It is guaranteed that no matter which state the ore is in, there is a path from that state to a terminal state. That is, the processing will always eventually end in a stable state. The ore starts in state 0. The denominator will fit within a signed 32-bit integer during the calculation, as long as the fraction is simplified regularly.

>For example, consider the matrix m:
[
  [0,1,0,0,0,1],  # s0, the initial state, goes to s1 and s5 with equal probability
  [4,0,0,3,2,0],  # s1 can become s0, s3, or s4, but with different probabilities
  [0,0,0,0,0,0],  # s2 is terminal, and unreachable (never observed in practice)
  [0,0,0,0,0,0],  # s3 is terminal
  [0,0,0,0,0,0],  # s4 is terminal
  [0,0,0,0,0,0],  # s5 is terminal
]
So, we can consider different paths to terminal states, such as:
s0 -> s1 -> s3
s0 -> s1 -> s0 -> s1 -> s0 -> s1 -> s4
s0 -> s1 -> s0 -> s5
Tracing the probabilities of each, we find that
s2 has probability 0
s3 has probability 3/14
s4 has probability 1/7
s5 has probability 9/14
So, putting that together, and making a common denominator, gives an answer in the form of
[s2.numerator, s3.numerator, s4.numerator, s5.numerator, denominator] which is
[0, 3, 2, 9, 14].

Languages

To provide a Java solution, edit Solution.java
  To provide a Python solution, edit solution.py

Test cases
==========
>Your code should pass the following test cases.
Note that it may also be run against hidden test cases not shown here.

>-- Java cases --
Input:
Solution.solution({{0, 2, 1, 0, 0}, {0, 0, 0, 3, 4}, {0, 0, 0, 0, 0}, {0, 0, 0, 0,0}, {0, 0, 0, 0, 0}})
Output:
    [7, 6, 8, 21]

>Input:
Solution.solution({{0, 1, 0, 0, 0, 1}, {4, 0, 0, 3, 2, 0}, {0, 0, 0, 0, 0, 0}, {0, 0, 0, 0, 0, 0}, {0, 0, 0, 0, 0, 0}, {0, 0, 0, 0, 0, 0}})
Output:
    [0, 3, 2, 9, 14]

>-- Python cases --
Input:
solution.solution([[0, 2, 1, 0, 0], [0, 0, 0, 3, 4], [0, 0, 0, 0, 0], [0, 0, 0, 0,0], [0, 0, 0, 0, 0]])
Output:
    [7, 6, 8, 21]

>Input:
solution.solution([[0, 1, 0, 0, 0, 1], [4, 0, 0, 3, 2, 0], [0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0]])
Output:
    [0, 3, 2, 9, 14]

>Use verify [file] to test your solution and see how it does. When you are finished editing your code, use submit [file] to submit your answer. If your solution passes the test cases, it will be removed from your home folder.

I have written the following code to solve it:

import java.util.ArrayList;
public class Solution {
    public static int[] solution(int[][] m) {
        double[][] mDouble = toDouble(m);
        //TODO: change the double back into an int
        // GOAL ONE: find Q matrix :
        // 1:seperate the input into two 2d arrays
        ArrayList<double[]> ters = new ArrayList<double[]>();
        ArrayList<double[]> nonTers = new ArrayList<double[]>();
        for(int i = 0; i < mDouble.length; i++){
            boolean isTerminal = true;
            int sum = 0;
            for(int j = 0; j < mDouble[0].length; j++){
                sum += mDouble[i][j];
                if(mDouble[i][j] != 0){
                    isTerminal = false;
                }
            }

            if(isTerminal){
                ters.add(mDouble[i]);
            }else{
                for(int j = 0; j < mDouble[0].length; j++){
                    mDouble[i][j] = mDouble[i][j]/sum;
                }
                nonTers.add(mDouble[i]);
            }
        }
        double[][] terminalStates = new double[ters.size()][m.length];
        double[][] nonTerminalStates = new double[nonTers.size()][m.length];

        for(int i = 0; i < ters.size(); i++){
            terminalStates[i] = ters.get(i);
        }
        for(int i = 0; i < nonTers.size(); i++){
            nonTerminalStates[i] = nonTers.get(i);
        }
        // 2: Plug into a function that will create the 2d array
        double[][] QMatrix = getQMatrix(nonTerminalStates);
        // GOAL TWO: find I matrix
        double[][] IMatrix = makeIMatrix(QMatrix.length);
        //GOAL 3: Find F matrix
        //1: subtract the q matrix from the I matrix
        double[][] AMatrix = SubtractMatrices(IMatrix, QMatrix);
        //2: find the inverse TODO WRITE FUNCTION
        double[][] FMatrix = invert(AMatrix);
        //GOAL 4: multiply by R Matrix
        //1: find r Matrx
        double[][] RMatrix = getRMatrix(nonTerminalStates, terminalStates.length);
        //2: use multiply function to get FR Matrix
        double[][] FRMatrix = multiplyMatrices(FMatrix, RMatrix);
        //GOAL 5: find answer array
        //1: get the one dimensional answer
        double[] unsimplifiedAns = FRMatrix[0];
        //2: get fractions for the answers
        int[] ans = fractionAns(unsimplifiedAns);
        return ans;
    }
    public static int[] fractionAns(double[] uAns){
        int[] ans = new int[uAns.length + 1];
        int[] denominators = new int[uAns.length];
        int[] numerators = new int[uAns.length];
        for(int i = 0; i < uAns.length; i++){
            denominators[i] = (int)(convertDecimalToFraction(uAns[i])[1]);
            numerators[i] = (int)(convertDecimalToFraction(uAns[i])[0]);
        }
        int lcm = (int) lcm_of_array_elements(denominators);
        for(int i = 0; i < uAns.length; i++){
            ans[i] = numerators[i]*(lcm/convertDecimalToFraction(uAns[i])[1]);
        }
        ans[ans.length-1] = lcm;
        return ans;
    }

    static private int[] convertDecimalToFraction(double x){
        double tolerance = 1.0E-10;
        double h1=1; double h2=0;
        double k1=0; double k2=1;
        double b = x;
        do {
            double a = Math.floor(b);
            double aux = h1; h1 = a*h1+h2; h2 = aux;
            aux = k1; k1 = a*k1+k2; k2 = aux;
            b = 1/(b-a);
        } while (Math.abs(x-h1/k1) > x*tolerance);

        return new int[]{(int)h1, (int)k1};
    }
   public static long lcm_of_array_elements(int[] element_array)
    {
        long lcm_of_array_elements = 1;
        int divisor = 2;

        while (true) {
            int counter = 0;
            boolean divisible = false;

            for (int i = 0; i < element_array.length; i++) {

                // lcm_of_array_elements (n1, n2, ... 0) = 0.
                // For negative number we convert into
                // positive and calculate lcm_of_array_elements.

                if (element_array[i] == 0) {
                    return 0;
                }
                else if (element_array[i] < 0) {
                    element_array[i] = element_array[i] * (-1);
                }
                if (element_array[i] == 1) {
                    counter++;
                }

                // Divide element_array by devisor if complete
                // division i.e. without remainder then replace
                // number with quotient; used for find next factor
                if (element_array[i] % divisor == 0) {
                    divisible = true;
                    element_array[i] = element_array[i] / divisor;
                }
            }

            // If divisor able to completely divide any number
            // from array multiply with lcm_of_array_elements
            // and store into lcm_of_array_elements and continue
            // to same divisor for next factor finding.
            // else increment divisor
            if (divisible) {
                lcm_of_array_elements = lcm_of_array_elements * divisor;
            }
            else {
                divisor++;
            }

            // Check if all element_array is 1 indicate
            // we found all factors and terminate while loop.
            if (counter == element_array.length) {
                return lcm_of_array_elements;
            }
        }
    }
    public static double[][] toDouble(int[][] ma){
        double[][] retArr = new double[ma.length][ma.length];
        for(int i = 0; i < retArr.length; i++){
            for(int j = 0; j < retArr[0].length; j++){
                retArr[i][j] = (ma[i][j]);
            }
        }
        return retArr;
    }
    public static double[][] getRMatrix(double[][] nonTerminals, int terminalLength){
        double[][] retArr = new double[nonTerminals.length][terminalLength];
        for(int i = 0; i < retArr.length; i++){
            for(int j = nonTerminals.length; j < nonTerminals[0].length; j++){
                retArr[i][j-nonTerminals.length] = (nonTerminals[i][j]);
            }
        }
        return retArr;
    }

    public static double[][] multiplyMatrices(double[][] firstMatrix, double[][] secondMatrix){
        int r1 = firstMatrix.length;
        int c1 = firstMatrix[0].length;
        int c2 = secondMatrix[0].length;
        double[][] product = new double[r1][c2];
        for(int i = 0; i < r1; i++) {
            for (int j = 0; j < c2; j++) {
                for (int k = 0; k < c1; k++) {
                    product[i][j] += firstMatrix[i][k] * secondMatrix[k][j];
                }
            }
        }

        return product;
    }
    public static double[][] inverseMatrix(double[][] Amatrix){
        return null;
    }
    public static double[][] SubtractMatrices(double[][] I, double[][] Q){
        double[][] retArr = new double[I.length][I.length];
        for(int i = 0; i < retArr.length; i++){
            for(int j = 0; j < retArr.length; j++){
                retArr[i][j] = I[i][j]-Q[i][j];
            }
        }
        return retArr;
    }
    public static double[][] getQMatrix(double[][] qArr){
        int size = qArr.length;
        double[][] retArr = new double[size][size];
        for(int i = 0; i < size; i++){
            for(int j = 0; j < size; j++){
                retArr[i][j] = qArr[i][j];
            }
        }
        return retArr;
    }
    public static double[][] makeIMatrix(int size){
        double[][] retArr = new double[size][size];
        for(int i = 0; i < size; i++){
            for(int j = 0; j < size; j++){
                if(i == j){
                    retArr[i][j] = 1;
                }else{
                    retArr[i][j] = 0;
                }
            }
        }
        return retArr;
    }
    public static double[][] invert(double a[][])
    {
        int n = a.length;
        double x[][] = new double[n][n];
        double b[][] = new double[n][n];
        int index[] = new int[n];
        for (int i=0; i<n; ++i)
            b[i][i] = 1;

 // Transform the matrix into an upper triangle
        gaussian(a, index);

 // Update the matrix b[i][j] with the ratios stored
        for (int i=0; i<n-1; ++i)
            for (int j=i+1; j<n; ++j)
                for (int k=0; k<n; ++k)
                    b[index[j]][k]
                            -= a[index[j]][i]*b[index[i]][k];

 // Perform backward substitutions
        for (int i=0; i<n; ++i)
        {
            x[n-1][i] = b[index[n-1]][i]/a[index[n-1]][n-1];
            for (int j=n-2; j>=0; --j)
            {
                x[j][i] = b[index[j]][i];
                for (int k=j+1; k<n; ++k)
                {
                    x[j][i] -= a[index[j]][k]*x[k][i];
                }
                x[j][i] /= a[index[j]][j];
            }
        }
        return x;
    }

// Method to carry out the partial-pivoting Gaussian
// elimination.  Here index[] stores pivoting order.

    public static void gaussian(double a[][], int index[])
    {
        int n = index.length;
        double c[] = new double[n];

 // Initialize the index
        for (int i=0; i<n; ++i)
            index[i] = i;

 // Find the rescaling factors, one from each row
        for (int i=0; i<n; ++i)
        {
            double c1 = 0;
            for (int j=0; j<n; ++j)
            {
                double c0 = Math.abs(a[i][j]);
                if (c0 > c1) c1 = c0;
            }
            c[i] = c1;
        }

 // Search the pivoting element from each column
        int k = 0;
        for (int j=0; j<n-1; ++j)
        {
            double pi1 = 0;
            for (int i=j; i<n; ++i)
            {
                double pi0 = Math.abs(a[index[i]][j]);
                pi0 /= c[index[i]];
                if (pi0 > pi1)
                {
                    pi1 = pi0;
                    k = i;
                }
            }

   // Interchange rows according to the pivoting order
            int itmp = index[j];
            index[j] = index[k];
            index[k] = itmp;
            for (int i=j+1; i<n; ++i)
            {
                double pj = a[index[i]][j]/a[index[j]][j];

 // Record pivoting ratios below the diagonal
                a[index[i]][j] = pj;

 // Modify other elements accordingly
                for (int l=j+1; l<n; ++l)
                    a[index[i]][l] -= pj*a[index[j]][l];
            }
        }
    }

}

It passes all the test cases but two hidden ones I cannot see.

I've tried all the test cases I possibly could to find the fault in my code but I cannot.

Are there any test cases here where my code fails?

Body:
The problem lies in the line

double[] unsimplifiedAns = FRMatrix[0];

The above is true only if state 0 is non-terminating.

Otherwise the output array will be all '0's except the first and last element as '1'.


来源: stackexchange

问:
Advice on domain modeling
New to DDD here and have a architecture question which should be a typical problem.

I have a StockItem entity and a Store entity. I assign a StockItem to multiple Stores and set different ParLevels for each store. I then need to have transactions (sale, purchase, transfer-in etc) that adjust the quantity on hand at the various store.

What is the preferred way to model this solution? In my DB I would have the following simplied table structure:
[StockItem]
StockItemID pk,
Name

[Store]
StoreID pk,
Name

[Store_StockItem]
StoreStockItemID pk,
StoreID fk,
StockItemID fk,
ParlLevel,
QuantityOnHand

[StockItemTransaction]
StockItemTransactionID pk,
StockItemStoreID fk,
StockItemTransactionTypeID fk,
Quantity

Any advice greatly appreciated

答:
I don't think that anyone here can really give you the answer you want, because the whole point of Domain Driven Design is that your own specific business domain should guide the object model.

The only piece of advice I would offer is that you shouldn't worry about how the database models these relationships when working on the domain. You should normalise your database according to the rules of normalisation, and create your domain model according to the actual business domain.

Remember that the data model and the domain model don't map to each other in a 1:1 way.

OwnerDisplayName: Chev

ContentLicense: CC BY-SA 2.5

Title: Advice on domain modeling

OwnerDisplayName: RibaldEddie


问:
Android listview highlight issue
I have to do a listing for items. Here is my XML for layout:

```<code><?xml version="1.0" encoding="utf-8"?>
<TableLayout
        xmlns:android="http://schemas.android.com/apk/res/android"
        android:layout_width="fill_parent"
        android:layout_height="fill_parent"
        android:gravity="center_horizontal"
        android:background="#FFFFFF"
        android:stretchColumns="0">
        <TableRow
            android:layout_width="fill_parent"
            android:layout_height="fill_parent"
            android:layout_gravity="center_horizontal"
            android:id="@+id/listbox">
            <LinearLayout
                android:layout_width="fill_parent"
                android:background="#FFFFFF"
                android:layout_gravity="center_horizontal"
                android:layout_height="fill_parent">
                    <ListView
                        android:id="@+id/android:list"
                        android:layout_width="fill_parent"
                        android:layout_height="fill_parent">
                    </ListView>
                    <LinearLayout
                        android:id="@android:id/empty"
                        android:layout_width="fill_parent"
                        android:background="#FFFFFF"
                        android:layout_gravity="center"
                        android:layout_height="fill_parent">
                        <TextView
                            android:id="@+id/emptytext"
                            android:paddingTop="100dp"
                            android:layout_width="fill_parent"
                            android:layout_height="fill_parent"
                            android:textColor="#818185"
                            android:textSize="25sp"
                            android:textStyle="bold"
                            android:background="#FFFFFF"
                            android:text="@string/empty_fav"
                            android:gravity="center_horizontal|center_vertical"
                            android:paddingLeft="6dip"
                        />
                    </LinearLayout>
            </LinearLayout>
        </TableRow>
</TableLayout>
</code>```

It is the xml for R.layout.custom_list_item_1.

```<code><TextView xmlns:android="http://schemas.android.com/apk/res/android"
    android:id="@android:id/text1"
    android:layout_width="fill_parent"
    android:layout_height="wrap_content"
    android:textColor="#000000"
    android:textSize="18sp"
    android:textStyle="bold"
    android:background="#FFFFFF"
    android:textAppearance="?android:attr/textAppearanceLarge"
    android:gravity="center_vertical"
    android:paddingLeft="6dip"
    android:minHeight="?android:attr/listPreferredItemHeight"
/>
</code>```

And here is my code.

```<code>public void onCreate(Bundle savedInstanceState) {

        super.onCreate(savedInstanceState);

        setContentView(R.layout.fav_layout);
        ....
        ........

        wordDataHelper = new WordDataHelper(getApplicationContext());
        favCursor  = wordDataHelper.getCursorFav();
        startManagingCursor(favCursor);

        favAdapter = new SimpleCursorAdapter(
                this,
                R.layout.custom_list_item_1,
                favCursor,
                new String[] {WordDataHelper.ENGWORD},
                new int[] {android.R.id.text1});

        setListAdapter(favAdapter);

        list = getListView();

        list.setOnItemClickListener(new OnItemClickListener() {

            @Override
            public void onItemClick(AdapterView<?> parent, View view, int position,
                    long id) {
                // TODO Auto-generated method stub

                showVideo(((TextView) view).getText());

            }});

        list.setOnCreateContextMenuListener(new OnCreateContextMenuListener() {
            // @Override
            public void onCreateContextMenu(ContextMenu menu, View v,
                    ContextMenu.ContextMenuInfo menuInfo) {
                menu.setHeaderTitle("Context Menu");
                menu.add(0, CONTEXT_DELETE, 1, "Delete Item");
            }
        });

        list.setTextFilterEnabled(true);

        list.setClickable(true);
}
</code>```

Now everything is showing fine and all other functionality is also
working fine. But if I am touching the any row, it is not highlighting
i.e. the color is not changing. Is there any problem in my code or
does it have to be implement separately? If so, how?

答:
You probably need to set the

```<code>android:listSelector
</code>```

attribute of the ListView.

Check this answer Android ListView Selector Color, it's well explained.

If that didn't work out (I haven't tried it), try setting a color selector as the background of the TextView defined in R.layout.custom_list_item_1

ContentLicense: CC BY-SA 3.0

Title: Android listview highlight issue

Body:
I have to do a listing for items. Here is my XML for layout:

<?xml version="1.0" encoding="utf-8"?>
<TableLayout
        xmlns:android="http://schemas.android.com/apk/res/android"
        android:layout_width="fill_parent"
        android:layout_height="fill_parent"
        android:gravity="center_horizontal"
        android:background="#FFFFFF"
        android:stretchColumns="0">
        <TableRow
            android:layout_width="fill_parent"
            android:layout_height="fill_parent"
            android:layout_gravity="center_horizontal"
            android:id="@+id/listbox">
            <LinearLayout
                android:layout_width="fill_parent"
                android:background="#FFFFFF"
                android:layout_gravity="center_horizontal"
                android:layout_height="fill_parent">
                    <ListView
                        android:id="@+id/android:list"
                        android:layout_width="fill_parent"
                        android:layout_height="fill_parent">
                    </ListView>
                    <LinearLayout
                        android:id="@android:id/empty"
                        android:layout_width="fill_parent"
                        android:background="#FFFFFF"
                        android:layout_gravity="center"
                        android:layout_height="fill_parent">
                        <TextView
                            android:id="@+id/emptytext"
                            android:paddingTop="100dp"
                            android:layout_width="fill_parent"
                            android:layout_height="fill_parent"
                            android:textColor="#818185"
                            android:textSize="25sp"
                            android:textStyle="bold"
                            android:background="#FFFFFF"
                            android:text="@string/empty_fav"
                            android:gravity="center_horizontal|center_vertical"
                            android:paddingLeft="6dip"
                        />
                    </LinearLayout>
            </LinearLayout>
        </TableRow>
</TableLayout>

It is the xml for R.layout.custom_list_item_1.

<TextView xmlns:android="http://schemas.android.com/apk/res/android"
    android:id="@android:id/text1"
    android:layout_width="fill_parent"
    android:layout_height="wrap_content"
    android:textColor="#000000"
    android:textSize="18sp"
    android:textStyle="bold"
    android:background="#FFFFFF"
    android:textAppearance="?android:attr/textAppearanceLarge"
    android:gravity="center_vertical"
    android:paddingLeft="6dip"
    android:minHeight="?android:attr/listPreferredItemHeight"
/>

And here is my code.

public void onCreate(Bundle savedInstanceState) {

        super.onCreate(savedInstanceState);

        setContentView(R.layout.fav_layout);
        ....
        ........

        wordDataHelper = new WordDataHelper(getApplicationContext());
        favCursor  = wordDataHelper.getCursorFav();
        startManagingCursor(favCursor);

        favAdapter = new SimpleCursorAdapter(
                this,
                R.layout.custom_list_item_1,
                favCursor,
                new String[] {WordDataHelper.ENGWORD},
                new int[] {android.R.id.text1});

        setListAdapter(favAdapter);

        list = getListView();

        list.setOnItemClickListener(new OnItemClickListener() {

            @Override
            public void onItemClick(AdapterView<?> parent, View view, int position,
                    long id) {
                // TODO Auto-generated method stub

                showVideo(((TextView) view).getText());

            }});

        list.setOnCreateContextMenuListener(new OnCreateContextMenuListener() {
            // @Override
            public void onCreateContextMenu(ContextMenu menu, View v,
                    ContextMenu.ContextMenuInfo menuInfo) {
                menu.setHeaderTitle("Context Menu");
                menu.add(0, CONTEXT_DELETE, 1, "Delete Item");
            }
        });

        list.setTextFilterEnabled(true);

        list.setClickable(true);
}

Now everything is showing fine and all other functionality is also
working fine. But if I am touching the any row, it is not highlighting
i.e. the color is not changing. Is there any problem in my code or
does it have to be implement separately? If so, how?

Body:
You probably need to set the

android:listSelector

attribute of the ListView.

Check this answer Android ListView Selector Color, it's well explained.

If that didn't work out (I haven't tried it), try setting a color selector as the background of the TextView defined in R.layout.custom_list_item_1


问:
Detox animation settings
Quick question about 16.6.0

In the release note of 16.6.0 is says "Disabling all native-Android animations on Android emulator"

Does that mean I don't have to set these globals anymore?

```<code>adb shell "settings put global window_animation_scale 0.0"
adb shell "settings put global transition_animation_scale 0.0"
adb shell "settings put global animator_duration_scale 0.0"
</code>```

Thanks

答: Indeed, this is exactly what it means. More so, Detox will ensure this will be applied even in an environment running tests on multiple emulators, concurrently.</p>

ContentLicense: CC BY-SA 4.0

Title: Detox animation settings

Body:
Quick question about 16.6.0

In the release note of 16.6.0 is says "Disabling all native-Android animations on Android emulator"

Does that mean I don't have to set these globals anymore?

adb shell "settings put global window_animation_scale 0.0"
adb shell "settings put global transition_animation_scale 0.0"
adb shell "settings put global animator_duration_scale 0.0"

Thanks


问:
Is there a way to tell how many tables are aggregated by wildcards in BigQuery Standard SQL?
Say I have tables in BigQuery with the format table_name_YYYYMMDD. I know in standard SQL you can use wildcards with _TABLE_SUFFIX to select a range of tables in a query like this:

```<code>SELECT
  user_id
FROM `project.dataset.table_name_*`
WHERE _TABLE_SUFFIX BETWEEN '20200520' AND '20200525'
</code>```

My question is whether there is a way to tell the number of tables that were actually selected in the end (i.e. 20200520, 20200521, 20200522, 20200523, 20200524, 20200525 would be 6 tables).

In my case, I have a query that uses 30 tables to create a monthly summary, but sometimes one or more of those 30 tables might not exist, in which case I would like to throw an error instead of silently querying less than the 30 tables I expect. (I know this can be done in Legacy SQL with TABLE_DATE_RANGE but I was wondering if there is a way to accomplish it in Standard SQL).

答:
In order to further contribute to the community and in addition to Mikhail's answer, I would like to provide you two other options.

First using the Python API for BigQuery. I created the following script:

```<code>import argparse
from google.cloud import bigquery

client = bigquery.Client()
dataset_ref = client.dataset("dataset_name", project="project_id")
dataset = client.get_dataset(dataset_ref)

parser = argparse.ArgumentParser()
parser.add_argument('-t', type=int,
                    help='number of tables that we need to be queried')

parsed = parser.parse_args()
#retrieve the integer you passed through the command line
arg = parsed.t

def check_tables(arg):
    #counting the number of tables give the desired wildcard
    query = """SELECT COUNT(DISTINCT _TABLE_SUFFIX)
                    FROM `project.dataset.table_*`
                    """
    query_job = client.query(query)
    result= query_job.result()
    for r in result:
        count=r[0]
        print("There are {} tables".format(count))
    if count==int(arg):
        return True
    else:
        return False

if check_tables(arg):
    print("In total BigQuery is querying {} tables".format(arg))
    #your new SQL query here, which will execute only if there are the amount of tables you specified
    #you can output the results to a new table
</code>```

As shown above, the script used argparse library from Python. Therefore, in order to run the script, you have to use python script_name.py <int64>, in my case I ran python query.py 3. In other words, you have to specify the number of tables you are expecting to query using wildcards. The script checks how many tables there are with the given wildcard. Then, it compares with the number of tables we expect and if the numbers are equal you can execute another query (after the last if). Otherwise, output a message.

The second option is using the BigQuery Console. We will be taking advantage of Scripting. The script will basically check the number of tables within the desired wildcard, evaluate if there are the expected number of tables. Then, if there are, execute the query. The syntax is below:

```<code>DECLARE num_tables INT64 DEFAULT 3;
DECLARE count_tables INT64;
SET count_tables = (SELECT COUNT(DISTINCT _TABLE_SUFFIX) FROM `test-proj-261014.sample.abril_airports_*`);

IF(num_tables = count_tables) THEN
SELECT *  FROM `project.dataset.table_*`;
END IF;
</code>```

Body:
Say I have tables in BigQuery with the format table_name_YYYYMMDD. I know in standard SQL you can use wildcards with _TABLE_SUFFIX to select a range of tables in a query like this:

SELECT
  user_id
FROM `project.dataset.table_name_*`
WHERE _TABLE_SUFFIX BETWEEN '20200520' AND '20200525'

My question is whether there is a way to tell the number of tables that were actually selected in the end (i.e. 20200520, 20200521, 20200522, 20200523, 20200524, 20200525 would be 6 tables).

In my case, I have a query that uses 30 tables to create a monthly summary, but sometimes one or more of those 30 tables might not exist, in which case I would like to throw an error instead of silently querying less than the 30 tables I expect. (I know this can be done in Legacy SQL with TABLE_DATE_RANGE but I was wondering if there is a way to accomplish it in Standard SQL).

Body:
In order to further contribute to the community and in addition to Mikhail's answer, I would like to provide you two other options.

First using the Python API for BigQuery. I created the following script:

import argparse
from google.cloud import bigquery

client = bigquery.Client()
dataset_ref = client.dataset("dataset_name", project="project_id")
dataset = client.get_dataset(dataset_ref)

parser = argparse.ArgumentParser()
parser.add_argument('-t', type=int,
                    help='number of tables that we need to be queried')

parsed = parser.parse_args()
#retrieve the integer you passed through the command line
arg = parsed.t

def check_tables(arg):
    #counting the number of tables give the desired wildcard
    query = """SELECT COUNT(DISTINCT _TABLE_SUFFIX)
                    FROM `project.dataset.table_*`
                    """
    query_job = client.query(query)
    result= query_job.result()
    for r in result:
        count=r[0]
        print("There are {} tables".format(count))
    if count==int(arg):
        return True
    else:
        return False

if check_tables(arg):
    print("In total BigQuery is querying {} tables".format(arg))
    #your new SQL query here, which will execute only if there are the amount of tables you specified
    #you can output the results to a new table

As shown above, the script used argparse library from Python. Therefore, in order to run the script, you have to use python script_name.py <int64>, in my case I ran python query.py 3. In other words, you have to specify the number of tables you are expecting to query using wildcards. The script checks how many tables there are with the given wildcard. Then, it compares with the number of tables we expect and if the numbers are equal you can execute another query (after the last if). Otherwise, output a message.

The second option is using the BigQuery Console. We will be taking advantage of Scripting. The script will basically check the number of tables within the desired wildcard, evaluate if there are the expected number of tables. Then, if there are, execute the query. The syntax is below:

DECLARE num_tables INT64 DEFAULT 3;
DECLARE count_tables INT64;
SET count_tables = (SELECT COUNT(DISTINCT _TABLE_SUFFIX) FROM `test-proj-261014.sample.abril_airports_*`);

IF(num_tables = count_tables) THEN
SELECT *  FROM `project.dataset.table_*`;
END IF;


问:
R fill in new column by filtered value by group
I want to create new column in my dat, filled in by values
satisfying a condition. So, I want to keep the value where where regime == "SA" and fill this value over the whole group in new column vals.sa.

This combines the filter and mutate, but I am unsure what is their correct combination? Filter just keeps rows satisfying condition, but how can I expand this value over all rows in a group?

My data:

```<code>dat <- data.frame(id = c(1,1,2,2,2,2,3,3),
                  regime = c("SA", "B", "SA", "B", "C", "F", "SA", "D"),
                  value = c(3,5,1,2,5,6,7,8))
</code>```

Process the data using dplyr

```<code>dat %>%
  # group data by id
  group_by(id) %>%
  # how to write this condition and get values instead or TRUE/FALSE?
  mutate(val.sa = regime == "SA")
</code>```

Expected output with filled in val.sa value corresponding regime == "SA"

```<code>     id regime value val.sa
  <dbl> <fct>  <dbl>  <dbl>
1     1 SA         3      3
2     1 B          5      3
3     2 SA         1      1
4     2 B          2      1
5     2 C          5      1
6     2 F          6      1
7     3 SA         7      7
8     3 D          8      7
</code>```

答:
Create a new column val.sa first with the value in the SA rows equal to the values in the value column. Then group by id and fill down. Like this:

```<code>dat %>%
  mutate(val.sa = ifelse(regime == 'SA', value, NA)) %>%
  group_by(id) %>%
  fill(val.sa)
## A tibble: 8 x 4
## Groups:   id [3]
#     id regime value val.sa
#  <dbl> <chr>  <dbl>  <dbl>
#1     1 SA         3      3
#2     1 B          5      3
#3     2 SA         1      1
#4     2 B          2      1
#5     2 C          5      1
#6     2 F          6      1
#7     3 SA         7      7
#8     3 D          8      7
</code>```

Body:
I want to create new column in my dat, filled in by values
satisfying a condition. So, I want to keep the value where where regime == "SA" and fill this value over the whole group in new column vals.sa.

This combines the filter and mutate, but I am unsure what is their correct combination? Filter just keeps rows satisfying condition, but how can I expand this value over all rows in a group?

My data:

dat <- data.frame(id = c(1,1,2,2,2,2,3,3),
                  regime = c("SA", "B", "SA", "B", "C", "F", "SA", "D"),
                  value = c(3,5,1,2,5,6,7,8))

Process the data using dplyr

dat %>%
  # group data by id
  group_by(id) %>%
  # how to write this condition and get values instead or TRUE/FALSE?
  mutate(val.sa = regime == "SA")

Expected output with filled in val.sa value corresponding regime == "SA"

     id regime value val.sa
  <dbl> <fct>  <dbl>  <dbl>
1     1 SA         3      3
2     1 B          5      3
3     2 SA         1      1
4     2 B          2      1
5     2 C          5      1
6     2 F          6      1
7     3 SA         7      7
8     3 D          8      7

Body:
Create a new column val.sa first with the value in the SA rows equal to the values in the value column. Then group by id and fill down. Like this:

dat %>%
  mutate(val.sa = ifelse(regime == 'SA', value, NA)) %>%
  group_by(id) %>%
  fill(val.sa)
## A tibble: 8 x 4
## Groups:   id [3]
#     id regime value val.sa
#  <dbl> <chr>  <dbl>  <dbl>
#1     1 SA         3      3
#2     1 B          5      3
#3     2 SA         1      1
#4     2 B          2      1
#5     2 C          5      1
#6     2 F          6      1
#7     3 SA         7      7
#8     3 D          8      7


问:
spring batch read jobParameters from command line and use it in job config
EDITS BASED ON SUGGESTION:
For brevity, I will remove older code and long part and re-phrase the issue.

I am trying to build the app (Spring boot + Spring Batch) taking the date and config information from command line. Based on suggestions, I can use the application properties?
The main aim is to use the same job (task of the job) to download different files form different host/time etc. So, properties file can give the information to use for download and compiled jar should read the info and do its tasks.

Main Entry point.

```<code>@SpringBootApplication
public class CoreApplication implements ApplicationRunner {

    @Autowired
    JobLauncher jobLauncher;

    @Autowired
    Job processJob;

    @Value("${rundate}")
    private String run_date;

    private static final Logger logger = LoggerFactory.getLogger(CoreApplication.class);

    public static void main(String[] args) {
        SpringApplication.run(CoreApplication.class, args);
    }

    @Override
    public void run(ApplicationArguments args) throws Exception {

        JobParameters jobParameters = new JobParametersBuilder()
                .addLong("JobID", System.currentTimeMillis())
                .addString("RunDate", run_date)
                .toJobParameters();

        try {
            jobLauncher.run(processJob, jobParameters);
        } catch (Exception e) {
            logger.error("Exception while running a batch job {}", e.getMessage());
        }

    }

}
</code>```

I rearranged the code, to use the values of server, user, etc from application.properties file. Please let me know if it is wrong way to inject the properties.

application.properties file:

```<code>spring.datasource.url=jdbc:postgresql://dbhost:1000/db
spring.datasource.username=username
spring.datasource.password=password
spring.datasource.platform=postgresql
spring.batch.job.enabled=false
local.directory="/my/local/path/"
file.name="file_name_20200601.csv"
remote.directory="/remote/ftp/location"
remote.host="remotehost"
remote.port=22
remote.user="remoteuser"
private.key.location="/key/file/location"
</code>```

My Batch Configuration:

```<code>@Configuration
@EnableBatchProcessing
@EnableIntegration
@EnableAutoConfiguration
public class BatchConfiguration {
    private Logger logger = LoggerFactory.getLogger(BatchConfiguration.class);

    @Autowired
    public JobBuilderFactory jobBuilderFactory;

    @Autowired
    public StepBuilderFactory stepBuilderFactory;

    @Bean
    public Job ftpJob() {
        return jobBuilderFactory.get("FTP Job")
                .incrementer(new RunIdIncrementer())
                .start(getFilesFromFTPServer())
                .build();
    }

    @Bean
    public Step getFilesFromFTPServer() {
        return stepBuilderFactory.get("Get file from server")
                .tasklet(new RemoteFileInboundTasklet())
                .build();

    }
}
</code>```

My Tasklet:

public class RemoteFileInboundTasklet implements Tasklet {

```<code>private Logger logger = LoggerFactory.getLogger(RemoteFileInboundTasklet.class);

@Value("${file.name}")
private String fileNamePattern;

private String clientName;
private boolean deleteLocalFiles = true;
private boolean retryIfNotFound = false;

@Value("${local.directory}")
private String local_directory_value;

private File localDirectory;
private int downloadFileAttempts = 12;
private long retryIntervalMilliseconds = 300000;

@Value("${remote.directory}")
private String remoteDirectory;

@Value("${remote.host}")
private String remoteHost;

@Value("${remote.user}")
private String remoteUser;

@Value("${remote.port}")
private int remotePort;

@Value("${private.key.location}")
private String private_key_file;

public SessionFactory<ChannelSftp.LsEntry> clientSessionFactory() {
    DefaultSftpSessionFactory ftpSessionFactory = new DefaultSftpSessionFactory();
    ftpSessionFactory.setHost(remoteHost);
    ftpSessionFactory.setPort(remotePort);
    ftpSessionFactory.setUser(remoteUser);
    ftpSessionFactory.setPrivateKey(new FileSystemResource(private_key_file));
    ftpSessionFactory.setAllowUnknownKeys(true);
    return ftpSessionFactory;
}

private SessionFactory sessionFactory = clientSessionFactory();

public SftpInboundFileSynchronizer sftpInboundFileSynchronizer() {
    SftpInboundFileSynchronizer sftpInboundFileSynchronizer = new SftpInboundFileSynchronizer(sessionFactory);
    sftpInboundFileSynchronizer.setDeleteRemoteFiles(false);
    sftpInboundFileSynchronizer.setRemoteDirectory(remoteDirectory);
    return sftpInboundFileSynchronizer;
}

private SftpInboundFileSynchronizer ftpInboundFileSynchronizer = sftpInboundFileSynchronizer();

private SftpInboundFileSynchronizingMessageSource sftpInboundFileSynchronizingMessageSource;

public boolean isDeleteLocalFiles() {
    return deleteLocalFiles;
}

public void setDeleteLocalFiles(boolean deleteLocalFiles) {
    this.deleteLocalFiles = deleteLocalFiles;
}

public SftpInboundFileSynchronizer getFtpInboundFileSynchronizer() {
    return ftpInboundFileSynchronizer;
}

public void setFtpInboundFileSynchronizer(SftpInboundFileSynchronizer ftpInboundFileSynchronizer) {
    this.ftpInboundFileSynchronizer = ftpInboundFileSynchronizer;
}

public SessionFactory getSessionFactory() {
    return sessionFactory;
}

public void setSessionFactory(SessionFactory sessionFactory) {
    this.sessionFactory = sessionFactory;
}

public SftpInboundFileSynchronizingMessageSource getSftpInboundFileSynchronizingMessageSource() {
    return sftpInboundFileSynchronizingMessageSource;
}

public void setSftpInboundFileSynchronizingMessageSource(SftpInboundFileSynchronizingMessageSource sftpInboundFileSynchronizingMessageSource) {
    this.sftpInboundFileSynchronizingMessageSource = sftpInboundFileSynchronizingMessageSource;
}

public String getRemoteDirectory() {
    return remoteDirectory;
}

public void setRemoteDirectory(String remoteDirectory) {
    this.remoteDirectory = remoteDirectory;
}

private SFTPGateway sftpGateway;

@ServiceActivator(inputChannel = "sftpChannel")
public MessageHandler clientMessageHandler() {
    SftpOutboundGateway sftpOutboundGateway = new SftpOutboundGateway(clientSessionFactory(), "mget", "payload");
    sftpOutboundGateway.setAutoCreateLocalDirectory(true);
    sftpOutboundGateway.setLocalDirectory(new File(local_directory_value));
    sftpOutboundGateway.setFileExistsMode(FileExistsMode.REPLACE_IF_MODIFIED);
    sftpOutboundGateway.setFilter(new AcceptOnceFileListFilter<>());
    return sftpOutboundGateway;
}

private void deleteLocalFiles()
{
    if (deleteLocalFiles)
    {
        localDirectory = new File(local_directory_value);
        SimplePatternFileListFilter filter = new SimplePatternFileListFilter(fileNamePattern);
        List<File> matchingFiles = filter.filterFiles(localDirectory.listFiles());
        if (CollectionUtils.isNotEmpty(matchingFiles))
        {
            for (File file : matchingFiles)
            {
                FileUtils.deleteQuietly(file);
            }
        }
    }
}

@Override
public RepeatStatus execute(StepContribution stepContribution, ChunkContext chunkContext) throws Exception {

    deleteLocalFiles();
    ftpInboundFileSynchronizer.synchronizeToLocalDirectory(localDirectory);
    if (retryIfNotFound) {

        SimplePatternFileListFilter filter = new SimplePatternFileListFilter(fileNamePattern);
        int attemptCount = 1;
        while (filter.filterFiles(localDirectory.listFiles()).size() == 0 && attemptCount <= downloadFileAttempts) {

            logger.info("File(s) matching " + fileNamePattern + " not found on remote site. Attempt " + attemptCount + " out of " + downloadFileAttempts);
            Thread.sleep(retryIntervalMilliseconds);
            ftpInboundFileSynchronizer.synchronizeToLocalDirectory(localDirectory);
            attemptCount++;
        }

        if (attemptCount >= downloadFileAttempts && filter.filterFiles(localDirectory.listFiles()).size() == 0) {
            throw new FileNotFoundException("Could not find remote file(s) matching " + fileNamePattern + " after " + downloadFileAttempts + " attempts.");
        }
    }
    return RepeatStatus.FINISHED;
}

public String getFileNamePattern() {
    return fileNamePattern;
}

public void setFileNamePattern(String fileNamePattern) {
    this.fileNamePattern = fileNamePattern;
}

public String getClientName() {
    return clientName;
}

public void setClientName(String clientName) {
    this.clientName = clientName;
}

public boolean isRetryIfNotFound() {
    return retryIfNotFound;
}

public void setRetryIfNotFound(boolean retryIfNotFound) {
    this.retryIfNotFound = retryIfNotFound;
}

public File getLocalDirectory() {
    return localDirectory;
}

public void setLocalDirectory(File localDirectory) {
    this.localDirectory = localDirectory;
}

public int getDownloadFileAttempts() {
    return downloadFileAttempts;
}

public void setDownloadFileAttempts(int downloadFileAttempts) {
    this.downloadFileAttempts = downloadFileAttempts;
}

public long getRetryIntervalMilliseconds() {
    return retryIntervalMilliseconds;
}

public void setRetryIntervalMilliseconds(long retryIntervalMilliseconds) {
    this.retryIntervalMilliseconds = retryIntervalMilliseconds;
}
</code>```

}

My understanding (please correct here if wrong) that the application.properties file properties can be injected in the tasklet (as above).
Then I try to build the package.

```<code>mvn clean package
</code>```

I get the following error:

```<code>Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.batch.core.Step]: Factory method 'getFilesFromFTPServer' threw exception; nested exception is java.lang.IllegalArgumentException: Path must not be null
    at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:185) ~[spring-beans-5.2.6.RELEASE.jar:5.2.6.RELEASE]
    at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:651) ~[spring-beans-5.2.6.RELEASE.jar:5.2.6.RELEASE]
    ... 122 common frames omitted
Caused by: java.lang.IllegalArgumentException: Path must not be null
    at org.springframework.util.Assert.notNull(Assert.java:198) ~[spring-core-5.2.6.RELEASE.jar:5.2.6.RELEASE]
    at org.springframework.core.io.FileSystemResource.<init>(FileSystemResource.java:80) ~[spring-core-5.2.6.RELEASE.jar:5.2.6.RELEASE]
    at com.my.batch.core.tasklet.RemoteFileInboundTasklet.clientSessionFactory(RemoteFileInboundTasklet.java:78) ~[classes/:na]
    at com.my.batch.core.tasklet.RemoteFileInboundTasklet.<init>(RemoteFileInboundTasklet.java:83) ~[classes/:na]
    at com.my.batch.core.BatchConfiguration.getFilesFromFTPServer(BatchConfiguration.java:71) ~[classes/:na]
    at com.my.batch.core.BatchConfiguration$$EnhancerBySpringCGLIB$$17d8a6d9.CGLIB$getFilesFromFTPServer$1(<generated>) ~[classes/:na]
</code>```

The line in the code is:

```<code>ftpSessionFactory.setPrivateKey(new FileSystemResource(private_key_file));
</code>```

called via BatchConfiguration.java -> getFilesFromFTPServer.

This means my values from applcation.properties is not passed?
What changes I need to do?

And, while compiling or building the jar, why is it checking the value of variable?

NEW EDITS:

I tried to declare my tasklet as a bean in Configuration and build the package again. However, it is giving the same error.

My application.properties file after change:

```<code>spring.datasource.url=jdbc:postgresql://dbhost:1000/db
spring.datasource.username=username
spring.datasource.password=password
spring.datasource.platform=postgresql
spring.batch.job.enabled=false
local.directory=/my/local/path/
file.name=file_name_20200601.csv
remote.directory=/remote/ftp/location
remote.host=remotehost
remote.port=22
remote.user=remoteuser
private.key.location=/key/file/location
</code>```

No change in tasklet.

Changed Configuration:

```<code>@Configuration
@EnableBatchProcessing
@EnableIntegration
@EnableAutoConfiguration
public class BatchConfiguration {
    private Logger logger = LoggerFactory.getLogger(BatchConfiguration.class);

    @Autowired
    public JobBuilderFactory jobBuilderFactory;

    @Autowired
    public StepBuilderFactory stepBuilderFactory;

    @Bean
    public RemoteFileInboundTasklet remoteFileInboundTasklet() {
        return new RemoteFileInboundTasklet();
    }

    @Bean
    public Job ftpJob() {
        return jobBuilderFactory.get("FTP Job")
                .incrementer(new RunIdIncrementer())
                .start(getFilesFromFTPServer())
                .build();
    }

    @Bean
    public Step getFilesFromFTPServer() {
        return stepBuilderFactory.get("Get file from server")
                .tasklet(remoteFileInboundTasklet())
                .build();

    }
}
</code>```

When I tried to build the package(mvn clean package), I still get the same error.

Path must not be null.

It is not able to read the properties. Any idea what is wrong?

EDITS BASED ON DIFFERENT APPROACH:

I tried to further see how to use configuration and found the following approach to use the @ConfigurationProperties annotation (How to access a value defined in the application.properties file in Spring Boot)

I created a new ftp config class:

```<code>import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.context.annotation.Configuration;

@ConfigurationProperties(prefix = "ftp")
@Configuration("coreFtpProperties")
public class CoreFtp {
    private String host;
    private String port;
    private String user;
    private String passwordKey;
    private String localDirectory;
    private String remoteDirectory;
    private String fileName;

    public String getHost() {
        return host;
    }

    public void setHost(String host) {
        this.host = host;
    }

    public String getPort() {
        return port;
    }

    public void setPort(String port) {
        this.port = port;
    }

    public String getUser() {
        return user;
    }

    public void setUser(String user) {
        this.user = user;
    }

    public String getPasswordKey() {
        return passwordKey;
    }

    public void setPasswordKey(String passwordKey) {
        this.passwordKey = passwordKey;
    }

    public String getLocalDirectory() {
        return localDirectory;
    }

    public void setLocalDirectory(String localDirectory) {
        this.localDirectory = localDirectory;
    }

    public String getRemoteDirectory() {
        return remoteDirectory;
    }

    public void setRemoteDirectory(String remoteDirectory) {
        this.remoteDirectory = remoteDirectory;
    }

    public String getFileName() {
        return fileName;
    }

    public void setFileName(String fileName) {
        this.fileName = fileName;
    }
}
</code>```

Minor change to application.properties file:

```<code>spring.datasource.url=jdbc:postgresql://dbhost:1000/db
spring.datasource.username=username
spring.datasource.password=password
spring.datasource.platform=postgresql
spring.batch.job.enabled=false
local.directory="/my/local/path/"
file.name="file_name_20200601.csv"
remote.directory="/remote/ftp/location"
remote.host="remotehost"
remote.port=22
remote.user="remoteuser"
private.key.location="/key/file/location"
</code>```0

In my batch configuration I made this changes:

```<code>spring.datasource.url=jdbc:postgresql://dbhost:1000/db
spring.datasource.username=username
spring.datasource.password=password
spring.datasource.platform=postgresql
spring.batch.job.enabled=false
local.directory="/my/local/path/"
file.name="file_name_20200601.csv"
remote.directory="/remote/ftp/location"
remote.host="remotehost"
remote.port=22
remote.user="remoteuser"
private.key.location="/key/file/location"
</code>```1

So, accordingly my Tasklet is changed as:

```<code>spring.datasource.url=jdbc:postgresql://dbhost:1000/db
spring.datasource.username=username
spring.datasource.password=password
spring.datasource.platform=postgresql
spring.batch.job.enabled=false
local.directory="/my/local/path/"
file.name="file_name_20200601.csv"
remote.directory="/remote/ftp/location"
remote.host="remotehost"
remote.port=22
remote.user="remoteuser"
private.key.location="/key/file/location"
</code>```2

Based on above changes, I am able to compile the code and create the necessary Jar, and run the code using the jar.

答:
You need to change getFilesFromFTPServer bean to JobScope and read all the job runtime parameters from there.

```<code>@Bean
@JobScope
public Step getFilesFromFTPServer() {
</code>```

Body:
EDITS BASED ON SUGGESTION:
For brevity, I will remove older code and long part and re-phrase the issue.

I am trying to build the app (Spring boot + Spring Batch) taking the date and config information from command line. Based on suggestions, I can use the application properties?
The main aim is to use the same job (task of the job) to download different files form different host/time etc. So, properties file can give the information to use for download and compiled jar should read the info and do its tasks.

Main Entry point.

@SpringBootApplication
public class CoreApplication implements ApplicationRunner {

    @Autowired
    JobLauncher jobLauncher;

    @Autowired
    Job processJob;

    @Value("${rundate}")
    private String run_date;

    private static final Logger logger = LoggerFactory.getLogger(CoreApplication.class);

    public static void main(String[] args) {
        SpringApplication.run(CoreApplication.class, args);
    }

    @Override
    public void run(ApplicationArguments args) throws Exception {

        JobParameters jobParameters = new JobParametersBuilder()
                .addLong("JobID", System.currentTimeMillis())
                .addString("RunDate", run_date)
                .toJobParameters();

        try {
            jobLauncher.run(processJob, jobParameters);
        } catch (Exception e) {
            logger.error("Exception while running a batch job {}", e.getMessage());
        }

    }

}

I rearranged the code, to use the values of server, user, etc from application.properties file. Please let me know if it is wrong way to inject the properties.

application.properties file:

spring.datasource.url=jdbc:postgresql://dbhost:1000/db
spring.datasource.username=username
spring.datasource.password=password
spring.datasource.platform=postgresql
spring.batch.job.enabled=false
local.directory="/my/local/path/"
file.name="file_name_20200601.csv"
remote.directory="/remote/ftp/location"
remote.host="remotehost"
remote.port=22
remote.user="remoteuser"
private.key.location="/key/file/location"

My Batch Configuration:

@Configuration
@EnableBatchProcessing
@EnableIntegration
@EnableAutoConfiguration
public class BatchConfiguration {
    private Logger logger = LoggerFactory.getLogger(BatchConfiguration.class);

    @Autowired
    public JobBuilderFactory jobBuilderFactory;

    @Autowired
    public StepBuilderFactory stepBuilderFactory;

    @Bean
    public Job ftpJob() {
        return jobBuilderFactory.get("FTP Job")
                .incrementer(new RunIdIncrementer())
                .start(getFilesFromFTPServer())
                .build();
    }

    @Bean
    public Step getFilesFromFTPServer() {
        return stepBuilderFactory.get("Get file from server")
                .tasklet(new RemoteFileInboundTasklet())
                .build();

    }
}

My Tasklet:

public class RemoteFileInboundTasklet implements Tasklet {

private Logger logger = LoggerFactory.getLogger(RemoteFileInboundTasklet.class);

@Value("${file.name}")
private String fileNamePattern;

private String clientName;
private boolean deleteLocalFiles = true;
private boolean retryIfNotFound = false;

@Value("${local.directory}")
private String local_directory_value;

private File localDirectory;
private int downloadFileAttempts = 12;
private long retryIntervalMilliseconds = 300000;

@Value("${remote.directory}")
private String remoteDirectory;

@Value("${remote.host}")
private String remoteHost;

@Value("${remote.user}")
private String remoteUser;

@Value("${remote.port}")
private int remotePort;

@Value("${private.key.location}")
private String private_key_file;

public SessionFactory<ChannelSftp.LsEntry> clientSessionFactory() {
    DefaultSftpSessionFactory ftpSessionFactory = new DefaultSftpSessionFactory();
    ftpSessionFactory.setHost(remoteHost);
    ftpSessionFactory.setPort(remotePort);
    ftpSessionFactory.setUser(remoteUser);
    ftpSessionFactory.setPrivateKey(new FileSystemResource(private_key_file));
    ftpSessionFactory.setAllowUnknownKeys(true);
    return ftpSessionFactory;
}

private SessionFactory sessionFactory = clientSessionFactory();

public SftpInboundFileSynchronizer sftpInboundFileSynchronizer() {
    SftpInboundFileSynchronizer sftpInboundFileSynchronizer = new SftpInboundFileSynchronizer(sessionFactory);
    sftpInboundFileSynchronizer.setDeleteRemoteFiles(false);
    sftpInboundFileSynchronizer.setRemoteDirectory(remoteDirectory);
    return sftpInboundFileSynchronizer;
}

private SftpInboundFileSynchronizer ftpInboundFileSynchronizer = sftpInboundFileSynchronizer();

private SftpInboundFileSynchronizingMessageSource sftpInboundFileSynchronizingMessageSource;

public boolean isDeleteLocalFiles() {
    return deleteLocalFiles;
}

public void setDeleteLocalFiles(boolean deleteLocalFiles) {
    this.deleteLocalFiles = deleteLocalFiles;
}

public SftpInboundFileSynchronizer getFtpInboundFileSynchronizer() {
    return ftpInboundFileSynchronizer;
}

public void setFtpInboundFileSynchronizer(SftpInboundFileSynchronizer ftpInboundFileSynchronizer) {
    this.ftpInboundFileSynchronizer = ftpInboundFileSynchronizer;
}

public SessionFactory getSessionFactory() {
    return sessionFactory;
}

public void setSessionFactory(SessionFactory sessionFactory) {
    this.sessionFactory = sessionFactory;
}

public SftpInboundFileSynchronizingMessageSource getSftpInboundFileSynchronizingMessageSource() {
    return sftpInboundFileSynchronizingMessageSource;
}

public void setSftpInboundFileSynchronizingMessageSource(SftpInboundFileSynchronizingMessageSource sftpInboundFileSynchronizingMessageSource) {
    this.sftpInboundFileSynchronizingMessageSource = sftpInboundFileSynchronizingMessageSource;
}

public String getRemoteDirectory() {
    return remoteDirectory;
}

public void setRemoteDirectory(String remoteDirectory) {
    this.remoteDirectory = remoteDirectory;
}

private SFTPGateway sftpGateway;

@ServiceActivator(inputChannel = "sftpChannel")
public MessageHandler clientMessageHandler() {
    SftpOutboundGateway sftpOutboundGateway = new SftpOutboundGateway(clientSessionFactory(), "mget", "payload");
    sftpOutboundGateway.setAutoCreateLocalDirectory(true);
    sftpOutboundGateway.setLocalDirectory(new File(local_directory_value));
    sftpOutboundGateway.setFileExistsMode(FileExistsMode.REPLACE_IF_MODIFIED);
    sftpOutboundGateway.setFilter(new AcceptOnceFileListFilter<>());
    return sftpOutboundGateway;
}

private void deleteLocalFiles()
{
    if (deleteLocalFiles)
    {
        localDirectory = new File(local_directory_value);
        SimplePatternFileListFilter filter = new SimplePatternFileListFilter(fileNamePattern);
        List<File> matchingFiles = filter.filterFiles(localDirectory.listFiles());
        if (CollectionUtils.isNotEmpty(matchingFiles))
        {
            for (File file : matchingFiles)
            {
                FileUtils.deleteQuietly(file);
            }
        }
    }
}

@Override
public RepeatStatus execute(StepContribution stepContribution, ChunkContext chunkContext) throws Exception {

    deleteLocalFiles();
    ftpInboundFileSynchronizer.synchronizeToLocalDirectory(localDirectory);
    if (retryIfNotFound) {

        SimplePatternFileListFilter filter = new SimplePatternFileListFilter(fileNamePattern);
        int attemptCount = 1;
        while (filter.filterFiles(localDirectory.listFiles()).size() == 0 && attemptCount <= downloadFileAttempts) {

            logger.info("File(s) matching " + fileNamePattern + " not found on remote site. Attempt " + attemptCount + " out of " + downloadFileAttempts);
            Thread.sleep(retryIntervalMilliseconds);
            ftpInboundFileSynchronizer.synchronizeToLocalDirectory(localDirectory);
            attemptCount++;
        }

        if (attemptCount >= downloadFileAttempts && filter.filterFiles(localDirectory.listFiles()).size() == 0) {
            throw new FileNotFoundException("Could not find remote file(s) matching " + fileNamePattern + " after " + downloadFileAttempts + " attempts.");
        }
    }
    return RepeatStatus.FINISHED;
}

public String getFileNamePattern() {
    return fileNamePattern;
}

public void setFileNamePattern(String fileNamePattern) {
    this.fileNamePattern = fileNamePattern;
}

public String getClientName() {
    return clientName;
}

public void setClientName(String clientName) {
    this.clientName = clientName;
}

public boolean isRetryIfNotFound() {
    return retryIfNotFound;
}

public void setRetryIfNotFound(boolean retryIfNotFound) {
    this.retryIfNotFound = retryIfNotFound;
}

public File getLocalDirectory() {
    return localDirectory;
}

public void setLocalDirectory(File localDirectory) {
    this.localDirectory = localDirectory;
}

public int getDownloadFileAttempts() {
    return downloadFileAttempts;
}

public void setDownloadFileAttempts(int downloadFileAttempts) {
    this.downloadFileAttempts = downloadFileAttempts;
}

public long getRetryIntervalMilliseconds() {
    return retryIntervalMilliseconds;
}

public void setRetryIntervalMilliseconds(long retryIntervalMilliseconds) {
    this.retryIntervalMilliseconds = retryIntervalMilliseconds;
}

}

My understanding (please correct here if wrong) that the application.properties file properties can be injected in the tasklet (as above).
Then I try to build the package.

mvn clean package

I get the following error:

Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.batch.core.Step]: Factory method 'getFilesFromFTPServer' threw exception; nested exception is java.lang.IllegalArgumentException: Path must not be null
    at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:185) ~[spring-beans-5.2.6.RELEASE.jar:5.2.6.RELEASE]
    at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:651) ~[spring-beans-5.2.6.RELEASE.jar:5.2.6.RELEASE]
    ... 122 common frames omitted
Caused by: java.lang.IllegalArgumentException: Path must not be null
    at org.springframework.util.Assert.notNull(Assert.java:198) ~[spring-core-5.2.6.RELEASE.jar:5.2.6.RELEASE]
    at org.springframework.core.io.FileSystemResource.<init>(FileSystemResource.java:80) ~[spring-core-5.2.6.RELEASE.jar:5.2.6.RELEASE]
    at com.my.batch.core.tasklet.RemoteFileInboundTasklet.clientSessionFactory(RemoteFileInboundTasklet.java:78) ~[classes/:na]
    at com.my.batch.core.tasklet.RemoteFileInboundTasklet.<init>(RemoteFileInboundTasklet.java:83) ~[classes/:na]
    at com.my.batch.core.BatchConfiguration.getFilesFromFTPServer(BatchConfiguration.java:71) ~[classes/:na]
    at com.my.batch.core.BatchConfiguration$$EnhancerBySpringCGLIB$$17d8a6d9.CGLIB$getFilesFromFTPServer$1(<generated>) ~[classes/:na]

The line in the code is:

ftpSessionFactory.setPrivateKey(new FileSystemResource(private_key_file));

called via BatchConfiguration.java -> getFilesFromFTPServer.

This means my values from applcation.properties is not passed?
What changes I need to do?

And, while compiling or building the jar, why is it checking the value of variable?

NEW EDITS:

I tried to declare my tasklet as a bean in Configuration and build the package again. However, it is giving the same error.

My application.properties file after change:

spring.datasource.url=jdbc:postgresql://dbhost:1000/db
spring.datasource.username=username
spring.datasource.password=password
spring.datasource.platform=postgresql
spring.batch.job.enabled=false
local.directory=/my/local/path/
file.name=file_name_20200601.csv
remote.directory=/remote/ftp/location
remote.host=remotehost
remote.port=22
remote.user=remoteuser
private.key.location=/key/file/location

No change in tasklet.

Changed Configuration:

@Configuration
@EnableBatchProcessing
@EnableIntegration
@EnableAutoConfiguration
public class BatchConfiguration {
    private Logger logger = LoggerFactory.getLogger(BatchConfiguration.class);

    @Autowired
    public JobBuilderFactory jobBuilderFactory;

    @Autowired
    public StepBuilderFactory stepBuilderFactory;

    @Bean
    public RemoteFileInboundTasklet remoteFileInboundTasklet() {
        return new RemoteFileInboundTasklet();
    }

    @Bean
    public Job ftpJob() {
        return jobBuilderFactory.get("FTP Job")
                .incrementer(new RunIdIncrementer())
                .start(getFilesFromFTPServer())
                .build();
    }

    @Bean
    public Step getFilesFromFTPServer() {
        return stepBuilderFactory.get("Get file from server")
                .tasklet(remoteFileInboundTasklet())
                .build();

    }
}

When I tried to build the package(mvn clean package), I still get the same error.

Path must not be null.

It is not able to read the properties. Any idea what is wrong?

EDITS BASED ON DIFFERENT APPROACH:

I tried to further see how to use configuration and found the following approach to use the @ConfigurationProperties annotation (How to access a value defined in the application.properties file in Spring Boot)

I created a new ftp config class:

import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.context.annotation.Configuration;

@ConfigurationProperties(prefix = "ftp")
@Configuration("coreFtpProperties")
public class CoreFtp {
    private String host;
    private String port;
    private String user;
    private String passwordKey;
    private String localDirectory;
    private String remoteDirectory;
    private String fileName;

    public String getHost() {
        return host;
    }

    public void setHost(String host) {
        this.host = host;
    }

    public String getPort() {
        return port;
    }

    public void setPort(String port) {
        this.port = port;
    }

    public String getUser() {
        return user;
    }

    public void setUser(String user) {
        this.user = user;
    }

    public String getPasswordKey() {
        return passwordKey;
    }

    public void setPasswordKey(String passwordKey) {
        this.passwordKey = passwordKey;
    }

    public String getLocalDirectory() {
        return localDirectory;
    }

    public void setLocalDirectory(String localDirectory) {
        this.localDirectory = localDirectory;
    }

    public String getRemoteDirectory() {
        return remoteDirectory;
    }

    public void setRemoteDirectory(String remoteDirectory) {
        this.remoteDirectory = remoteDirectory;
    }

    public String getFileName() {
        return fileName;
    }

    public void setFileName(String fileName) {
        this.fileName = fileName;
    }
}

Minor change to application.properties file:

spring.datasource.url=jdbc:postgresql://dbhost:1000/db
spring.datasource.username=username
spring.datasource.password=password
spring.datasource.platform=postgresql
spring.batch.job.enabled=false
ftp.local_directory=/my/local/path/
ftp.file_name=file_name_20200601.csv
ftp.remote_directory=/remote/ftp/location
ftp.host=remotehost
ftp.port=22
ftp.user=remoteuser
ftp.password_key=/key/file/location

In my batch configuration I made this changes:

@Configuration
@EnableBatchProcessing
@EnableIntegration
public class BatchConfiguration {
    private Logger logger = LoggerFactory.getLogger(BatchConfiguration.class);

    @Autowired
    public JobBuilderFactory jobBuilderFactory;

    @Autowired
    public StepBuilderFactory stepBuilderFactory;

    @Autowired
    private CoreFtp coreFtpProperties;

    @Bean
    public RemoteFileInboundTasklet remoteFileInboundTasklet() {
        RemoteFileInboundTasklet ftpTasklet = new RemoteFileInboundTasklet();
        ftpTasklet.setRetryIfNotFound(true);
        ftpTasklet.setDownloadFileAttempts(3);
        ftpTasklet.setRetryIntervalMilliseconds(10000);
        ftpTasklet.setFileNamePattern(coreFtpProperties.getFileName());
        ftpTasklet.setRemoteDirectory(coreFtpProperties.getRemoteDirectory());
        ftpTasklet.setLocalDirectory(new File(coreFtpProperties.getLocalDirectory()));
        ftpTasklet.setSessionFactory(clientSessionFactory());
        ftpTasklet.setFtpInboundFileSynchronizer(sftpInboundFileSynchronizer());
        ftpTasklet.setSftpInboundFileSynchronizingMessageSource(new SftpInboundFileSynchronizingMessageSource(sftpInboundFileSynchronizer()));

        return ftpTasklet;
    }

    @Bean
    public SftpInboundFileSynchronizer sftpInboundFileSynchronizer() {
        SftpInboundFileSynchronizer sftpInboundFileSynchronizer = new SftpInboundFileSynchronizer(clientSessionFactory());
        sftpInboundFileSynchronizer.setDeleteRemoteFiles(false);
        sftpInboundFileSynchronizer.setRemoteDirectory(coreFtpProperties.getRemoteDirectory());
        return sftpInboundFileSynchronizer;
    }

    @Bean(name = "clientSessionFactory")
    public SessionFactory<LsEntry> clientSessionFactory() {
        DefaultSftpSessionFactory ftpSessionFactory = new DefaultSftpSessionFactory();
        ftpSessionFactory.setHost(coreFtpProperties.getHost());
        ftpSessionFactory.setPort(Integer.parseInt(coreFtpProperties.getPort()));
        ftpSessionFactory.setUser(coreFtpProperties.getUser());
        ftpSessionFactory.setPrivateKey(new FileSystemResource(coreFtpProperties.getPasswordKey()));
        ftpSessionFactory.setPassword("");
        ftpSessionFactory.setAllowUnknownKeys(true);
        return ftpSessionFactory;
    }

    @Bean
    @ServiceActivator(inputChannel = "sftpChannel")
    public MessageHandler clientMessageHandler() {
        SftpOutboundGateway sftpOutboundGateway = new SftpOutboundGateway(clientSessionFactory(), "mget", "payload");
        sftpOutboundGateway.setAutoCreateLocalDirectory(true);
        sftpOutboundGateway.setLocalDirectory(new File(coreFtpProperties.getLocalDirectory()));
        sftpOutboundGateway.setFileExistsMode(FileExistsMode.REPLACE_IF_MODIFIED);
        sftpOutboundGateway.setFilter(new AcceptOnceFileListFilter<>());
        return sftpOutboundGateway;
    }

    @Bean
    public Job ftpJob() {
        return jobBuilderFactory.get("FTP Job")
                .incrementer(new RunIdIncrementer())
                .start(getFilesFromFTPServer())
                .build();
    }

    @Bean
    public Step getFilesFromFTPServer() {
        return stepBuilderFactory.get("Get file from server")
                .tasklet(remoteFileInboundTasklet())
                .build();

    }

}

So, accordingly my Tasklet is changed as:

public class RemoteFileInboundTasklet implements Tasklet {

    private Logger logger = LoggerFactory.getLogger(RemoteFileInboundTasklet.class);

    private String fileNamePattern;

    private String clientName;
    private boolean deleteLocalFiles = true;
    private boolean retryIfNotFound = false;

    private File localDirectory;

    private int downloadFileAttempts = 12;
    private long retryIntervalMilliseconds = 300000;

    private String remoteDirectory;

    private SessionFactory sessionFactory;
    private SftpInboundFileSynchronizer ftpInboundFileSynchronizer;

    private SftpInboundFileSynchronizingMessageSource sftpInboundFileSynchronizingMessageSource;

    public boolean isDeleteLocalFiles() {
        return deleteLocalFiles;
    }

    public void setDeleteLocalFiles(boolean deleteLocalFiles) {
        this.deleteLocalFiles = deleteLocalFiles;
    }

    public SftpInboundFileSynchronizer getFtpInboundFileSynchronizer() {
        return ftpInboundFileSynchronizer;
    }

    public void setFtpInboundFileSynchronizer(SftpInboundFileSynchronizer ftpInboundFileSynchronizer) {
        this.ftpInboundFileSynchronizer = ftpInboundFileSynchronizer;
    }

    public SessionFactory getSessionFactory() {
        return sessionFactory;
    }

    public void setSessionFactory(SessionFactory sessionFactory) {
        this.sessionFactory = sessionFactory;
    }

    public SftpInboundFileSynchronizingMessageSource getSftpInboundFileSynchronizingMessageSource() {
        return sftpInboundFileSynchronizingMessageSource;
    }

    public void setSftpInboundFileSynchronizingMessageSource(SftpInboundFileSynchronizingMessageSource sftpInboundFileSynchronizingMessageSource) {
        this.sftpInboundFileSynchronizingMessageSource = sftpInboundFileSynchronizingMessageSource;
    }

    public String getRemoteDirectory() {
        return remoteDirectory;
    }

    public void setRemoteDirectory(String remoteDirectory) {
        this.remoteDirectory = remoteDirectory;
    }

    private SFTPGateway sftpGateway;

    private void deleteLocalFiles()
    {
        if (deleteLocalFiles)
        {
            SimplePatternFileListFilter filter = new SimplePatternFileListFilter(fileNamePattern);
            List<File> matchingFiles = filter.filterFiles(localDirectory.listFiles());
            if (CollectionUtils.isNotEmpty(matchingFiles))
            {
                for (File file : matchingFiles)
                {
                    FileUtils.deleteQuietly(file);
                }
            }
        }
    }

    @Override
    public RepeatStatus execute(StepContribution stepContribution, ChunkContext chunkContext) throws Exception {

        deleteLocalFiles();

        ftpInboundFileSynchronizer.synchronizeToLocalDirectory(localDirectory);

        if (retryIfNotFound) {

            SimplePatternFileListFilter filter = new SimplePatternFileListFilter(fileNamePattern);
            int attemptCount = 1;
            while (filter.filterFiles(localDirectory.listFiles()).size() == 0 && attemptCount <= downloadFileAttempts) {

                logger.info("File(s) matching " + fileNamePattern + " not found on remote site. Attempt " + attemptCount + " out of " + downloadFileAttempts);
                Thread.sleep(retryIntervalMilliseconds);
                ftpInboundFileSynchronizer.synchronizeToLocalDirectory(localDirectory);
                attemptCount++;
            }

            if (attemptCount >= downloadFileAttempts && filter.filterFiles(localDirectory.listFiles()).size() == 0) {
                throw new FileNotFoundException("Could not find remote file(s) matching " + fileNamePattern + " after " + downloadFileAttempts + " attempts.");
            }
        }
        return RepeatStatus.FINISHED;
    }
}

Based on above changes, I am able to compile the code and create the necessary Jar, and run the code using the jar.

Body:
You need to change getFilesFromFTPServer bean to JobScope and read all the job runtime parameters from there.

@Bean
@JobScope
public Step getFilesFromFTPServer() {


问:
AttributeError: 'KerasRegressor' object has no attribute 'model'
I have this piece of code. But when I try to run the prediction value code there's an error

```<code># Creating a data structure with n timesteps
X_test = []
for i in range(5, 25):
    X_test.append(inputs[i-5:i, 0])
X_test = np.array(X_test)
# Reshape to a new dimension
X_test = np.reshape(X_test, (X_test.shape[0], X_test.shape[1], 1))

# Identify the predicted values
predicted_number = regressor.predict(X_test)
# Inverse the scaling to put them back to the normal values
predicted_number = sc.inverse_transform(predicted_number)
</code>```

the error is like this

```<code>AttributeError Traceback (most recent call last)
<ipython-input-364-17fa061596c6> in <module>()
      1 # Identify the predicted values
----> 2 predicted_number = regressor.predict(X_test)
      3 # Inverse the scaling to put them back to the normal values
      4 predicted_number = sc.inverse_transform(predicted_number)
      5 KerasRegressor.model
~\Anaconda3\lib\site-packages\keras\wrappers\scikit_learn.py in predict(self, x, **kwargs)
    320         """
    321         kwargs = self.filter_sk_params(Sequential.predict, kwargs)
--> 322         preds = np.array(self.model.predict(x, **kwargs))
    323         if preds.shape[-1] == 1:
    324             return np.squeeze(preds, axis=-1)

AttributeError: 'KerasRegressor' object has no attribute 'model'
</code>```

in case if needed, below is the full script

```<code>import numpy as np
import matplotlib.pyplot as plt
import pandas as pd

dataset_train = pd.read_csv('Datatraining.csv')
training_set = dataset_train.iloc[:, 1:2].values

from sklearn.preprocessing import MinMaxScaler
sc = MinMaxScaler(feature_range = (0, 1))
training_set_scaled = sc.fit_transform(training_set)

X_train = []
y_train = []
for i in range(60, 72):
    X_train.append(training_set_scaled[i-60:i, 0])
    y_train.append(training_set_scaled[i, 0])
X_train, y_train = np.array(X_train), np.array(y_train)

X_train = np.reshape(X_train, newshape = (X_train.shape[0], X_train.shape[1], 1))

from keras.models import Sequential
from keras.layers import Dense, LSTM, Dropout

regressor = Sequential()

# Adding the first LSTM layer and some Dropout regularisation
regressor.add(LSTM( units = 50, return_sequences = True, input_shape = (X_train.shape[1], 1) ))
regressor.add(Dropout(0.2))

# Adding the second LSTM layer and some Dropout regularisation
regressor.add(LSTM(units = 50, return_sequences = True))
regressor.add(Dropout(0.2))

# Adding the third LSTM layer and some Dropout regularisation
regressor.add(LSTM(units = 50, return_sequences = True))
regressor.add(Dropout(0.2))

# Adding the fourth LSTM layer and some Dropout regularisation
regressor.add(LSTM(units = 50))
regressor.add(Dropout(0.2))

regressor.add(Dense(units = 1))

regressor.compile(optimizer = 'adam', loss = 'mean_squared_error')

regressor.fit(X_train, y_train, epochs = 300, batch_size = 32)

dataset_test = pd.read_csv('Datatesting.csv')
real_number_arrivals = dataset_test.iloc[:, 1:2].values

dataset_total = pd.concat( (downloads['China'], dataset_test['China']), axis = 0 )
inputs = dataset_total[len(dataset_total) - len(dataset_test) - 72:].values
inputs = inputs.reshape(-1, 1)
inputs = sc.transform(inputs)

# Creating a data structure with n timesteps
X_test = []
for i in range(5, 25):
    X_test.append(inputs[i-5:i, 0])
X_test = np.array(X_test)
# Reshape to a new dimension
X_test = np.reshape(X_test, (X_test.shape[0], X_test.shape[1], 1))

# Identify the predicted values
predicted_number = regressor.predict(X_test)
# Inverse the scaling to put them back to the normal values
predicted_number = sc.inverse_transform(predicted_number)
</code>```

any solution would be really helpful. Thanks in advance

答:
You have forgotten to fit the model first.

You did not share the whole code, but I believe you have some X_train and Y_train somewhere in your code. So try this line:

regressor.fit(X_train, Y_train)

And then you can run the prediction.

Body:
I have this piece of code. But when I try to run the prediction value code there's an error

# Creating a data structure with n timesteps
X_test = []
for i in range(5, 25):
    X_test.append(inputs[i-5:i, 0])
X_test = np.array(X_test)
# Reshape to a new dimension
X_test = np.reshape(X_test, (X_test.shape[0], X_test.shape[1], 1))

# Identify the predicted values
predicted_number = regressor.predict(X_test)
# Inverse the scaling to put them back to the normal values
predicted_number = sc.inverse_transform(predicted_number)

the error is like this

AttributeError Traceback (most recent call last)
<ipython-input-364-17fa061596c6> in <module>()
      1 # Identify the predicted values
----> 2 predicted_number = regressor.predict(X_test)
      3 # Inverse the scaling to put them back to the normal values
      4 predicted_number = sc.inverse_transform(predicted_number)
      5 KerasRegressor.model
~\Anaconda3\lib\site-packages\keras\wrappers\scikit_learn.py in predict(self, x, **kwargs)
    320         """
    321         kwargs = self.filter_sk_params(Sequential.predict, kwargs)
--> 322         preds = np.array(self.model.predict(x, **kwargs))
    323         if preds.shape[-1] == 1:
    324             return np.squeeze(preds, axis=-1)

AttributeError: 'KerasRegressor' object has no attribute 'model'

in case if needed, below is the full script

import numpy as np
import matplotlib.pyplot as plt
import pandas as pd

dataset_train = pd.read_csv('Datatraining.csv')
training_set = dataset_train.iloc[:, 1:2].values

from sklearn.preprocessing import MinMaxScaler
sc = MinMaxScaler(feature_range = (0, 1))
training_set_scaled = sc.fit_transform(training_set)

X_train = []
y_train = []
for i in range(60, 72):
    X_train.append(training_set_scaled[i-60:i, 0])
    y_train.append(training_set_scaled[i, 0])
X_train, y_train = np.array(X_train), np.array(y_train)

X_train = np.reshape(X_train, newshape = (X_train.shape[0], X_train.shape[1], 1))

from keras.models import Sequential
from keras.layers import Dense, LSTM, Dropout

regressor = Sequential()

# Adding the first LSTM layer and some Dropout regularisation
regressor.add(LSTM( units = 50, return_sequences = True, input_shape = (X_train.shape[1], 1) ))
regressor.add(Dropout(0.2))

# Adding the second LSTM layer and some Dropout regularisation
regressor.add(LSTM(units = 50, return_sequences = True))
regressor.add(Dropout(0.2))

# Adding the third LSTM layer and some Dropout regularisation
regressor.add(LSTM(units = 50, return_sequences = True))
regressor.add(Dropout(0.2))

# Adding the fourth LSTM layer and some Dropout regularisation
regressor.add(LSTM(units = 50))
regressor.add(Dropout(0.2))

regressor.add(Dense(units = 1))

regressor.compile(optimizer = 'adam', loss = 'mean_squared_error')

regressor.fit(X_train, y_train, epochs = 300, batch_size = 32)

dataset_test = pd.read_csv('Datatesting.csv')
real_number_arrivals = dataset_test.iloc[:, 1:2].values

dataset_total = pd.concat( (downloads['China'], dataset_test['China']), axis = 0 )
inputs = dataset_total[len(dataset_total) - len(dataset_test) - 72:].values
inputs = inputs.reshape(-1, 1)
inputs = sc.transform(inputs)

# Creating a data structure with n timesteps
X_test = []
for i in range(5, 25):
    X_test.append(inputs[i-5:i, 0])
X_test = np.array(X_test)
# Reshape to a new dimension
X_test = np.reshape(X_test, (X_test.shape[0], X_test.shape[1], 1))

# Identify the predicted values
predicted_number = regressor.predict(X_test)
# Inverse the scaling to put them back to the normal values
predicted_number = sc.inverse_transform(predicted_number)

any solution would be really helpful. Thanks in advance