问:
How can assigning a variable result in a serious performance drop while the execution order is (nearly) untouched?
When playing around with multithreading, I could observe some unexpected but serious performance issues related to AtomicLong (and classes using it, such as java.util.Random), for which I currently have no explanation. However, I created a minimalistic example, which basically consists of two classes: a class "Container", which keeps a reference to a volatile variable, and a class "DemoThread", which operates on an instance of "Container" during thread execution. Note that the references to "Container" and the volatile long are private, and never shared between threads (I know that there's no need to use volatile here, it's just for demonstration purposes) - thus, multiple instances of "DemoThread" should run perfectly parallel on a multiprocessor machine, but for some reason, they do not (Complete example is at the bottom of this post).

```<code>private static class Container  {

    private volatile long value;

    public long getValue() {
        return value;
    }

    public final void set(long newValue) {
        value = newValue;
    }
}

private static class DemoThread extends Thread {

    private Container variable;

    public void prepare() {
        this.variable = new Container();
    }

    public void run() {
        for(int j = 0; j < 10000000; j++) {
            variable.set(variable.getValue() + System.nanoTime());
        }
    }
}
</code>```

During my test, I repeatedly create 4 DemoThreads, which are then started and joined. The only difference in each loop is the time when "prepare()" gets called (which is obviously required for the thread to run, as it otherwise would result in a NullPointerException):

```<code>DemoThread[] threads = new DemoThread[numberOfThreads];
    for(int j = 0; j < 100; j++) {
        boolean prepareAfterConstructor = j % 2 == 0;
        for(int i = 0; i < threads.length; i++) {
            threads[i] = new DemoThread();
            if(prepareAfterConstructor) threads[i].prepare();
        }

        for(int i = 0; i < threads.length; i++) {
            if(!prepareAfterConstructor) threads[i].prepare();
            threads[i].start();
        }
        joinThreads(threads);
    }
</code>```

For some reason, if prepare() is executed immediately before starting the thread, it will take twice as more time to finish, and even without the "volatile" keyword, the performance differences were significant, at  least on two of the machines and OS'es I tested the code. Here's a short summary:

Mac OS Summary:

Java Version: 1.6.0_24

Java Class Version: 50.0

VM Vendor: Sun Microsystems Inc.

VM Version: 19.1-b02-334

VM Name: Java HotSpot(TM) 64-Bit Server VM

OS Name: Mac OS X

OS Arch: x86_64

OS Version: 10.6.5

Processors/Cores: 8

With volatile keyword:

Final results:

31979 ms. when prepare() was called after instantiation.

96482 ms. when prepare() was called before execution.

Without volatile keyword:

Final results:

26009 ms. when prepare() was called after instantiation.

35196 ms. when prepare() was called before execution.

Windows Summary:

Java Version: 1.6.0_24

Java Class Version: 50.0

VM Vendor: Sun Microsystems Inc.

VM Version: 19.1-b02

VM Name: Java HotSpot(TM) 64-Bit Server VM

OS Name: Windows 7

OS Arch: amd64

OS Version: 6.1

Processors/Cores: 4

With volatile keyword:

Final results:

18120 ms. when prepare() was called after instantiation.

36089 ms. when prepare() was called before execution.

Without volatile keyword:

Final results:

10115 ms. when prepare() was called after instantiation.

10039 ms. when prepare() was called before execution.

Linux Summary:

Java Version: 1.6.0_20

Java Class Version: 50.0

VM Vendor: Sun Microsystems Inc.

VM Version: 19.0-b09

VM Name: OpenJDK 64-Bit Server VM

OS Name: Linux

OS Arch: amd64

OS Version: 2.6.32-28-generic

Processors/Cores: 4

With volatile keyword:

Final results:

45848 ms. when prepare() was called after instantiation.

110754 ms. when prepare() was called before execution.

Without volatile keyword:

Final results:

37862 ms. when prepare() was called after instantiation.

39357 ms. when prepare() was called before execution.

Mac OS Details (volatile):

Test 1, 4 threads, setting variable in creation loop

Thread-2 completed after 653 ms.

Thread-3 completed after 653 ms.

Thread-4 completed after 653 ms.

Thread-5 completed after 653 ms.

Overall time: 654 ms.

Test 2, 4 threads, setting variable in start loop

Thread-7 completed after 1588 ms.

Thread-6 completed after 1589 ms.

Thread-8 completed after 1593 ms.

Thread-9 completed after 1593 ms.

Overall time: 1594 ms.

Test 3, 4 threads, setting variable in creation loop

Thread-10 completed after 648 ms.

Thread-12 completed after 648 ms.

Thread-13 completed after 648 ms.

Thread-11 completed after 648 ms.

Overall time: 648 ms.

Test 4, 4 threads, setting variable in start loop

Thread-17 completed after 1353 ms.

Thread-16 completed after 1957 ms.

Thread-14 completed after 2170 ms.

Thread-15 completed after 2169 ms.

Overall time: 2172 ms.

(and so on, sometimes one or two of the threads in the 'slow' loop finish as expected, but most times they don't).

The given example looks theoretically, as it is of no use, and 'volatile' is not needed here - however, if you'd use a 'java.util.Random'-Instance instead of the 'Container'-Class and call, for instance, nextInt() multiple times, the same effects will occur: The thread will be executed fast if you create the object in the Thread's constructor, but slow if you create it within the run()-method. I believe that the performance issues described in Java Random Slowdowns on Mac OS more than a year ago are related to this effect, but I have no idea why it is as it is - besides that I'm sure that it shouldn't be like that, as it would mean that it's always dangerous to create a new object within the run-method of a thread, unless you know that no volatile variables will get involved within the object graph. Profiling doesn't help, as the problem disappears in this case (same observation as in Java Random Slowdowns on Mac OS cont'd), and it also does not happen on a single-core-PC - so I'd guess that it's kind of a thread synchronization problem... however, the strange thing is that there's actually nothing to synchronize, as all variables are thread-local.

Really looking forward for any hints - and just in case you want to confirm or falsify the problem, see the test case below.

Thanks,

Stephan

```<code>public class UnexpectedPerformanceIssue {

private static class Container  {

    // Remove the volatile keyword, and the problem disappears (on windows)
    // or gets smaller (on mac os)
    private volatile long value;

    public long getValue() {
        return value;
    }

    public final void set(long newValue) {
        value = newValue;
    }
}

private static class DemoThread extends Thread {

    private Container variable;

    public void prepare() {
        this.variable = new Container();
    }

    @Override
    public void run() {
        long start = System.nanoTime();
        for(int j = 0; j < 10000000; j++) {
            variable.set(variable.getValue() + System.nanoTime());
        }
        long end = System.nanoTime();
        System.out.println(this.getName() + " completed after "
                +  ((end - start)/1000000) + " ms.");
    }
}

public static void main(String[] args) {
    System.out.println("Java Version: " + System.getProperty("java.version"));
    System.out.println("Java Class Version: " + System.getProperty("java.class.version"));

    System.out.println("VM Vendor: " + System.getProperty("java.vm.specification.vendor"));
    System.out.println("VM Version: " + System.getProperty("java.vm.version"));
    System.out.println("VM Name: " + System.getProperty("java.vm.name"));

    System.out.println("OS Name: " + System.getProperty("os.name"));
    System.out.println("OS Arch: " + System.getProperty("os.arch"));
    System.out.println("OS Version: " + System.getProperty("os.version"));
    System.out.println("Processors/Cores: " + Runtime.getRuntime().availableProcessors());

    System.out.println();
    int numberOfThreads = 4;

    System.out.println("\nReference Test (single thread):");
    DemoThread t = new DemoThread();
    t.prepare();
    t.run();

    DemoThread[] threads = new DemoThread[numberOfThreads];
    long createTime = 0, startTime = 0;
    for(int j = 0; j < 100; j++) {
        boolean prepareAfterConstructor = j % 2 == 0;
        long overallStart = System.nanoTime();
        if(prepareAfterConstructor) {
            System.out.println("\nTest " + (j+1) + ", " + numberOfThreads + " threads, setting variable in creation loop");
        } else {
            System.out.println("\nTest " + (j+1) + ", " + numberOfThreads + " threads, setting variable in start loop");
        }

        for(int i = 0; i < threads.length; i++) {
            threads[i] = new DemoThread();
            // Either call DemoThread.prepare() here (in odd loops)...
            if(prepareAfterConstructor) threads[i].prepare();
        }

        for(int i = 0; i < threads.length; i++) {
            // or here (in even loops). Should make no difference, but does!
            if(!prepareAfterConstructor) threads[i].prepare();
            threads[i].start();
        }
        joinThreads(threads);
        long overallEnd = System.nanoTime();
        long overallTime = (overallEnd - overallStart);
        if(prepareAfterConstructor) {
            createTime += overallTime;
        } else {
            startTime += overallTime;
        }
        System.out.println("Overall time: " + (overallTime)/1000000 + " ms.");
    }
    System.out.println("Final results:");
    System.out.println(createTime/1000000 + " ms. when prepare() was called after instantiation.");
    System.out.println(startTime/1000000 + " ms. when prepare() was called before execution.");
}

private static void joinThreads(Thread[] threads) {
    for(int i = 0; i < threads.length; i++) {
        try {
            threads[i].join();
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
    }
}
</code>```

}

答:
It's likely that two volatile variables a and b are too close to each other, they fall in the same cache line; although CPU A only reads/writes variable a, and CPU B only reads/writes variable b, they are still coupled to each other through the same cache line. Such problems are called false sharing.

In your example, we have two allocation schemes:

```<code>new Thread                               new Thread
new Container               vs           new Thread
new Thread                               ....
new Container                            new Container
....                                     new Container
</code>```

In the first scheme, it's very unlikely that two volatile variables are close to each other. In the 2nd scheme, it's almost certainly the case.

CPU caches don't work with individual words; instead, they deal with cache lines. A cache line is a continuous chunk of memory, say 64 neighboring bytes. Usually this is nice - if a CPU accessed a cell, it's very likely that it will access the neighboring cells too. Except in your example, that assumption is not only invalid, but detrimental.

Suppose a and b fall in the same cache line L. When CPU A updates a, it notifies other CPUs that L is dirty. Since B caches L too, because it's working on b, B must drop its cached L. So next time B needs to read b, it must reload L, which is costly.

If B must access main memory to reload, that is extremely costly, it's usually 100X slower.

Fortunately, A and B can communicate directly about the new values without going through main memory. Nevertheless it takes extra time.

To verify this theory, you can stuff extra 128 bytes in Container, so that two volatile variable of two Container will not fall in the same cache line; then you should observe that the two schemes take about the same time to execute.

Lession learned: usually CPUs assume that adjecent variables are related. If we want independent variables, we better place them far away from each other.

Body:
When playing around with multithreading, I could observe some unexpected but serious performance issues related to AtomicLong (and classes using it, such as java.util.Random), for which I currently have no explanation. However, I created a minimalistic example, which basically consists of two classes: a class "Container", which keeps a reference to a volatile variable, and a class "DemoThread", which operates on an instance of "Container" during thread execution. Note that the references to "Container" and the volatile long are private, and never shared between threads (I know that there's no need to use volatile here, it's just for demonstration purposes) - thus, multiple instances of "DemoThread" should run perfectly parallel on a multiprocessor machine, but for some reason, they do not (Complete example is at the bottom of this post).

private static class Container  {

    private volatile long value;

    public long getValue() {
        return value;
    }

    public final void set(long newValue) {
        value = newValue;
    }
}

private static class DemoThread extends Thread {

    private Container variable;

    public void prepare() {
        this.variable = new Container();
    }

    public void run() {
        for(int j = 0; j < 10000000; j++) {
            variable.set(variable.getValue() + System.nanoTime());
        }
    }
}

During my test, I repeatedly create 4 DemoThreads, which are then started and joined. The only difference in each loop is the time when "prepare()" gets called (which is obviously required for the thread to run, as it otherwise would result in a NullPointerException):

DemoThread[] threads = new DemoThread[numberOfThreads];
    for(int j = 0; j < 100; j++) {
        boolean prepareAfterConstructor = j % 2 == 0;
        for(int i = 0; i < threads.length; i++) {
            threads[i] = new DemoThread();
            if(prepareAfterConstructor) threads[i].prepare();
        }

        for(int i = 0; i < threads.length; i++) {
            if(!prepareAfterConstructor) threads[i].prepare();
            threads[i].start();
        }
        joinThreads(threads);
    }

For some reason, if prepare() is executed immediately before starting the thread, it will take twice as more time to finish, and even without the "volatile" keyword, the performance differences were significant, at  least on two of the machines and OS'es I tested the code. Here's a short summary:

Mac OS Summary:

Java Version: 1.6.0_24

Java Class Version: 50.0

VM Vendor: Sun Microsystems Inc.

VM Version: 19.1-b02-334

VM Name: Java HotSpot(TM) 64-Bit Server VM

OS Name: Mac OS X

OS Arch: x86_64

OS Version: 10.6.5

Processors/Cores: 8

With volatile keyword:

Final results:

31979 ms. when prepare() was called after instantiation.

96482 ms. when prepare() was called before execution.

Without volatile keyword:

Final results:

26009 ms. when prepare() was called after instantiation.

35196 ms. when prepare() was called before execution.

Windows Summary:

Java Version: 1.6.0_24

Java Class Version: 50.0

VM Vendor: Sun Microsystems Inc.

VM Version: 19.1-b02

VM Name: Java HotSpot(TM) 64-Bit Server VM

OS Name: Windows 7

OS Arch: amd64

OS Version: 6.1

Processors/Cores: 4

With volatile keyword:

Final results:

18120 ms. when prepare() was called after instantiation.

36089 ms. when prepare() was called before execution.

Without volatile keyword:

Final results:

10115 ms. when prepare() was called after instantiation.

10039 ms. when prepare() was called before execution.

Linux Summary:

Java Version: 1.6.0_20

Java Class Version: 50.0

VM Vendor: Sun Microsystems Inc.

VM Version: 19.0-b09

VM Name: OpenJDK 64-Bit Server VM

OS Name: Linux

OS Arch: amd64

OS Version: 2.6.32-28-generic

Processors/Cores: 4

With volatile keyword:

Final results:

45848 ms. when prepare() was called after instantiation.

110754 ms. when prepare() was called before execution.

Without volatile keyword:

Final results:

37862 ms. when prepare() was called after instantiation.

39357 ms. when prepare() was called before execution.

Mac OS Details (volatile):

Test 1, 4 threads, setting variable in creation loop

Thread-2 completed after 653 ms.

Thread-3 completed after 653 ms.

Thread-4 completed after 653 ms.

Thread-5 completed after 653 ms.

Overall time: 654 ms.

Test 2, 4 threads, setting variable in start loop

Thread-7 completed after 1588 ms.

Thread-6 completed after 1589 ms.

Thread-8 completed after 1593 ms.

Thread-9 completed after 1593 ms.

Overall time: 1594 ms.

Test 3, 4 threads, setting variable in creation loop

Thread-10 completed after 648 ms.

Thread-12 completed after 648 ms.

Thread-13 completed after 648 ms.

Thread-11 completed after 648 ms.

Overall time: 648 ms.

Test 4, 4 threads, setting variable in start loop

Thread-17 completed after 1353 ms.

Thread-16 completed after 1957 ms.

Thread-14 completed after 2170 ms.

Thread-15 completed after 2169 ms.

Overall time: 2172 ms.

(and so on, sometimes one or two of the threads in the 'slow' loop finish as expected, but most times they don't).

The given example looks theoretically, as it is of no use, and 'volatile' is not needed here - however, if you'd use a 'java.util.Random'-Instance instead of the 'Container'-Class and call, for instance, nextInt() multiple times, the same effects will occur: The thread will be executed fast if you create the object in the Thread's constructor, but slow if you create it within the run()-method. I believe that the performance issues described in Java Random Slowdowns on Mac OS more than a year ago are related to this effect, but I have no idea why it is as it is - besides that I'm sure that it shouldn't be like that, as it would mean that it's always dangerous to create a new object within the run-method of a thread, unless you know that no volatile variables will get involved within the object graph. Profiling doesn't help, as the problem disappears in this case (same observation as in Java Random Slowdowns on Mac OS cont'd), and it also does not happen on a single-core-PC - so I'd guess that it's kind of a thread synchronization problem... however, the strange thing is that there's actually nothing to synchronize, as all variables are thread-local.

Really looking forward for any hints - and just in case you want to confirm or falsify the problem, see the test case below.

Thanks,

Stephan

public class UnexpectedPerformanceIssue {

private static class Container  {

    // Remove the volatile keyword, and the problem disappears (on windows)
    // or gets smaller (on mac os)
    private volatile long value;

    public long getValue() {
        return value;
    }

    public final void set(long newValue) {
        value = newValue;
    }
}

private static class DemoThread extends Thread {

    private Container variable;

    public void prepare() {
        this.variable = new Container();
    }

    @Override
    public void run() {
        long start = System.nanoTime();
        for(int j = 0; j < 10000000; j++) {
            variable.set(variable.getValue() + System.nanoTime());
        }
        long end = System.nanoTime();
        System.out.println(this.getName() + " completed after "
                +  ((end - start)/1000000) + " ms.");
    }
}

public static void main(String[] args) {
    System.out.println("Java Version: " + System.getProperty("java.version"));
    System.out.println("Java Class Version: " + System.getProperty("java.class.version"));

    System.out.println("VM Vendor: " + System.getProperty("java.vm.specification.vendor"));
    System.out.println("VM Version: " + System.getProperty("java.vm.version"));
    System.out.println("VM Name: " + System.getProperty("java.vm.name"));

    System.out.println("OS Name: " + System.getProperty("os.name"));
    System.out.println("OS Arch: " + System.getProperty("os.arch"));
    System.out.println("OS Version: " + System.getProperty("os.version"));
    System.out.println("Processors/Cores: " + Runtime.getRuntime().availableProcessors());

    System.out.println();
    int numberOfThreads = 4;

    System.out.println("\nReference Test (single thread):");
    DemoThread t = new DemoThread();
    t.prepare();
    t.run();

    DemoThread[] threads = new DemoThread[numberOfThreads];
    long createTime = 0, startTime = 0;
    for(int j = 0; j < 100; j++) {
        boolean prepareAfterConstructor = j % 2 == 0;
        long overallStart = System.nanoTime();
        if(prepareAfterConstructor) {
            System.out.println("\nTest " + (j+1) + ", " + numberOfThreads + " threads, setting variable in creation loop");
        } else {
            System.out.println("\nTest " + (j+1) + ", " + numberOfThreads + " threads, setting variable in start loop");
        }

        for(int i = 0; i < threads.length; i++) {
            threads[i] = new DemoThread();
            // Either call DemoThread.prepare() here (in odd loops)...
            if(prepareAfterConstructor) threads[i].prepare();
        }

        for(int i = 0; i < threads.length; i++) {
            // or here (in even loops). Should make no difference, but does!
            if(!prepareAfterConstructor) threads[i].prepare();
            threads[i].start();
        }
        joinThreads(threads);
        long overallEnd = System.nanoTime();
        long overallTime = (overallEnd - overallStart);
        if(prepareAfterConstructor) {
            createTime += overallTime;
        } else {
            startTime += overallTime;
        }
        System.out.println("Overall time: " + (overallTime)/1000000 + " ms.");
    }
    System.out.println("Final results:");
    System.out.println(createTime/1000000 + " ms. when prepare() was called after instantiation.");
    System.out.println(startTime/1000000 + " ms. when prepare() was called before execution.");
}

private static void joinThreads(Thread[] threads) {
    for(int i = 0; i < threads.length; i++) {
        try {
            threads[i].join();
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
    }
}

}

Body:
It's likely that two volatile variables a and b are too close to each other, they fall in the same cache line; although CPU A only reads/writes variable a, and CPU B only reads/writes variable b, they are still coupled to each other through the same cache line. Such problems are called false sharing.

In your example, we have two allocation schemes:

new Thread                               new Thread
new Container               vs           new Thread
new Thread                               ....
new Container                            new Container
....                                     new Container

In the first scheme, it's very unlikely that two volatile variables are close to each other. In the 2nd scheme, it's almost certainly the case.

CPU caches don't work with individual words; instead, they deal with cache lines. A cache line is a continuous chunk of memory, say 64 neighboring bytes. Usually this is nice - if a CPU accessed a cell, it's very likely that it will access the neighboring cells too. Except in your example, that assumption is not only invalid, but detrimental.

Suppose a and b fall in the same cache line L. When CPU A updates a, it notifies other CPUs that L is dirty. Since B caches L too, because it's working on b, B must drop its cached L. So next time B needs to read b, it must reload L, which is costly.

If B must access main memory to reload, that is extremely costly, it's usually 100X slower.

Fortunately, A and B can communicate directly about the new values without going through main memory. Nevertheless it takes extra time.

To verify this theory, you can stuff extra 128 bytes in Container, so that two volatile variable of two Container will not fall in the same cache line; then you should observe that the two schemes take about the same time to execute.

Lession learned: usually CPUs assume that adjecent variables are related. If we want independent variables, we better place them far away from each other.


来源: stackexchange

问:
How to extract historical weather data based on specific longitudes and latitudes?
I need to extract historical weather data at a monthly basis, from 2001 to 2018, based on specific locations in Europe (all locations are in the sea). I have the longitude and latitude stored in separate columns:

```<code>longitude    latitude

55.2000       6.8500
52.6450       1.7870
53.1350       1.1470
55.3430       10.95580
</code>```

I looked into RNCEP package in R, which stores weather data. But to extract it I have to insert the interval for latitude and longitude (for example from 0 to 60), which gets the weather data at increments of 2.5 for latitude and longitude. How can I extract it for the precise latitude and longitude I need?

This is the code that extracts the weather data at the 2.5 increment.

```<code>#Define limits for latitude and longitude
min_lat <- min(data$latitude, na.rm = TRUE)
max_lat <- max(data$latitude, na.rm = TRUE)

min_lon <- min(data$longitude, na.rm = TRUE)
max_lon <- max(data$longitude, na.rm = TRUE)

# define arguments for latitude and longitude
lat_range <- c(min_lat, max_lat)
lon_range <-c(min_lon, max_lon)

# get monthly air temperature between 2001 and 2018
weather <- NCEP.gather(variable = "air.sig995", level = "surface", months.minmax = c(1,12),
                       years.minmax = c(2001,2018), lat.southnorth =lat_range,
                       lon.westeast = lon_range)

# dimensions (obs. at time 00:00, 6:00, 12:00, 18:00 each day)
dim(weather) #creates 3 dimensions [lat, lon, time]

# extract date and time based on created weather dataset
date_time <- dimnames(weather)[[3]]
# format UTC date
date_time <- ymd_h(date_time)

# extract longitude & latitude based on created weather dataset
lat <- dimnames(weather)[[1]] # in increments of 2.5
lon <- dimnames(weather)[[2]] # in increments of 2.5

#Calculate the mean daily air temp. of the different times of day
w <- NCEP.aggregate(weather, YEARS = TRUE, MONTHS = TRUE, HOURS = FALSE, fxn='mean')

#Visualize temperature as heatmap for 1 day
NCEP.vis.area(w, layer = 1, show.pts = TRUE, draw.contours = TRUE, cols = heat.colors(64), transparency = 0.4)
</code>```

My result just extracts the historical weather data for the entire region, setting the range for longitude and latitude. But I need the weather conditions (such as mean temperature for the month) for the precise locations, based on my longitude and latitude columns for each month for all the years (2001 to 2018). Is it possible to do this with RNCEP package? Or what other options could I try?

Final results should be similar to this:

```<code>longitude    latitude   month  year  temperature

55.2000       6.8500     1     2001    20
55.2000       6.8500     2     2001    20
55.2000       6.8500     3     2001    20

...

55.2000       6.8500     1     2018    20

...

52.6450       1.7870     2

...

</code>```

I am open to any suggestion that could bring be closer to the solution, not only the final solution to the problem. Thanks.

答:
It is not possible to get more granular with RNCEP. That module is querying from the NCEP/NCAR Reanalysis and reanalysis 2 data set, and digging through those sites it looks like you cannot get more granular than 2.5 x 2.5 with the surface level data: Reanalysis 2/Reanalysis.

If you filter the NOAA data sets for temperature, and select the attribute view there are a handful of other data sets that you may be able to use. Some have different time granularity, and some are not kept up to the present.
I am going to try this GHCNCAMS data set for my needs because it has a grainularity of 0.5x0.5 degrees. To access the data directly(, you need to access NOAA via ftp, then read about netCDF file format/tools. There are a a lot of links/pages on NOAA's site as well

Also check out this opendata.stackexchange answers for some other places you could find this data.

ContentLicense: CC BY-SA 4.0

Body:
I need to extract historical weather data at a monthly basis, from 2001 to 2018, based on specific locations in Europe (all locations are in the sea). I have the longitude and latitude stored in separate columns:

longitude    latitude

55.2000       6.8500
52.6450       1.7870
53.1350       1.1470
55.3430       10.95580

I looked into RNCEP package in R, which stores weather data. But to extract it I have to insert the interval for latitude and longitude (for example from 0 to 60), which gets the weather data at increments of 2.5 for latitude and longitude. How can I extract it for the precise latitude and longitude I need?

This is the code that extracts the weather data at the 2.5 increment.

#Define limits for latitude and longitude
min_lat <- min(data$latitude, na.rm = TRUE)
max_lat <- max(data$latitude, na.rm = TRUE)

min_lon <- min(data$longitude, na.rm = TRUE)
max_lon <- max(data$longitude, na.rm = TRUE)

# define arguments for latitude and longitude
lat_range <- c(min_lat, max_lat)
lon_range <-c(min_lon, max_lon)

# get monthly air temperature between 2001 and 2018
weather <- NCEP.gather(variable = "air.sig995", level = "surface", months.minmax = c(1,12),
                       years.minmax = c(2001,2018), lat.southnorth =lat_range,
                       lon.westeast = lon_range)

# dimensions (obs. at time 00:00, 6:00, 12:00, 18:00 each day)
dim(weather) #creates 3 dimensions [lat, lon, time]

# extract date and time based on created weather dataset
date_time <- dimnames(weather)[[3]]
# format UTC date
date_time <- ymd_h(date_time)

# extract longitude & latitude based on created weather dataset
lat <- dimnames(weather)[[1]] # in increments of 2.5
lon <- dimnames(weather)[[2]] # in increments of 2.5

#Calculate the mean daily air temp. of the different times of day
w <- NCEP.aggregate(weather, YEARS = TRUE, MONTHS = TRUE, HOURS = FALSE, fxn='mean')

#Visualize temperature as heatmap for 1 day
NCEP.vis.area(w, layer = 1, show.pts = TRUE, draw.contours = TRUE, cols = heat.colors(64), transparency = 0.4)

My result just extracts the historical weather data for the entire region, setting the range for longitude and latitude. But I need the weather conditions (such as mean temperature for the month) for the precise locations, based on my longitude and latitude columns for each month for all the years (2001 to 2018). Is it possible to do this with RNCEP package? Or what other options could I try?

Final results should be similar to this:

longitude    latitude   month  year  temperature

55.2000       6.8500     1     2001    20
55.2000       6.8500     2     2001    20
55.2000       6.8500     3     2001    20

...

55.2000       6.8500     1     2018    20

...

52.6450       1.7870     2

...

I am open to any suggestion that could bring be closer to the solution, not only the final solution to the problem. Thanks.


问:
What is better to use just one method with switch or 4 different methods?
I have a object and i get some fields from it, i made a method with a switch statement, the idea was make it generic and just call this method through parameters but now I´m not sure.

The options are

```<code>private String getCode(Row row, String code) {
    String result;
    switch (code) {
      case code1:
        result = row.getString("constant1");
        break;
      case code2:
        result = row.getString(constant2);
        break;
      case code3:
        result = row.getString(constant3);
        break;
      case code4:
        result = row.getString(constant4);
        break;
      default:
        result = null;
    }
    return result;
  }
</code>```

or

```<code>private String getcode1(Row row){
    return row.getString("constant1")
  }

  private String getcode1(Row row){
    return row.getString("constant2")
  }
  private String getcode1(Row row){
    return row.getString("constant3")
  }

  private String getcode1(Row row){
    return row.getString("constant4")
  }
</code>```

I wand to use the better way, I´m a little confuse

答:
The answer is neither of the them.

What you have here is a value conversion process. How this works? Well, as far I understand, you need to store the data in form of key-value pair where key must be unique. So this is the definition of a HashMap in java.

Also, because you are using Switch I'm assuming that you can identify your data with some unique key. This meas for each code there is only one constant. But how we can apply this to my issue? Well, I think you have a collection of data from where you extracted a single row. Now, from this row you want to access to a value (lets call it codeValue) using a constant, but to get this constant you need a code. Like this:  code->constant->codeValue

How can I implement this??

Well, we gonna need a HasMap() called constants, which defines its keys a codes and the values as constants. Now you easily get each constant if you know its code. Obviously, if you get the constant you can also get the codeValue of each `row, like this:

public class TestClass {
    public static void main(String[] args) {

        // Create a HashMap object called constants
        Map<String, String> constants = new HashMap<String, String>();

        // Add keys and values (code, constant)
        constants.put("code1", "constant1");
        constants.put("code2", "constant2");
        constants.put("code3", "constant3");
        constants.put("code4", "constant4");

        System.out.println(constants);

        Row row  = loadRow();
        String code = loadCode();

        //How to use it
        String result = getCodeValue(row, code);

        System.out.println(result);
    }

    // Now you get the code from the Map
    private String getCodeValue(Row row, String code) {
        return row.getString(constants.get(code));
    }
}
```

Body:
I have a object and i get some fields from it, i made a method with a switch statement, the idea was make it generic and just call this method through parameters but now I´m not sure.

The options are

private String getCode(Row row, String code) {
    String result;
    switch (code) {
      case code1:
        result = row.getString("constant1");
        break;
      case code2:
        result = row.getString(constant2);
        break;
      case code3:
        result = row.getString(constant3);
        break;
      case code4:
        result = row.getString(constant4);
        break;
      default:
        result = null;
    }
    return result;
  }

or

private String getcode1(Row row){
    return row.getString("constant1")
  }

  private String getcode1(Row row){
    return row.getString("constant2")
  }
  private String getcode1(Row row){
    return row.getString("constant3")
  }

  private String getcode1(Row row){
    return row.getString("constant4")
  }

I wand to use the better way, I´m a little confuse


问:
How Can I Use Order By In Aggregate Function
I have a sql table seems like below

```<code>ID  AccountName Credit Debit CurrentBalance
1   John         10     0     10
2   Smith        20     0     20
3   Angel        0      10    20
4   John         30     0     40
5   Smith        0      10    10
6   Angel        30     0     50
</code>```

Expected Result

```<code>AccountName TotalCredit TotalDebit CurrentBalance
John        40          0          40
Smith       20          10         10
Angel       30          10         50
</code>```

I want the sum of credit and debit of every account and also the latest currentbalance of each account.

I have used this query

```<code>select AccountName,
       SUM(CreditAmount) As CreditAmount,
       SUM(DebitAmount) As DebitAmount,
       MAX(CurrentBalance) As CurrentBalance
from TransactionInfo
GROUP BY AccountName;
</code>```

答:
If your database supports ROW_NUMBER, then here is one way to do this:

```<code>WITH cte AS (
    SELECT *, ROW_NUMBER() OVER (PARTITION BY AccountNumber ORDER BY ID DESC) rn
    FROM TransactionInfo
)

SELECT
    AccountNumber,
    SUM(Credit) AS TotalCredit,
    SUM(Debit) AS TotalDebit,
    MAX(CASE WHEN rn = 1 THEN CurrentBalance END) AS CurrentBalance
FROM cte
GROUP BY
    AccountNumber;
</code>```

Body:
I have a sql table seems like below

ID  AccountName Credit Debit CurrentBalance
1   John         10     0     10
2   Smith        20     0     20
3   Angel        0      10    20
4   John         30     0     40
5   Smith        0      10    10
6   Angel        30     0     50

Expected Result

AccountName TotalCredit TotalDebit CurrentBalance
John        40          0          40
Smith       20          10         10
Angel       30          10         50

I want the sum of credit and debit of every account and also the latest currentbalance of each account.

I have used this query

select AccountName,
       SUM(CreditAmount) As CreditAmount,
       SUM(DebitAmount) As DebitAmount,
       MAX(CurrentBalance) As CurrentBalance
from TransactionInfo
GROUP BY AccountName;

Body:
If your database supports ROW_NUMBER, then here is one way to do this:

WITH cte AS (
    SELECT *, ROW_NUMBER() OVER (PARTITION BY AccountNumber ORDER BY ID DESC) rn
    FROM TransactionInfo
)

SELECT
    AccountNumber,
    SUM(Credit) AS TotalCredit,
    SUM(Debit) AS TotalDebit,
    MAX(CASE WHEN rn = 1 THEN CurrentBalance END) AS CurrentBalance
FROM cte
GROUP BY
    AccountNumber;


问:
What are the .hi and .o files generated by GHC (on Linux)?
I am just getting started with Haskell and functional programming in general. After compiling and running my first Haskell program, I noticed that GHC (on Linux) generates three separate files from a single .hs source file: a .hi file, a .o file, and finally an executable. What is the purpose of each of these files? Practically, when are .hi and .o files actually used?

From my little experience in C, I think .o is an object file. Searching Google shows that .hi is an "interface file".

答:
The .o is exactly the same as C's object files; the .hi file is an "interface file"; it contains information about the .o that GHC would need, if you compile other modules, to be able to link against that .o file (said information cannot be stored in a standard .o file).

You could say that the .hi file is the equivalent of C's header files (that is, with .h extension), only these are generated by GHC from the original Haskell source.

Thus, the .hi is used when GHC compiles other modules, and the .o is used when linking all modules together to produce the executable.

You can safely delete the .hi and .o files once you have successfully generated the executable (or keep them if you want to make some small change and rebuild quickly - it will save time in unneeded recompilations).


问:
C# regex to accept current/voltage value
I need a regex which can validate a input string if it is valid voltage or current.
Required format will be any number followed by 'V' or 'A' is valid.

I was thinking to split string and validate it but it not robust solution and hard to implement.

答:
You can use something like this:

```<code>^[0-9]+\s?[vaVA]$
</code>```

Another, more compact, expression would be

```<code>^\d+\s?[vaVA]$
</code>```

Or, you could simply include the characters [va] and mark the expressing as case insensitive.

If you don’t want to accept input with space between the number and the unit, leave the \s? out. So the expression would be

```<code>^\d+[vaVA]$
</code>```

If you want to allow negative numbers, include the dash as optional in front of the number, like so:

```<code>^-?\d+\s?[vaVA]$
</code>```

And if you would like to accept decimal values:

```<code> ^-?\d+(\.\d+)?\s?[vaVA]$
</code>```

Since the question is marked C#, make sure to either escape the RegEx definitions or use the @ operator with the string (ie. @"^\d+[avAV]$")

Update

To strictly match, you can use the start of line(^) and end of line anchors ($) respectively. I’ve updated the example expression to include these.

Body:
You can use something like this:

^[0-9]+\s?[vaVA]$

Another, more compact, expression would be

^\d+\s?[vaVA]$

Or, you could simply include the characters [va] and mark the expressing as case insensitive.

If you don’t want to accept input with space between the number and the unit, leave the \s? out. So the expression would be

^\d+[vaVA]$

If you want to allow negative numbers, include the dash as optional in front of the number, like so:

^-?\d+\s?[vaVA]$

And if you would like to accept decimal values:

 ^-?\d+(\.\d+)?\s?[vaVA]$

Since the question is marked C#, make sure to either escape the RegEx definitions or use the @ operator with the string (ie. @"^\d+[avAV]$")

Update

To strictly match, you can use the start of line(^) and end of line anchors ($) respectively. I’ve updated the example expression to include these.


问:
IEnumerable<T> null coalescing Extension
I frequently face the problem to check whether an IEnumerable<T> is null before iterating over it through foreach or LINQ queries, then I often come into codes like this:

```<code>var myProjection = (myList ?? Enumerable.Empty<T>()).Select(x => x.Foo)...
</code>```

Hence, I thought to add this extension-method to an Extensions class:

```<code>public static class MyExtensions
{
    public static IEnumerable<T> AsEmptyIfNull<T>(this IEnumerable<T> source)
    {
        return source ?? Enumerable.Empty<T>();
    }
}
</code>```

A little issue comes immediately in my mind looking at this code, i.e., given the "instance-methods aspect" of extension-methods, it should be implemented as a mere static method, otherwise something like this would be perfectly legal:

```<code>IEnumerable<int> list = null;
list.AsEmptyIfNull();
</code>```

Do you see any other drawback in using it ?

Can such extension leading to some kind of bad-trend in the developer(s), if massively used?

Bonus question:

Can you suggest a better name to it ?  :)

(English is not my first language, then I'm not so good in naming...)

Thanks in advance.

答:
Methods that return an IEnumerable<T> should return an empty one, instead of null. So you wouldn't need this.

See this question : Is it better to return null or empty collection?

Otherwise, your code seems ok.

ContentLicense: CC BY-SA 3.0

Body:
I frequently face the problem to check whether an IEnumerable<T> is null before iterating over it through foreach or LINQ queries, then I often come into codes like this:

var myProjection = (myList ?? Enumerable.Empty<T>()).Select(x => x.Foo)...

Hence, I thought to add this extension-method to an Extensions class:

public static class MyExtensions
{
    public static IEnumerable<T> AsEmptyIfNull<T>(this IEnumerable<T> source)
    {
        return source ?? Enumerable.Empty<T>();
    }
}

A little issue comes immediately in my mind looking at this code, i.e., given the "instance-methods aspect" of extension-methods, it should be implemented as a mere static method, otherwise something like this would be perfectly legal:

IEnumerable<int> list = null;
list.AsEmptyIfNull();

Do you see any other drawback in using it ?

Can such extension leading to some kind of bad-trend in the developer(s), if massively used?

Bonus question:

Can you suggest a better name to it ?  :)

(English is not my first language, then I'm not so good in naming...)

Thanks in advance.


答:
.o files are indeed object files.  Basically chunks of bytecode ready to be linked together.

.hi files are interface files.  The short version is that they hold something like compiled type signatures along with information that lets GHC perform optimization across file boundaries.

I personally find having these files in my working directory annoying enough that I add -outputdir ../tmp to my ghc invocation.

Title: What are the .hi and .o files generated by GHC (on Linux)?

Body:
I am just getting started with Haskell and functional programming in general. After compiling and running my first Haskell program, I noticed that GHC (on Linux) generates three separate files from a single .hs source file: a .hi file, a .o file, and finally an executable. What is the purpose of each of these files? Practically, when are .hi and .o files actually used?

From my little experience in C, I think .o is an object file. Searching Google shows that .hi is an "interface file".


问:
In pytorch, how to fill a tensor with another tensor?
I'm looking for a way to expand the size of an image by adding 0 values to the right & lower edges of it. My initial plan is to use nn.padding to add the edge, until I encounter this error:

```<code>  File "/home/shared/virtualenv/dl-torch/lib/python3.7/site-packages/torch/nn/functional.py", line 2796, in pad
    assert len(pad) % 2 == 0, 'Padding length must be divisible by 2'
AssertionError: Padding length must be divisible by 2
</code>```

It appears that torch tries to pad the image from both side! Is there an easy way to override this and fill the tensor into the upper-left side of another image?

Body:
I'm looking for a way to expand the size of an image by adding 0 values to the right & lower edges of it. My initial plan is to use nn.padding to add the edge, until I encounter this error:

  File "/home/shared/virtualenv/dl-torch/lib/python3.7/site-packages/torch/nn/functional.py", line 2796, in pad
    assert len(pad) % 2 == 0, 'Padding length must be divisible by 2'
AssertionError: Padding length must be divisible by 2

It appears that torch tries to pad the image from both side! Is there an easy way to override this and fill the tensor into the upper-left side of another image?

Body:
With nn.ConstantPad2d, you can specify the number of padding elements in all four directions separately.

>>> t = torch.randn(2,3)
>>> t
tensor([[ 0.1254,  0.6358,  0.3243],
        [ 0.7005, -0.4931,  1.0582]])
>>> p = torch.nn.ConstantPad2d((0, 4, 0, 2), 0)
>>> p(t)
tensor([[ 0.1254,  0.6358,  0.3243,  0.0000,  0.0000,  0.0000,  0.0000],
        [ 0.7005, -0.4931,  1.0582,  0.0000,  0.0000,  0.0000,  0.0000],
        [ 0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000],
        [ 0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000]])


问:
Wired shift caused by when using pd.concat(x,y)
I have a data set x and y. x is a one list that applies to all y data. Using the pd.concat([pd.Series(x),pd.DataFrame(y)], axis=1), I hope to attain the following:

```<code>x1   y1   ya
x2   y2   yb
x3   y3   yc
x4   y4   yd
x5   y5   ye
x6   y6   yf
</code>```

x contains x1,x2,x3,x4,x5,x6.

y contains [y1,y2,y3,y4,y5,y6],    [ya,yb,yc,yd,ye,yf].

x and y are in pandas DataFrame.

Running either pd.concat([pd.Series(x),pd.DataFrame(y)], axis=1) or pd.concat([pd.DataFrame(x),pd.DataFrame(y)], axis=1) results in the following

```<code>x1   Nan   Nan
x2   Nan   Nan
x3   y1   ya
x4   y2   yb
x5   y3   yc
x6   y4   yd
     y5   ye
     y6   yf
</code>```

So the code y values are shifted to the axis=0 direction by two cells. I have index in this DataFrame. I have removed the index. No difference was observed.

This shift is observed only once I concatenate x and y.

How could I shift the y values upward by two?

Thanks!

答:
This happens when the indexes of the 2 series are not aligned. Try after resetting the index (reset_index(drop=True))so both series or dataframes have indexes aligned:

```<code>pd.concat([pd.Series(x).reset_index(drop=True),pd.DataFrame(y)
               .reset_index(drop=True)], axis=1)
</code>```

Or:

```<code>pd.concat([pd.DataFrame(x).reset_index(drop=True),pd.DataFrame(y)
                              .reset_index(drop=True)],axis=1)
</code>```

Adding an example:

```<code>s1=pd.Series([1,2,3]) #normal indexed
s2=pd.Series([4,5,6],index=[2,3,4]) #index starts from 2
</code>```

Normal concat on axis=1

```<code>pd.concat([s1,s2],axis=1)
</code>```

```<code>     0    1
0  1.0  NaN
1  2.0  NaN
2  3.0  4.0
3  NaN  5.0
4  NaN  6.0
</code>```

Concat with reset_index()

```<code>pd.concat([s1.reset_index(drop=True),s2.reset_index(drop=True)],axis=1)
</code>```

```<code>   0  1
0  1  4
1  2  5
2  3  6
</code>```

Body:
I have a data set x and y. x is a one list that applies to all y data. Using the pd.concat([pd.Series(x),pd.DataFrame(y)], axis=1), I hope to attain the following:

x1   y1   ya
x2   y2   yb
x3   y3   yc
x4   y4   yd
x5   y5   ye
x6   y6   yf

x contains x1,x2,x3,x4,x5,x6.

y contains [y1,y2,y3,y4,y5,y6],    [ya,yb,yc,yd,ye,yf].

x and y are in pandas DataFrame.

Running either pd.concat([pd.Series(x),pd.DataFrame(y)], axis=1) or pd.concat([pd.DataFrame(x),pd.DataFrame(y)], axis=1) results in the following

x1   Nan   Nan
x2   Nan   Nan
x3   y1   ya
x4   y2   yb
x5   y3   yc
x6   y4   yd
     y5   ye
     y6   yf

So the code y values are shifted to the axis=0 direction by two cells. I have index in this DataFrame. I have removed the index. No difference was observed.

This shift is observed only once I concatenate x and y.

How could I shift the y values upward by two?

Thanks!

Body:
This happens when the indexes of the 2 series are not aligned. Try after resetting the index (reset_index(drop=True))so both series or dataframes have indexes aligned:

pd.concat([pd.Series(x).reset_index(drop=True),pd.DataFrame(y)
               .reset_index(drop=True)], axis=1)

Or:

pd.concat([pd.DataFrame(x).reset_index(drop=True),pd.DataFrame(y)
                              .reset_index(drop=True)],axis=1)

Adding an example:

s1=pd.Series([1,2,3]) #normal indexed
s2=pd.Series([4,5,6],index=[2,3,4]) #index starts from 2

Normal concat on axis=1

pd.concat([s1,s2],axis=1)

     0    1
0  1.0  NaN
1  2.0  NaN
2  3.0  4.0
3  NaN  5.0
4  NaN  6.0

Concat with reset_index()

pd.concat([s1.reset_index(drop=True),s2.reset_index(drop=True)],axis=1)

   0  1
0  1  4
1  2  5
2  3  6


问:
Test an error response in a method that's internal to a struct
I'm writing unit tests, and I want to write a unit test that asserts that a public method on the struct (Foo.Start) properly handles error responses from an internal method on the struct (Foo.internal).

Essentially I want to get test coverage on the this section of my code:

if err != nil {
    return err
}
```

<p>Here's an example of code and associated test that doesn't work (but would work in say, Python)</p>

<pre class="lang-golang prettyprint-override"><code># example.go

package example

import "fmt"

type FooAPI interface {
    Start() error
    internal(string) (string, error)
}

type Foo struct {
    FooAPI
}

func (foo Foo) Start() (err error) {

    data, err := foo.internal("bar")
    if err != nil {
        return err
    }

    fmt.Println(data)
    return err
}

func (foo Foo) internal(input string) (output string, err error) {
    return output, err
}
</code>```

```<code># example_test.go

package example

import (
    "testing"

    "github.com/pkg/errors"
)

type MockFoo struct {
    FooAPI
}

func (foo MockFoo) internal(input string) (output string, err error) {
    return output, errors.New("big error")
}

func TestStart(t *testing.T) {
    tdata := []struct {
        testCase        string
        expectedAnError bool
        foo             FooAPI
    }{
        {
            testCase:        "standard_case",
            expectedAnError: false,
            foo:             Foo{},
        },
        {
            testCase:        "error_case",
            expectedAnError: true,
            foo:             MockFoo{},
        },
    }
    for _, test := range tdata {
        t.Run(test.testCase, func(t *testing.T) {
            // The function under test
            test.foo.Start = Foo.Start // <= this doesn't work
            err := test.foo.Start()

            // Assertion 1
            if test.expectedAnError == false && err != nil {
                t.Error(err.Error())
            }

            // Assertion 2
            if test.expectedAnError == true && err == nil {
                t.Error("expected an error, but there was none!")
            }
        })
    }
}
</code>```

I'm less interested in correcting the specific example, more-so my goal is to get test coverage on Foo.Start's error handling. I feel like there's some trick with interfaces or pointers that will get me across the finish line here?

答:
I figured out one solution, inspired by https://stackoverflow.com/a/48206430/3055558

Using a internal{{ your struct }} struct and associated interface, and mocking that.

# example.go

package example

import "fmt"

type internalFooAPI interface {
    internalBehavior(string) (string, error)
}

type Foo struct {
    internal internalFooAPI
}

type internalFoo struct{}

func NewFoo(internal internalFooAPI) Foo {
    return Foo{
        internal: internal,
    }
}

func (foo Foo) Start() (err error) {

    data, err := foo.internal.internalBehavior("bar")
    if err != nil {
        return err
    }

    fmt.Println(data)
    return err
}

func (foo internalFoo) internalBehavior(input string) (output string, err error) {
    return output, err
}
```

```# example_test.go

package example

import (
    "testing"

    "github.com/pkg/errors"
)

type mockInternalFoo struct{}

type mockInternalFooWithErrors struct{}

func (foo mockInternalFoo) internalBehavior(input string) (output string, err error) {
    return output, err
}

func (foo mockInternalFooWithErrors) internalBehavior(input string) (output string, err error) {
    return output, errors.New("big error")
}

func TestStart(t *testing.T) {
    tdata := []struct {
        testCase        string
        expectedAnError bool
        foo             Foo
    }{
        {
            testCase:        "standard_case",
            expectedAnError: false,
            foo:             NewFoo(internalFoo{}),
        },
        {
            testCase:        "mock_case",
            expectedAnError: false,
            foo:             NewFoo(mockInternalFoo{}),
        },
        {
            testCase:        "error_case",
            expectedAnError: true,
            foo:             NewFoo(mockInternalFooWithErrors{}),
        },
    }
    for _, test := range tdata {
        t.Run(test.testCase, func(t *testing.T) {
            // The function under test
            err := test.foo.Start()

            // Assertion 1
            if test.expectedAnError == false && err != nil {
                t.Error(err.Error())
            }

            // Assertion 2
            if test.expectedAnError == true && err == nil {
                t.Error("expected an error, but there was none!")
            }
        })
    }
}
```

Body:
I'm writing unit tests, and I want to write a unit test that asserts that a public method on the struct (Foo.Start) properly handles error responses from an internal method on the struct (Foo.internal).

Essentially I want to get test coverage on the this section of my code:

if err != nil {
    return err
}

Here's an example of code and associated test that doesn't work (but would work in say, Python)

# example.go

package example

import "fmt"

type FooAPI interface {
    Start() error
    internal(string) (string, error)
}

type Foo struct {
    FooAPI
}

func (foo Foo) Start() (err error) {

    data, err := foo.internal("bar")
    if err != nil {
        return err
    }

    fmt.Println(data)
    return err
}

func (foo Foo) internal(input string) (output string, err error) {
    return output, err
}

# example_test.go

package example

import (
    "testing"

    "github.com/pkg/errors"
)

type MockFoo struct {
    FooAPI
}

func (foo MockFoo) internal(input string) (output string, err error) {
    return output, errors.New("big error")
}

func TestStart(t *testing.T) {
    tdata := []struct {
        testCase        string
        expectedAnError bool
        foo             FooAPI
    }{
        {
            testCase:        "standard_case",
            expectedAnError: false,
            foo:             Foo{},
        },
        {
            testCase:        "error_case",
            expectedAnError: true,
            foo:             MockFoo{},
        },
    }
    for _, test := range tdata {
        t.Run(test.testCase, func(t *testing.T) {
            // The function under test
            test.foo.Start = Foo.Start // <= this doesn't work
            err := test.foo.Start()

            // Assertion 1
            if test.expectedAnError == false && err != nil {
                t.Error(err.Error())
            }

            // Assertion 2
            if test.expectedAnError == true && err == nil {
                t.Error("expected an error, but there was none!")
            }
        })
    }
}

I'm less interested in correcting the specific example, more-so my goal is to get test coverage on Foo.Start's error handling. I feel like there's some trick with interfaces or pointers that will get me across the finish line here?

Body:
I figured out one solution, inspired by https://stackoverflow.com/a/48206430/3055558

Using a internal{{ your struct }} struct and associated interface, and mocking that.

# example.go

package example

import "fmt"

type internalFooAPI interface {
    internalBehavior(string) (string, error)
}

type Foo struct {
    internal internalFooAPI
}

type internalFoo struct{}

func NewFoo(internal internalFooAPI) Foo {
    return Foo{
        internal: internal,
    }
}

func (foo Foo) Start() (err error) {

    data, err := foo.internal.internalBehavior("bar")
    if err != nil {
        return err
    }

    fmt.Println(data)
    return err
}

func (foo internalFoo) internalBehavior(input string) (output string, err error) {
    return output, err
}

# example_test.go

package example

import (
    "testing"

    "github.com/pkg/errors"
)

type mockInternalFoo struct{}

type mockInternalFooWithErrors struct{}

func (foo mockInternalFoo) internalBehavior(input string) (output string, err error) {
    return output, err
}

func (foo mockInternalFooWithErrors) internalBehavior(input string) (output string, err error) {
    return output, errors.New("big error")
}

func TestStart(t *testing.T) {
    tdata := []struct {
        testCase        string
        expectedAnError bool
        foo             Foo
    }{
        {
            testCase:        "standard_case",
            expectedAnError: false,
            foo:             NewFoo(internalFoo{}),
        },
        {
            testCase:        "mock_case",
            expectedAnError: false,
            foo:             NewFoo(mockInternalFoo{}),
        },
        {
            testCase:        "error_case",
            expectedAnError: true,
            foo:             NewFoo(mockInternalFooWithErrors{}),
        },
    }
    for _, test := range tdata {
        t.Run(test.testCase, func(t *testing.T) {
            // The function under test
            err := test.foo.Start()

            // Assertion 1
            if test.expectedAnError == false && err != nil {
                t.Error(err.Error())
            }

            // Assertion 2
            if test.expectedAnError == true && err == nil {
                t.Error("expected an error, but there was none!")
            }
        })
    }
}


问:
Python 3: Pass variable from one CLASS to another
I have a beginner problem here. I have a parent Class that contains a call to SQLite DB and to ConfigParser ini file and I want my child Class to access all those pieces of information.

Below is my code and what I've tried so far with no success. I understand instantiation but only with minimal examples like employees/salaries lol.

My code works if I copy all the calls to DB and config files into the child Class and it's not how it should be done, I know. Could you please help me?

```<code>class SimpleGrid(gridlib.Grid): ##, mixins.GridAutoEditMixin):
    def __init__(self, parent, log):
        gridlib.Grid.__init__(self, parent, -1)

        self.wxFont = "9, wx.FONTFAMILY_DEFAULT, wx.NORMAL, wx.NORMAL"

        path =os.path.dirname(os.path.realpath(__file__))
        configFile = os.path.join(path , "config.ini")

        dbfile = os.path.join(path , "report.db")
        self.db_conn = sqlite3.connect(dbfile)
        self.theCursor =  self.db_conn.cursor()

        config = configparser.ConfigParser()
        config.read(configFile)

        if config.get('Network', 'taboola') == "true" and config.get('Network', 'outbrain') == "true":
            network = ""
        elif config.get('Network', 'taboola') == "true" and config.get('Network', 'outbrain') == "false":
            network = " and CAMPAIGN LIKE '%TB%'"
        elif config.get('Network', 'outbrain') == "true" and config.get('Network', 'taboola') == "false":
            network = " and CAMPAIGN LIKE '%OB%'"
        else:
            network = ""

        if config.get('Sort', 'value') == "Impr.":
            sort = "IMPR"
        elif config.get('Sort', 'value') == "Clicks":
            sort = "CLICKS"
        elif config.get('Sort', 'value') == "CTR":
            sort = "CTR"
        elif config.get('Sort', 'value') == "CPC":
            sort = "CPC"
        elif config.get('Sort', 'value') == "eCPC":
            sort = "eCPC"
        elif config.get('Sort', 'value') == "Spent":
            sort = "SPENT"
        elif config.get('Sort', 'value') == "Revenue":
            sort = "GA_REV"
        elif config.get('Sort', 'value') == "GA Impr.":
            sort = "GA_IMPR"
        elif config.get('Sort', 'value') == "GA Clicks":
            sort = "GA_CLICKS"
        elif config.get('Sort', 'value') == "GA CTR":
            sort = "GA_CTR"
        elif config.get('Sort', 'value') == "Rev. /1000":
            sort = "GA_RPM"
        else:
            sort = "SPENT"

        #['Impr.', 'Clicks', 'CTR', 'CPC', 'eCPC', 'Spent', 'Revenue', 'GA Impr.', 'GA Clicks', 'GA CTR', 'Rev. /1000']

        if config.get('Filter', 'column') == "Campaign":
            column = "CAMPAIGN"
        elif config.get('Filter', 'column') == "Impr.":
            column = "IMPR"
        elif config.get('Filter', 'column') == "Clicks":
            column = "CLICKS"
        elif config.get('Filter', 'column') == "CTR":
            column = "CTR"
        elif config.get('Filter', 'column') == "CPC":
            column = "CPC"
        elif config.get('Filter', 'column') == "eCPC":
            column = "eCPC"
        elif config.get('Filter', 'column') == "Spent":
            column = "SPENT"
        elif config.get('Filter', 'column') == "Revenue":
            column = "GA_REV"
        elif config.get('Filter', 'column') == "GA Impr.":
            column = "GA_IMPR"
        elif config.get('Filter', 'column') == "GA Clicks":
            column = "GA_CLICKS"
        elif config.get('Filter', 'column') == "GA CTR":
            column = "GA_CTR"
        elif config.get('Filter', 'column') == "Rev. /1000":
            column = "GA_RPM"
        else:
            column = ""

        if config.get('Filter', 'operator') == "Contains":
            query = "and " + column + " LIKE '%" + config.get('Filter', 'string') + "%'"
        elif config.get('Filter', 'operator') == "Doesn't Contain":
            query = "and " + column + " NOT LIKE '%" + config.get('Filter', 'string') + "%'"

        if config.get('Filter', 'operator') == "<":
            query = "and " + column + " < " + config.get('Filter', 'string')
        elif config.get('Filter', 'operator') == ">":
            query = "and " + column + " > " + config.get('Filter', 'string')

        #To instantiate
        queryFiltre = "SELECT * FROM Report WHERE  GA_RPM > 0 " + query + network + "  and STATUS = '1' ORDER BY " + sort + " DESC"
        print(queryFiltre)

        rows = self.db_conn.execute(queryFiltre)
        rowsCount = len(rows.fetchall())

class TestFrame(wx.Frame):
    def __init__(self, parent, log):
        wx.Frame.__init__(self, parent, 0, "V1.0", size=(1400,800))
        self.grid = SimpleGrid(self, log)
</code>```

What I've done so far:

```<code>class SimpleGrid(gridlib.Grid): ##, mixins.GridAutoEditMixin):
    def __init__(self, parent, log):
        gridlib.Grid.__init__(self, parent, -1)

        self.wxFont = "9, wx.FONTFAMILY_DEFAULT, wx.NORMAL, wx.NORMAL"
        self.queryFiltre = ""

        def setQueryFiltre(queryFiltre):
            path =os.path.dirname(os.path.realpath(__file__))
            configFile = os.path.join(path , "config.ini")
            dbfile = os.path.join(path , "report.db")
            self.db_conn = sqlite3.connect(dbfile)
            self.theCursor =  self.db_conn.cursor()

            config = configparser.ConfigParser()
            config.read(configFile)

            #All the calls to DB and config file here

            return queryFiltre

        def getQueryFiltre(queryFiltre):
            queryFiltre = queryFiltre

        rows = db_conn.execute(queryFiltre)
        rowsCount = len(rows.fetchall())

class TestFrame(wx.Frame):
    def __init__(self, parent, log):
        wx.Frame.__init__(self, parent, 0, "V1.0", size=(1400,800))
        self.grid = SimpleGrid(self, log)

        self.wxFont = SimpleGrid(self.wxFont)
        self.queryFiltre = SimpleGrid.getQueryFiltre()
</code>```

But here the first error I get is the db_conn is not defined and I'm sure I will still have more to handel if this one is taken car of. In fact all the variables within the def setQueryFiltre should be accessible from within both the parent and the child Classes.

Thank you,

EDIT:

With oe without self. i still get the same error

答:
You declared 2 functions inside your constructor, which never get called: setQueryFiltre and getQueryFiltre. Moreover, your test calls one of them as if it was a class method. It's not, and it will fail becuase that declaration is only visible inside the function. What you probably meant to write is this:

```<code>class SimpleGrid(gridlib.Grid): ##, mixins.GridAutoEditMixin):
    def __init__(self, parent, log):
        gridlib.Grid.__init__(self, parent, -1)
        self.setQueryFiltre(None)

        self.wxFont = "9, wx.FONTFAMILY_DEFAULT, wx.NORMAL, wx.NORMAL"
        self.queryFiltre = ""

    def setQueryFiltre(self, queryFiltre):
        path = os.path.dirname(os.path.realpath(__file__))
        configFile = os.path.join(path , "config.ini")
        dbfile = os.path.join(path , "report.db")
        self.db_conn = sqlite3.connect(dbfile)
        self.theCursor =  self.db_conn.cursor()

        config = configparser.ConfigParser()
        config.read(configFile)

        #All the calls to DB and config file here
        return queryFiltre # why are you returning the argument? For chaining?

    def getQueryFiltre(self, queryFiltre):
        queryFiltre = queryFiltre

        rows = self.db_conn.execute(queryFiltre)
        rowsCount = len(rows.fetchall())
        # you probably want to return rows here?

class TestFrame(wx.Frame):
    def __init__(self, parent, log):
        wx.Frame.__init__(self, parent, 0, "V1.0", size=(1400,800))
        self.grid = SimpleGrid(self, log)

        # self.wxFont = SimpleGrid(self.wxFont) # what does this try to do?
        self.queryFiltre = self.grid.getQueryFiltre()
</code>```

In Python, indentation is important.

Note that I also moved db initialization to constructor, where it belongs. Because otherwise, if get method gets called before set, you will still have the same crash. I also made a couple other comments that you'll probably get bitten by next.

Body:
I have a beginner problem here. I have a parent Class that contains a call to SQLite DB and to ConfigParser ini file and I want my child Class to access all those pieces of information.

Below is my code and what I've tried so far with no success. I understand instantiation but only with minimal examples like employees/salaries lol.

My code works if I copy all the calls to DB and config files into the child Class and it's not how it should be done, I know. Could you please help me?

class SimpleGrid(gridlib.Grid): ##, mixins.GridAutoEditMixin):
    def __init__(self, parent, log):
        gridlib.Grid.__init__(self, parent, -1)

        self.wxFont = "9, wx.FONTFAMILY_DEFAULT, wx.NORMAL, wx.NORMAL"

        path =os.path.dirname(os.path.realpath(__file__))
        configFile = os.path.join(path , "config.ini")

        dbfile = os.path.join(path , "report.db")
        self.db_conn = sqlite3.connect(dbfile)
        self.theCursor =  self.db_conn.cursor()

        config = configparser.ConfigParser()
        config.read(configFile)

        if config.get('Network', 'taboola') == "true" and config.get('Network', 'outbrain') == "true":
            network = ""
        elif config.get('Network', 'taboola') == "true" and config.get('Network', 'outbrain') == "false":
            network = " and CAMPAIGN LIKE '%TB%'"
        elif config.get('Network', 'outbrain') == "true" and config.get('Network', 'taboola') == "false":
            network = " and CAMPAIGN LIKE '%OB%'"
        else:
            network = ""

        if config.get('Sort', 'value') == "Impr.":
            sort = "IMPR"
        elif config.get('Sort', 'value') == "Clicks":
            sort = "CLICKS"
        elif config.get('Sort', 'value') == "CTR":
            sort = "CTR"
        elif config.get('Sort', 'value') == "CPC":
            sort = "CPC"
        elif config.get('Sort', 'value') == "eCPC":
            sort = "eCPC"
        elif config.get('Sort', 'value') == "Spent":
            sort = "SPENT"
        elif config.get('Sort', 'value') == "Revenue":
            sort = "GA_REV"
        elif config.get('Sort', 'value') == "GA Impr.":
            sort = "GA_IMPR"
        elif config.get('Sort', 'value') == "GA Clicks":
            sort = "GA_CLICKS"
        elif config.get('Sort', 'value') == "GA CTR":
            sort = "GA_CTR"
        elif config.get('Sort', 'value') == "Rev. /1000":
            sort = "GA_RPM"
        else:
            sort = "SPENT"

        #['Impr.', 'Clicks', 'CTR', 'CPC', 'eCPC', 'Spent', 'Revenue', 'GA Impr.', 'GA Clicks', 'GA CTR', 'Rev. /1000']

        if config.get('Filter', 'column') == "Campaign":
            column = "CAMPAIGN"
        elif config.get('Filter', 'column') == "Impr.":
            column = "IMPR"
        elif config.get('Filter', 'column') == "Clicks":
            column = "CLICKS"
        elif config.get('Filter', 'column') == "CTR":
            column = "CTR"
        elif config.get('Filter', 'column') == "CPC":
            column = "CPC"
        elif config.get('Filter', 'column') == "eCPC":
            column = "eCPC"
        elif config.get('Filter', 'column') == "Spent":
            column = "SPENT"
        elif config.get('Filter', 'column') == "Revenue":
            column = "GA_REV"
        elif config.get('Filter', 'column') == "GA Impr.":
            column = "GA_IMPR"
        elif config.get('Filter', 'column') == "GA Clicks":
            column = "GA_CLICKS"
        elif config.get('Filter', 'column') == "GA CTR":
            column = "GA_CTR"
        elif config.get('Filter', 'column') == "Rev. /1000":
            column = "GA_RPM"
        else:
            column = ""

        if config.get('Filter', 'operator') == "Contains":
            query = "and " + column + " LIKE '%" + config.get('Filter', 'string') + "%'"
        elif config.get('Filter', 'operator') == "Doesn't Contain":
            query = "and " + column + " NOT LIKE '%" + config.get('Filter', 'string') + "%'"

        if config.get('Filter', 'operator') == "<":
            query = "and " + column + " < " + config.get('Filter', 'string')
        elif config.get('Filter', 'operator') == ">":
            query = "and " + column + " > " + config.get('Filter', 'string')

        #To instantiate
        queryFiltre = "SELECT * FROM Report WHERE  GA_RPM > 0 " + query + network + "  and STATUS = '1' ORDER BY " + sort + " DESC"
        print(queryFiltre)

        rows = self.db_conn.execute(queryFiltre)
        rowsCount = len(rows.fetchall())

class TestFrame(wx.Frame):
    def __init__(self, parent, log):
        wx.Frame.__init__(self, parent, 0, "V1.0", size=(1400,800))
        self.grid = SimpleGrid(self, log)

What I've done so far:

class SimpleGrid(gridlib.Grid): ##, mixins.GridAutoEditMixin):
    def __init__(self, parent, log):
        gridlib.Grid.__init__(self, parent, -1)

        self.wxFont = "9, wx.FONTFAMILY_DEFAULT, wx.NORMAL, wx.NORMAL"
        self.queryFiltre = ""

        def setQueryFiltre(queryFiltre):
            path =os.path.dirname(os.path.realpath(__file__))
            configFile = os.path.join(path , "config.ini")
            dbfile = os.path.join(path , "report.db")
            self.db_conn = sqlite3.connect(dbfile)
            self.theCursor =  self.db_conn.cursor()

            config = configparser.ConfigParser()
            config.read(configFile)

            #All the calls to DB and config file here

            return queryFiltre

        def getQueryFiltre(queryFiltre):
            queryFiltre = queryFiltre

        rows = db_conn.execute(queryFiltre)
        rowsCount = len(rows.fetchall())

class TestFrame(wx.Frame):
    def __init__(self, parent, log):
        wx.Frame.__init__(self, parent, 0, "V1.0", size=(1400,800))
        self.grid = SimpleGrid(self, log)

        self.wxFont = SimpleGrid(self.wxFont)
        self.queryFiltre = SimpleGrid.getQueryFiltre()

But here the first error I get is the db_conn is not defined and I'm sure I will still have more to handel if this one is taken car of. In fact all the variables within the def setQueryFiltre should be accessible from within both the parent and the child Classes.

Thank you,

EDIT:

With oe without self. i still get the same error

Body:
You declared 2 functions inside your constructor, which never get called: setQueryFiltre and getQueryFiltre. Moreover, your test calls one of them as if it was a class method. It's not, and it will fail becuase that declaration is only visible inside the function. What you probably meant to write is this:

class SimpleGrid(gridlib.Grid): ##, mixins.GridAutoEditMixin):
    def __init__(self, parent, log):
        gridlib.Grid.__init__(self, parent, -1)
        self.setQueryFiltre(None)

        self.wxFont = "9, wx.FONTFAMILY_DEFAULT, wx.NORMAL, wx.NORMAL"
        self.queryFiltre = ""

    def setQueryFiltre(self, queryFiltre):
        path = os.path.dirname(os.path.realpath(__file__))
        configFile = os.path.join(path , "config.ini")
        dbfile = os.path.join(path , "report.db")
        self.db_conn = sqlite3.connect(dbfile)
        self.theCursor =  self.db_conn.cursor()

        config = configparser.ConfigParser()
        config.read(configFile)

        #All the calls to DB and config file here
        return queryFiltre # why are you returning the argument? For chaining?

    def getQueryFiltre(self, queryFiltre):
        queryFiltre = queryFiltre

        rows = self.db_conn.execute(queryFiltre)
        rowsCount = len(rows.fetchall())
        # you probably want to return rows here?

class TestFrame(wx.Frame):
    def __init__(self, parent, log):
        wx.Frame.__init__(self, parent, 0, "V1.0", size=(1400,800))
        self.grid = SimpleGrid(self, log)

        # self.wxFont = SimpleGrid(self.wxFont) # what does this try to do?
        self.queryFiltre = self.grid.getQueryFiltre()

In Python, indentation is important.

Note that I also moved db initialization to constructor, where it belongs. Because otherwise, if get method gets called before set, you will still have the same crash. I also made a couple other comments that you'll probably get bitten by next.


答:
if you Want to ordering the output record in aggregate function , just write the ORDER BY [columnindex from your query] e.g. in your query , if you want to ordering your output by AccountName then you can use ORDER BY 1 [AccountName], ORDER BY 2 [SUM(CreditAmount)], ORDER BY 3
SUM(DebitAmount)  etc...</p>

Title: How Can I Use Order By In Aggregate Function


问:
Pointer arithmetic in 2D array
I am learning C++. So, please excuse me for this long question.

I have written the below program for arrays. I understand the base address of the array is returned and the name of the array points to that. So, in the case of a 1D array, I can understand that

arr + 1

returns the address of next item of the array i.e.

arr + 4bytes

However, in the case of a 2D array, it is a little different. What I understand from the below cout statements that in memory arr2D[0], arr2D[1]..arr2D[4] all will be allocated 20bytes one after another as there are 5 cols i.e. 5 * 4 bytes

arr2D & arr2D[0] points to the same base address as expected; lines 43 & 45

So, if I do

arr2D[0] + 5

I get the address of arr2D[1]; line 47. But, when I do

arr2D + 5

I get a memory address 100 bytes from the base address; line 50. I don't understand why. Could anyone explain to me this?

```<code>#include <iostream>

using namespace std;

int main() {

int arr[5] = {0,1,2,3,4};

cout << "Printing arr" << endl;
for (int i = 0; i < 5; i++) {
    cout << *(arr + i) << " ";
}
cout << endl;

cout << "Base address of 'arr': " << (unsigned long) arr
     << " EQUALS " << "&arr: " << (unsigned long) &arr
     << endl;

cout << "Address of 2nd item i.e. i = 1 is: " << (unsigned long) (arr+1)
     << endl;

cout << "arr[1] = " << arr[1] << ", "
     << "1[arr] = " << 1[arr] << ", "
     << "*(arr + 1) = " << *(arr + 1)
     << endl << endl;

int arr2D[5][5] = {{1,2,3,4,5},
                   {3,4,5,6,7},
                   {5,6,7,8,9},
                   {7,8,9,10,11},
                   {9,10,11,12,13}};

cout << "Printing arr2D" << endl;
for (int i = 0; i < 5; i++) {
    for (int j = 0; j < 5; j++) {
        cout << *(arr2D[i] + j) << " ";
    }
    cout << endl;
}
cout << endl;

cout << "Base address of arr2D = " << (unsigned long) arr2D << endl; // this is fine

cout << "Address of arr2D[0] = " << (unsigned long) &arr2D[0] << endl; // this is fine

cout << "Address of arr2D[1] = " << (unsigned long) arr2D[1] << " EQUALS "
     << (unsigned long)(arr2D[0] + 5) << endl; // this is fine

cout << "Address of arr2D[1] = " << (unsigned long) arr2D[1] << " NOT EQUALS "
     << (unsigned long)(arr2D + 5) << endl; // I do not understand this, as arr2D and arr2D[0] is same

cout << "Address of arr2D[1][0] = " << (unsigned long)&arr2D[1][0] << endl;

cout << "arr2D[1][0] = " << *((arr2D[0] + 5) + 0) << endl;
}
</code>```

答:
From cppreference:

When array-to-pointer decay is applied, a multidimensional array is
converted to a pointer to its first element (e.g., a pointer to its
first row or to its first plane): array-to-pointer decay is applied
only once

So arr2D decayed into a pointer to an int array of size five, and the size of that is 20.

Title: Pointer arithmetic in 2D array

Body:
I am learning C++. So, please excuse me for this long question.

I have written the below program for arrays. I understand the base address of the array is returned and the name of the array points to that. So, in the case of a 1D array, I can understand that

arr + 1

returns the address of next item of the array i.e.

arr + 4bytes

However, in the case of a 2D array, it is a little different. What I understand from the below cout statements that in memory arr2D[0], arr2D[1]..arr2D[4] all will be allocated 20bytes one after another as there are 5 cols i.e. 5 * 4 bytes

arr2D & arr2D[0] points to the same base address as expected; lines 43 & 45

So, if I do

arr2D[0] + 5

I get the address of arr2D[1]; line 47. But, when I do

arr2D + 5

I get a memory address 100 bytes from the base address; line 50. I don't understand why. Could anyone explain to me this?

#include <iostream>

using namespace std;

int main() {

int arr[5] = {0,1,2,3,4};

cout << "Printing arr" << endl;
for (int i = 0; i < 5; i++) {
    cout << *(arr + i) << " ";
}
cout << endl;

cout << "Base address of 'arr': " << (unsigned long) arr
     << " EQUALS " << "&arr: " << (unsigned long) &arr
     << endl;

cout << "Address of 2nd item i.e. i = 1 is: " << (unsigned long) (arr+1)
     << endl;

cout << "arr[1] = " << arr[1] << ", "
     << "1[arr] = " << 1[arr] << ", "
     << "*(arr + 1) = " << *(arr + 1)
     << endl << endl;

int arr2D[5][5] = {{1,2,3,4,5},
                   {3,4,5,6,7},
                   {5,6,7,8,9},
                   {7,8,9,10,11},
                   {9,10,11,12,13}};

cout << "Printing arr2D" << endl;
for (int i = 0; i < 5; i++) {
    for (int j = 0; j < 5; j++) {
        cout << *(arr2D[i] + j) << " ";
    }
    cout << endl;
}
cout << endl;

cout << "Base address of arr2D = " << (unsigned long) arr2D << endl; // this is fine

cout << "Address of arr2D[0] = " << (unsigned long) &arr2D[0] << endl; // this is fine

cout << "Address of arr2D[1] = " << (unsigned long) arr2D[1] << " EQUALS "
     << (unsigned long)(arr2D[0] + 5) << endl; // this is fine

cout << "Address of arr2D[1] = " << (unsigned long) arr2D[1] << " NOT EQUALS "
     << (unsigned long)(arr2D + 5) << endl; // I do not understand this, as arr2D and arr2D[0] is same

cout << "Address of arr2D[1][0] = " << (unsigned long)&arr2D[1][0] << endl;

cout << "arr2D[1][0] = " << *((arr2D[0] + 5) + 0) << endl;
}


问:
How to create a tuple of corresponding elements of a 2D numpy array in an efficient way
I have two 2D numpy arrays, one for latitude, another for longitude.

```<code>a = [ [1, 2, 3, 4],                  b = [ [a, b, c, d],
      [1, 2, 3, 4],                        [a, b, c, d],
      [1, 2, 3, 4],                        [a, b, c, d],
      [1, 2, 3, 4] ]                       [a, b, c, d] ]
</code>```

I want the following 2 kinds of outputs (a list and a 2D array of tuples)

```<code>output1 = [ (1,a), (2,a), (3,a), (4,a), (1,b), (2,b), (3,b), (4,b), . . . (3,d), (4,d) ]

output2 = [ [(1,a), (2,a), (3,a), (4,a)],
            [(1,b), (2,b), (3,b), (4,b)],
            [(1,c), (2,c), (3,c), (4,c)],
            [(1,d), (2,d), (3,d), (4,d)] ]
</code>```

How should I do it efficiently without loops?

This is the code for a and b generation:

```<code>aa = np.array([1,2,3,4])
bb = np.array(['a','b','c','d'])
a, b = np.meshgrid(aa,bb)
</code>```

Body:
I have two 2D numpy arrays, one for latitude, another for longitude.

a = [ [1, 2, 3, 4],                  b = [ [a, b, c, d],
      [1, 2, 3, 4],                        [a, b, c, d],
      [1, 2, 3, 4],                        [a, b, c, d],
      [1, 2, 3, 4] ]                       [a, b, c, d] ]

I want the following 2 kinds of outputs (a list and a 2D array of tuples)

output1 = [ (1,a), (2,a), (3,a), (4,a), (1,b), (2,b), (3,b), (4,b), . . . (3,d), (4,d) ]

output2 = [ [(1,a), (2,a), (3,a), (4,a)],
            [(1,b), (2,b), (3,b), (4,b)],
            [(1,c), (2,c), (3,c), (4,c)],
            [(1,d), (2,d), (3,d), (4,d)] ]

How should I do it efficiently without loops?

This is the code for a and b generation:

aa = np.array([1,2,3,4])
bb = np.array(['a','b','c','d'])
a, b = np.meshgrid(aa,bb)

Body:
Your 2 arrays:

In [88]: aa = np.array([1,2,3,4])
    ...: bb = np.array(['a','b','c','d'])
    ...: a, b = np.meshgrid(aa,bb)
In [89]: a
Out[89]:
array([[1, 2, 3, 4],
       [1, 2, 3, 4],
       [1, 2, 3, 4],
       [1, 2, 3, 4]])
In [90]: b
Out[90]:
array([['a', 'a', 'a', 'a'],
       ['b', 'b', 'b', 'b'],
       ['c', 'c', 'c', 'c'],
       ['d', 'd', 'd', 'd']], dtype='<U1')

Some version of concatenate can be used to join the two arrays.  stack with axis 2 does it, so does np.array((a,b)) followed by a transpose.

In [91]: np.stack((a,b),axis=2)
Out[91]:
array([[['1', 'a'],
        ['2', 'a'],
        ['3', 'a'],
        ['4', 'a']],

       [['1', 'b'],
        ['2', 'b'],
        ['3', 'b'],
        ['4', 'b']],

       [['1', 'c'],
        ['2', 'c'],
        ['3', 'c'],
        ['4', 'c']],

       [['1', 'd'],
        ['2', 'd'],
        ['3', 'd'],
        ['4', 'd']]], dtype='<U21')

But the combined array is string dtype, not a mix of integers and strings.  And not tuples.  It's a (4,4,2) array.

But if we convert the arrays to object dtype:

In [93]: np.stack((a.astype(object),b.astype(object)),axis=2)
Out[93]:
array([[[1, 'a'],
        [2, 'a'],
        [3, 'a'],
        [4, 'a']],

       [[1, 'b'],
        [2, 'b'],
       ...
        [4, 'd']]], dtype=object)

We can reshape it to (-1,2) and then tolist() to produce the list version.

Or we could make a structured array, with a integer field and string field.  Its print display uses tuples, to mark the inner records (which are distinct from the last size 2 dimension above).

In [98]: arr = np.zeros(a.shape, dtype='int,U5')
In [99]: arr
Out[99]:
array([[(0, ''), (0, ''), (0, ''), (0, '')],
       [(0, ''), (0, ''), (0, ''), (0, '')],
       [(0, ''), (0, ''), (0, ''), (0, '')],
       [(0, ''), (0, ''), (0, ''), (0, '')]],
      dtype=[('f0', '<i8'), ('f1', '<U5')])
In [100]: arr['f0']=a
In [101]: arr['f1']=b
In [102]: arr
Out[102]:
array([[(1, 'a'), (2, 'a'), (3, 'a'), (4, 'a')],
       [(1, 'b'), (2, 'b'), (3, 'b'), (4, 'b')],
       [(1, 'c'), (2, 'c'), (3, 'c'), (4, 'c')],
       [(1, 'd'), (2, 'd'), (3, 'd'), (4, 'd')]],
      dtype=[('f0', '<i8'), ('f1', '<U5')])

In [103]: print(arr)
[[(1, 'a') (2, 'a') (3, 'a') (4, 'a')]
 [(1, 'b') (2, 'b') (3, 'b') (4, 'b')]
 [(1, 'c') (2, 'c') (3, 'c') (4, 'c')]
 [(1, 'd') (2, 'd') (3, 'd') (4, 'd')]]

zip list comprehension does the list version nicely:

In [105]: [item for item in zip(a.flat,b.flat)]
Out[105]:
[(1, 'a'),
 (2, 'a'),
 (3, 'a'),
 (4, 'a'),
 (1, 'b'),
 ...
 (4, 'd')]

For this sample size, the list comprehension is faster.  Trying to avoid loops isn't always the most efficient way.

===

Elements of arr are np.void objects, though they display as tuples.  To get actual tuples we have to convert to list or object dtype:

In [109]: arr.tolist()
Out[109]:
[[(1, 'a'), (2, 'a'), (3, 'a'), (4, 'a')],
 [(1, 'b'), (2, 'b'), (3, 'b'), (4, 'b')],
 [(1, 'c'), (2, 'c'), (3, 'c'), (4, 'c')],
 [(1, 'd'), (2, 'd'), (3, 'd'), (4, 'd')]]
In [110]: arr.astype(object)
Out[110]:
array([[(1, 'a'), (2, 'a'), (3, 'a'), (4, 'a')],
       [(1, 'b'), (2, 'b'), (3, 'b'), (4, 'b')],
       [(1, 'c'), (2, 'c'), (3, 'c'), (4, 'c')],
       [(1, 'd'), (2, 'd'), (3, 'd'), (4, 'd')]], dtype=object)
In [111]: type(_[0,0])
Out[111]: tuple

The fields of the structured array can be accessed by name:

In [112]: arr['f0']
Out[112]:
array([[1, 2, 3, 4],
       [1, 2, 3, 4],
       [1, 2, 3, 4],
       [1, 2, 3, 4]])


问:
Is there a better way to make a wait command? Cause importing time and then sleep seems to cause my hold script to sleep rather than just one line
importing time and then sleep seems to cause all of my scripts to sleep rather than just the one line. I was asking if there was a better way to call a wait. A new import entirely would also work.</p>

答: I can’t get your question clearly but in nodejs and angularjs , not to wait for all scripts to execute or sleep all scripts. Which scripts you want to wait for results then those are added to promise. Nodejs and angularjs are synchronous,so to make asynchronous, we use await-promise.It holds only those scripts which return results waiting for other scripts to process.</p>


问:
Oracle APEX - apex_application.g_print_success_message displays the message twice on refresh
I am utilizing apex_application.g_print_success_message in one of my APEX processes to display a message to the user.

The process gets executes on page submit when Request=VALUE1.

The code checks if a variable is greater than 0, then displays a message. So the message is supposed to only be displayed when Request=VALUE1 and if :P1_ITEM > 0:

```<code>BEGIN
      IF TO_NUMBER(:P1_ITEM) > 0 THEN
            apex_application.g_print_success_message := 'Test';
      END IF;
      :P_ITEM := 0;
END;
</code>```

After displaying the message I set the page item to zero.

I submit the page, process executes, message gets displayed. Everything works fine. But then if I go ahead and refresh the page, the message displays again. Not sure why as one of the conditions is TO_NUMBER(:P1_ITEM) > 0 and I did set it to 0. Also when I do a refresh the request cannot be VALUE1 so I know the process does not get executed. But then why is the message getting displayed again?

答:
Did you try with clear session state process?

Or maybe

Body:
I am utilizing apex_application.g_print_success_message in one of my APEX processes to display a message to the user.

The process gets executes on page submit when Request=VALUE1.

The code checks if a variable is greater than 0, then displays a message. So the message is supposed to only be displayed when Request=VALUE1 and if :P1_ITEM > 0:

BEGIN
      IF TO_NUMBER(:P1_ITEM) > 0 THEN
            apex_application.g_print_success_message := 'Test';
      END IF;
      :P_ITEM := 0;
END;

After displaying the message I set the page item to zero.

I submit the page, process executes, message gets displayed. Everything works fine. But then if I go ahead and refresh the page, the message displays again. Not sure why as one of the conditions is TO_NUMBER(:P1_ITEM) > 0 and I did set it to 0. Also when I do a refresh the request cannot be VALUE1 so I know the process does not get executed. But then why is the message getting displayed again?


问:
How to delete a row if it shares the value of another row in one column and has one value in other column in R?
I have a the following dataframe, let's call it Servers.

Server <-c("AUSTRALIA", "AUSTRALIA", "AUSTRALIA", "AUSTRALIA", "AUSTRALIA","CHINA","COLOMBIA","COLOMBIA","ECUADOR","ECUADOR","ECUADOR","COREA","COREA","COREA","COREA","BRASIL","BRASIL","BRASIL","ALEMANIA","ALEMANIA","ALEMANIA","ALEMANIA")
Product <- c("021-08183","79P-01747","C5E-00746","7NQ-00302","7JQ-00341","021-08183","021-08183","79P-01747","79P-01747","021-08183","6QH-00002","79P-01747","021-08183","#N/A","6QH-00002","79P-01747","76P-00742","021-08183","021-08183","79P-01747","C5E-00746","7NQ-00302")
Servers <- data.frame(Server, Product)

It looks like this

Table

What I need is the following. Whenever I have the same value in Server, and the values 021-08183 and 79P-01747, I need to remove the line that has the 021-08183 value, and keep only the one that has 79P-01747.

Output:

Thanks a lot!

PS: Sorry, I had some issues uploading the image for the original table

答:
Here's a way with dplyr package -

```<code>Servers %>%
  group_by(Server) %>%
  filter(
    (all(c("021-08183", "79P-01747") %in% Product) & (Product != "021-08183")) | n() == 1
  ) %>%
  ungroup()

# A tibble: 16 x 2
   Server    Product
   <chr>     <chr>
 1 AUSTRALIA 79P-01747
 2 AUSTRALIA C5E-00746
 3 AUSTRALIA 7NQ-00302
 4 AUSTRALIA 7JQ-00341
 5 CHINA     021-08183
 6 COLOMBIA  79P-01747
 7 ECUADOR   79P-01747
 8 ECUADOR   6QH-00002
 9 COREA     79P-01747
10 COREA     #N/A
11 COREA     6QH-00002
12 BRASIL    79P-01747
13 BRASIL    76P-00742
14 ALEMANIA  79P-01747
15 ALEMANIA  C5E-00746
16 ALEMANIA  7NQ-00302
</code>```

Body:
Here's a way with dplyr package -

Servers %>%
  group_by(Server) %>%
  filter(
    (all(c("021-08183", "79P-01747") %in% Product) & (Product != "021-08183")) | n() == 1
  ) %>%
  ungroup()

# A tibble: 16 x 2
   Server    Product
   <chr>     <chr>
 1 AUSTRALIA 79P-01747
 2 AUSTRALIA C5E-00746
 3 AUSTRALIA 7NQ-00302
 4 AUSTRALIA 7JQ-00341
 5 CHINA     021-08183
 6 COLOMBIA  79P-01747
 7 ECUADOR   79P-01747
 8 ECUADOR   6QH-00002
 9 COREA     79P-01747
10 COREA     #N/A
11 COREA     6QH-00002
12 BRASIL    79P-01747
13 BRASIL    76P-00742
14 ALEMANIA  79P-01747
15 ALEMANIA  C5E-00746
16 ALEMANIA  7NQ-00302


答:
We create a logical condition after grouping by 'Servers', filter out the rows that have "021-08183" in the Product column when both ""021-08183", "79P-01747" are present in the column

```<code>library(dplyr)
Servers %>%
   group_by(Server) %>%
   filter(if(all(c("021-08183", "79P-01747") %in% Product))
     Product != "021-08183" else TRUE)
# A tibble: 16 x 2
# Groups:   Server [7]
#   Server    Product
#   <fct>     <fct>
# 1 AUSTRALIA 79P-01747
# 2 AUSTRALIA C5E-00746
# 3 AUSTRALIA 7NQ-00302
# 4 AUSTRALIA 7JQ-00341
# 5 CHINA     021-08183
# 6 COLOMBIA  79P-01747
# 7 ECUADOR   79P-01747
# 8 ECUADOR   6QH-00002
# 9 COREA     79P-01747
#10 COREA     #N/A
#11 COREA     6QH-00002
#12 BRASIL    79P-01747
#13 BRASIL    76P-00742
#14 ALEMANIA  79P-01747
#15 ALEMANIA  C5E-00746
#16 ALEMANIA  7NQ-00302
</code>```

Or a similar option with data.table

```<code>library(data.table)
setDT(Servers)[, .SD[if(all(c("021-08183", "79P-01747") %in% Product))
        Product != "021-08183" else TRUE], Server]
</code>```

NOTE: It gives the expected output as in the OP's post while keeping the row for 'CHINA'

Title: How to delete a row if it shares the value of another row in one column and has one value in other column in R?

Body:
I have a the following dataframe, let's call it Servers.

Server <-c("AUSTRALIA", "AUSTRALIA", "AUSTRALIA", "AUSTRALIA", "AUSTRALIA","CHINA","COLOMBIA","COLOMBIA","ECUADOR","ECUADOR","ECUADOR","COREA","COREA","COREA","COREA","BRASIL","BRASIL","BRASIL","ALEMANIA","ALEMANIA","ALEMANIA","ALEMANIA")
Product <- c("021-08183","79P-01747","C5E-00746","7NQ-00302","7JQ-00341","021-08183","021-08183","79P-01747","79P-01747","021-08183","6QH-00002","79P-01747","021-08183","#N/A","6QH-00002","79P-01747","76P-00742","021-08183","021-08183","79P-01747","C5E-00746","7NQ-00302")
Servers <- data.frame(Server, Product)

It looks like this

Table

What I need is the following. Whenever I have the same value in Server, and the values 021-08183 and 79P-01747, I need to remove the line that has the 021-08183 value, and keep only the one that has 79P-01747.

Output:

Thanks a lot!

PS: Sorry, I had some issues uploading the image for the original table

Body:
We create a logical condition after grouping by 'Servers', filter out the rows that have "021-08183" in the Product column when both ""021-08183", "79P-01747" are present in the column

library(dplyr)
Servers %>%
   group_by(Server) %>%
   filter(if(all(c("021-08183", "79P-01747") %in% Product))
     Product != "021-08183" else TRUE)
# A tibble: 16 x 2
# Groups:   Server [7]
#   Server    Product
#   <fct>     <fct>
# 1 AUSTRALIA 79P-01747
# 2 AUSTRALIA C5E-00746
# 3 AUSTRALIA 7NQ-00302
# 4 AUSTRALIA 7JQ-00341
# 5 CHINA     021-08183
# 6 COLOMBIA  79P-01747
# 7 ECUADOR   79P-01747
# 8 ECUADOR   6QH-00002
# 9 COREA     79P-01747
#10 COREA     #N/A
#11 COREA     6QH-00002
#12 BRASIL    79P-01747
#13 BRASIL    76P-00742
#14 ALEMANIA  79P-01747
#15 ALEMANIA  C5E-00746
#16 ALEMANIA  7NQ-00302

Or a similar option with data.table

library(data.table)
setDT(Servers)[, .SD[if(all(c("021-08183", "79P-01747") %in% Product))
        Product != "021-08183" else TRUE], Server]

NOTE: It gives the expected output as in the OP's post while keeping the row for 'CHINA'


问:
React - Unable to setState inside a Promise then
I'm trying to set the state inside a then() of a Promise, but the state value is not getting saved and is not accessible outside of the then().

Below is the UPDATED code:

```<code>handleSelect = location => {
  this.setState({ location });
  geocodeByAddress(
    location
  )
  .then(results => getLatLng(results[0]))
  .then(latLng => {
    this.setState({
      location_latitude: latLng.lat,
      location_longitude: latLng.lng,
    })
    console.log(this.state.location_latitude); // returns the correct value
  })
  console.log(this.state.location_latitude); // returns null (the originally declared value)
}
</code>```

If I console.log(this.state.location_latitude) I get a null value. But if I console.log inside the then() block, I get the correct value. How do I set the state in this case?

UPDATED explanation:

To give a better context to the whole logic here: I'm using a Google Places Autocomplete component which allows the user to select a location from the dropdown. The above method handleSelect is called when the user selects a location from the dropdown. I then extract the latitude and longitude of the place selected by the user using the geocodeByAddress method, which is a promise. I need to then retrieve the latitude and longitude and set the state accordingly, which is later used to send to the database. I can't submit the value to the database yet because there are other form inputs that the user needs to fill (which are also stored in the state) and once he clicks on Submit, I'll be submitting all the state elements in a single call to the backend. So, there is nothing I want to update with componentDidMount() or nothing is getting updated yet to use componentDidUpdate(). In any case, I can only store the state value inside the then of geocodeByAddress (please correct me if I'm wrong here). So, I HAVE to be able to modify the state value inside the then block.

答:
The thing is when you set-state you can't see the state result immediately after that. That's why the console.log returns null. Make sure your console.log is in the render block. This is because set-state actions are asynchronous and are batched for performance gains. This is explained in documentation of setState.
Take a look at this => Why is setState in reactjs Async instead of Sync?

Body:
I'm trying to set the state inside a then() of a Promise, but the state value is not getting saved and is not accessible outside of the then().

Below is the UPDATED code:

handleSelect = location => {
  this.setState({ location });
  geocodeByAddress(
    location
  )
  .then(results => getLatLng(results[0]))
  .then(latLng => {
    this.setState({
      location_latitude: latLng.lat,
      location_longitude: latLng.lng,
    })
    console.log(this.state.location_latitude); // returns the correct value
  })
  console.log(this.state.location_latitude); // returns null (the originally declared value)
}

If I console.log(this.state.location_latitude) I get a null value. But if I console.log inside the then() block, I get the correct value. How do I set the state in this case?

UPDATED explanation:

To give a better context to the whole logic here: I'm using a Google Places Autocomplete component which allows the user to select a location from the dropdown. The above method handleSelect is called when the user selects a location from the dropdown. I then extract the latitude and longitude of the place selected by the user using the geocodeByAddress method, which is a promise. I need to then retrieve the latitude and longitude and set the state accordingly, which is later used to send to the database. I can't submit the value to the database yet because there are other form inputs that the user needs to fill (which are also stored in the state) and once he clicks on Submit, I'll be submitting all the state elements in a single call to the backend. So, there is nothing I want to update with componentDidMount() or nothing is getting updated yet to use componentDidUpdate(). In any case, I can only store the state value inside the then of geocodeByAddress (please correct me if I'm wrong here). So, I HAVE to be able to modify the state value inside the then block.