问:
Building Breakout Game (python) but I can't get bricks to delete_?
I am a new coder and I am building a "Breakout" game in python and i got most of it to except getting the bricks to delete. *What is odd, is after the ball touches the bottom wall (out of bounds) and regenerates, the bricks are then able to be deleted by playing the game! Also, I need to stop the game after 3 chances. I'm stuck, with those 2 problems. Help please. Here is my code:

import tkinter
import time

```<code># How big is the playing area?
CANVAS_WIDTH = 600  # Width of drawing canvas in pixels
CANVAS_HEIGHT = 800  # Height of drawing canvas in pixels

# Constants for the bricks
N_ROWS = 8  # How many rows of bricks are there?
N_COLS = 10  # How many columns of bricks are there?
SPACING = 5  # How much space is there between each brick?
BRICK_START_Y = 50  # The y coordinate of the top-most brick
BRICK_HEIGHT = 20  # How many pixels high is each brick
BRICK_WIDTH = (CANVAS_WIDTH - (N_COLS + 1) * SPACING) // N_COLS

# Constants for the ball and paddle
BALL_SIZE = 70
PADDLE_Y = CANVAS_HEIGHT - 40
PADDLE_WIDTH = 200

def main():
    canvas = make_canvas(CANVAS_WIDTH, CANVAS_HEIGHT, 'Brick Breaker')
    # Makes a ball
    ball = canvas.create_oval(300, 300, 350, 350, fill="red", outline="red")
    # Makes a paddle
    paddle = canvas.create_rectangle(299, PADDLE_Y, PADDLE_WIDTH, CANVAS_HEIGHT - 20, fill="black")
    # Change_X
    dx = 6
    # Change_Y
    dy = 6
    for row in range(N_ROWS):
        # Draws columns of brick
        for col in range(N_COLS):
            draw_brick(canvas, row, col)

    while True:
        # Mouse location and respond to movement
        mouse_x = canvas.winfo_pointerx()
        # Move Paddle to X location
        canvas.moveto(paddle, mouse_x, PADDLE_Y)
        # Ball movement
        canvas.move(ball, dx, dy)
        # If ball hits left of right wall, change X location
        if hit_left_wall(canvas, ball) or hit_right_wall(canvas, ball):
            dx *= -1
        # If ball hits top wall, then change Y location
        elif hit_top_wall(canvas, ball):
            dy *= -1
        elif hit_brick(canvas, ball, paddle):
            dy *= -1
        if hit_bottom(canvas, ball):
            canvas.delete(ball)
            ball = make_ball(canvas)

        # Recreates canvas
        canvas.update()
        # Pause time
        time.sleep(1 / 50.)

    canvas.mainloop()

# Finds coordinates of paddle
def hit_paddle(canvas, ball, paddle):
    paddle_coords = canvas.coords(paddle)
    x1 = paddle_coords[0]
    y1 = paddle_coords[1]
    x2 = paddle_coords[2]
    y2 = paddle_coords[3]
    # If any object begins to overlap with paddle, create a Hit
    result = canvas.find_overlapping(x1, y1, x2, y2)
    return len(result) > 1

def make_ball(canvas):
    return canvas.create_oval(300, 300, 350, 350, fill="red", outline="red")

def hit_brick(canvas, ball, paddle):
    ball_coord = canvas.coords(ball)
    x_1 = ball_coord[0]
    y_1 = ball_coord[1]
    x_2 = ball_coord[2]
    y_2 = ball_coord[3]
    results = canvas.find_overlapping(x_1, y_1, x_2, y_2)
    for object in results:
        if object == paddle or object == ball:
            return len(results) > 1
        else:
            canvas.delete(object)

def moveto(canvas, oval, x, y):
    # Get current position
    x0, y0, x1, y1 = canvas.coords(oval)
    # Sets new position
    canvas.move(oval, x - x0, y - y0)

def hit_bottom(canvas, ball):
    return get_bottom_y(canvas, ball) >= CANVAS_HEIGHT

def hit_left_wall(canvas, ball):
    return get_left_x(canvas, ball) <= 0

def hit_right_wall(canvas, ball):
    return get_right_x(canvas, ball) >= CANVAS_WIDTH

def hit_top_wall(canvas, ball):
    return get_top_y(canvas, ball) <= 0

def draw_brick(canvas, row, col):
    x = col * (BRICK_WIDTH + SPACING)
    y = row * (BRICK_HEIGHT + SPACING)
    color = "blue"
    canvas.create_rectangle(x, y, x + BRICK_WIDTH, y + BRICK_HEIGHT, fill=color, outline=color)

def get_bottom_y(canvas, ball):
    return canvas.coords(ball)[3]

def get_top_y(canvas, ball):
    """
    This friendly method returns the y coordinate of the top of an object.
    Recall that canvas.coords(object) returns a list of the object
    bounding box: [x_1, y_1, x_2, y_2]. The element at index 1 is the top-y
    """
    return canvas.coords(ball)[1]

def get_left_x(canvas, ball):
    """
    This friendly method returns the x coordinate of the left of an object.
    Recall that canvas.coords(object) returns a list of the object
    bounding box: [x_1, y_1, x_2, y_2]. The element at index 0 is the left-x
    """
    return canvas.coords(ball)[0]

def get_right_x(canvas, ball):
    """
    This friendly method returns the x coordinate of the right of an object.
    Recall that canvas.coords(object) returns a list of the object
    bounding box: [x_1, y_1, x_2, y_2]. The element at index 2 is the right-x
    """
    return canvas.coords(ball)[2]

def make_canvas(width, height, title):
    """
    Creates and returns a drawing canvas
    of the given int size with a blue border,
    ready for drawing.
    """
    top = tkinter.Tk()
    top.minsize(width=width, height=height)
    top.title(title)
    canvas = tkinter.Canvas(top, width=width + 1, height=height + 1)
    canvas.pack()
    return canvas

if __name__ == '__main__':
    main()
</code>```

答:
The first problem is due to the if statement in the for loop inside hit_brick():

```<code>def hit_brick(canvas, ball, paddle):
    ball_coord = canvas.coords(ball)
    x_1 = ball_coord[0]
    y_1 = ball_coord[1]
    x_2 = ball_coord[2]
    y_2 = ball_coord[3]
    results = canvas.find_overlapping(x_1, y_1, x_2, y_2)
    for object in results:
        if object == paddle or object == ball: # <-- problem here
            return len(results) > 1
        else:
            canvas.delete(object)
</code>```

As the values of ball and paddle are 1 and 2 (as they are the first two canvas items created) and so results is something like (1, N) when the ball hit one of the bricks.
So the if statement returns true for the first checking and then the function exits by the return statement.

Now let the ball hits the bottom and it will be recreated with id greater than existing canvas items.  The results will be something like (N, ball) when the ball hits one of the bricks.
This time the if statement will return false and the brick is deleted.

So hit_brick() should be modified as below:

```<code>def hit_brick(canvas, ball, paddle):
    ball_coord = canvas.coords(ball)
    results = canvas.find_overlapping(*ball_coord)
    for object in results:
        if object not in (paddle, ball):
            canvas.delete(object)
    return len(results) > 1
</code>```

For the second problem, you need to declare a variable, for example lives = 3, before the while loop and decrease it by one if the ball hits the bottom.
The while loop should be terminated if lives == 0:

```<code>def main():
    ...
    lives = 3
    while lives > 0:
        ...
        if hit_bottom(canvas.ball):
            ...
            lives -= 1
</code>```

Body:
I am a new coder and I am building a "Breakout" game in python and i got most of it to except getting the bricks to delete. *What is odd, is after the ball touches the bottom wall (out of bounds) and regenerates, the bricks are then able to be deleted by playing the game! Also, I need to stop the game after 3 chances. I'm stuck, with those 2 problems. Help please. Here is my code:

import tkinter
import time

# How big is the playing area?
CANVAS_WIDTH = 600  # Width of drawing canvas in pixels
CANVAS_HEIGHT = 800  # Height of drawing canvas in pixels

# Constants for the bricks
N_ROWS = 8  # How many rows of bricks are there?
N_COLS = 10  # How many columns of bricks are there?
SPACING = 5  # How much space is there between each brick?
BRICK_START_Y = 50  # The y coordinate of the top-most brick
BRICK_HEIGHT = 20  # How many pixels high is each brick
BRICK_WIDTH = (CANVAS_WIDTH - (N_COLS + 1) * SPACING) // N_COLS

# Constants for the ball and paddle
BALL_SIZE = 70
PADDLE_Y = CANVAS_HEIGHT - 40
PADDLE_WIDTH = 200

def main():
    canvas = make_canvas(CANVAS_WIDTH, CANVAS_HEIGHT, 'Brick Breaker')
    # Makes a ball
    ball = canvas.create_oval(300, 300, 350, 350, fill="red", outline="red")
    # Makes a paddle
    paddle = canvas.create_rectangle(299, PADDLE_Y, PADDLE_WIDTH, CANVAS_HEIGHT - 20, fill="black")
    # Change_X
    dx = 6
    # Change_Y
    dy = 6
    for row in range(N_ROWS):
        # Draws columns of brick
        for col in range(N_COLS):
            draw_brick(canvas, row, col)

    while True:
        # Mouse location and respond to movement
        mouse_x = canvas.winfo_pointerx()
        # Move Paddle to X location
        canvas.moveto(paddle, mouse_x, PADDLE_Y)
        # Ball movement
        canvas.move(ball, dx, dy)
        # If ball hits left of right wall, change X location
        if hit_left_wall(canvas, ball) or hit_right_wall(canvas, ball):
            dx *= -1
        # If ball hits top wall, then change Y location
        elif hit_top_wall(canvas, ball):
            dy *= -1
        elif hit_brick(canvas, ball, paddle):
            dy *= -1
        if hit_bottom(canvas, ball):
            canvas.delete(ball)
            ball = make_ball(canvas)

        # Recreates canvas
        canvas.update()
        # Pause time
        time.sleep(1 / 50.)

    canvas.mainloop()

# Finds coordinates of paddle
def hit_paddle(canvas, ball, paddle):
    paddle_coords = canvas.coords(paddle)
    x1 = paddle_coords[0]
    y1 = paddle_coords[1]
    x2 = paddle_coords[2]
    y2 = paddle_coords[3]
    # If any object begins to overlap with paddle, create a Hit
    result = canvas.find_overlapping(x1, y1, x2, y2)
    return len(result) > 1

def make_ball(canvas):
    return canvas.create_oval(300, 300, 350, 350, fill="red", outline="red")

def hit_brick(canvas, ball, paddle):
    ball_coord = canvas.coords(ball)
    x_1 = ball_coord[0]
    y_1 = ball_coord[1]
    x_2 = ball_coord[2]
    y_2 = ball_coord[3]
    results = canvas.find_overlapping(x_1, y_1, x_2, y_2)
    for object in results:
        if object == paddle or object == ball:
            return len(results) > 1
        else:
            canvas.delete(object)

def moveto(canvas, oval, x, y):
    # Get current position
    x0, y0, x1, y1 = canvas.coords(oval)
    # Sets new position
    canvas.move(oval, x - x0, y - y0)

def hit_bottom(canvas, ball):
    return get_bottom_y(canvas, ball) >= CANVAS_HEIGHT

def hit_left_wall(canvas, ball):
    return get_left_x(canvas, ball) <= 0

def hit_right_wall(canvas, ball):
    return get_right_x(canvas, ball) >= CANVAS_WIDTH

def hit_top_wall(canvas, ball):
    return get_top_y(canvas, ball) <= 0

def draw_brick(canvas, row, col):
    x = col * (BRICK_WIDTH + SPACING)
    y = row * (BRICK_HEIGHT + SPACING)
    color = "blue"
    canvas.create_rectangle(x, y, x + BRICK_WIDTH, y + BRICK_HEIGHT, fill=color, outline=color)

def get_bottom_y(canvas, ball):
    return canvas.coords(ball)[3]

def get_top_y(canvas, ball):
    """
    This friendly method returns the y coordinate of the top of an object.
    Recall that canvas.coords(object) returns a list of the object
    bounding box: [x_1, y_1, x_2, y_2]. The element at index 1 is the top-y
    """
    return canvas.coords(ball)[1]

def get_left_x(canvas, ball):
    """
    This friendly method returns the x coordinate of the left of an object.
    Recall that canvas.coords(object) returns a list of the object
    bounding box: [x_1, y_1, x_2, y_2]. The element at index 0 is the left-x
    """
    return canvas.coords(ball)[0]

def get_right_x(canvas, ball):
    """
    This friendly method returns the x coordinate of the right of an object.
    Recall that canvas.coords(object) returns a list of the object
    bounding box: [x_1, y_1, x_2, y_2]. The element at index 2 is the right-x
    """
    return canvas.coords(ball)[2]

def make_canvas(width, height, title):
    """
    Creates and returns a drawing canvas
    of the given int size with a blue border,
    ready for drawing.
    """
    top = tkinter.Tk()
    top.minsize(width=width, height=height)
    top.title(title)
    canvas = tkinter.Canvas(top, width=width + 1, height=height + 1)
    canvas.pack()
    return canvas

if __name__ == '__main__':
    main()

Body:
The first problem is due to the if statement in the for loop inside hit_brick():

def hit_brick(canvas, ball, paddle):
    ball_coord = canvas.coords(ball)
    x_1 = ball_coord[0]
    y_1 = ball_coord[1]
    x_2 = ball_coord[2]
    y_2 = ball_coord[3]
    results = canvas.find_overlapping(x_1, y_1, x_2, y_2)
    for object in results:
        if object == paddle or object == ball: # <-- problem here
            return len(results) > 1
        else:
            canvas.delete(object)

As the values of ball and paddle are 1 and 2 (as they are the first two canvas items created) and so results is something like (1, N) when the ball hit one of the bricks.
So the if statement returns true for the first checking and then the function exits by the return statement.

Now let the ball hits the bottom and it will be recreated with id greater than existing canvas items.  The results will be something like (N, ball) when the ball hits one of the bricks.
This time the if statement will return false and the brick is deleted.

So hit_brick() should be modified as below:

def hit_brick(canvas, ball, paddle):
    ball_coord = canvas.coords(ball)
    results = canvas.find_overlapping(*ball_coord)
    for object in results:
        if object not in (paddle, ball):
            canvas.delete(object)
    return len(results) > 1

For the second problem, you need to declare a variable, for example lives = 3, before the while loop and decrease it by one if the ball hits the bottom.
The while loop should be terminated if lives == 0:

def main():
    ...
    lives = 3
    while lives > 0:
        ...
        if hit_bottom(canvas.ball):
            ...
            lives -= 1


来源: stackexchange

问:
How do I decide on a suitable TLABSIZE setting for a Java application?
My Java application on an single cpu arm7 (32bit) device using Java 14 is occasionally crashing
after running under load for a number of hours, and is always failing in ThreadLocalAllocBuffer::resize()

```<code>  A fatal error has been detected by the Java Runtime Environment:
    #
    SIGSEGV (0xb) at pc=0xb6cd515e, pid=1725, tid=1733
    #
    JRE version: OpenJDK Runtime Environment (14.0+36) (build 14+36)
    Java VM: OpenJDK Client VM (14+36, mixed mode, serial gc, linux-arm)
    Problematic frame:
    V
    #
    No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
    #
    If you would like to submit a bug report, please visit:
    https://bugreport.java.com/bugreport/crash.jsp
    #

--------------- S U M M A R Y ------------

Command Line: -Duser.home=/mnt/app/share/log -Djdk.lang.Process.launchMechanism=vfork -Xms150m -Xmx900m -Dcom.mchange.v2.log.MLog=com.mchange.v2.log.jdk14logging.Jdk14MLog -Dorg.jboss.logging.provider=jdk -Djava.util.logging.config.class=com.jthink.songkong.logging.StandardLogging --add-opens=java.base/java.lang=ALL-UNNAMED lib/songkong-6.9.jar -r

Host: Marvell PJ4Bv7 Processor rev 1 (v7l), 1 cores, 1G, Buildroot 2014.11-rc1
Time: Fri Apr 24 19:36:54 2020 BST elapsed time: 37456 seconds (0d 10h 24m 16s)

    --------------- T H R E A D ---------------

    Current thread (0xb6582a30): VMThread "VM Thread" [stack: 0x7b716000,0x7b796000] [id=3625] _threads_hazard_ptr=0x7742f140

    Stack: [0x7b716000,0x7b796000], sp=0x7b7946b0, free space=505k
    Native frames: (J=compiled Java code, A=aot compiled Java code, j=interpreted, Vv=VM code, C=native code)
    V [libjvm.so+0x48015e] ThreadLocalAllocBuffer::resize()+0x85

    [error occurred during error reporting (printing native stack), id 0xb, SIGSEGV (0xb) at pc=0xb6b4ccae]
</code>```

Now this must surely be bug in JVM, but as its not one of the standard Java platforms and I dont have a simple test case I cannot see it getting fixed anytime soon, so I am trying to workaround it. Its also worth noting  that it crashed with ThreadLocalAllocBuffer::accumulate_statistics_before_gc() when I used Java 11 which is why I moved to Java 14 to try and resolve the issue.

As the the issue is with TLABs one solution is to disable TLABS with -XX:-UseTLAB but that makes the code run slower on an already slow machine.

So I think another solution is to disable resizing with -XX:-ResizeTLAB, but then I need to know work out a suitable size and specify that using -XX:TLABSize=N. But I am not sure what N actually represents and what would be a suitable size to set

I tried setting  -XX:TLABSize=1000000 which seems to me to be quite large ?

I have some logging set with

```<code> -Xlog:tlab*=debug,tlab*=trace:file=gc.log:time:filecount=7,filesize=8M
</code>```

but I don't really understand the output.

```<code>[2020-05-19T15:43:43.836+0100] ThreadLocalAllocBuffer::compute_size(132) returns 250132
[2020-05-19T15:43:43.837+0100] TLAB: fill thread: 0x0026d548 [id: 871] desired_size: 976KB slow allocs: 0  refill waste: 15624B alloc: 0.25725     1606KB refills: 1 waste  0.0% gc: 0B slow: 0B fast: 0B
[2020-05-19T15:43:43.853+0100] ThreadLocalAllocBuffer::compute_size(6) returns 250006
[2020-05-19T15:43:43.854+0100] TLAB: fill thread: 0xb669be48 [id: 32635] desired_size: 976KB slow allocs: 0  refill waste: 15624B alloc: 0.00002        0KB refills: 1 waste  0.0% gc: 0B slow: 0B fast: 0B
[2020-05-19T15:43:43.910+0100] ThreadLocalAllocBuffer::compute_size(4) returns 250004
[2020-05-19T15:43:43.911+0100] TLAB: fill thread: 0x76c1d6f8 [id: 917] desired_size: 976KB slow allocs: 0  refill waste: 15624B alloc: 0.91261     8085KB refills: 1 waste  0.0% gc: 0B slow: 0B fast: 0B
[2020-05-19T15:43:43.962+0100] ThreadLocalAllocBuffer::compute_size(2052) returns 252052
[2020-05-19T15:43:43.962+0100] TLAB: fill thread: 0x76e06f10 [id: 534] desired_size: 976KB slow allocs: 4  refill waste: 15688B alloc: 0.13977     1612KB refills: 2 waste  0.2% gc: 0B slow: 4520B fast: 0B
[2020-05-19T15:43:43.982+0100] ThreadLocalAllocBuffer::compute_size(28878) returns 278878
[2020-05-19T15:43:43.983+0100] TLAB: fill thread: 0x76e06f10 [id: 534] desired_size: 976KB slow allocs: 4  refill waste: 15624B alloc: 0.13977     1764KB refills: 3 waste  0.3% gc: 0B slow: 10424B fast: 0B
[2020-05-19T15:43:44.023+0100] ThreadLocalAllocBuffer::compute_size(4) returns 250004
[2020-05-19T15:43:44.023+0100] TLAB: fill thread: 0x7991df20 [id: 32696] desired_size: 976KB slow allocs: 0  refill waste: 15624B alloc: 0.00132       19KB refills: 1 waste  0.0% gc: 0B slow: 0B fast: 0B
</code>```

Update

I reran with -XX:+HeapDumpOnOutOfMemoryError option added, and this time it showed:

```<code>java.lang.OutOfMemoryError: Java heap space
Dumping heap to java_pid1600.hprof ...
</code>```

but then the dump itself failed with

```<code>#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0xb6a81b9a, pid=1600, tid=1606
#
# JRE version: OpenJDK Runtime Environment (14.0+36) (build 14+36)
# Java VM: OpenJDK Client VM (14+36, mixed mode, serial gc, linux-arm)
# Problematic frame:
# V  [libjvm.so+0x22eb9a]  DumperSupport::dump_field_value(DumpWriter*, char, oopDesc*, int)+0x91
#
# No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /mnt/system/config/Apps/SongKong/songkong/hs_err_pid1600.log
#
# If you would like to submit a bug report, please visit:
#   https://bugreport.java.com/bugreport/crash.jsp
</code>```

I am not clear if the dump failed because of ulimit or soemthing else, but
 java_pid1600.hprof was created but was empty

I was also monitoring the process with jstat -gc, and jstat -gcutil. I paste the end of the putput here, to me it does not look like there was a particular memory problem before the crash, although I am only checking every 5 seconds so maybe that is the issue ?

```<code>[root@N1-0247 bin]# ./jstat -gc 1600 5s

 S0C    S1C    S0U    S1U      EC       EU        OC         OU       MC     MU    CCSC   CCSU   YGC     YGCT    FGC    FGCT    CGC    CGCT     GCT
........
30720.0 30720.0  0.0    0.0   245760.0 236647.2  614400.0   494429.2  50136.0 49436.9  0.0    0.0     5084 3042.643  155   745.523   -          - 3788.166
30720.0 30720.0  0.0   28806.1 245760.0 244460.2  614400.0   506541.7  50136.0 49436.9  0.0    0.0     5085 3043.887  156   745.523   -          - 3789.410
30720.0 30720.0 28760.4  0.0   245760.0 245760.0  614400.0   514809.7  50136.0 49437.2  0.0    0.0     5086 3044.895  157   751.204   -          - 3796.098
30720.0 30720.0  0.0   231.1  245760.0 234781.8  614400.0   514809.7  50136.0 49437.2  0.0    0.0     5087 3044.895  157   755.042   -          - 3799.936
30720.0 30720.0  0.0    0.0   245760.0 190385.5  614400.0   519650.7  50136.0 49449.6  0.0    0.0     5087 3045.905  159   758.890   -          - 3804.795
30720.0 30720.0  0.0    0.0   245760.0 190385.5  614400.0   519650.7  50136.0 49449.6  0.0    0.0     5087 3045.905  159   758.890   -          - 3804.795

[root@N1-0247 bin]# ./jstat -gc 1600 5s
     S0     S1     E      O      M     CCS    YGC     YGCT    FGC    FGCT    CGC    CGCT     GCT
..............
     99.70   0.00 100.00  75.54  98.56      -   5080 3037.321   150  724.674     -        - 3761.995
      0.00  29.93  99.30  75.55  98.56      -   5081 3038.403   151  728.584     -        - 3766.987
      0.00 100.00  99.30  75.94  98.56      -   5081 3039.405   152  728.584     -        - 3767.989
    100.00   0.00  99.14  76.14  98.56      -   5082 3040.366   153  734.088     -        - 3774.454
      0.00  96.58  99.87  78.50  98.57      -   5083 3041.366   154  737.960     -        - 3779.325
     56.99   0.00 100.00  78.50  98.58      -   5084 3041.366   154  741.880     -        - 3783.246
      0.00   0.00  96.29  80.47  98.61      -   5084 3042.643   155  745.523     -        - 3788.166
      0.00  93.77  99.47  82.44  98.61      -   5085 3043.887   156  745.523     -        - 3789.410
     93.62   0.00 100.00  83.79  98.61      -   5086 3044.895   157  751.204     -        - 3796.098
      0.00   0.76  95.53  83.79  98.61      -   5087 3044.895   157  755.042     -        - 3799.936
      0.00   0.00  77.47  84.58  98.63      -   5087 3045.905   159  758.890     -        - 3804.795
      0.00   0.00  77.47  84.58  98.63      -   5087 3045.905   159  758.890     -        - 3804.795
</code>```

Update Latest run

Configured gclogging, i get many

```<code>Pause Young (Allocation Failure)
</code>```

errors, does this indicate I need to make the eden space larger?

```<code>[2020-05-29T14:00:22.668+0100] GC(44) Pause Young (GCLocker Initiated GC)
[2020-05-29T14:00:22.739+0100] GC(44) DefNew: 43230K(46208K)->4507K(46208K) Eden: 41088K(41088K)->0K(41088K) From: 2142K(5120K)->4507K(5120K)
[2020-05-29T14:00:22.739+0100] GC(44) Tenured: 50532K(102400K)->50532K(102400K)
[2020-05-29T14:00:22.740+0100] GC(44) Metaspace: 40054K(40536K)->40054K(40536K)
[2020-05-29T14:00:22.740+0100] GC(44) Pause Young (GCLocker Initiated GC) 91M->53M(145M) 72.532ms
[2020-05-29T14:00:22.741+0100] GC(44) User=0.07s Sys=0.00s Real=0.07s
[2020-05-29T14:00:25.196+0100] GC(45) Pause Young (Allocation Failure)
[2020-05-29T14:00:25.306+0100] GC(45) DefNew: 45595K(46208K)->2150K(46208K) Eden: 41088K(41088K)->0K(41088K) From: 4507K(5120K)->2150K(5120K)
[2020-05-29T14:00:25.306+0100] GC(45) Tenured: 50532K(102400K)->53861K(102400K)
[2020-05-29T14:00:25.307+0100] GC(45) Metaspace: 40177K(40664K)->40177K(40664K)
[2020-05-29T14:00:25.307+0100] GC(45) Pause Young (Allocation Failure) 93M->54M(145M) 111.252ms
[2020-05-29T14:00:25.308+0100] GC(45) User=0.08s Sys=0.02s Real=0.11s
[2020-05-29T14:00:29.248+0100] GC(46) Pause Young (Allocation Failure)
[2020-05-29T14:00:29.404+0100] GC(46) DefNew: 43238K(46208K)->4318K(46208K) Eden: 41088K(41088K)->0K(41088K) From: 2150K(5120K)->4318K(5120K)
[2020-05-29T14:00:29.405+0100] GC(46) Tenured: 53861K(102400K)->53861K(102400K)
[2020-05-29T14:00:29.405+0100] GC(46) Metaspace: 40319K(40792K)->40319K(40792K)
[2020-05-29T14:00:29.406+0100] GC(46) Pause Young (Allocation Failure) 94M->56M(145M) 157.614ms
[2020-05-29T14:00:29.406+0100] GC(46) User=0.07s Sys=0.00s Real=0.16s
[2020-05-29T14:00:36.466+0100] GC(47) Pause Young (Allocation Failure)
[2020-05-29T14:00:36.661+0100] GC(47) DefNew: 45406K(46208K)->5120K(46208K) Eden: 41088K(41088K)->0K(41088K) From: 4318K(5120K)->5120K(5120K)
[2020-05-29T14:00:36.662+0100] GC(47) Tenured: 53861K(102400K)->55125K(102400K)
[2020-05-29T14:00:36.662+0100] GC(47) Metaspace: 40397K(40920K)->40397K(40920K)
[2020-05-29T14:00:36.663+0100] GC(47) Pause Young (Allocation Failure) 96M->58M(145M) 196.531ms
[2020-05-29T14:00:36.663+0100] GC(47) User=0.09s Sys=0.01s Real=0.19s
[2020-05-29T14:00:40.523+0100] GC(48) Pause Young (Allocation Failure)
[2020-05-29T14:00:40.653+0100] GC(48) DefNew: 44274K(46208K)->2300K(46208K) Eden: 39154K(41088K)->0K(41088K) From: 5120K(5120K)->2300K(5120K)
[2020-05-29T14:00:40.653+0100] GC(48) Tenured: 55125K(102400K)->59965K(102400K)
[2020-05-29T14:00:40.654+0100] GC(48) Metaspace: 40530K(41048K)->40530K(41048K)
[2020-05-29T14:00:40.654+0100] GC(48) Pause Young (Allocation Failure) 97M->60M(145M) 131.365ms
[2020-05-29T14:00:40.655+0100] GC(48) User=0.11s Sys=0.01s Real=0.14s
[2020-05-29T14:00:43.936+0100] GC(49) Pause Young (Allocation Failure)
[2020-05-29T14:00:44.100+0100] GC(49) DefNew: 43388K(46208K)->5120K(46208K) Eden: 41088K(41088K)->0K(41088K) From: 2300K(5120K)->5120K(5120K)
</code>```

Updated with gc analysis done by gceasy

Okay so this is useful I uploaded log to gceasy.org and it clearly shows that shortly before it crashed heap size was significantly higher and approaching the 900mb limit,even after a number of full gcs, so I think basically it ran out of heap space.

What is a little frustrating is I have the

-XX:+HeapDumpOnOutOfMemoryError

option enabled, but when it crashes it reports an issue trying to do create the dump file so I cannot get one.

And when I process the same file on Windows with the same setting for heap size it suceeds without failure, But Im goinf to run again ewith gclogging enabled and see if it reaches simailr levels even if it doesnt actually fall over.

Ran again (this is building on chnages made in previous run and doesnt show start of run) but to me  the memory usage is higher but looks quite normal (sawtooth pattern) with no particular differenc ebefore the crash.

Update

With last run I reduced max heap from 900MB to 600MB, but I also monitored with vmstat, Yo can see clearly below where the applciation crashed but It doesn't seem we were approaching particularly ow memory at this point.

```<code>        procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
     3  0      0  57072   7812 1174128    0    0  5360     0  211  558 96  4  0  0  0
     1  0      0  55220   7812 1176184    0    0  2048     0  203  467 79 21  0  0  0
     3  0      0  61296   7812 1169096    0    0  2036    44  193  520 96  4  0  0  0
     2  0      0  59808   7812 1171144    0    0  2048    32  212  522 96  4  0  0  0
     1  0      0  59436   7812 1171144    0    0     0     0  180  307 83 17  0  0  0
     1  0      0  59436   7812 1171144    0    0     0     0  179  173 100  0  0  0  0
     1  0      0  59436   7812 1171128    0    0     0     0  179  184 100  0  0  0  0
     2  1      0  51764   7816 1158452    0    0  4124    52  190  490 80 20  0  0  0
     3  0      0  63428   7612 1146388    0    0 20472    48  251  533 86 14  0  0  0
     2  0      0  63428   7616 1146412    0    0     4     0  196  508 99  1  0  0  0
     2  0      0  84136   7616 1146400    0    0     0     0  186  461 84 16  0  0  0
     2  0      0  61436   7608 1148960    0    0 24601     0  325  727 77 23  0  0  0
     4  0      0  60196   7648 1150204    0    0  1160    76  232  611 98  2  0  0  0
     4  0      0  59204   7656 1151052    0    0    52   376  305  570 80 20  0  0  0
     3  0      0  59204   7656 1151052    0    0     0     0  378  433 96  4  0  0  0
     1  0      0 762248   7768 1151420    0    0   106     0  253  660 74 26  0  0  0
     0  0      0 859272   8188 1151892    0    0   417     0  302  550  9 26 64  1  0
     0  0      0 859272   8188 1151892    0    0     0     0  111  132  0  0 100  0  0
</code>```

答:
Based on your jstat data and their explanation here: https://docs.oracle.com/en/java/javase/11/tools/jstat.html#GUID-5F72A7F9-5D5A-4486-8201-E1D1BA8ACCB5

I would not expect OutOfMemoryError just yet from the HeapSpace based on the slow and steady rate of the Old Generation filling up and the small size of the from and to space (not that I know whether your application might allocate a huge array anytime soon) unless:

initial heap size (-Xms) is smaller than the max (-Xmx) and

Linux has overcomitted virtual memory

If you do overcommit (and who doesn't) maybe you should keep an eye on Linux with vmstat 1 or gathering data frequently for sar

But I do wonder why you are not using Garbage Collection logging with -Xlog:gc*:stderr or to a file with -Xlog:gc*:file= and maybe analyze that with https://gceasy.io/ as it is very low overhead (unless writing to the logfile is slow) and very precise. For more information on the logging syntax see: https://openjdk.java.net/jeps/158 and https://openjdk.java.net/jeps/271

java -Xlog:gc*:stderr -jar yourapp.jar

and analyze those logs with great ease with tools like these:

https://gceasy.io/

JClarity Censum

This should give similar information as jstack and more in realtime (as far as I know)

ContentLicense: CC BY-SA 4.0

Body:
My Java application on an single cpu arm7 (32bit) device using Java 14 is occasionally crashing
after running under load for a number of hours, and is always failing in ThreadLocalAllocBuffer::resize()

  A fatal error has been detected by the Java Runtime Environment:
    #
    SIGSEGV (0xb) at pc=0xb6cd515e, pid=1725, tid=1733
    #
    JRE version: OpenJDK Runtime Environment (14.0+36) (build 14+36)
    Java VM: OpenJDK Client VM (14+36, mixed mode, serial gc, linux-arm)
    Problematic frame:
    V
    #
    No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
    #
    If you would like to submit a bug report, please visit:
    https://bugreport.java.com/bugreport/crash.jsp
    #

--------------- S U M M A R Y ------------

Command Line: -Duser.home=/mnt/app/share/log -Djdk.lang.Process.launchMechanism=vfork -Xms150m -Xmx900m -Dcom.mchange.v2.log.MLog=com.mchange.v2.log.jdk14logging.Jdk14MLog -Dorg.jboss.logging.provider=jdk -Djava.util.logging.config.class=com.jthink.songkong.logging.StandardLogging --add-opens=java.base/java.lang=ALL-UNNAMED lib/songkong-6.9.jar -r

Host: Marvell PJ4Bv7 Processor rev 1 (v7l), 1 cores, 1G, Buildroot 2014.11-rc1
Time: Fri Apr 24 19:36:54 2020 BST elapsed time: 37456 seconds (0d 10h 24m 16s)

    --------------- T H R E A D ---------------

    Current thread (0xb6582a30): VMThread "VM Thread" [stack: 0x7b716000,0x7b796000] [id=3625] _threads_hazard_ptr=0x7742f140

    Stack: [0x7b716000,0x7b796000], sp=0x7b7946b0, free space=505k
    Native frames: (J=compiled Java code, A=aot compiled Java code, j=interpreted, Vv=VM code, C=native code)
    V [libjvm.so+0x48015e] ThreadLocalAllocBuffer::resize()+0x85

    [error occurred during error reporting (printing native stack), id 0xb, SIGSEGV (0xb) at pc=0xb6b4ccae]

Now this must surely be bug in JVM, but as its not one of the standard Java platforms and I dont have a simple test case I cannot see it getting fixed anytime soon, so I am trying to workaround it. Its also worth noting  that it crashed with ThreadLocalAllocBuffer::accumulate_statistics_before_gc() when I used Java 11 which is why I moved to Java 14 to try and resolve the issue.

As the the issue is with TLABs one solution is to disable TLABS with -XX:-UseTLAB but that makes the code run slower on an already slow machine.

So I think another solution is to disable resizing with -XX:-ResizeTLAB, but then I need to know work out a suitable size and specify that using -XX:TLABSize=N. But I am not sure what N actually represents and what would be a suitable size to set

I tried setting  -XX:TLABSize=1000000 which seems to me to be quite large ?

I have some logging set with

 -Xlog:tlab*=debug,tlab*=trace:file=gc.log:time:filecount=7,filesize=8M

but I don't really understand the output.

[2020-05-19T15:43:43.836+0100] ThreadLocalAllocBuffer::compute_size(132) returns 250132
[2020-05-19T15:43:43.837+0100] TLAB: fill thread: 0x0026d548 [id: 871] desired_size: 976KB slow allocs: 0  refill waste: 15624B alloc: 0.25725     1606KB refills: 1 waste  0.0% gc: 0B slow: 0B fast: 0B
[2020-05-19T15:43:43.853+0100] ThreadLocalAllocBuffer::compute_size(6) returns 250006
[2020-05-19T15:43:43.854+0100] TLAB: fill thread: 0xb669be48 [id: 32635] desired_size: 976KB slow allocs: 0  refill waste: 15624B alloc: 0.00002        0KB refills: 1 waste  0.0% gc: 0B slow: 0B fast: 0B
[2020-05-19T15:43:43.910+0100] ThreadLocalAllocBuffer::compute_size(4) returns 250004
[2020-05-19T15:43:43.911+0100] TLAB: fill thread: 0x76c1d6f8 [id: 917] desired_size: 976KB slow allocs: 0  refill waste: 15624B alloc: 0.91261     8085KB refills: 1 waste  0.0% gc: 0B slow: 0B fast: 0B
[2020-05-19T15:43:43.962+0100] ThreadLocalAllocBuffer::compute_size(2052) returns 252052
[2020-05-19T15:43:43.962+0100] TLAB: fill thread: 0x76e06f10 [id: 534] desired_size: 976KB slow allocs: 4  refill waste: 15688B alloc: 0.13977     1612KB refills: 2 waste  0.2% gc: 0B slow: 4520B fast: 0B
[2020-05-19T15:43:43.982+0100] ThreadLocalAllocBuffer::compute_size(28878) returns 278878
[2020-05-19T15:43:43.983+0100] TLAB: fill thread: 0x76e06f10 [id: 534] desired_size: 976KB slow allocs: 4  refill waste: 15624B alloc: 0.13977     1764KB refills: 3 waste  0.3% gc: 0B slow: 10424B fast: 0B
[2020-05-19T15:43:44.023+0100] ThreadLocalAllocBuffer::compute_size(4) returns 250004
[2020-05-19T15:43:44.023+0100] TLAB: fill thread: 0x7991df20 [id: 32696] desired_size: 976KB slow allocs: 0  refill waste: 15624B alloc: 0.00132       19KB refills: 1 waste  0.0% gc: 0B slow: 0B fast: 0B

Update

I reran with -XX:+HeapDumpOnOutOfMemoryError option added, and this time it showed:

java.lang.OutOfMemoryError: Java heap space
Dumping heap to java_pid1600.hprof ...

but then the dump itself failed with

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0xb6a81b9a, pid=1600, tid=1606
#
# JRE version: OpenJDK Runtime Environment (14.0+36) (build 14+36)
# Java VM: OpenJDK Client VM (14+36, mixed mode, serial gc, linux-arm)
# Problematic frame:
# V  [libjvm.so+0x22eb9a]  DumperSupport::dump_field_value(DumpWriter*, char, oopDesc*, int)+0x91
#
# No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /mnt/system/config/Apps/SongKong/songkong/hs_err_pid1600.log
#
# If you would like to submit a bug report, please visit:
#   https://bugreport.java.com/bugreport/crash.jsp

I am not clear if the dump failed because of ulimit or soemthing else, but
 java_pid1600.hprof was created but was empty

I was also monitoring the process with jstat -gc, and jstat -gcutil. I paste the end of the putput here, to me it does not look like there was a particular memory problem before the crash, although I am only checking every 5 seconds so maybe that is the issue ?

[root@N1-0247 bin]# ./jstat -gc 1600 5s

 S0C    S1C    S0U    S1U      EC       EU        OC         OU       MC     MU    CCSC   CCSU   YGC     YGCT    FGC    FGCT    CGC    CGCT     GCT
........
30720.0 30720.0  0.0    0.0   245760.0 236647.2  614400.0   494429.2  50136.0 49436.9  0.0    0.0     5084 3042.643  155   745.523   -          - 3788.166
30720.0 30720.0  0.0   28806.1 245760.0 244460.2  614400.0   506541.7  50136.0 49436.9  0.0    0.0     5085 3043.887  156   745.523   -          - 3789.410
30720.0 30720.0 28760.4  0.0   245760.0 245760.0  614400.0   514809.7  50136.0 49437.2  0.0    0.0     5086 3044.895  157   751.204   -          - 3796.098
30720.0 30720.0  0.0   231.1  245760.0 234781.8  614400.0   514809.7  50136.0 49437.2  0.0    0.0     5087 3044.895  157   755.042   -          - 3799.936
30720.0 30720.0  0.0    0.0   245760.0 190385.5  614400.0   519650.7  50136.0 49449.6  0.0    0.0     5087 3045.905  159   758.890   -          - 3804.795
30720.0 30720.0  0.0    0.0   245760.0 190385.5  614400.0   519650.7  50136.0 49449.6  0.0    0.0     5087 3045.905  159   758.890   -          - 3804.795

[root@N1-0247 bin]# ./jstat -gc 1600 5s
     S0     S1     E      O      M     CCS    YGC     YGCT    FGC    FGCT    CGC    CGCT     GCT
..............
     99.70   0.00 100.00  75.54  98.56      -   5080 3037.321   150  724.674     -        - 3761.995
      0.00  29.93  99.30  75.55  98.56      -   5081 3038.403   151  728.584     -        - 3766.987
      0.00 100.00  99.30  75.94  98.56      -   5081 3039.405   152  728.584     -        - 3767.989
    100.00   0.00  99.14  76.14  98.56      -   5082 3040.366   153  734.088     -        - 3774.454
      0.00  96.58  99.87  78.50  98.57      -   5083 3041.366   154  737.960     -        - 3779.325
     56.99   0.00 100.00  78.50  98.58      -   5084 3041.366   154  741.880     -        - 3783.246
      0.00   0.00  96.29  80.47  98.61      -   5084 3042.643   155  745.523     -        - 3788.166
      0.00  93.77  99.47  82.44  98.61      -   5085 3043.887   156  745.523     -        - 3789.410
     93.62   0.00 100.00  83.79  98.61      -   5086 3044.895   157  751.204     -        - 3796.098
      0.00   0.76  95.53  83.79  98.61      -   5087 3044.895   157  755.042     -        - 3799.936
      0.00   0.00  77.47  84.58  98.63      -   5087 3045.905   159  758.890     -        - 3804.795
      0.00   0.00  77.47  84.58  98.63      -   5087 3045.905   159  758.890     -        - 3804.795

Update Latest run

Configured gclogging, i get many

Pause Young (Allocation Failure)

errors, does this indicate I need to make the eden space larger?

[2020-05-29T14:00:22.668+0100] GC(44) Pause Young (GCLocker Initiated GC)
[2020-05-29T14:00:22.739+0100] GC(44) DefNew: 43230K(46208K)->4507K(46208K) Eden: 41088K(41088K)->0K(41088K) From: 2142K(5120K)->4507K(5120K)
[2020-05-29T14:00:22.739+0100] GC(44) Tenured: 50532K(102400K)->50532K(102400K)
[2020-05-29T14:00:22.740+0100] GC(44) Metaspace: 40054K(40536K)->40054K(40536K)
[2020-05-29T14:00:22.740+0100] GC(44) Pause Young (GCLocker Initiated GC) 91M->53M(145M) 72.532ms
[2020-05-29T14:00:22.741+0100] GC(44) User=0.07s Sys=0.00s Real=0.07s
[2020-05-29T14:00:25.196+0100] GC(45) Pause Young (Allocation Failure)
[2020-05-29T14:00:25.306+0100] GC(45) DefNew: 45595K(46208K)->2150K(46208K) Eden: 41088K(41088K)->0K(41088K) From: 4507K(5120K)->2150K(5120K)
[2020-05-29T14:00:25.306+0100] GC(45) Tenured: 50532K(102400K)->53861K(102400K)
[2020-05-29T14:00:25.307+0100] GC(45) Metaspace: 40177K(40664K)->40177K(40664K)
[2020-05-29T14:00:25.307+0100] GC(45) Pause Young (Allocation Failure) 93M->54M(145M) 111.252ms
[2020-05-29T14:00:25.308+0100] GC(45) User=0.08s Sys=0.02s Real=0.11s
[2020-05-29T14:00:29.248+0100] GC(46) Pause Young (Allocation Failure)
[2020-05-29T14:00:29.404+0100] GC(46) DefNew: 43238K(46208K)->4318K(46208K) Eden: 41088K(41088K)->0K(41088K) From: 2150K(5120K)->4318K(5120K)
[2020-05-29T14:00:29.405+0100] GC(46) Tenured: 53861K(102400K)->53861K(102400K)
[2020-05-29T14:00:29.405+0100] GC(46) Metaspace: 40319K(40792K)->40319K(40792K)
[2020-05-29T14:00:29.406+0100] GC(46) Pause Young (Allocation Failure) 94M->56M(145M) 157.614ms
[2020-05-29T14:00:29.406+0100] GC(46) User=0.07s Sys=0.00s Real=0.16s
[2020-05-29T14:00:36.466+0100] GC(47) Pause Young (Allocation Failure)
[2020-05-29T14:00:36.661+0100] GC(47) DefNew: 45406K(46208K)->5120K(46208K) Eden: 41088K(41088K)->0K(41088K) From: 4318K(5120K)->5120K(5120K)
[2020-05-29T14:00:36.662+0100] GC(47) Tenured: 53861K(102400K)->55125K(102400K)
[2020-05-29T14:00:36.662+0100] GC(47) Metaspace: 40397K(40920K)->40397K(40920K)
[2020-05-29T14:00:36.663+0100] GC(47) Pause Young (Allocation Failure) 96M->58M(145M) 196.531ms
[2020-05-29T14:00:36.663+0100] GC(47) User=0.09s Sys=0.01s Real=0.19s
[2020-05-29T14:00:40.523+0100] GC(48) Pause Young (Allocation Failure)
[2020-05-29T14:00:40.653+0100] GC(48) DefNew: 44274K(46208K)->2300K(46208K) Eden: 39154K(41088K)->0K(41088K) From: 5120K(5120K)->2300K(5120K)
[2020-05-29T14:00:40.653+0100] GC(48) Tenured: 55125K(102400K)->59965K(102400K)
[2020-05-29T14:00:40.654+0100] GC(48) Metaspace: 40530K(41048K)->40530K(41048K)
[2020-05-29T14:00:40.654+0100] GC(48) Pause Young (Allocation Failure) 97M->60M(145M) 131.365ms
[2020-05-29T14:00:40.655+0100] GC(48) User=0.11s Sys=0.01s Real=0.14s
[2020-05-29T14:00:43.936+0100] GC(49) Pause Young (Allocation Failure)
[2020-05-29T14:00:44.100+0100] GC(49) DefNew: 43388K(46208K)->5120K(46208K) Eden: 41088K(41088K)->0K(41088K) From: 2300K(5120K)->5120K(5120K)

Updated with gc analysis done by gceasy

Okay so this is useful I uploaded log to gceasy.org and it clearly shows that shortly before it crashed heap size was significantly higher and approaching the 900mb limit,even after a number of full gcs, so I think basically it ran out of heap space.

What is a little frustrating is I have the

-XX:+HeapDumpOnOutOfMemoryError

option enabled, but when it crashes it reports an issue trying to do create the dump file so I cannot get one.

And when I process the same file on Windows with the same setting for heap size it suceeds without failure, But Im goinf to run again ewith gclogging enabled and see if it reaches simailr levels even if it doesnt actually fall over.

Ran again (this is building on chnages made in previous run and doesnt show start of run) but to me  the memory usage is higher but looks quite normal (sawtooth pattern) with no particular differenc ebefore the crash.

Update

With last run I reduced max heap from 900MB to 600MB, but I also monitored with vmstat, Yo can see clearly below where the applciation crashed but It doesn't seem we were approaching particularly ow memory at this point.

        procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
     3  0      0  57072   7812 1174128    0    0  5360     0  211  558 96  4  0  0  0
     1  0      0  55220   7812 1176184    0    0  2048     0  203  467 79 21  0  0  0
     3  0      0  61296   7812 1169096    0    0  2036    44  193  520 96  4  0  0  0
     2  0      0  59808   7812 1171144    0    0  2048    32  212  522 96  4  0  0  0
     1  0      0  59436   7812 1171144    0    0     0     0  180  307 83 17  0  0  0
     1  0      0  59436   7812 1171144    0    0     0     0  179  173 100  0  0  0  0
     1  0      0  59436   7812 1171128    0    0     0     0  179  184 100  0  0  0  0
     2  1      0  51764   7816 1158452    0    0  4124    52  190  490 80 20  0  0  0
     3  0      0  63428   7612 1146388    0    0 20472    48  251  533 86 14  0  0  0
     2  0      0  63428   7616 1146412    0    0     4     0  196  508 99  1  0  0  0
     2  0      0  84136   7616 1146400    0    0     0     0  186  461 84 16  0  0  0
     2  0      0  61436   7608 1148960    0    0 24601     0  325  727 77 23  0  0  0
     4  0      0  60196   7648 1150204    0    0  1160    76  232  611 98  2  0  0  0
     4  0      0  59204   7656 1151052    0    0    52   376  305  570 80 20  0  0  0
     3  0      0  59204   7656 1151052    0    0     0     0  378  433 96  4  0  0  0
     1  0      0 762248   7768 1151420    0    0   106     0  253  660 74 26  0  0  0
     0  0      0 859272   8188 1151892    0    0   417     0  302  550  9 26 64  1  0
     0  0      0 859272   8188 1151892    0    0     0     0  111  132  0  0 100  0  0


问:
python/django - assigning to global variable in decorator
I use thread-locals in order to implement user-based theming. First I wrote a middleware and have had no issues. Now I am converting it to a decorator and having an embarrassing issue. Here's the simplified code:

```<code># views.py

_thread_locals = threading.local()

def foo_decorator(f):
    def _wrapper(*args, **kwargs):
        global _thread_locals
        _thread_locals.bar = kwargs['bar']
        return f(*args, **kwargs)
    return wraps(f)(_wrapper)

@foo_decorator
def some_view(request, bar):
    # Stuff here
    # Here _thread_locals.bar exists

# This is called by the template loader:
def get_theme():
    return _thread_locals.bar # Error
</code>```

What I get when get_theme is called is a complaint that bar attribute does not exist. Where is it gone? Most probably I am missing something related to scoping in closures, but can't figure out what.

答:
Your code is working for me, but only if some_view is called before get_theme.

The devserver kills your process and restarts a new one every time you modify a file in your project directory.  Your thread data is of course lost when this is done.  If you see the "Validating models..." message, your thread data is already lost.

ContentLicense: CC BY-SA 3.0

Body:
I use thread-locals in order to implement user-based theming. First I wrote a middleware and have had no issues. Now I am converting it to a decorator and having an embarrassing issue. Here's the simplified code:

# views.py

_thread_locals = threading.local()

def foo_decorator(f):
    def _wrapper(*args, **kwargs):
        global _thread_locals
        _thread_locals.bar = kwargs['bar']
        return f(*args, **kwargs)
    return wraps(f)(_wrapper)

@foo_decorator
def some_view(request, bar):
    # Stuff here
    # Here _thread_locals.bar exists

# This is called by the template loader:
def get_theme():
    return _thread_locals.bar # Error

What I get when get_theme is called is a complaint that bar attribute does not exist. Where is it gone? Most probably I am missing something related to scoping in closures, but can't figure out what.


问:
Configure AIX Syslog to send application logs to Logstash on Remote Server
I am trying to ship an application log files to logstash. It runs on AIX, so I can't use filebeat. Has anyone configured syslog to send to logstash before?

I have an application that will write files to /var/mqsi/log/application.log
I want to pick that log up and forward to a remote server.

```<code>syslog.conf

mqsi.debug  /var/mqsi/log/application.log
mqsi.debug       @10.123.445.217
</code>```

Any help is much appreciated.

答:
I want to ship logs from AIX server to kafka broker.

Below conf for logstash is not working

```<code> "output" {
  "kafka" {
    "bootstrap_servers" => "10.23.***.***:9092",
    "topic_id" =>  "my-topic",
    "ssl ca": "./usr/semasupp/LogShipper/PANTOMATH.GLOBAL.STANDARDCHARTERED.COM.cer",
    "ssl key": "./usr/semasupp/LogShipper/pantomath.global.standardchartered.com.ssl.key",
  }
},

  "files": [
    {
      "paths": [
        "/prd/ccsq/xib/data/trace/257/*"
      ],
      "fields": { "type": "syslog" }
    }
  ]
}
</code>```

Body:
I am trying to ship an application log files to logstash. It runs on AIX, so I can't use filebeat. Has anyone configured syslog to send to logstash before?

I have an application that will write files to /var/mqsi/log/application.log
I want to pick that log up and forward to a remote server.

syslog.conf

mqsi.debug  /var/mqsi/log/application.log
mqsi.debug       @10.123.445.217

Any help is much appreciated.

Body:
I want to ship logs from AIX server to kafka broker.

Below conf for logstash is not working

 "output" {
  "kafka" {
    "bootstrap_servers" => "10.23.***.***:9092",
    "topic_id" =>  "my-topic",
    "ssl ca": "./usr/semasupp/LogShipper/PANTOMATH.GLOBAL.STANDARDCHARTERED.COM.cer",
    "ssl key": "./usr/semasupp/LogShipper/pantomath.global.standardchartered.com.ssl.key",
  }
},

  "files": [
    {
      "paths": [
        "/prd/ccsq/xib/data/trace/257/*"
      ],
      "fields": { "type": "syslog" }
    }
  ]
}


问:
Java patten draw within for loop
I am very new in the programming world and in Javaland too.
I am trying to learn for loop and solving pattern with it.

I want to draw this pattern using for loop :

```<code>*
**
***
****
*****
</code>```

I am using this code to achieve this:

```<code>public class PatterWithForLoop {

    public static void main(String args[]){
        for(int i = 0; i <= 5; i++){
            System.out.println("*" + "\n");
        }
    }
}

</code>```

But, I am only getting * in 15 lines, not in the pattern.
Please, anyone can help to solve this issue here?

Thanks in advance.

答:
If you are using JDK 11, you can write the following code:

```<code>for(int i = 0; i < 5; i++) {
    System.out.println("*".repeat(i+1));
}
</code>```

With String.repeat(), you can do the same thing with less code.

Title: Java patten draw within for loop

Body:
I am very new in the programming world and in Javaland too.
I am trying to learn for loop and solving pattern with it.

I want to draw this pattern using for loop :

*
**
***
****
*****

I am using this code to achieve this:

public class PatterWithForLoop {

    public static void main(String args[]){
        for(int i = 0; i <= 5; i++){
            System.out.println("*" + "\n");
        }
    }
}

But, I am only getting * in 15 lines, not in the pattern.
Please, anyone can help to solve this issue here?

Thanks in advance.

Body:
If you are using JDK 11, you can write the following code:

for(int i = 0; i < 5; i++) {
    System.out.println("*".repeat(i+1));
}

With String.repeat(), you can do the same thing with less code.


问:
how to filter an object with RegExp and return a new object
I have an object:

```<code>object1 = {
dEabc: 1,
dEasdf: 2,
dEjfhs: 5,
bhsn: 7,
oki: 9
}

</code>```

My goal is to get a new object which contains only the keys with "dE", that means:

```<code>object1 = {
dEabc: 1,
dEasdf: 2,
dEjfhs: 5,
}
</code>```

is the object that I expected.

What I have tried is,
firstly I got all key from this object with

```<code>Object.keys(object);
</code>```

and then I used a RegExp to check if properties in the object with relevant keys.
if not I use

```<code>delete object[key];
</code>```

Any solutions?

答:
With reduce, it can be done this way:

object1 = {
dEabc: 1,
dEasdf: 2,
dEjfhs: 5,
bhsn: 7,
oki: 9
}

let finalObj = Object.keys(object1).filter(i=>i.includes('dE')).reduce((obj, curr)=>{
     return {...obj, ...{[curr]: object1[curr]}};
},{});
console.log(finalObj);```

Body:
I have an object:

object1 = {
dEabc: 1,
dEasdf: 2,
dEjfhs: 5,
bhsn: 7,
oki: 9
}

My goal is to get a new object which contains only the keys with "dE", that means:

object1 = {
dEabc: 1,
dEasdf: 2,
dEjfhs: 5,
}

is the object that I expected.

What I have tried is,
firstly I got all key from this object with

Object.keys(object);

and then I used a RegExp to check if properties in the object with relevant keys.
if not I use

delete object[key];

Any solutions?


问:
How to maintain an aspect ratio of a div inside another div
So I have a div and which contains two divs one on top of the other. I want the one on top to have an aspect ratio of 4:5 with 100% width as it will contain an image of that ratio and then remaining height goes goes  to the second div.

This is as far I could go

You can find all the code here

note: I wish to maintain that aspect ratio as the user shrink the screen size, try to shrink the width on the sandbox example to see

```<code>&_product{
                    overflow: hidden;
                    padding-top: 800.34px / 591.44px * 100%;
                    background: rgb(207, 206, 206);
                    position: relative;
                    max-width: 232px;
                    // max-height: 317.313px;
                    &_inside {
                      display: grid;
                      // grid-template-columns: 1fr
                      // grid-template-rows: 1fr 80px;;
                      position: absolute;
                      top: 0;
                      left: 0;
                      width: 100%;
                      height: 100%;
                    }

                    border: 1px solid rgb(131, 72, 72);
                    position:relative;
                    width:100%;
                    height:fit-content;
                    box-shadow: 0 5px 15px rgba(0,0,0,.25);
                    overflow:hidden;
                    &_imgbox{
                      height: 100% ;
                      border: 1px solid red;
                      box-sizing:border-box;
                      position: relative;
                      // img{
                      //   width:100%;
                      //   display:block;
                      //   margin:0;
                      // }
                      &_aspect {
                        position: absolute;
                        width: 100%;
                        height: 5px/4px*100%;   // My attempted calculation
                        background: blue;
                        top: 0;
                        // left: 0;

                      }
                  }
                }
</code>```

html

```<code>    <div class="Recommendations_container_recomendation_box_product">
                            <div class="Recommendations_container_recomendation_box_product_inside">
                                <div class="Recommendations_container_recomendation_box_product_imgbox">
                                    <div class="Recommendations_container_recomendation_box_product_imgbox_aspect">

                                        {/* <img src="https://images.sportsdirect.com/images/products/62310640_l.jpg" /> */}
                                    </div>

                                </div>
                                <div class="Recommendations_container_recomendation_box_details">
                                    <div class="Recommendations_container_recomendation_box_details_price">$55.99</div>
                                    <h2>Brand Name<br /><span>Mens Designer T-shirt</span></h2>

                                </div>
                            </div>

                        </div>
</code>```

答:
I had this fixed by calculating the ratio on padding top instead of the height

fixed here

```<code>padding-top: height: 5px/4px * 100%;
</code>```

Body:
So I have a div and which contains two divs one on top of the other. I want the one on top to have an aspect ratio of 4:5 with 100% width as it will contain an image of that ratio and then remaining height goes goes  to the second div.

This is as far I could go

You can find all the code here

note: I wish to maintain that aspect ratio as the user shrink the screen size, try to shrink the width on the sandbox example to see

&_product{
                    overflow: hidden;
                    padding-top: 800.34px / 591.44px * 100%;
                    background: rgb(207, 206, 206);
                    position: relative;
                    max-width: 232px;
                    // max-height: 317.313px;
                    &_inside {
                      display: grid;
                      // grid-template-columns: 1fr
                      // grid-template-rows: 1fr 80px;;
                      position: absolute;
                      top: 0;
                      left: 0;
                      width: 100%;
                      height: 100%;
                    }

                    border: 1px solid rgb(131, 72, 72);
                    position:relative;
                    width:100%;
                    height:fit-content;
                    box-shadow: 0 5px 15px rgba(0,0,0,.25);
                    overflow:hidden;
                    &_imgbox{
                      height: 100% ;
                      border: 1px solid red;
                      box-sizing:border-box;
                      position: relative;
                      // img{
                      //   width:100%;
                      //   display:block;
                      //   margin:0;
                      // }
                      &_aspect {
                        position: absolute;
                        width: 100%;
                        height: 5px/4px*100%;   // My attempted calculation
                        background: blue;
                        top: 0;
                        // left: 0;

                      }
                  }
                }

html

                                        {/*  */}

$55.99

Brand Name
Mens Designer T-shirt

Body:
I had this fixed by calculating the ratio on padding top instead of the height

fixed here

padding-top: height: 5px/4px * 100%;


问:
Javascript and sqlite request
I have a little Problem. I got the following simple SQL request:

```<code>async userexists(pUsername) {
    await this._sequilze.query("SELECT username, userpw, grade FROM users WHERE username = pUser",
    {replacements: {pUser: pUsername}})
    .then(result => {
        console.log(result)
    })
}
</code>```

But I don't know why I get this Errormessage:

UnhandledPromiseRejectionWarning: SequelizeDatabaseError: SQLITE_ERROR: no such column: pUser

I mean, I replace the pUser with my parameter pUsername. It works when inserted into the database.
A request with hardcoded Data is still working too like WHERE grade = 1

I hope you can help me

Thanks in advance,

JN

答:
According to this reference for Sequelize (which it seems you are using):
Sequelize replacements doc

You should use ":key" in your SQL query, in your case replace "pUser" with ":pUser" to have it replaced

```<code>async userexists(pUsername) {
    await this._sequilze.query("SELECT username, userpw, grade FROM users WHERE username = :pUser",
    {replacements: {pUser: pUsername}})
    .then(result => {
        console.log(result)
    })
</code>```

}

Title: Javascript and sqlite request

Body:
I have a little Problem. I got the following simple SQL request:

async userexists(pUsername) {
    await this._sequilze.query("SELECT username, userpw, grade FROM users WHERE username = pUser",
    {replacements: {pUser: pUsername}})
    .then(result => {
        console.log(result)
    })
}

But I don't know why I get this Errormessage:

UnhandledPromiseRejectionWarning: SequelizeDatabaseError: SQLITE_ERROR: no such column: pUser

I mean, I replace the pUser with my parameter pUsername. It works when inserted into the database.
A request with hardcoded Data is still working too like WHERE grade = 1

I hope you can help me

Thanks in advance,

JN

Body:
According to this reference for Sequelize (which it seems you are using):
Sequelize replacements doc

You should use ":key" in your SQL query, in your case replace "pUser" with ":pUser" to have it replaced

async userexists(pUsername) {
    await this._sequilze.query("SELECT username, userpw, grade FROM users WHERE username = :pUser",
    {replacements: {pUser: pUsername}})
    .then(result => {
        console.log(result)
    })

}


问:
Split a data.table at position
I want to calculate the mean of a column in a data.table way by using row numbers or a vector with position.

Here is a sample data and a postion vector:

```<code>x <- data.table(a = c(1,2,3,4,5,6,7,8))
pos <- c(3,5)
</code>```

I tried:

```<code>x[mean(a), by = pos]
</code>```

So i want the mean from row 1:2, 3:4 and 5:8.

答:
You can use findInterval/cut to create groups based on pos :

```<code>library(data.table)
x[, mean(a), findInterval(a, pos)]

#   findInterval  V1
#1:            0 1.5
#2:            1 3.5
#3:            2 6.5
</code>```

Title: Split a data.table at position

Body:
I want to calculate the mean of a column in a data.table way by using row numbers or a vector with position.

Here is a sample data and a postion vector:

x <- data.table(a = c(1,2,3,4,5,6,7,8))
pos <- c(3,5)

I tried:

x[mean(a), by = pos]

So i want the mean from row 1:2, 3:4 and 5:8.

Body:
You can use findInterval/cut to create groups based on pos :

library(data.table)
x[, mean(a), findInterval(a, pos)]

#   findInterval  V1
#1:            0 1.5
#2:            1 3.5
#3:            2 6.5


问:
Can't upload pdf to slack "no_file_data" with Elixir and Hackney
I'm writing a slack bot in Elixir that has to upload a pdf to a channel.

Here's what I've come up with:

```<code>    {:ok, ref} = :hackney.request(
      :post,
      "https://slack.com/api/files.upload?"
      <> Plug.Conn.Query.encode(%{
          "channels" => channel_id,
          "filename" => "test.pdf",
          "filetype" => "pdf"
        }),
      [
        {"Content-Type", "multipart/form-data"},
        {"Authorization", "Bearer xoxb-XXXXX"},
      ],
      :stream_multipart,
      [])

    :hackney.send_multipart_body(
      ref,
      {:file, "./test.pdf"})

    {:ok, _status, _headers, ref} = :hackney.start_response(ref)

    {:ok, body} = :hackney.body(ref)

    body |> Poison.decode! |> IO.inspect
</code>```

But I get this error:
%{"error" => "no_file_data", "ok" => false}

I've used this question as a basis: Streaming a file using hackney

Version:

Elixir: 1.10.2

OTP: 22.0.7

Hackney: 1.15

PLug: 1.10

答:
Problem solved. I Just needed to remove {"Content-Type", "multipart/form-data"} in the header of the request. Hackney already provides one with a boundary.

My Content-Type Looked like this:

```<code>Content-Type: multipart/form-data
</code>```

When it should have looked like this:

```<code>Content-Type: multipart/form-data; boundary=---------------------------tlzhgtmgeelwolfm
</code>```

Here is the corrected code:

```<code>    {:ok, ref} = :hackney.request(
      :post,
      "https://slack.com/api/files.upload?"
      <> Plug.Conn.Query.encode(%{
          "channels" => channel_id,
          "filename" => "test.pdf",
          "filetype" => "pdf"
        }),
      [
        {"Authorization", "Bearer xoxb-XXXXX"},
      ],
      :stream_multipart,
      [])

    :hackney.send_multipart_body(
      ref,
      {:file, "./test.pdf"})

    {:ok, _status, _headers, ref} = :hackney.start_response(ref)

    {:ok, body} = :hackney.body(ref)

    body |> Poison.decode! |> IO.inspect
</code>```

Body:
I'm writing a slack bot in Elixir that has to upload a pdf to a channel.

Here's what I've come up with:

    {:ok, ref} = :hackney.request(
      :post,
      "https://slack.com/api/files.upload?"
      <> Plug.Conn.Query.encode(%{
          "channels" => channel_id,
          "filename" => "test.pdf",
          "filetype" => "pdf"
        }),
      [
        {"Content-Type", "multipart/form-data"},
        {"Authorization", "Bearer xoxb-XXXXX"},
      ],
      :stream_multipart,
      [])

    :hackney.send_multipart_body(
      ref,
      {:file, "./test.pdf"})

    {:ok, _status, _headers, ref} = :hackney.start_response(ref)

    {:ok, body} = :hackney.body(ref)

    body |> Poison.decode! |> IO.inspect

But I get this error:
%{"error" => "no_file_data", "ok" => false}

I've used this question as a basis: Streaming a file using hackney

Version:

Elixir: 1.10.2

OTP: 22.0.7

Hackney: 1.15

PLug: 1.10

Body:
Problem solved. I Just needed to remove {"Content-Type", "multipart/form-data"} in the header of the request. Hackney already provides one with a boundary.

My Content-Type Looked like this:

Content-Type: multipart/form-data

When it should have looked like this:

Content-Type: multipart/form-data; boundary=---------------------------tlzhgtmgeelwolfm

Here is the corrected code:

    {:ok, ref} = :hackney.request(
      :post,
      "https://slack.com/api/files.upload?"
      <> Plug.Conn.Query.encode(%{
          "channels" => channel_id,
          "filename" => "test.pdf",
          "filetype" => "pdf"
        }),
      [
        {"Authorization", "Bearer xoxb-XXXXX"},
      ],
      :stream_multipart,
      [])

    :hackney.send_multipart_body(
      ref,
      {:file, "./test.pdf"})

    {:ok, _status, _headers, ref} = :hackney.start_response(ref)

    {:ok, body} = :hackney.body(ref)

    body |> Poison.decode! |> IO.inspect


问:
Delete XElement based on attribute
I'm still playing around with xml.  Now I have a file that looks like this:

```<code><?xml version="1.0" encoding="utf-8"?>
<Attributes>
 <AttributeSet id="10110">
  <Attribute id="1">some text here</Attribute>
  <Attribute id="2">some text here</Attribute>
  <!-- 298 more Attribute nodes follow -->
  <!-- note that the value for the id attribute is numbered consecutively -->
 </AttributeSet>
</Attributes>
</code>```

There are 300 Attribute nodes total, most of which I don't need.  What I'd like to do is remove all Attribute nodes that don't have a specified value for the id attribute.  I have established a string array with about 10 values.  These values represent the attributes that I'd like to keep in the xml.  The rest I'd like to remove.

What I'm trying to do with the code below is modify the xml by removing all Attribute nodes that I don't want to use:

```<code>Dim ss() As String = New String() {"39", "41", "38", "111", "148", "222", "256", "270", "283", "284"} 'keep the Attributes whose id value is one of these numbers
Dim rv As New List(Of String)'will hold Attribute ids to remove
Dim bool As Boolean = False
For Each x As XElement In doc...<eb:Attribute>
 For Each s As String In ss
  If x.@id = s Then
   bool = True
   Exit For
  End If
 Next
 If bool = True Then
  'do nothing
 Else 'no attribute matched any of the attribute ids listed mark xelement for removal
  rv.Add(x.@id)
 End If
Next
'now remove the xelement
For Each tr As String In rv
 Dim h As String = tr
 doc...<eb:Attribute>.Where(Function(g) g.@id = h).Remove()
Next
'save the xml
doc.Save("C:\myXMLFile.xml")
</code>```

For some reason, my code doesn't work.  None of the undesired Attribute nodes are removed.

What's the problem?  How can I remove the Attribute nodes whose id attribute values don't match any number in my string array?

Thanks in advance.

P.S. - I hope I made myself clear in describing my problem.

答:
Never mind.  I figured it out.  Here what I did:</p>

```<code>For Each x As XElement In doc...<eb:Attribute>
 **bool = False 'I simply added this line of code and everything worked perfectly**
 For Each s As String In ss
  If x.@id = s Then
   bool = True
   Exit For
  End If
 Next
 If bool = True Then
  'do nothing
 Else 'no attribute matched any of the attribute ids listed so remove the xelement
  rv.Add(x.@id)
 End If
Next
</code>```

OwnerDisplayName: Dude-Dastic

ContentLicense: CC BY-SA 2.5

Title: Delete XElement based on attribute

Body:
I'm still playing around with xml.  Now I have a file that looks like this:

<?xml version="1.0" encoding="utf-8"?>
<Attributes>
 <AttributeSet id="10110">
  <Attribute id="1">some text here</Attribute>
  <Attribute id="2">some text here</Attribute>

 </AttributeSet>
</Attributes>

There are 300 Attribute nodes total, most of which I don't need.  What I'd like to do is remove all Attribute nodes that don't have a specified value for the id attribute.  I have established a string array with about 10 values.  These values represent the attributes that I'd like to keep in the xml.  The rest I'd like to remove.

What I'm trying to do with the code below is modify the xml by removing all Attribute nodes that I don't want to use:

Dim ss() As String = New String() {"39", "41", "38", "111", "148", "222", "256", "270", "283", "284"} 'keep the Attributes whose id value is one of these numbers
Dim rv As New List(Of String)'will hold Attribute ids to remove
Dim bool As Boolean = False
For Each x As XElement In doc...<eb:Attribute>
 For Each s As String In ss
  If x.@id = s Then
   bool = True
   Exit For
  End If
 Next
 If bool = True Then
  'do nothing
 Else 'no attribute matched any of the attribute ids listed mark xelement for removal
  rv.Add(x.@id)
 End If
Next
'now remove the xelement
For Each tr As String In rv
 Dim h As String = tr
 doc...<eb:Attribute>.Where(Function(g) g.@id = h).Remove()
Next
'save the xml
doc.Save("C:\myXMLFile.xml")

For some reason, my code doesn't work.  None of the undesired Attribute nodes are removed.

What's the problem?  How can I remove the Attribute nodes whose id attribute values don't match any number in my string array?

Thanks in advance.

P.S. - I hope I made myself clear in describing my problem.

Body:
Never mind.  I figured it out.  Here what I did:

For Each x As XElement In doc...<eb:Attribute>
 **bool = False 'I simply added this line of code and everything worked perfectly**
 For Each s As String In ss
  If x.@id = s Then
   bool = True
   Exit For
  End If
 Next
 If bool = True Then
  'do nothing
 Else 'no attribute matched any of the attribute ids listed so remove the xelement
  rv.Add(x.@id)
 End If
Next


问:
How to get post values from first page to third page ,as post
i have 3 pages

Page1.php,page2.php,page3.php

In page1.php, i have some hidden values, for example 'name'

After the submission of page1.php, it will go to page2.

Then after some process in page2.php, it should need to automatically submit to page3.php(where page3.php is in another sever)

Finally,when i print the $_POST  variables in page3.php, i need to get the variable 'name'

答:
This seems straightforward to me, page one has a hidden value called name. Page 2 should retrieve the post $_POST['name'] and print it on page 2 as a hidden field. Once you post it to Page 3 you can retrieve it the same way $_POST['name'].

Realistically if the data is exactly the same and is being carried all the way to page 3, why do you even need it? Can you not just declare it on page 3?


问:
Kubernetes scaling pods by number of active connections
I have a kubernetes cluster that runs some legacy containers (windows containers) .
To simplify , let's say that the container can handle max 5 requests at a time something like

```<code>handleRequest(){
   requestLock(semaphore_Of_5)
   sleep(2s)
   return "result"
}
</code>```

So the cpu is not spiked . I need to scale based on nr of active connections

I can see from the documentation https://kubernetes.io/docs/concepts/services-networking/service/#proxy-mode-iptables

```<code>You can use Pod readiness probes to verify that backend Pods are working OK, so that kube-proxy in iptables mode only sees backends that test out as healthy. Doing this means you avoid having traffic sent via kube-proxy to a Pod that’s known to have failed.
</code>```

So there is a mechanism to make pods available for routing new requests but it is the livenessProbe that actually mark the pod as unhealthy and subject to restart policy. But my pods are just busy. They don't need restarting.

How can I increase the nr of pods in this case ?

答:
You can enable HPA for the deployment.

You can autoscale on the no of requests metrics and perform autoscaling on this metric.

https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough/#autoscaling-on-metrics-not-related-to-kubernetes-objects

I would also recommend to configure liveness probe failureThreshold and timeoutSeconds, check if it helps.

https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/#configure-probes

Body:
I have a kubernetes cluster that runs some legacy containers (windows containers) .
To simplify , let's say that the container can handle max 5 requests at a time something like

handleRequest(){
   requestLock(semaphore_Of_5)
   sleep(2s)
   return "result"
}

So the cpu is not spiked . I need to scale based on nr of active connections

I can see from the documentation https://kubernetes.io/docs/concepts/services-networking/service/#proxy-mode-iptables

You can use Pod readiness probes to verify that backend Pods are working OK, so that kube-proxy in iptables mode only sees backends that test out as healthy. Doing this means you avoid having traffic sent via kube-proxy to a Pod that’s known to have failed.

So there is a mechanism to make pods available for routing new requests but it is the livenessProbe that actually mark the pod as unhealthy and subject to restart policy. But my pods are just busy. They don't need restarting.

How can I increase the nr of pods in this case ?


问:
column-wise correlation between all pairs of columns of two data frame
Hi so I have created a function to check the correlation between 2 variables, anyone knows how can I create a new data frame from this?

```<code>In [1]:from scipy.stats import pearsonr
for colY in Y.columns:
    for colX in X.columns:
    #print('Pearson Correlation')
        corr, _ = pearsonr(numerical_cols_target[colX], numerical_cols_target[colY])
        alpha = 0.05
        print('Pearson Correlation', (alpha, corr))
        if corr <= alpha:
            print(colX +' and ' +colY+ ' two ariables are not correlated ')
        else:
            print(colX +' and ' +colY+ ' two variables are highly correlated ')
        print('\n')
    print('\n')
</code>```

here's a sample output from the correlation function:

```<code>Out [1]:
Pearson Correlation (0.05, -0.1620045985125294)
banana and orange are not correlated

Pearson Correlation (0.05, 0.2267582070839226)
apple and orange are highly correlated
```
```

答:
I would avoid using two for loops. Depending on the size of your dataset this will be very slow.

Pandas provides a correlation function with might come in hand here:

```<code>import pandas as pd

df = pd.DataFrame({'A': range(4), 'B': [2*i for i in range(4)]})
</code>```

using corr() will give you the pairwise correlations then and returns a new dataframe as well:

```<code>df.corr()
</code>```

For more infos you can check the manual: https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.corr.html

Body:
Hi so I have created a function to check the correlation between 2 variables, anyone knows how can I create a new data frame from this?

In [1]:from scipy.stats import pearsonr
for colY in Y.columns:
    for colX in X.columns:
    #print('Pearson Correlation')
        corr, _ = pearsonr(numerical_cols_target[colX], numerical_cols_target[colY])
        alpha = 0.05
        print('Pearson Correlation', (alpha, corr))
        if corr <= alpha:
            print(colX +' and ' +colY+ ' two ariables are not correlated ')
        else:
            print(colX +' and ' +colY+ ' two variables are highly correlated ')
        print('\n')
    print('\n')

here's a sample output from the correlation function:

Out [1]:
Pearson Correlation (0.05, -0.1620045985125294)
banana and orange are not correlated

Pearson Correlation (0.05, 0.2267582070839226)
apple and orange are highly correlated
```

Body:
I would avoid using two for loops. Depending on the size of your dataset this will be very slow.

Pandas provides a correlation function with might come in hand here:

import pandas as pd

df = pd.DataFrame({'A': range(4), 'B': [2*i for i in range(4)]})

using corr() will give you the pairwise correlations then and returns a new dataframe as well:

df.corr()

For more infos you can check the manual: https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.corr.html


问:
It is possible that flock returns false with LOCK_EX?
I found at the flock manual the following description:

By default, this function will block until the requested lock is
  acquired

Further below I found the following example code:

```<code><?php

$fp = fopen("/tmp/lock.txt", "r+");

if (flock($fp, LOCK_EX)) {  // acquire an exclusive lock
    ftruncate($fp, 0);      // truncate file
    fwrite($fp, "Write something here\n");
    fflush($fp);            // flush output before releasing the lock
    flock($fp, LOCK_UN);    // release the lock
} else {
    echo "Couldn't get the lock!";
}

fclose($fp);

?>
</code>```

But is there any case where the script would actually return "Couldn't get the lock!"? I thought it waits until the file lock.txt gets unlocked. If the file never gets unlocked, then the script waits forever, right?

Further, I found this answer explaining the difference between exclusive and shared lock on unix: https://stackoverflow.com/a/11837714/2311074
Do these 4 rules also apply to flock in PHP (for example "If one or more shared locks already exist, exclusive locks cannot be obtained")?

答:
On SunOS, if you opened the file in read mode, flock with LOCK_EX will always return false, even if the file is not locked. There is a comment to this effect on the PHP Manual flock page. The comment is down-voted to -2, but it is correct and bears repeating here:

flock on Solaris is slightly strange: it will fail if you try to get an exclusive lock on a file not opened for writing. That is, for reading files, you MUST use a shared lock. From the Solaris man page for flock: "Read permission is required on a file to obtain a shared lock, and write  permission is required to obtain an exclusive lock." In contrast, this is from the Linux man page for flock: "The mode used to open the file doesn’t matter to flock." So, beware...

Many flock examples show the file being opened in read mode, but I can confirm the above comment. If you use flock with LOCK_EX, it will return return false on SunOS if you opened the file in read mode. Since Linux doesn't care which mode you use, it seems to me that one should always open the file in write mode when using flock with LOCK_EX.

Body:
I found at the flock manual the following description:

By default, this function will block until the requested lock is
  acquired

Further below I found the following example code:

<?php

$fp = fopen("/tmp/lock.txt", "r+");

if (flock($fp, LOCK_EX)) {  // acquire an exclusive lock
    ftruncate($fp, 0);      // truncate file
    fwrite($fp, "Write something here\n");
    fflush($fp);            // flush output before releasing the lock
    flock($fp, LOCK_UN);    // release the lock
} else {
    echo "Couldn't get the lock!";
}

fclose($fp);

?>

But is there any case where the script would actually return "Couldn't get the lock!"? I thought it waits until the file lock.txt gets unlocked. If the file never gets unlocked, then the script waits forever, right?

Further, I found this answer explaining the difference between exclusive and shared lock on unix: https://stackoverflow.com/a/11837714/2311074
Do these 4 rules also apply to flock in PHP (for example "If one or more shared locks already exist, exclusive locks cannot be obtained")?


问:
Read .csv that contains commas
I have a .csv file that contains multiple columns with texts in it. These texts contain commas, which makes things messy when I try to read the file into Python.

When I tried:

```<code>import pandas as pd
directory = 'some directory'

dataset = pd.read_csv(directory)
</code>```

I got the following error:

```<code>ParserError: Error tokenizing data. C error: Expected 3 fields in line 42, saw 5
</code>```

After doing some research, I found the clevercsv package.

So, I ran:

```<code>import clevercsv as csv

dataset = csv.read_csv(directory)
</code>```

Running this, I got the error:

```<code>UnicodeDecodeError: 'charmap' codec can't decode byte 0x8f in position 4359705: character maps to <undefined>
</code>```

To overcome this, I tried:

```<code>dataset = csv.read_csv(directory, encoding="utf8")
</code>```

However, 10 hours later my computer was still working on reading it. So I expect that something went wrong there.

Furthermore, when I open the file in Excel, it does split cells well. Therefore, What I tried was to save the .csv file as a .xlsx and then save it as .csv in Python with an uncommon delimiter ('~'). However, when I save my .csv file as a .xlsx file, the size of the file gets smaller, which indicates that only a part of the file is saved and that is not what I want.

Lastly, I have tried the solutions given here and here. But neither seem to work for my problem.

Given that Excel reads in the file without problems, I do expect that it should be possible to read it into Python as well. Who can help me with this?

UPDATE:

When using dataset = pd.read_csv(directory, sep = ',', error_bad_lines=False)I manage to read in the .csv. But many lines are skipped. Is there a better way for this?

答:
panda should be work https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_csv.html

Dou you tried somthing like dataset = pd.read_csv(directory, sep = ',', header = None)

Regards

Title: Read .csv that contains commas

Body:
I have a .csv file that contains multiple columns with texts in it. These texts contain commas, which makes things messy when I try to read the file into Python.

When I tried:

import pandas as pd
directory = 'some directory'

dataset = pd.read_csv(directory)

I got the following error:

ParserError: Error tokenizing data. C error: Expected 3 fields in line 42, saw 5

After doing some research, I found the clevercsv package.

So, I ran:

import clevercsv as csv

dataset = csv.read_csv(directory)

Running this, I got the error:

UnicodeDecodeError: 'charmap' codec can't decode byte 0x8f in position 4359705: character maps to <undefined>

To overcome this, I tried:

dataset = csv.read_csv(directory, encoding="utf8")

However, 10 hours later my computer was still working on reading it. So I expect that something went wrong there.

Furthermore, when I open the file in Excel, it does split cells well. Therefore, What I tried was to save the .csv file as a .xlsx and then save it as .csv in Python with an uncommon delimiter ('~'). However, when I save my .csv file as a .xlsx file, the size of the file gets smaller, which indicates that only a part of the file is saved and that is not what I want.

Lastly, I have tried the solutions given here and here. But neither seem to work for my problem.

Given that Excel reads in the file without problems, I do expect that it should be possible to read it into Python as well. Who can help me with this?

UPDATE:

When using dataset = pd.read_csv(directory, sep = ',', error_bad_lines=False)I manage to read in the .csv. But many lines are skipped. Is there a better way for this?


问:
How to execute activate and deactivate methods of OSGI bundle in a single thread
I have an OSGI bundle of a following structure:

```<code>//...

public ScheduledExecutorService executor = Executors.newSingleThreadScheduledExecutor();

//...

@Activate
public void activate() {
   executor.submit(new Runnable() {
     @Override
     public void run() {
      //call 3 functions and log the data
     }
   }
}

@Deactivate
public void deactivate(){
  //call 2 other functions
}
</code>```

The executor in the activate method makes sure that 3 functions are called in a separate from all other bundles thread, because those functions actually implement some sophisticated Windows-message-loop, i.e. a while true loop, that's why, in order not to block other bundles, it is activated in a separate thread. Now what I've sadly noticed, that in order to run 2 functions in deactivate method I need to run them in the same thread, in which 3 functions in activate method were run. Simply speaking, I need to be sure, that activate and deactivate methods of my bundle run in the one same thread, but still to keep this bundle activation separated (in an own thread) from the other bundles.

My question is: how to implement this?

I am not a guru in concurrency in Java, I've tried simply to call this executor in the deactivate method too but I don't know how to do it with one Runnable task, since in deactivate I have only to call 2 functions and in activate only 3 functions and no other calls should be made.

UPD: sorry, I forgot to mention, that there is a routine in another bundle, which calls in certain situations context.getBundle(0).stop() in order to call a deactivation for all bundles. If I want just to add the same submit routine in the deactivate method as is in activate, then in such situation I could clearly see, that those 2 functions from deactivate method of my bundle in the submit's body were not called.

答:
Simply do another executor.submit in deactivate. As it is a single threaded executor it will make sure only one thread processes both.

The only question is how to shut down the executor reliably. Normally after deactivate a component should have closed all its resources.

Body:
I have an OSGI bundle of a following structure:

//...

public ScheduledExecutorService executor = Executors.newSingleThreadScheduledExecutor();

//...

@Activate
public void activate() {
   executor.submit(new Runnable() {
     @Override
     public void run() {
      //call 3 functions and log the data
     }
   }
}

@Deactivate
public void deactivate(){
  //call 2 other functions
}

The executor in the activate method makes sure that 3 functions are called in a separate from all other bundles thread, because those functions actually implement some sophisticated Windows-message-loop, i.e. a while true loop, that's why, in order not to block other bundles, it is activated in a separate thread. Now what I've sadly noticed, that in order to run 2 functions in deactivate method I need to run them in the same thread, in which 3 functions in activate method were run. Simply speaking, I need to be sure, that activate and deactivate methods of my bundle run in the one same thread, but still to keep this bundle activation separated (in an own thread) from the other bundles.

My question is: how to implement this?

I am not a guru in concurrency in Java, I've tried simply to call this executor in the deactivate method too but I don't know how to do it with one Runnable task, since in deactivate I have only to call 2 functions and in activate only 3 functions and no other calls should be made.

UPD: sorry, I forgot to mention, that there is a routine in another bundle, which calls in certain situations context.getBundle(0).stop() in order to call a deactivation for all bundles. If I want just to add the same submit routine in the deactivate method as is in activate, then in such situation I could clearly see, that those 2 functions from deactivate method of my bundle in the submit's body were not called.