问:
PostgreSQL GROUP BY different from MySQL?
I've been migrating some of my MySQL queries to PostgreSQL to use Heroku. Most of my queries work fine, but I keep having a similar recurring error when I use group by:

ERROR:  column "XYZ" must appear in the GROUP BY clause or be used in
  an aggregate function

Could someone tell me what I'm doing wrong?

MySQL which works 100%:

SELECT `availables`.*
FROM `availables`
INNER JOIN `rooms` ON `rooms`.id = `availables`.room_id
WHERE (rooms.hotel_id = 5056 AND availables.bookdate BETWEEN '2009-11-22' AND '2009-11-24')
GROUP BY availables.bookdate
ORDER BY availables.updated_at
```

<p><br>
PostgreSQL error:</p>

<blockquote>
  <p>ActiveRecord::StatementInvalid: PGError: ERROR:  column
  "availables.id" must appear in the GROUP BY clause or be used in an
  aggregate function:<br>
  SELECT "availables".* FROM "availables"   INNER
  JOIN "rooms" ON "rooms".id = "availables".room_id  WHERE
  (rooms.hotel_id = 5056 AND availables.bookdate BETWEEN E'2009-10-21'
  AND E'2009-10-23')  GROUP BY availables.bookdate ORDER BY
  availables.updated_at</p>
</blockquote>

<p><br>
Ruby code generating the SQL:</p>

<pre class="lang-rb prettyprint-override"><code>expiration = Available.find(:all,
    :joins => [ :room ],
    :conditions => [ "rooms.hotel_id = ? AND availables.bookdate BETWEEN ? AND ?", hostel_id, date.to_s, (date+days-1).to_s ],
    :group => 'availables.bookdate',
    :order => 'availables.updated_at')
</code>```

Expected Output (from working MySQL query):

```
+-----+-------+-------+------------+---------+---------------+---------------+
| id  | price | spots | bookdate   | room_id | created_at    | updated_at    |
+-----+-------+-------+------------+---------+---------------+---------------+
| 414 | 38.0  | 1     | 2009-11-22 | 1762    | 2009-11-20... | 2009-11-20... |
| 415 | 38.0  | 1     | 2009-11-23 | 1762    | 2009-11-20... | 2009-11-20... |
| 416 | 38.0  | 2     | 2009-11-24 | 1762    | 2009-11-20... | 2009-11-20... |
+-----+-------+-------+------------+---------+---------------+---------------+
3 rows in set
```

答: According to MySQL's "Debuking GROUP BY Myths" http://dev.mysql.com/tech-resources/articles/debunking-group-by-myths.html. SQL (2003 version of the standard) doesn't requires columns referenced in the SELECT list of a query to also appear in the GROUP BY clause.</p>

Body:
I've been migrating some of my MySQL queries to PostgreSQL to use Heroku. Most of my queries work fine, but I keep having a similar recurring error when I use group by:

ERROR:  column "XYZ" must appear in the GROUP BY clause or be used in
  an aggregate function

Could someone tell me what I'm doing wrong?

MySQL which works 100%:

SELECT `availables`.*
FROM `availables`
INNER JOIN `rooms` ON `rooms`.id = `availables`.room_id
WHERE (rooms.hotel_id = 5056 AND availables.bookdate BETWEEN '2009-11-22' AND '2009-11-24')
GROUP BY availables.bookdate
ORDER BY availables.updated_at

PostgreSQL error:

ActiveRecord::StatementInvalid: PGError: ERROR:  column
  "availables.id" must appear in the GROUP BY clause or be used in an
  aggregate function:

  SELECT "availables".* FROM "availables"   INNER
  JOIN "rooms" ON "rooms".id = "availables".room_id  WHERE
  (rooms.hotel_id = 5056 AND availables.bookdate BETWEEN E'2009-10-21'
  AND E'2009-10-23')  GROUP BY availables.bookdate ORDER BY
  availables.updated_at

Ruby code generating the SQL:

expiration = Available.find(:all,
    :joins => [ :room ],
    :conditions => [ "rooms.hotel_id = ? AND availables.bookdate BETWEEN ? AND ?", hostel_id, date.to_s, (date+days-1).to_s ],
    :group => 'availables.bookdate',
    :order => 'availables.updated_at')

Expected Output (from working MySQL query):

+-----+-------+-------+------------+---------+---------------+---------------+
| id  | price | spots | bookdate   | room_id | created_at    | updated_at    |
+-----+-------+-------+------------+---------+---------------+---------------+
| 414 | 38.0  | 1     | 2009-11-22 | 1762    | 2009-11-20... | 2009-11-20... |
| 415 | 38.0  | 1     | 2009-11-23 | 1762    | 2009-11-20... | 2009-11-20... |
| 416 | 38.0  | 2     | 2009-11-24 | 1762    | 2009-11-20... | 2009-11-20... |
+-----+-------+-------+------------+---------+---------------+---------------+
3 rows in set


来源: stackexchange

问:
watch not getting all in Vue
```<code><template>
 <v-form :model='agency'>
  <v-layout row wrap>
    <v-flex xs12 sm12 lg12 >
      <v-layout row wrap>
       <v-flex xs12 md12 class="add-col-padding-right">
        <v-radio-group
         v-model="agency.location"
         :mandatory="true"
         label="Please select one of the options below">
           <v-radio label="NYC" value="nyc"></v-radio>
           <v-radio label="SAN JOSE" value="san_jose"></v-radio>
        </v-radio-group>
       </v-flex>
      </v-layout>
    </v-flex>
  </v-layout>
 </v-form>
</template>

<script>

  export default {

   data: function () {
    return {
      agency: {
        id: '',
        name: '',
        location: '',
        contacts: {
          name: '',
          number: '',
          address: ''
        }
      },
      final_location_contact: {
       name: '',
       number: '',
       address: ''
      }
    }
   }
   watch: {
    agency: {
      handler(val){
       if (this.agency.location === "nyc") {
        this.final_location_contact = this.agency.contacts;
       }
    }
   },
   created: function() {
    this.fetchAgency();
   },
   method: {
    fetchAgency() {
      this.$axios.get('/agency.json')
      .then(response => {
        if (Object.keys(response.data).length > 0) {
          this.agency = response.data;
        }
      }).
      then(() => {
      });
    },
   }
  }
</script>
</code>```

The issue I am facing here is when the page loads, and the fetchAgency method runs and I am able to get the agency data. When I select first option from the radio button, then I am able to assign the value agency.contacts values to final_location_contact. But the issue arrises when I first select the San Jose Radio button option and after that I select nyc radio button then I am  not able to assign the value agency.contacts values to final_location_contact as I checked using  debugger the value of agency.contacts is empty. But it not empty if I select nyc the first time. Any Idea where could be the issue?

```<code>'agency.location': {
  handler (newValue, oldValue) {
    if (newValue === "nyc") {
      this.final_location_contact = this.agency.contacts;
    }
    else {
      this.final_location_contact.name: '',
      this.final_location_contact.number: '',
      this.final_location_contact.address: ''
    }
  }
 }
</code>```

答:
As Jacob said, the watcher is not being triggered because agency itself was not changed.

In the comment i've suggested to use deep watcher, but that's not the best approach. You can listen specifically to changes made on 'agency.location' this way:

watch: {
  'agency.location': {
    handler (newValue, oldValue) {
      // this will be triggered every time `agency.location` is changed
    }
  }
}
```

<p>This is better than deep watching, which would look like this</p>

<pre class="lang-js prettyprint-override"><code>watch: {
  deep: true,
  agency (newValue, oldValue) {
    // logic
  }
}
</code>```

because deep watcher will be triggered any time either agency or any of it's properties will change.

Here is fiddle which demonstrates the difference and how it's implemented (make sure to open the console).

If the watcher not being triggered is the issue here, then no changes to the model / template is necessary.

ContentLicense: CC BY-SA 4.0

Title: watch not getting all in Vue

Body:
<template>
 <v-form :model='agency'>
  <v-layout row wrap>
    <v-flex xs12 sm12 lg12 >
      <v-layout row wrap>
       <v-flex xs12 md12 class="add-col-padding-right">
        <v-radio-group
         v-model="agency.location"
         :mandatory="true"
         label="Please select one of the options below">
           <v-radio label="NYC" value="nyc"></v-radio>
           <v-radio label="SAN JOSE" value="san_jose"></v-radio>
        </v-radio-group>
       </v-flex>
      </v-layout>
    </v-flex>
  </v-layout>
 </v-form>
</template>

The issue I am facing here is when the page loads, and the fetchAgency method runs and I am able to get the agency data. When I select first option from the radio button, then I am able to assign the value agency.contacts values to final_location_contact. But the issue arrises when I first select the San Jose Radio button option and after that I select nyc radio button then I am  not able to assign the value agency.contacts values to final_location_contact as I checked using  debugger the value of agency.contacts is empty. But it not empty if I select nyc the first time. Any Idea where could be the issue?

'agency.location': {
  handler (newValue, oldValue) {
    if (newValue === "nyc") {
      this.final_location_contact = this.agency.contacts;
    }
    else {
      this.final_location_contact.name: '',
      this.final_location_contact.number: '',
      this.final_location_contact.address: ''
    }
  }
 }

Body:
As Jacob said, the watcher is not being triggered because agency itself was not changed.

In the comment i've suggested to use deep watcher, but that's not the best approach. You can listen specifically to changes made on 'agency.location' this way:

watch: {
  'agency.location': {
    handler (newValue, oldValue) {
      // this will be triggered every time `agency.location` is changed
    }
  }
}

This is better than deep watching, which would look like this

watch: {
  deep: true,
  agency (newValue, oldValue) {
    // logic
  }
}

because deep watcher will be triggered any time either agency or any of it's properties will change.

Here is fiddle which demonstrates the difference and how it's implemented (make sure to open the console).

If the watcher not being triggered is the issue here, then no changes to the model / template is necessary.


问:
Converting eval() to new Function()
Given an arbitrary eval(), is there a way to convert it to use new Function(), without breaking compatibility with any input (except for the change in scope)?

The only issue I can think of is that eval allows multiple statements, and returns the result of the last one, without having to add an explicit return. I cannot think of a way to emulate this with new Function():

```<code>let code = '4; 5'
> eval(code);
5
> new Function(code)();
undefined
> new Function('return ' + code)();
4
> new Function('return (' + code + ')')();
SyntaxError: Unexpected token ;
</code>```

Is there something I am missing or is eval() incompatible with new Function() in this case?

答:
new Function() can indeed handle multiple statements...  but it can only operate on local vars and vars that are passed into it.. which means you can only return a single result.

```<code>var outerVar1 = 2;
var outerVar2 = 3;
var newFunc = new Function('a1','a2', 'var result = a1 * a2; return(result);');
console.log('Result = ' + newFunc(outerVar1, outerVar2));
</code>```

The above works... the below doesn't... at least in most engines...

```<code>var outerVar1 = 2;
var outerVar2 = 3;
var newFunc = new Function('var result = outerVar1 * outerVar2; return(result);');
console.log('Result = ' + newFunc());
</code>```

Title: Converting eval() to new Function()

Body:
Given an arbitrary eval(), is there a way to convert it to use new Function(), without breaking compatibility with any input (except for the change in scope)?

The only issue I can think of is that eval allows multiple statements, and returns the result of the last one, without having to add an explicit return. I cannot think of a way to emulate this with new Function():

let code = '4; 5'
> eval(code);
5
> new Function(code)();
undefined
> new Function('return ' + code)();
4
> new Function('return (' + code + ')')();
SyntaxError: Unexpected token ;

Is there something I am missing or is eval() incompatible with new Function() in this case?

Body:
new Function() can indeed handle multiple statements...  but it can only operate on local vars and vars that are passed into it.. which means you can only return a single result.

var outerVar1 = 2;
var outerVar2 = 3;
var newFunc = new Function('a1','a2', 'var result = a1 * a2; return(result);');
console.log('Result = ' + newFunc(outerVar1, outerVar2));

The above works... the below doesn't... at least in most engines...

var outerVar1 = 2;
var outerVar2 = 3;
var newFunc = new Function('var result = outerVar1 * outerVar2; return(result);');
console.log('Result = ' + newFunc());


问:
(OS 10048)Only one usage of each socket address (protocol/network address/port) is normally permitted. Windows apache
When trying to execute httpd.exe on my windows server, I get this error:

(OS 10048)Only one usage of each socket address (protocol/network address/port) is normally permitted.  : AH00072: make_sock: could not bind to address (myIpAddress):80

AH00451: no listening sockets available, shutting down

AH00015: Unable to open logs

This only happens when in the httpd.conf I set:

Listening (myIPAddress):80

but the I don't get this error when it's set to:

Listening 80

答:
I only stopped Skype! this worked...no changes were required to do in Apache config file. Skype is somehow affecting the Apache on port 80
https://www.apachelounge.com/viewtopic.php?t=767
Thanks</p>


问:
dcast() - adding a column that doesnt exist in R
I have encountered a problem, that i am sure has a easy solution, but i cannot find it. I basically summarise my table to get the sum of an value per level of a factor variable:

```<code> NOdependants <- unique(claimsMonthly[policyID == policy, .(exposure = sum(exposure)),
                                        by = c("productID", "Year", "product", "QualityCheck", "dependant")][order(Year)])

   productID       Year product QualityCheck dependant exposure
1:           1      2016  ELI18            0  EMPLOYEE 17.041096
2:           1      2016  ELI18            0    SPOUSE 40.484932
3:           1      2016  ELI18            0     CHILD 5.164384
</code>```

and then i do the following:

```<code>NOdependants <- dcast(NOdependants,  productID + Year ~ dependant, value.var = "exposure", fill = 0, drop = FALSE, fun.aggregate = sum)
setnames(NOdependants, c("CHILD", "EMPLOYEE", "SPOUSE"), c("childno", "employeeno", "spouseno"), skip_absent=TRUE)

> NOdependants
   productRank startYear  childno employeeno spouseno
1:           1      2016 5.164384   17.041096 41.484932
</code>```

This is all good so far. The problem is when a product doesnt have any data on one of the factor in dependant. Lets say there are no children:

```<code> NOdependants <- unique(claimsMonthly[policyID == policy, .(exposure = sum(exposure)),
                                        by = c("productID", "Year", "product", "QualityCheck", "dependant")][order(Year)])

   productID       Year product QualityCheck dependant exposure
1:           1      2016  ELI18            0  EMPLOYEE 17.041096
2:           1      2016  ELI18            0    SPOUSE 40.484932
</code>```

Then my dcast does the following:

```<code>> NOdependants
   productRank startYear  employeeno spouseno
1:           1      2016  17.041096 41.484932
</code>```

and this is a problem for me, I need to have all three columns. SO what i need is to artifically create an additional columns in case a factor level is without data (like child in here), so i would get something this:

```<code>> NOdependants
   productRank startYear  childno employeeno spouseno
1:           1      2016       0   17.041096 41.484932
</code>```

for now i have create a workround, where i first create an empty data.table and then use rbindlist  with fill=0 to merge theses, but there must be some easier solution.

Any ideas?

Note: i am working with a lot of data and this operation is part of a loop which will be repeated around 80 times or so, so ideally something efficient is possible.

SIMPLIFIED EXAMPLE WITH DATA:

  #

```<code>> claimsMonthly <- data.table(productID = c(rep(1,6), rep(2,3), rep(3,2)),
+                      Year = c(rep(2015,9), 2016, 2016),
+                      product = c(rep("ELI18",6), rep("JCI22",3), rep("ZDP01",2)),
+                      dependant = c(rep(c("EMPLOYEE", "SPOUSE", "CHILD"), 3),"EMPLOYEE", "SPOUSE"),
+                      QualityCheck = c(rep(0,11)),
+                      exposure = c(abs(rnorm(11))))
>
> productIDs <- unique(claimsMonthly$productID)
> for(prod in productIDs){
+
+   NOdependants <- unique(claimsMonthly[ productID == prod, .(exposure = sum(exposure)),
+                                         by = c("productID", "Year", "product", "QualityCheck", "dependant")][order(Year)])
+
+   NOdependants <- dcast(NOdependants,  productID + Year ~ dependant, value.var = "exposure", fill = 0, drop = FALSE, fun.aggregate = sum)
+   setnames(NOdependants, c("CHILD", "EMPLOYEE", "SPOUSE"), c("childno", "employeeno", "spouseno"), skip_absent=TRUE)
+
+   NOdependants[order(childno)]
+
+ }
Error in .checkTypos(e, names_x) :
  Object 'childno' not found amongst productID, Year, employeeno, spouseno
</code>```

答:
Your use of 'unique' outside of data.table brackets may perplex data.table here. Please See: https://www.rdocumentation.org/packages/data.table/versions/1.12.8/topics/duplicated

I am wondering if your code could be simpler and achieve your result just as well. Some of the  beauty of rdata.table is its ability to eliminate the need for loops and control structures. Using your sample data for 'claimsMonthly':

```<code>claimsMonthly[, .(exposure = sum(exposure)),
.(productID,Year,product,QualityCheck,dependant)][
,dcast(.SD, productID + Year ~ dependant,
value.var = "exposure", drop = FALSE, fun.aggregate = sum)][
         CHILD == 0 &
         EMPLOYEE == 0 &
         SPOUSE == 0,.(productID,Year,CHILD,EMPLOYEE,SPOUSE)]

       productID Year CHILD EMPLOYEE SPOUSE
    1:         1 2016     0        0      0
    2:         2 2016     0        0      0
    3:         3 2015     0        0      0
</code>```

Body:
I have encountered a problem, that i am sure has a easy solution, but i cannot find it. I basically summarise my table to get the sum of an value per level of a factor variable:

 NOdependants <- unique(claimsMonthly[policyID == policy, .(exposure = sum(exposure)),
                                        by = c("productID", "Year", "product", "QualityCheck", "dependant")][order(Year)])

   productID       Year product QualityCheck dependant exposure
1:           1      2016  ELI18            0  EMPLOYEE 17.041096
2:           1      2016  ELI18            0    SPOUSE 40.484932
3:           1      2016  ELI18            0     CHILD 5.164384

and then i do the following:

NOdependants <- dcast(NOdependants,  productID + Year ~ dependant, value.var = "exposure", fill = 0, drop = FALSE, fun.aggregate = sum)
setnames(NOdependants, c("CHILD", "EMPLOYEE", "SPOUSE"), c("childno", "employeeno", "spouseno"), skip_absent=TRUE)

> NOdependants
   productRank startYear  childno employeeno spouseno
1:           1      2016 5.164384   17.041096 41.484932

This is all good so far. The problem is when a product doesnt have any data on one of the factor in dependant. Lets say there are no children:

 NOdependants <- unique(claimsMonthly[policyID == policy, .(exposure = sum(exposure)),
                                        by = c("productID", "Year", "product", "QualityCheck", "dependant")][order(Year)])

   productID       Year product QualityCheck dependant exposure
1:           1      2016  ELI18            0  EMPLOYEE 17.041096
2:           1      2016  ELI18            0    SPOUSE 40.484932

Then my dcast does the following:

> NOdependants
   productRank startYear  employeeno spouseno
1:           1      2016  17.041096 41.484932

and this is a problem for me, I need to have all three columns. SO what i need is to artifically create an additional columns in case a factor level is without data (like child in here), so i would get something this:

> NOdependants
   productRank startYear  childno employeeno spouseno
1:           1      2016       0   17.041096 41.484932

for now i have create a workround, where i first create an empty data.table and then use rbindlist  with fill=0 to merge theses, but there must be some easier solution.

Any ideas?

Note: i am working with a lot of data and this operation is part of a loop which will be repeated around 80 times or so, so ideally something efficient is possible.

SIMPLIFIED EXAMPLE WITH DATA:

  #

> claimsMonthly <- data.table(productID = c(rep(1,6), rep(2,3), rep(3,2)),
+                      Year = c(rep(2015,9), 2016, 2016),
+                      product = c(rep("ELI18",6), rep("JCI22",3), rep("ZDP01",2)),
+                      dependant = c(rep(c("EMPLOYEE", "SPOUSE", "CHILD"), 3),"EMPLOYEE", "SPOUSE"),
+                      QualityCheck = c(rep(0,11)),
+                      exposure = c(abs(rnorm(11))))
>
> productIDs <- unique(claimsMonthly$productID)
> for(prod in productIDs){
+
+   NOdependants <- unique(claimsMonthly[ productID == prod, .(exposure = sum(exposure)),
+                                         by = c("productID", "Year", "product", "QualityCheck", "dependant")][order(Year)])
+
+   NOdependants <- dcast(NOdependants,  productID + Year ~ dependant, value.var = "exposure", fill = 0, drop = FALSE, fun.aggregate = sum)
+   setnames(NOdependants, c("CHILD", "EMPLOYEE", "SPOUSE"), c("childno", "employeeno", "spouseno"), skip_absent=TRUE)
+
+   NOdependants[order(childno)]
+
+ }
Error in .checkTypos(e, names_x) :
  Object 'childno' not found amongst productID, Year, employeeno, spouseno

Body:
Your use of 'unique' outside of data.table brackets may perplex data.table here. Please See: https://www.rdocumentation.org/packages/data.table/versions/1.12.8/topics/duplicated

I am wondering if your code could be simpler and achieve your result just as well. Some of the  beauty of rdata.table is its ability to eliminate the need for loops and control structures. Using your sample data for 'claimsMonthly':

claimsMonthly[, .(exposure = sum(exposure)),
.(productID,Year,product,QualityCheck,dependant)][
,dcast(.SD, productID + Year ~ dependant,
value.var = "exposure", drop = FALSE, fun.aggregate = sum)][
         CHILD == 0 &
         EMPLOYEE == 0 &
         SPOUSE == 0,.(productID,Year,CHILD,EMPLOYEE,SPOUSE)]

       productID Year CHILD EMPLOYEE SPOUSE
    1:         1 2016     0        0      0
    2:         2 2016     0        0      0
    3:         3 2015     0        0      0


问:
Camera movement in a 2d game
I really wanna know how to make 2d camera movement in html and javascript, so i wrote this code which is just a character and a block so i can test if the camera movement is working. This is my code:

var c = document.getElementById("screen");
var ctx = c.getContext("2d");
var rightPressed;
var leftPressed;
var x = (c.width - 75) / 2;
var speed = 10;

function block() {
  ctx.fillStyle = "#fff";
  ctx.fillRect(390, 225, 80, 25);
}

function character() {
  ctx.fillStyle = "#00f"
  ctx.fillRect(x, 250, 50, 50);
}

function keyUpHandler(event) {
  if (event.keyCode == 37) {
    leftPressed = false;
  } else if (event.keyCode == 39) {
    rightPressed = false;
  }
}

function keyDownHandler(event) {
  if (event.keyCode == 37) {
    leftPressed = true;
  } else if (event.keyCode == 39) {
    rightPressed = true;
  }
}

document.addEventListener("keyup", keyUpHandler, false);
document.addEventListener("keydown", keyDownHandler, false);

function draw() {
  ctx.clearRect(0, 0, c.width, c.height);
  block();
  character();

  if (rightPressed && (x + 50) < c.width) {
    x += speed;
  } else if (leftPressed && x > 0) {
    x -= speed;
  }

  requestAnimationFrame(draw);
}

requestAnimationFrame(draw);```
<pre class="snippet-code-css lang-css prettyprint-override"><code>#screen {
  display: block;
  margin: auto;
  background-color: #0f0;
  border: 2px solid black;
}

#headline {
  text-align: center;
}</code>```

2d Camera movement

<canvas id="screen" width="500" height="300"></canvas>```

Is there anyone who knows how to do this then please tell me, also it would be nice if you could put it into this code :)

答:
I created two classes, a Game and a Ball. The balls will move when a player presses the left or right key.

The balls know their own position and speed, the game does not care.

Just tell the ball to update according to the game's current state.

I adapted a good portion of code from here:

"Quick Tip: How to Make a Game Loop in JavaScript"

I added ES6 classes, vector support and dynamic game element support.

Demo

const canvasTxt = window.canvasTxt.default;

const KeyCodeMap = {
  37: 'left',
  38: 'up',
  39: 'right',
  40: 'down',
  65: 'left',
  68: 'right',
  83: 'down',
  87: 'up'
};

Object.assign(canvasTxt, {
  align: 'left',
  vAlign: 'top',
  font: 'monospace',
  fontSize: 24
});

const main = () => {
  Game.DEBUG = true; // Enable global DEBUG mode.

  const ctx = document.getElementById('screen').getContext('2d');
  const game = new Game(ctx);

  game.gameElements.push(...[
    new Ball({
      speed: new Victor(10, 0),
      position: new Victor(game.getWidth() / 2, game.getHeight() / 2),
      color: 'red',
      size: 20
    }),
    new Ball({
      speed: new Victor(20, 0),
      position: new Victor(game.getWidth() / 2, game.getHeight() / 4),
      color: 'green',
      size: 30
    }),
    new Ball({
      speed: new Victor(20, 50),
      position: new Victor(100, 140),
      color: 'cyan',
      size: 25
    }),
  ]);

  game.redraw();
};

class Ball {
  constructor(options) {
    let opts = Object.assign({}, Ball.defaultOptions, options);
    this.position = opts.position;
    this.speed = opts.speed;
    this.size = opts.size;
    this.color = opts.color;
  }
  update(container, state) {
    if (state.pressedKeys.left && !state.pressedKeys.right) {
      this.position.subtract(this.speed);
    }
    if (state.pressedKeys.right && !state.pressedKeys.left) {
      this.position.add(this.speed);
    }
    this.checkBounds(container); // Make sure object is in-bounds...
  }
  checkBounds(container) {
    if (this.position.x > container.width) {
      this.position.x = 0;
    }
    if (this.position.x < 0) {
      this.position.x = container.width;
    }
    if (this.position.y > container.height) {
      this.position.y = 0;
    }
    if (this.position.y < 0) {
      this.position.y = container.height;
    }
  }
  draw(ctx) {
    ctx.save();
    ctx.beginPath();
    ctx.arc(this.position.x, this.position.y, this.size / 2, 0, 2 * Math.PI, false);
    ctx.fillStyle = this.color;
    ctx.fill();
    //ctx.lineWidth = 1;
    //ctx.strokeStyle = '#000000';
    ctx.stroke();
    ctx.restore();
  }
}
Ball.defaultOptions = {
  position: new Victor(0, 0),
  speed: new Victor(0, 0),
  size: 1,
  color: '#FFFFFF'
};

class Game {
  constructor(ctx) {
    this.ctx = ctx;
    this.gameElements = [];
    this.resize();
    this.state = {
      pressedKeys: {} // Capture key pressed
    };
    this.__lastRender = 0;
    this.__animationId = null;

    window.onresize = (e) => this.resize(e);
    document.addEventListener('click', (e) => this.clickHandler(e), false);
    document.addEventListener('keyup', (e) => this.keyUpHandler(e), false);
    document.addEventListener('keydown', (e) => this.keyDownHandler(e), false);
  }

  clickHandler(e) {
    this.isRunning() ? this.pause() : this.run(); // Pause, if running
  }

  keyUpHandler(event) {
    let key = KeyCodeMap[event.keyCode];
    delete this.state.pressedKeys[key];
  }

  keyDownHandler(event) {
    let key = KeyCodeMap[event.keyCode];
    if (key) this.state.pressedKeys[key] = true;
  }

  getWidth() {
    return this.ctx.canvas.width;
  }

  getHeight() {
    return this.ctx.canvas.height;
  }

  resize(event) {
    this.ctx.canvas.width = window.innerWidth * 2
    this.ctx.canvas.height = window.innerHeight * 2
  }

  update(progress) {
    this.gameElements.forEach(gameElement => gameElement.update(this.ctx.canvas, this.state));
  }

  redraw() {
    this.ctx.clearRect(0, 0, this.getWidth(), this.getHeight());
    this.gameElements.forEach(gameElement => gameElement.draw(this.ctx));

    if (Game.DEBUG) {
      this.__renderDebugText(this.ctx);
    }
  }

  /* @private */ __renderDebugText(ctx) {
    ctx.save();
    let text = 'DEBUG OUTPUT:\n\n' + JSON.stringify(Object.assign({
      running: this.isRunning()
    }, this.state), null, 2);
    let offset = { x: 16, y: 16 };
    let bounds = { width: 280,  height: 320 };
    ctx.fillStyle = 'rgba(16, 16, 16, 0.8)';
    ctx.fillRect(offset.x, offset.y, bounds.width, bounds.height);
    ctx.lineWidth = 1;
    ctx.strokeStyle = '#333';
    ctx.strokeRect(offset.x, offset.y, bounds.width, bounds.height);
    ctx.fillStyle = '#FFF';
    canvasTxt.drawText(ctx, text, offset.x + 4, offset.y + 4, bounds.width, bounds.height);
    ctx.restore();
  }

  loop(timestamp) {
    this.update(timestamp - this.__lastRender);
    this.redraw();
    this.__lastRender = timestamp;
    this.__animationId = window.requestAnimationFrame((ts) => this.loop(ts));
  }

  isRunning() {
    return this.__animationId != null;
  }

  run() {
    this.__animationId = window.requestAnimationFrame((ts) => this.loop(ts));
  }

  pause() {
    cancelAnimationFrame(this.__animationId);
    this.__animationId = null;
    this.redraw();
  }
}
Game.DEBUG = false; // Default is off

main();```
<pre class="snippet-code-css lang-css prettyprint-override"><code>html,
body {
  margin: 0;
  padding: 0;
}

canvas {
  background: #000;
  height: 100%;
  width: 100%;
  display: block
}</code>```

<canvas id="screen" />```

Title: Camera movement in a 2d game

Body:
I really wanna know how to make 2d camera movement in html and javascript, so i wrote this code which is just a character and a block so i can test if the camera movement is working. This is my code:

var c = document.getElementById("screen");
var ctx = c.getContext("2d");
var rightPressed;
var leftPressed;
var x = (c.width - 75) / 2;
var speed = 10;

function block() {
  ctx.fillStyle = "#fff";
  ctx.fillRect(390, 225, 80, 25);
}

function character() {
  ctx.fillStyle = "#00f"
  ctx.fillRect(x, 250, 50, 50);
}

function keyUpHandler(event) {
  if (event.keyCode == 37) {
    leftPressed = false;
  } else if (event.keyCode == 39) {
    rightPressed = false;
  }
}

function keyDownHandler(event) {
  if (event.keyCode == 37) {
    leftPressed = true;
  } else if (event.keyCode == 39) {
    rightPressed = true;
  }
}

document.addEventListener("keyup", keyUpHandler, false);
document.addEventListener("keydown", keyDownHandler, false);

function draw() {
  ctx.clearRect(0, 0, c.width, c.height);
  block();
  character();

  if (rightPressed && (x + 50) < c.width) {
    x += speed;
  } else if (leftPressed && x > 0) {
    x -= speed;
  }

  requestAnimationFrame(draw);
}

requestAnimationFrame(draw);

#screen {
  display: block;
  margin: auto;
  background-color: #0f0;
  border: 2px solid black;
}

#headline {
  text-align: center;
}

2d Camera movement

<canvas id="screen" width="500" height="300"></canvas>

Is there anyone who knows how to do this then please tell me, also it would be nice if you could put it into this code :)

Body:
I created two classes, a Game and a Ball. The balls will move when a player presses the left or right key.

The balls know their own position and speed, the game does not care.

Just tell the ball to update according to the game's current state.

I adapted a good portion of code from here:

"Quick Tip: How to Make a Game Loop in JavaScript"

I added ES6 classes, vector support and dynamic game element support.

Demo

const canvasTxt = window.canvasTxt.default;

const KeyCodeMap = {
  37: 'left',
  38: 'up',
  39: 'right',
  40: 'down',
  65: 'left',
  68: 'right',
  83: 'down',
  87: 'up'
};

Object.assign(canvasTxt, {
  align: 'left',
  vAlign: 'top',
  font: 'monospace',
  fontSize: 24
});

const main = () => {
  Game.DEBUG = true; // Enable global DEBUG mode.

  const ctx = document.getElementById('screen').getContext('2d');
  const game = new Game(ctx);

  game.gameElements.push(...[
    new Ball({
      speed: new Victor(10, 0),
      position: new Victor(game.getWidth() / 2, game.getHeight() / 2),
      color: 'red',
      size: 20
    }),
    new Ball({
      speed: new Victor(20, 0),
      position: new Victor(game.getWidth() / 2, game.getHeight() / 4),
      color: 'green',
      size: 30
    }),
    new Ball({
      speed: new Victor(20, 50),
      position: new Victor(100, 140),
      color: 'cyan',
      size: 25
    }),
  ]);

  game.redraw();
};

class Ball {
  constructor(options) {
    let opts = Object.assign({}, Ball.defaultOptions, options);
    this.position = opts.position;
    this.speed = opts.speed;
    this.size = opts.size;
    this.color = opts.color;
  }
  update(container, state) {
    if (state.pressedKeys.left && !state.pressedKeys.right) {
      this.position.subtract(this.speed);
    }
    if (state.pressedKeys.right && !state.pressedKeys.left) {
      this.position.add(this.speed);
    }
    this.checkBounds(container); // Make sure object is in-bounds...
  }
  checkBounds(container) {
    if (this.position.x > container.width) {
      this.position.x = 0;
    }
    if (this.position.x < 0) {
      this.position.x = container.width;
    }
    if (this.position.y > container.height) {
      this.position.y = 0;
    }
    if (this.position.y < 0) {
      this.position.y = container.height;
    }
  }
  draw(ctx) {
    ctx.save();
    ctx.beginPath();
    ctx.arc(this.position.x, this.position.y, this.size / 2, 0, 2 * Math.PI, false);
    ctx.fillStyle = this.color;
    ctx.fill();
    //ctx.lineWidth = 1;
    //ctx.strokeStyle = '#000000';
    ctx.stroke();
    ctx.restore();
  }
}
Ball.defaultOptions = {
  position: new Victor(0, 0),
  speed: new Victor(0, 0),
  size: 1,
  color: '#FFFFFF'
};

class Game {
  constructor(ctx) {
    this.ctx = ctx;
    this.gameElements = [];
    this.resize();
    this.state = {
      pressedKeys: {} // Capture key pressed
    };
    this.__lastRender = 0;
    this.__animationId = null;

    window.onresize = (e) => this.resize(e);
    document.addEventListener('click', (e) => this.clickHandler(e), false);
    document.addEventListener('keyup', (e) => this.keyUpHandler(e), false);
    document.addEventListener('keydown', (e) => this.keyDownHandler(e), false);
  }

  clickHandler(e) {
    this.isRunning() ? this.pause() : this.run(); // Pause, if running
  }

  keyUpHandler(event) {
    let key = KeyCodeMap[event.keyCode];
    delete this.state.pressedKeys[key];
  }

  keyDownHandler(event) {
    let key = KeyCodeMap[event.keyCode];
    if (key) this.state.pressedKeys[key] = true;
  }

  getWidth() {
    return this.ctx.canvas.width;
  }

  getHeight() {
    return this.ctx.canvas.height;
  }

  resize(event) {
    this.ctx.canvas.width = window.innerWidth * 2
    this.ctx.canvas.height = window.innerHeight * 2
  }

  update(progress) {
    this.gameElements.forEach(gameElement => gameElement.update(this.ctx.canvas, this.state));
  }

  redraw() {
    this.ctx.clearRect(0, 0, this.getWidth(), this.getHeight());
    this.gameElements.forEach(gameElement => gameElement.draw(this.ctx));

    if (Game.DEBUG) {
      this.__renderDebugText(this.ctx);
    }
  }

  /* @private */ __renderDebugText(ctx) {
    ctx.save();
    let text = 'DEBUG OUTPUT:\n\n' + JSON.stringify(Object.assign({
      running: this.isRunning()
    }, this.state), null, 2);
    let offset = { x: 16, y: 16 };
    let bounds = { width: 280,  height: 320 };
    ctx.fillStyle = 'rgba(16, 16, 16, 0.8)';
    ctx.fillRect(offset.x, offset.y, bounds.width, bounds.height);
    ctx.lineWidth = 1;
    ctx.strokeStyle = '#333';
    ctx.strokeRect(offset.x, offset.y, bounds.width, bounds.height);
    ctx.fillStyle = '#FFF';
    canvasTxt.drawText(ctx, text, offset.x + 4, offset.y + 4, bounds.width, bounds.height);
    ctx.restore();
  }

  loop(timestamp) {
    this.update(timestamp - this.__lastRender);
    this.redraw();
    this.__lastRender = timestamp;
    this.__animationId = window.requestAnimationFrame((ts) => this.loop(ts));
  }

  isRunning() {
    return this.__animationId != null;
  }

  run() {
    this.__animationId = window.requestAnimationFrame((ts) => this.loop(ts));
  }

  pause() {
    cancelAnimationFrame(this.__animationId);
    this.__animationId = null;
    this.redraw();
  }
}
Game.DEBUG = false; // Default is off

main();

html,
body {
  margin: 0;
  padding: 0;
}

canvas {
  background: #000;
  height: 100%;
  width: 100%;
  display: block
}

<script src="https://cdnjs.cloudflare.com/ajax/libs/victor/1.1.0/victor.min.js"></script>
<script src="https://unpkg.com/canvas-txt@2.0.6/build/index.js"></script>
<canvas id="screen" />


问:
Why does java.lang.ArrayIndexOutOfBoundsException happen on this piece of code?
```<code>public boolean felGissningar(char gi, int antalfel){
    if(antalfel>0){
    int test = 0;
    for (int i = 0; i<felGiss1.length; i++){
        if(gi == felGiss1[i]){
            test+=1;
        }
    }
    felGiss2 = new char[antalfel];
    for (int i = 0; i<felGiss1.length; i++){
        felGiss2[i]=felGiss1[i];
    }

    felGiss1 = new char[antalfel+1];
    for (int i =0;i<antalfel; i++){
        felGiss1[i]=felGiss2[i];
    }
    felGiss1[antalfel] = gi;
    System.out.println("1: "+String.valueOf(felGiss1));
    System.out.println("2: "+String.valueOf(felGiss2));
    if(test>0){
        return false;
    } else {
        return true;
    }
    }else {
        felGiss1 = new char[1];
        felGiss1[0]=gi;
        return true;
    }
}
</code>```

antalfel means number of incorrect answers

After this boolean has returned false once it says java.lang.ArrayIndexOutOfBoundsException

what im trying to do is to not allow for the same wrong character twise.

答:
Over here:

    felGiss2 = new char[antalfel];
    for (int i = 0; i<felGiss1.length; i++){
        felGiss2[i]=felGiss1[i];
    }
```

<p>Since you didn't post where <code>felGiss1</code> was created, if <code>felGiss1.length</code> is greater than <code>felGiss2.length</code> then it will produce an exception. Please change to:</p>

<pre class="lang-java prettyprint-override"><code>    felGiss2 = new char[antalfel];
    for (int i = 0; i<felGiss1.length && i < fellGiss2.length; i++){
        felGiss2[i]=felGiss1[i];
    }
</code>```

Body:
public boolean felGissningar(char gi, int antalfel){
    if(antalfel>0){
    int test = 0;
    for (int i = 0; i<felGiss1.length; i++){
        if(gi == felGiss1[i]){
            test+=1;
        }
    }
    felGiss2 = new char[antalfel];
    for (int i = 0; i<felGiss1.length; i++){
        felGiss2[i]=felGiss1[i];
    }

    felGiss1 = new char[antalfel+1];
    for (int i =0;i<antalfel; i++){
        felGiss1[i]=felGiss2[i];
    }
    felGiss1[antalfel] = gi;
    System.out.println("1: "+String.valueOf(felGiss1));
    System.out.println("2: "+String.valueOf(felGiss2));
    if(test>0){
        return false;
    } else {
        return true;
    }
    }else {
        felGiss1 = new char[1];
        felGiss1[0]=gi;
        return true;
    }
}

antalfel means number of incorrect answers

After this boolean has returned false once it says java.lang.ArrayIndexOutOfBoundsException

what im trying to do is to not allow for the same wrong character twise.

Body:
Over here:

    felGiss2 = new char[antalfel];
    for (int i = 0; i<felGiss1.length; i++){
        felGiss2[i]=felGiss1[i];
    }

Since you didn't post where felGiss1 was created, if felGiss1.length is greater than felGiss2.length then it will produce an exception. Please change to:

    felGiss2 = new char[antalfel];
    for (int i = 0; i<felGiss1.length && i < fellGiss2.length; i++){
        felGiss2[i]=felGiss1[i];
    }


问:
Android: How can I use send data from Room to Retrofit without LiveData?
my situation is that I am trying to get a list of strings List<String> from my Room database and send it over to Retrofit to retrieve information.

I don't want to use LiveData because I would need to have a nested observer in the View to observe the data in the database and another to get the list of data from Retrofit.

I am trying to use coroutines but am unsure how to return data from the viewModelScope.launch function when it is finally done getting the values.

答:
I don't want to use LiveData because I would need to have a nested observer in the View to observe the data in the database and another to get the list of data from Retrofit.

You can define your observers as top-level classes instead of nesting them. You can do this either as a private class in the same file as the class where they are used or as a public class in a separate file. This is a good solution if your concern is an overly complex, deeply-nested, long source file. It will allow you to more easily share code if necessary.

For example, if you have the following observer in your view:

```<code>final Observer<String> nameObserver = new Observer<String>() {
    @Override
    public void onChanged(@Nullable final String newName) {
        // Update the UI, in this case, a TextView.
        nameTextView.setText(newName);
    }
};
</code>```

Then you create a named class:

```<code>class MyObserver extends Observer<String> {
    @Override
    public void onChanged(@Nullable final String newName) {
        // Update the UI, in this case, a TextView.
        nameTextView.setText(newName);
    }
}
</code>```

and then in your view:

```<code>final Observer<String> nameObserver = new MyObserver();
</code>```

If you insist on avoiding LiveData, you can do what you want with a AsyncTask or roll your own threaded implementation.

Body:
I don't want to use LiveData because I would need to have a nested observer in the View to observe the data in the database and another to get the list of data from Retrofit.

You can define your observers as top-level classes instead of nesting them. You can do this either as a private class in the same file as the class where they are used or as a public class in a separate file. This is a good solution if your concern is an overly complex, deeply-nested, long source file. It will allow you to more easily share code if necessary.

For example, if you have the following observer in your view:

final Observer<String> nameObserver = new Observer<String>() {
    @Override
    public void onChanged(@Nullable final String newName) {
        // Update the UI, in this case, a TextView.
        nameTextView.setText(newName);
    }
};

Then you create a named class:

class MyObserver extends Observer<String> {
    @Override
    public void onChanged(@Nullable final String newName) {
        // Update the UI, in this case, a TextView.
        nameTextView.setText(newName);
    }
}

and then in your view:

final Observer<String> nameObserver = new MyObserver();

If you insist on avoiding LiveData, you can do what you want with a AsyncTask or roll your own threaded implementation.


问:
Laravel - How to resolve special Date issue from External API
In my Laravel-5.8 project, I am having this model: Employee

Employee

```<code> class Employee extends Model
{
    protected $table = 'employees';

    protected $primaryKey = 'id';

   protected $fillable = [
          'staff_code',
          'first_name',
          'last_name',
       'date_of_birth',
      ];

}
</code>```

That is,:

```<code>App\Employee

 CREATE TABLE `employees` (
     `id` int NOT NULL auto_increment,
     `staff_code` varchar(255) UNIQUE NOT NULL,
     `first_name` varchar(255) UNIQUE NOT NULL,
     `last_name` varchar(255) NOT NULL,
     `date_of_birth` date NOT NULL

) ENGINE=MyISAM DEFAULT CHARSET=latin1;

Also I have an external api that comes in form of JSON get request.

https://api.employees.net/allemployees
</code>```

I have viewed it with postman get request and I have something like this:

```<code> {
    "ID": "1",
    "StaffCode": "STC001",
    "FirstName": "Japheth",
    "LastName": "Shalom",
    "DateOfBirth": "1992-07-11T00:00:00",
 },
 {
    "ID": "2",
    "StaffCode": "STC002",
   "FirstName": "Ahitophel",
   "last_name": "Nedum",
   "DateOfBirth": "1991-10-23T00:00:00",
},
{
    "ID": "3",
    "StaffCode": "STC003",
    "FirstName": "Joash",
    "FirstName": "Nathan",
    "DateOfBirth": "1979-09-22T00:00:00",
 },
</code>```

and so on... this continues

Already I have created this function:

use App\Employee;

```<code> public function index()
 {
     $client = new GuzzleHttp\Client();
     $res = $client->request('GET','https://api.employees.net/allemployees');
     $clientdatas = json_decode($res->getBody()->getContents(), true);

     foreach($clientdatas as $clientdata)
     {
        $employee = Employee::firstOrNew(['id' => $clientdata['ID']]);
        $employee->staff_code = $clientdata['StaffCode'];
        $employee->first_name = $clientdata['FirstName'];
        $employee->last_name = $clientdata['LastName'];
        $employee->date_of_birth = $clientdata['DateOfBirth'];
        $employee->save();
     }
 }
</code>```

The DateOfBirth that is coming from the external API is carrying a special character (T) along with it.  And the data type for date_of_birth in the database is date.

I want to save the data from the external API into the local database.

How do I format DateOfBirth from the API into date data type and remove T00:00:00 from it?

Thank you

答:
Just take what you need from that string:</p>

```<code>$employee->date_of_birth = substr($clientdata['DateOfBirth'], 0,10);
</code>```

Body:
In my Laravel-5.8 project, I am having this model: Employee

Employee

 class Employee extends Model
{
    protected $table = 'employees';

    protected $primaryKey = 'id';

   protected $fillable = [
          'staff_code',
          'first_name',
          'last_name',
       'date_of_birth',
      ];

}

That is,:

App\Employee

 CREATE TABLE `employees` (
     `id` int NOT NULL auto_increment,
     `staff_code` varchar(255) UNIQUE NOT NULL,
     `first_name` varchar(255) UNIQUE NOT NULL,
     `last_name` varchar(255) NOT NULL,
     `date_of_birth` date NOT NULL

) ENGINE=MyISAM DEFAULT CHARSET=latin1;

Also I have an external api that comes in form of JSON get request.

https://api.employees.net/allemployees

I have viewed it with postman get request and I have something like this:

 {
    "ID": "1",
    "StaffCode": "STC001",
    "FirstName": "Japheth",
    "LastName": "Shalom",
    "DateOfBirth": "1992-07-11T00:00:00",
 },
 {
    "ID": "2",
    "StaffCode": "STC002",
   "FirstName": "Ahitophel",
   "last_name": "Nedum",
   "DateOfBirth": "1991-10-23T00:00:00",
},
{
    "ID": "3",
    "StaffCode": "STC003",
    "FirstName": "Joash",
    "FirstName": "Nathan",
    "DateOfBirth": "1979-09-22T00:00:00",
 },

and so on... this continues

Already I have created this function:

use App\Employee;

 public function index()
 {
     $client = new GuzzleHttp\Client();
     $res = $client->request('GET','https://api.employees.net/allemployees');
     $clientdatas = json_decode($res->getBody()->getContents(), true);

     foreach($clientdatas as $clientdata)
     {
        $employee = Employee::firstOrNew(['id' => $clientdata['ID']]);
        $employee->staff_code = $clientdata['StaffCode'];
        $employee->first_name = $clientdata['FirstName'];
        $employee->last_name = $clientdata['LastName'];
        $employee->date_of_birth = $clientdata['DateOfBirth'];
        $employee->save();
     }
 }

The DateOfBirth that is coming from the external API is carrying a special character (T) along with it.  And the data type for date_of_birth in the database is date.

I want to save the data from the external API into the local database.

How do I format DateOfBirth from the API into date data type and remove T00:00:00 from it?

Thank you

Body:
Just take what you need from that string:

$employee->date_of_birth = substr($clientdata['DateOfBirth'], 0,10);


问:
NullPointerException in hashCode() when deserializing an object graph
I have a final field in a Java class that is sure to be initialized. When I serialize and deserialize a corresponding object graph, I get a NPE because the field is used in the hashCode() method but apparently not yet read back. I created a minimal test case that also contains the serialize()/deserialize() methods (in case those are buggy), but I can't seem to be able to understand what the (underlying) problem is or how to work around it.

```<code>public class TestSerializerTest {

private static final org.slf4j.Logger logger = org.slf4j.LoggerFactory.getLogger(TestSerializerTest.class);

private File workDir;

@Before
public void setUp() {
    workDir = new File("target/tmp");
}

@After
public void tearDown() {
    workDir.delete();
}

private static class ManagingContainer implements Serializable {

    private static final long serialVersionUID = 1L;

    private final Set<Contained> containedElements = new HashSet<Contained>();

    public Contained getContained(List<String> descriptors) {
        Contained contained = new Contained(this, descriptors);
        containedElements.add(contained);
        return contained;
    }
}

private static class Contained implements Serializable {

    private static final long serialVersionUID = 1L;

    @SuppressWarnings("unused")
    private final ManagingContainer container;
    private final List<String> descriptors;

    Contained(ManagingContainer container, List<String> descriptors) {
        this.container = container;
        if (descriptors == null) {
            throw new NullPointerException();
        }
        this.descriptors = new ArrayList<String>(descriptors);
    }

    @Override
    public int hashCode() {
        return descriptors.hashCode();
    }

    @Override
    public boolean equals(Object other) {
        if (this == other) {
            return true;
        }
        if (other == null || !(other instanceof Contained)) {
            return false;
        }
        return this.descriptors.equals(((Contained) other).descriptors);
    }
}

private static class OtherContainer implements Serializable {
    private static final long serialVersionUID = 1L;

    private final ArrayList<Contained> containedElements = new ArrayList<Contained>();

    public OtherContainer(Contained initialElement) {
        this.containedElements.add(initialElement);
    }

    public void addContained(Contained nextElement) {
        containedElements.add(nextElement);
    }
}

void serializeObjectToFile(Serializable serializable, File file) {
    logger.info("Saving object '{}' to file '{}'.", serializable, file.getAbsolutePath());
    ObjectOutputStream stream = null;
    try {
        if (!file.getParentFile().exists()) {
            logger.info("Creating directory '{}'.", file.getParentFile().getAbsolutePath());
            file.getParentFile().mkdirs();
        }
        stream = new ObjectOutputStream(new FileOutputStream(file));
        stream.writeObject(serializable);
        stream.flush();
    } catch (Exception exception) {
        throw new RuntimeException(exception);
    } finally {
        if (stream != null) {
            try {
                stream.close();
            } catch (IOException exc) {
                logger.error("Error closing stream.", exc);
            }
        }
    }
}

Object deserializeObjectFromFile(File file) {
    ObjectInputStream oiStream = null;
    try {
        logger.info("Loading object from file '{}'.", file.getAbsolutePath());
        oiStream = new ObjectInputStream(new FileInputStream(file));
        return oiStream.readObject();
    } catch (Exception exc) {
        logger.error("Exception loading object from file '{}'. Ignoring file!", file.getAbsolutePath(), exc);
        throw new RuntimeException(exc);
    } finally {
        if (oiStream != null) {
            try {
                oiStream.close();
            } catch (IOException exc) {
                logger.error("Error closing stream.", exc);
            }
        }
    }
}

@Test
public void testSerializeDeserialize() {
    ManagingContainer container = new ManagingContainer();
    OtherContainer serializable = new OtherContainer(container.getContained(new ArrayList<String>()));
    serializable.addContained(container.getContained(new ArrayList<String>()));
    File file = new File(workDir, "test.ser");
    serializeObjectToFile(serializable, file);
    OtherContainer result = (OtherContainer) deserializeObjectFromFile(file);
}
}
</code>```

Executing this test creates the following NullPointerException:

```<code>java.lang.NullPointerException: null
at TestSerializerTest$Contained.hashCode(TestSerializerTest.java:67) ~[test-classes/:na]
at java.util.HashMap.put(HashMap.java:372) ~[na:1.6.0_29]
at java.util.HashSet.readObject(HashSet.java:292) ~[na:1.6.0_29]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.6.0_29]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) ~[na:1.6.0_29]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) ~[na:1.6.0_29]
at java.lang.reflect.Method.invoke(Method.java:597) ~[na:1.6.0_29]
at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:974) ~[na:1.6.0_29]
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1848) ~[na:1.6.0_29]
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752) ~[na:1.6.0_29]
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328) ~[na:1.6.0_29]
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1946) ~[na:1.6.0_29]
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1870) ~[na:1.6.0_29]
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752) ~[na:1.6.0_29]
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328) ~[na:1.6.0_29]
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1946) ~[na:1.6.0_29]
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1870) ~[na:1.6.0_29]
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752) ~[na:1.6.0_29]
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328) ~[na:1.6.0_29]
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350) ~[na:1.6.0_29]
at java.util.LinkedList.readObject(LinkedList.java:964) ~[na:1.6.0_29]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.6.0_29]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) ~[na:1.6.0_29]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) ~[na:1.6.0_29]
at java.lang.reflect.Method.invoke(Method.java:597) ~[na:1.6.0_29]
at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:974) ~[na:1.6.0_29]
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1848) ~[na:1.6.0_29]
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752) ~[na:1.6.0_29]
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328) ~[na:1.6.0_29]
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1946) ~[na:1.6.0_29]
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1870) ~[na:1.6.0_29]
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752) ~[na:1.6.0_29]
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328) ~[na:1.6.0_29]
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350) ~[na:1.6.0_29]
at TestSerializerTest.deserializeObjectFromFile(TestSerializerTest.java:128) [test-classes/:na]
at TestSerializerTest.testSerializeDeserialize(TestSerializerTest.java:151) [test-classes/:na]
</code>```

As far as I understand serialization, this should not be a problem (see also Does Java Serialization work for cyclic references? and https://softwareengineering.stackexchange.com/questions/151055/what-happens-if-we-serialize-and-deserialize-two-objects-which-references-to-eac).

Currently I have no idea how to proceed. Any hints are highly appreciated! Thanks!

答:
I encountered this issue with a complex object graph had bi-directional circular references using HashMaps. Since the key of the HashMap had not fully loaded when hashCode() was called on it, I also got a NullPointerException:

```<code>private String id;

int hashCode() {
    return id.hashCode(); // NPE during deserialization
}
</code>```

I "fixed" (read: "avoided") the problem by also caching and serializing the hashCode int itself:

```<code>private String id;
private int hashCode;

public void setId( String id ) {
  this.id = id;
  this.hashCode = id.hashCode();
}

public int hashCode() {
  return this.hashCode;
}
</code>```

Primitives are in-lined during serialization. They're loaded immediately, before any object references, such as the id String above. So the hashCode int will be available even if no referenced Objects have loaded.

Bad news, this does mean that hashCode will need to be updated whenever the hashCode-providing Object is. For Objects with clearly defined access/update, such as immutable Strings, this is pretty straightforward (see setId(...) above). For Objects with complex interactions, like Collections, this is harder to control, since it may depend on changes to the contents of the Collection and also the hashCodes of the individual Objects in the Collection.

Good news, as long as memory is cheap, this may lead to a slight performance improvement if hashCode() is called repeatedly.

ContentLicense: CC BY-SA 3.0

Body:
I have a final field in a Java class that is sure to be initialized. When I serialize and deserialize a corresponding object graph, I get a NPE because the field is used in the hashCode() method but apparently not yet read back. I created a minimal test case that also contains the serialize()/deserialize() methods (in case those are buggy), but I can't seem to be able to understand what the (underlying) problem is or how to work around it.

public class TestSerializerTest {

private static final org.slf4j.Logger logger = org.slf4j.LoggerFactory.getLogger(TestSerializerTest.class);

private File workDir;

@Before
public void setUp() {
    workDir = new File("target/tmp");
}

@After
public void tearDown() {
    workDir.delete();
}

private static class ManagingContainer implements Serializable {

    private static final long serialVersionUID = 1L;

    private final Set<Contained> containedElements = new HashSet<Contained>();

    public Contained getContained(List<String> descriptors) {
        Contained contained = new Contained(this, descriptors);
        containedElements.add(contained);
        return contained;
    }
}

private static class Contained implements Serializable {

    private static final long serialVersionUID = 1L;

    @SuppressWarnings("unused")
    private final ManagingContainer container;
    private final List<String> descriptors;

    Contained(ManagingContainer container, List<String> descriptors) {
        this.container = container;
        if (descriptors == null) {
            throw new NullPointerException();
        }
        this.descriptors = new ArrayList<String>(descriptors);
    }

    @Override
    public int hashCode() {
        return descriptors.hashCode();
    }

    @Override
    public boolean equals(Object other) {
        if (this == other) {
            return true;
        }
        if (other == null || !(other instanceof Contained)) {
            return false;
        }
        return this.descriptors.equals(((Contained) other).descriptors);
    }
}

private static class OtherContainer implements Serializable {
    private static final long serialVersionUID = 1L;

    private final ArrayList<Contained> containedElements = new ArrayList<Contained>();

    public OtherContainer(Contained initialElement) {
        this.containedElements.add(initialElement);
    }

    public void addContained(Contained nextElement) {
        containedElements.add(nextElement);
    }
}

void serializeObjectToFile(Serializable serializable, File file) {
    logger.info("Saving object '{}' to file '{}'.", serializable, file.getAbsolutePath());
    ObjectOutputStream stream = null;
    try {
        if (!file.getParentFile().exists()) {
            logger.info("Creating directory '{}'.", file.getParentFile().getAbsolutePath());
            file.getParentFile().mkdirs();
        }
        stream = new ObjectOutputStream(new FileOutputStream(file));
        stream.writeObject(serializable);
        stream.flush();
    } catch (Exception exception) {
        throw new RuntimeException(exception);
    } finally {
        if (stream != null) {
            try {
                stream.close();
            } catch (IOException exc) {
                logger.error("Error closing stream.", exc);
            }
        }
    }
}

Object deserializeObjectFromFile(File file) {
    ObjectInputStream oiStream = null;
    try {
        logger.info("Loading object from file '{}'.", file.getAbsolutePath());
        oiStream = new ObjectInputStream(new FileInputStream(file));
        return oiStream.readObject();
    } catch (Exception exc) {
        logger.error("Exception loading object from file '{}'. Ignoring file!", file.getAbsolutePath(), exc);
        throw new RuntimeException(exc);
    } finally {
        if (oiStream != null) {
            try {
                oiStream.close();
            } catch (IOException exc) {
                logger.error("Error closing stream.", exc);
            }
        }
    }
}

@Test
public void testSerializeDeserialize() {
    ManagingContainer container = new ManagingContainer();
    OtherContainer serializable = new OtherContainer(container.getContained(new ArrayList<String>()));
    serializable.addContained(container.getContained(new ArrayList<String>()));
    File file = new File(workDir, "test.ser");
    serializeObjectToFile(serializable, file);
    OtherContainer result = (OtherContainer) deserializeObjectFromFile(file);
}
}

Executing this test creates the following NullPointerException:

java.lang.NullPointerException: null
at TestSerializerTest$Contained.hashCode(TestSerializerTest.java:67) ~[test-classes/:na]
at java.util.HashMap.put(HashMap.java:372) ~[na:1.6.0_29]
at java.util.HashSet.readObject(HashSet.java:292) ~[na:1.6.0_29]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.6.0_29]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) ~[na:1.6.0_29]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) ~[na:1.6.0_29]
at java.lang.reflect.Method.invoke(Method.java:597) ~[na:1.6.0_29]
at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:974) ~[na:1.6.0_29]
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1848) ~[na:1.6.0_29]
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752) ~[na:1.6.0_29]
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328) ~[na:1.6.0_29]
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1946) ~[na:1.6.0_29]
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1870) ~[na:1.6.0_29]
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752) ~[na:1.6.0_29]
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328) ~[na:1.6.0_29]
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1946) ~[na:1.6.0_29]
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1870) ~[na:1.6.0_29]
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752) ~[na:1.6.0_29]
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328) ~[na:1.6.0_29]
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350) ~[na:1.6.0_29]
at java.util.LinkedList.readObject(LinkedList.java:964) ~[na:1.6.0_29]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.6.0_29]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) ~[na:1.6.0_29]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) ~[na:1.6.0_29]
at java.lang.reflect.Method.invoke(Method.java:597) ~[na:1.6.0_29]
at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:974) ~[na:1.6.0_29]
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1848) ~[na:1.6.0_29]
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752) ~[na:1.6.0_29]
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328) ~[na:1.6.0_29]
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1946) ~[na:1.6.0_29]
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1870) ~[na:1.6.0_29]
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752) ~[na:1.6.0_29]
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328) ~[na:1.6.0_29]
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350) ~[na:1.6.0_29]
at TestSerializerTest.deserializeObjectFromFile(TestSerializerTest.java:128) [test-classes/:na]
at TestSerializerTest.testSerializeDeserialize(TestSerializerTest.java:151) [test-classes/:na]

As far as I understand serialization, this should not be a problem (see also Does Java Serialization work for cyclic references? and https://softwareengineering.stackexchange.com/questions/151055/what-happens-if-we-serialize-and-deserialize-two-objects-which-references-to-eac).

Currently I have no idea how to proceed. Any hints are highly appreciated! Thanks!

Body:
I encountered this issue with a complex object graph had bi-directional circular references using HashMaps. Since the key of the HashMap had not fully loaded when hashCode() was called on it, I also got a NullPointerException:

private String id;

int hashCode() {
    return id.hashCode(); // NPE during deserialization
}

I "fixed" (read: "avoided") the problem by also caching and serializing the hashCode int itself:

private String id;
private int hashCode;

public void setId( String id ) {
  this.id = id;
  this.hashCode = id.hashCode();
}

public int hashCode() {
  return this.hashCode;
}

Primitives are in-lined during serialization. They're loaded immediately, before any object references, such as the id String above. So the hashCode int will be available even if no referenced Objects have loaded.

Bad news, this does mean that hashCode will need to be updated whenever the hashCode-providing Object is. For Objects with clearly defined access/update, such as immutable Strings, this is pretty straightforward (see setId(...) above). For Objects with complex interactions, like Collections, this is harder to control, since it may depend on changes to the contents of the Collection and also the hashCodes of the individual Objects in the Collection.

Good news, as long as memory is cheap, this may lead to a slight performance improvement if hashCode() is called repeatedly.


问:
istio 1.4.8: strange 400 error when used with AWS Load balancer
I'm getting a strange 400 error when I try to connect to an Istio Gateway when behind an AWS load balancer.

I don't see any activity in the istio-ingresgateway logs (even with debug settings on), but when I run sysdig on the ingressgateway pod, I see weird semi-random text with often with "QUIT !T" in it.

I get this when I try to make an http request via a browser or curl from outside the cluster.

The same Istio configuration works when I try to make the request in minikube or in Azure.

I'm also able to use the same AWS lb to point to a Nginx ingress controller and it works just fine.

```<code>sudo sysdig -s2000 -A -c echo_fds fd.ip=10.1.2.3
</code>```

Sometimes there is no GET request in the output

```<code>------ Read 100B from   10.1.1.3:44404->10.1.2.3:80 (envoy)

QUIT
!T
C
ct>
------ Write 66B to   10.1.1.3:44404->10.1.2.3:80 (envoy)

HTTP/1.1 400 Bad Request
content-length: 0
connection: close
</code>```

And, sometimes this happens right before the GET request

```<code>------ Read 3.39KB from   10.1.1.3:35430->10.1.2.3:80 (envoy)

QUIT
!T
C
atfI>GET /myapp/ HTTP/1.1
</code>```

I'm wondering if the weird characters are causing the envoy routes not to match, but I have no idea where this could be coming from.

Any advice as to what this might be?

Any general strategies for debugging Istio ingress?

Any help is much appreciated.

答:
So I found the answer to this question. The garbage in the request was a read herring.

A little more info about the setup:

The AWS load balancer was terminating TLS, so all the traffic behind it was going over the http port 31380. I was already passing the X-Forwarded-Proto: https header via the istio VirtualService setting, so the backend applications could assume that the traffic was already-terminated TLS traffic.

The solution:

The issue in my case was that the AWS Target Group had the Proxy protocol v2: Enabled set on it.  Turning this off solved the issue.

Body:
I'm getting a strange 400 error when I try to connect to an Istio Gateway when behind an AWS load balancer.

I don't see any activity in the istio-ingresgateway logs (even with debug settings on), but when I run sysdig on the ingressgateway pod, I see weird semi-random text with often with "QUIT !T" in it.

I get this when I try to make an http request via a browser or curl from outside the cluster.

The same Istio configuration works when I try to make the request in minikube or in Azure.

I'm also able to use the same AWS lb to point to a Nginx ingress controller and it works just fine.

sudo sysdig -s2000 -A -c echo_fds fd.ip=10.1.2.3

Sometimes there is no GET request in the output

------ Read 100B from   10.1.1.3:44404->10.1.2.3:80 (envoy)

QUIT
!T
C
ct>
------ Write 66B to   10.1.1.3:44404->10.1.2.3:80 (envoy)

HTTP/1.1 400 Bad Request
content-length: 0
connection: close

And, sometimes this happens right before the GET request

------ Read 3.39KB from   10.1.1.3:35430->10.1.2.3:80 (envoy)

QUIT
!T
C
atfI>GET /myapp/ HTTP/1.1

I'm wondering if the weird characters are causing the envoy routes not to match, but I have no idea where this could be coming from.

Any advice as to what this might be?

Any general strategies for debugging Istio ingress?

Any help is much appreciated.


问:
How to remove portions of Web page title using Javascript
I'm new at this Javascript stuff, so please excuse my basic question. I'm trying to find a way to remove some portion of a web page title using Javascript in a userscript. Take for example the title of this page:

How to remove portions of Web page title using Javascript - Stack Overflow

I want to yank out the '-' and Website name so that it becomes:

How to remove portions of Web page title using Javascript

I know how to change the entire title using document.title = "new title";, but that's about the closest I can get. Can someone please help?

Thanks in advance.

答:
This should do it:

```<code>document.title.substr(0,document.title.lastIndexOf("-"));
</code>```

It removes everything from the last occurrence of "-".

http://jsfiddle.net/niklasvh/TA69F/

Body:
This should do it:

document.title.substr(0,document.title.lastIndexOf("-"));

It removes everything from the last occurrence of "-".

http://jsfiddle.net/niklasvh/TA69F/


问:
how should I prevent the UI from being stuck when unzipping file with ZIP in swift
I am trying to print the progress on unzipping a file on the swiftUI. but each time the UI gets stuck while doing so. I am using ZIP as a 3rd part library.
I tried this code to make unzipping on a different thread

```<code>    utils.initWindow()
    DispatchQueue.main.async {
        self.utils.unzipFile(fileLocation: URL(string: "/Users/user/Downloads/Archive.zip")!, destination: URL(string: "/Users/user/Downloads/testunzip")!)
    }
</code>```

and this is how I implemented the function unzipFile

```<code>    public func unzipFile(fileLocation: URL, destination: URL){
    self.launcher.addLog(log: "Retrieving files")
    do{
        try Zip.unzipFile(fileLocation, destination: destination, overwrite: true, password: "", progress: { (progress) -> () in
            print(progress)
            self.launcher.setState(state: progress)
        })
    }
    catch {
        self.launcher.setState(state: progress)
        print("Something went wrong")
    }
}
</code>```

PS: setState basically sets a value of a @Published variable on a @ObservedObject that control the Text on the UI and it works perfectly fine when I am printing the progress on a different function that prints the progress on downloading a file

答:
Don't perform unzip or zip on main thread as it can be a heavy operation. Instead of main thread use

```<code>        DispatchQueue.global(qos: .background).async {
        }
</code>```

Happy Coding!

Body:
I am trying to print the progress on unzipping a file on the swiftUI. but each time the UI gets stuck while doing so. I am using ZIP as a 3rd part library.
I tried this code to make unzipping on a different thread

    utils.initWindow()
    DispatchQueue.main.async {
        self.utils.unzipFile(fileLocation: URL(string: "/Users/user/Downloads/Archive.zip")!, destination: URL(string: "/Users/user/Downloads/testunzip")!)
    }

and this is how I implemented the function unzipFile

    public func unzipFile(fileLocation: URL, destination: URL){
    self.launcher.addLog(log: "Retrieving files")
    do{
        try Zip.unzipFile(fileLocation, destination: destination, overwrite: true, password: "", progress: { (progress) -> () in
            print(progress)
            self.launcher.setState(state: progress)
        })
    }
    catch {
        self.launcher.setState(state: progress)
        print("Something went wrong")
    }
}

PS: setState basically sets a value of a @Published variable on a @ObservedObject that control the Text on the UI and it works perfectly fine when I am printing the progress on a different function that prints the progress on downloading a file

Body:
Don't perform unzip or zip on main thread as it can be a heavy operation. Instead of main thread use

        DispatchQueue.global(qos: .background).async {
        }

Happy Coding!


问:
How to get the correct input_file_name in PySpark after a repartition
I'm developing a PySpark job that read text files and write parquet files on a AWS S3 bucket.

I need to add a column to each parquet line that describe the source file of the original text line.

I'm using the input_file_name but i don't see the correct source file name in the output column. Seems that use the same filename for an entire tasks or for the wrong partition.

I don't know if this is caused by the repartition that i need to do or because i'm reading an entire folder (prefix) in S3.

Here an example of my code:

```<code>rdd = spark.sparkContext.textFile("s3a://source-bucket/prefix/")
rdd = rdd.repartition(numPartitions=(120))
rdd = rdd.map(lambda line: custom_regex_parser)

df = spark.createDataFrame(rdd, schema=custom_source_schema)
df = df..withColumn("source_file_name", input_file_name)
df.write.mode('overwrite').parquet("s3://dest-bucket/prefix/")
</code>```

In the S3 bucket i have lots of GZ files, of various dimension (from few KB to 50 MB compressed). I need to do a repartition to avoid random memory issues on some Spark tasks.

Could be this repartition the problem with input_file_name?

After that, do you have any suggestion to better spread data across executors homogeneously ? This is quite time consuming...

答:
I resolved this issue by adding a new column in data frame while reading the data itself.

```<code>var df1 = sc.textFile("s3://xxxx/xxx/*").toDF().withColumn("filename", input_file_name)
</code>```

In your code, I see two issues

```<code>df = df..withColumn("source_file_name", input_file_name)
</code>```

You have two ".", which is a syntax error.

More importantly, you have already repartitioned the data and have lost the original partition information.

Body:
I'm developing a PySpark job that read text files and write parquet files on a AWS S3 bucket.

I need to add a column to each parquet line that describe the source file of the original text line.

I'm using the input_file_name but i don't see the correct source file name in the output column. Seems that use the same filename for an entire tasks or for the wrong partition.

I don't know if this is caused by the repartition that i need to do or because i'm reading an entire folder (prefix) in S3.

Here an example of my code:

rdd = spark.sparkContext.textFile("s3a://source-bucket/prefix/")
rdd = rdd.repartition(numPartitions=(120))
rdd = rdd.map(lambda line: custom_regex_parser)

df = spark.createDataFrame(rdd, schema=custom_source_schema)
df = df..withColumn("source_file_name", input_file_name)
df.write.mode('overwrite').parquet("s3://dest-bucket/prefix/")

In the S3 bucket i have lots of GZ files, of various dimension (from few KB to 50 MB compressed). I need to do a repartition to avoid random memory issues on some Spark tasks.

Could be this repartition the problem with input_file_name?

After that, do you have any suggestion to better spread data across executors homogeneously ? This is quite time consuming...

Body:
I resolved this issue by adding a new column in data frame while reading the data itself.

var df1 = sc.textFile("s3://xxxx/xxx/*").toDF().withColumn("filename", input_file_name)

In your code, I see two issues

df = df..withColumn("source_file_name", input_file_name)

You have two ".", which is a syntax error.

More importantly, you have already repartitioned the data and have lost the original partition information.


问:
Changing app navigation structure from version 4 to 5 in react native
I was working on an old app using react navigation version 4 the app contains a register and login in page obviously and then the content of the app.

recently I started remaking the content of the app using react navigation version 5 in order to use the shared element animation and the bottom tab navigator and it was fairly simple.

but I struggled with converting the login part to version 5 since the app structure is somewhat complicated and I am somewhat new to react navigation version 5.

i will leave a figure of the app structure bellow a long with samples of the code used.

App.js :

```<code>import { setNavigator } from "./app/navigationRef";
const articleListFlow = createStackNavigator({
  Main: MainScreen, // screen with diffrent articles categories
  ResultsShow: ResultShowScreen, // article details screen
});
const loginFlow = createStackNavigator({
  Signup: SignupScreen,
  Signin: SigninScreen,
});
loginFlow.navigationOptions = () => {
  return {
    headerShown: false,
  };
};

articleListFlow.navigationOptions = {
  title: "News Feed",
  tabBarIcon: ({ tintColor }) => (
    <View>
      <Icon style={[{ color: tintColor }]} size={25} name={"ios-cart"} />
    </View>
  ),
  activeColor: "#ffffff",
  inactiveColor: "#ebaabd",
  barStyle: { backgroundColor: "#d13560" },
};
const switchNavigator = createSwitchNavigator({
  ResolveAuth: ResolveAuthScreen,
  MainloginFlow: createSwitchNavigator({
    //WelcomeScreen: WeclomeScreen,
    loginFlow: loginFlow,
  }),

  mainFlow: createMaterialBottomTabNavigator(
    {
      articleListFlow: articleListFlow,
      ArticleSave: ArticleSaveScreen, // we dont need this one
      Account: AccountScreen,
    },
    {
      activeColor: "#ffffff",
      inactiveColor: "#bda1f7",
      barStyle: { backgroundColor: "#6948f4" },
    }
  ),
});
const App = createAppContainer(switchNavigator);
export default () => {
  return (
    <AuthProvider>
      <App
        ref={(navigator) => {
          setNavigator(navigator);
        }}
      />
    </AuthProvider>
  );
};
</code>```

NavigationRef.js :

```<code>import { NavigationActions } from "react-navigation";

let navigator;

export const setNavigator = (nav) => {
  navigator = nav;
};

export const navigate = (routeName, params) => {
  navigator.dispatch(
    NavigationActions.navigate({
      routeName,
      params,
    })
  );
};

// routename is the name of the routes singin singup accountscreen
// params information we want to pass to the screen we want to show
</code>```

AuthContext.js

```<code>import { AsyncStorage } from "react-native";
import createDataContext from "./createDataContext";
import userAPI from "../api/user";

// using navigate to access the navigator and redirect the user
import { navigate } from "../navigationRef";

// AUTHENTICATION REDUCERS
const authReducer = (state, action) => {
  switch (action.type) {
    case "add_error": {
      return {
        ...state,
        errorMessage: action.payload,
      };
    }

    case "clear_error_message": {
      return {
        ...state,
        errorMessage: "",
      };
    }
    case "signin": {
      return {
        errorMessage: "",
        token: action.payload,
      };
    }

    default:
      return state;
  }
};

// CLEARING ERROR MESSAGES WHEN SWITCHING SIGNIN-SIGNUP
const clearErrorMessage = (dispatch) => () => {
  dispatch({ type: "clear_error_message" });
};

// AUTOMATIC SIGNIN ONLY USING TOKENS ON USER DEVICE
const tryLocalSignin = (dispatch) => async () => {
  const token = await AsyncStorage.getItem("token");
  if (token) {
    // if token exists
    dispatch({ type: "signin", payload: token });

    navigate("Main");
  } else {
    // if token doesnt exist
    navigate("WelcomeScreen");
  }
};

// SIGNUP
const signup = (dispatch) => async ({ email, password }) => {
  try {
    const response = await userAPI.post("/signup", { email, password });
    await AsyncStorage.setItem("token", response.data.token);
    dispatch({ type: "signin", payload: response.data.token });

    // making use of the navigate component to access navigation
    // and redirect the user
    navigate("Main");
  } catch (err) {
    dispatch({
      type: "add_error",
      payload: "Something went wrong with sign up",
    });
  }
};

// SIGNIN
const signin = (dispatch) => async ({ email, password }) => {
  try {
    const response = await userAPI.post("/signin", { email, password });
    await AsyncStorage.setItem("token", response.data.token);
    // using signin since the logic is the same
    dispatch({ type: "signin", payload: response.data.token });

    // making use of the navigate component to access navigation
    // and redirect the user
    navigate("Main");
  } catch (err) {
    console.log(err);
    dispatch({
      type: "add_error",
      payload: "Something went wrong with sign in",
    });
  }
};

// SIGNOUT
const signout = (dispatch) => async () => {
  // removing the token makes identification not work again
  await AsyncStorage.removeItem("token");
  dispatch({ type: "signout" });

  navigate("loginFlow");
};

// CREATING CONTEXT AND PROVIDER OBJECTS FOR AUTHENTICATION
export const { Provider, Context } = createDataContext(
  authReducer,
  {
    signin,
    signup,
    signout,
    clearErrorMessage,
    tryLocalSignin,
  },
  {
    token: null,
    errorMessage: "",
  }
);
</code>```

createDataContext.js

```<code>import React, { useReducer } from "react";

export default (reducer, actions, defaultValue) => {
  const Context = React.createContext();

  const Provider = ({ children }) => {
    const [state, dispatch] = useReducer(reducer, defaultValue);

    const boundActions = {};

    for (let action in actions) {
      // for every action in the actions, call it with dispatch
      boundActions[action] = actions[action](dispatch);
    }

    return (
      <Context.Provider value={{ state, ...boundActions }}>
        {children}
      </Context.Provider>
    );
  };

  return { Context, Provider };
};
</code>```

My appologies for the long code and thank you in advance for anyone who can help.

答:
There are several things that you need to consider when moving from V4 to V5 it involves some changes and also you can consider using features like the hooks.

The first change will be removing the Switch Navigator and conditionally render the navigator in its place. This will be done in your App.js. As you already have a reducer based implementation you can use the state values to take this decision.

The next change will be the creation of stacks, in V4 you create the navigation by passing the screen, now everything is a component and you pass the screens as children.

The option are also sent as props to either the navigator or the screen itself.

The usage of navigation ref is still possible but you can also use hooks like usenavigation inside components and for your authentication flow you wont be using this as you conditionally render the navigators.

I have made a simplified version based on your code.
App.js

```<code>const AuthStack = createStackNavigator();
const AppTabs = createMaterialBottomTabNavigator();
const ArticleStack = createStackNavigator();

const Articles = () => {
  return (
    <ArticleStack.Navigator>
      <AppTabs.Screen name="ArticlesList" component={ArticleList} />
      <AppTabs.Screen name="ArticlesDetails" component={ArticleDetail} />
    </ArticleStack.Navigator>
  );
};

export default function App() {
  const [state, dispatch] = React.useReducer(authReducer, {
    isLoading: true,
    token: null,
    errorMessage: '',
  });

  React.useEffect(() => {
    const bootstrapAsync = async () => {
      const userToken = await AsyncStorage.getItem('userToken');
      dispatch({ type: 'RESTORE_TOKEN', token: userToken });
    };

    bootstrapAsync();
  }, []);
  const authContext = React.useMemo(
    () => ({
      signIn: async (data) => {
        dispatch({ type: 'SIGN_IN', token: 'dummy-auth-token' });
      },
      signOut: () => dispatch({ type: 'SIGN_OUT' }),
      signUp: async (data) => {
        dispatch({ type: 'SIGN_IN', token: 'dummy-auth-token' });
      },
    }),
    []
  );

  return (
    <AuthContext.Provider value={authContext}>
      <NavigationContainer>
        {state.token === null ? (
          <AuthStack.Navigator headerMode="none">
            {state.isLoading ? (
              <AuthStack.Screen name="Welcome" component={WelcomeScreen} />
            ) : (
              <>
                <AuthStack.Screen name="SignIn" component={SignInScreen} />
                <AuthStack.Screen name="SignUp" component={SingUpScreen} />
              </>
            )}
          </AuthStack.Navigator>
        ) : (
          <AppTabs.Navigator
            activeColor="#f0edf6"
            inactiveColor="#3e2465"
            barStyle={{ backgroundColor: '#694fad' }}>
            <AppTabs.Screen
              name="Articles"
              component={Articles}
              options={{
                tabBarLabel: 'Home',
                tabBarIcon: ({ color, size }) => (
                  <MaterialCommunityIcons
                    name="home"
                    color={color}
                    size={size}
                  />
                ),
              }}
            />
            <AppTabs.Screen name="Search" component={SearchScreen} />
            <AppTabs.Screen name="Save" component={SaveScreen} />
            <AppTabs.Screen name="Account" component={AccountScreen} />
          </AppTabs.Navigator>
        )}
      </NavigationContainer>
    </AuthContext.Provider>
  );
}
</code>```

Auth Context

```<code>const AuthContext = React.createContext();
export default AuthContext;
</code>```

Auth Reducer

```<code>export const authReducer = (state, action) => {
  switch (action.type) {
    case 'RESTORE_TOKEN':
      return {
        ...state,
        token: action.token,
        isLoading: false,
      };

    case 'SIGN_IN': {
      return {
        errorMessage: '',
        token: action.payload,
      };
    }

    case 'SIGN_OUT': {
      return {
        errorMessage: '',
        token: null,
      };
    }

    default:
      return state;
  }
};
</code>```

As you can see the flow will be showing the welcome screen till the token is loaded from async storage and then based on that show the tabs or the login screen. Also the parameters are passed as props. I've moved the actions to app.js but it can be separated as well.

You can see a fully running sample here
https://snack.expo.io/@guruparan/navigation-sample-3

Hope this helps, Feel free to ask if there are any questions.

Body:
I was working on an old app using react navigation version 4 the app contains a register and login in page obviously and then the content of the app.

recently I started remaking the content of the app using react navigation version 5 in order to use the shared element animation and the bottom tab navigator and it was fairly simple.

but I struggled with converting the login part to version 5 since the app structure is somewhat complicated and I am somewhat new to react navigation version 5.

i will leave a figure of the app structure bellow a long with samples of the code used.

App.js :

import { setNavigator } from "./app/navigationRef";
const articleListFlow = createStackNavigator({
  Main: MainScreen, // screen with diffrent articles categories
  ResultsShow: ResultShowScreen, // article details screen
});
const loginFlow = createStackNavigator({
  Signup: SignupScreen,
  Signin: SigninScreen,
});
loginFlow.navigationOptions = () => {
  return {
    headerShown: false,
  };
};

articleListFlow.navigationOptions = {
  title: "News Feed",
  tabBarIcon: ({ tintColor }) => (
    <View>
      <Icon style={[{ color: tintColor }]} size={25} name={"ios-cart"} />
    </View>
  ),
  activeColor: "#ffffff",
  inactiveColor: "#ebaabd",
  barStyle: { backgroundColor: "#d13560" },
};
const switchNavigator = createSwitchNavigator({
  ResolveAuth: ResolveAuthScreen,
  MainloginFlow: createSwitchNavigator({
    //WelcomeScreen: WeclomeScreen,
    loginFlow: loginFlow,
  }),

  mainFlow: createMaterialBottomTabNavigator(
    {
      articleListFlow: articleListFlow,
      ArticleSave: ArticleSaveScreen, // we dont need this one
      Account: AccountScreen,
    },
    {
      activeColor: "#ffffff",
      inactiveColor: "#bda1f7",
      barStyle: { backgroundColor: "#6948f4" },
    }
  ),
});
const App = createAppContainer(switchNavigator);
export default () => {
  return (
    <AuthProvider>
      <App
        ref={(navigator) => {
          setNavigator(navigator);
        }}
      />
    </AuthProvider>
  );
};

NavigationRef.js :

import { NavigationActions } from "react-navigation";

let navigator;

export const setNavigator = (nav) => {
  navigator = nav;
};

export const navigate = (routeName, params) => {
  navigator.dispatch(
    NavigationActions.navigate({
      routeName,
      params,
    })
  );
};

// routename is the name of the routes singin singup accountscreen
// params information we want to pass to the screen we want to show

AuthContext.js

import { AsyncStorage } from "react-native";
import createDataContext from "./createDataContext";
import userAPI from "../api/user";

// using navigate to access the navigator and redirect the user
import { navigate } from "../navigationRef";

// AUTHENTICATION REDUCERS
const authReducer = (state, action) => {
  switch (action.type) {
    case "add_error": {
      return {
        ...state,
        errorMessage: action.payload,
      };
    }

    case "clear_error_message": {
      return {
        ...state,
        errorMessage: "",
      };
    }
    case "signin": {
      return {
        errorMessage: "",
        token: action.payload,
      };
    }

    default:
      return state;
  }
};

// CLEARING ERROR MESSAGES WHEN SWITCHING SIGNIN-SIGNUP
const clearErrorMessage = (dispatch) => () => {
  dispatch({ type: "clear_error_message" });
};

// AUTOMATIC SIGNIN ONLY USING TOKENS ON USER DEVICE
const tryLocalSignin = (dispatch) => async () => {
  const token = await AsyncStorage.getItem("token");
  if (token) {
    // if token exists
    dispatch({ type: "signin", payload: token });

    navigate("Main");
  } else {
    // if token doesnt exist
    navigate("WelcomeScreen");
  }
};

// SIGNUP
const signup = (dispatch) => async ({ email, password }) => {
  try {
    const response = await userAPI.post("/signup", { email, password });
    await AsyncStorage.setItem("token", response.data.token);
    dispatch({ type: "signin", payload: response.data.token });

    // making use of the navigate component to access navigation
    // and redirect the user
    navigate("Main");
  } catch (err) {
    dispatch({
      type: "add_error",
      payload: "Something went wrong with sign up",
    });
  }
};

// SIGNIN
const signin = (dispatch) => async ({ email, password }) => {
  try {
    const response = await userAPI.post("/signin", { email, password });
    await AsyncStorage.setItem("token", response.data.token);
    // using signin since the logic is the same
    dispatch({ type: "signin", payload: response.data.token });

    // making use of the navigate component to access navigation
    // and redirect the user
    navigate("Main");
  } catch (err) {
    console.log(err);
    dispatch({
      type: "add_error",
      payload: "Something went wrong with sign in",
    });
  }
};

// SIGNOUT
const signout = (dispatch) => async () => {
  // removing the token makes identification not work again
  await AsyncStorage.removeItem("token");
  dispatch({ type: "signout" });

  navigate("loginFlow");
};

// CREATING CONTEXT AND PROVIDER OBJECTS FOR AUTHENTICATION
export const { Provider, Context } = createDataContext(
  authReducer,
  {
    signin,
    signup,
    signout,
    clearErrorMessage,
    tryLocalSignin,
  },
  {
    token: null,
    errorMessage: "",
  }
);

createDataContext.js

import React, { useReducer } from "react";

export default (reducer, actions, defaultValue) => {
  const Context = React.createContext();

  const Provider = ({ children }) => {
    const [state, dispatch] = useReducer(reducer, defaultValue);

    const boundActions = {};

    for (let action in actions) {
      // for every action in the actions, call it with dispatch
      boundActions[action] = actions[action](dispatch);
    }

    return (
      <Context.Provider value={{ state, ...boundActions }}>
        {children}
      </Context.Provider>
    );
  };

  return { Context, Provider };
};

My appologies for the long code and thank you in advance for anyone who can help.

Body:
There are several things that you need to consider when moving from V4 to V5 it involves some changes and also you can consider using features like the hooks.

The first change will be removing the Switch Navigator and conditionally render the navigator in its place. This will be done in your App.js. As you already have a reducer based implementation you can use the state values to take this decision.

The next change will be the creation of stacks, in V4 you create the navigation by passing the screen, now everything is a component and you pass the screens as children.

The option are also sent as props to either the navigator or the screen itself.

The usage of navigation ref is still possible but you can also use hooks like usenavigation inside components and for your authentication flow you wont be using this as you conditionally render the navigators.

I have made a simplified version based on your code.
App.js

const AuthStack = createStackNavigator();
const AppTabs = createMaterialBottomTabNavigator();
const ArticleStack = createStackNavigator();

const Articles = () => {
  return (
    <ArticleStack.Navigator>
      <AppTabs.Screen name="ArticlesList" component={ArticleList} />
      <AppTabs.Screen name="ArticlesDetails" component={ArticleDetail} />
    </ArticleStack.Navigator>
  );
};

export default function App() {
  const [state, dispatch] = React.useReducer(authReducer, {
    isLoading: true,
    token: null,
    errorMessage: '',
  });

  React.useEffect(() => {
    const bootstrapAsync = async () => {
      const userToken = await AsyncStorage.getItem('userToken');
      dispatch({ type: 'RESTORE_TOKEN', token: userToken });
    };

    bootstrapAsync();
  }, []);
  const authContext = React.useMemo(
    () => ({
      signIn: async (data) => {
        dispatch({ type: 'SIGN_IN', token: 'dummy-auth-token' });
      },
      signOut: () => dispatch({ type: 'SIGN_OUT' }),
      signUp: async (data) => {
        dispatch({ type: 'SIGN_IN', token: 'dummy-auth-token' });
      },
    }),
    []
  );

  return (
    <AuthContext.Provider value={authContext}>
      <NavigationContainer>
        {state.token === null ? (
          <AuthStack.Navigator headerMode="none">
            {state.isLoading ? (
              <AuthStack.Screen name="Welcome" component={WelcomeScreen} />
            ) : (
              <>
                <AuthStack.Screen name="SignIn" component={SignInScreen} />
                <AuthStack.Screen name="SignUp" component={SingUpScreen} />
              </>
            )}
          </AuthStack.Navigator>
        ) : (
          <AppTabs.Navigator
            activeColor="#f0edf6"
            inactiveColor="#3e2465"
            barStyle={{ backgroundColor: '#694fad' }}>
            <AppTabs.Screen
              name="Articles"
              component={Articles}
              options={{
                tabBarLabel: 'Home',
                tabBarIcon: ({ color, size }) => (
                  <MaterialCommunityIcons
                    name="home"
                    color={color}
                    size={size}
                  />
                ),
              }}
            />
            <AppTabs.Screen name="Search" component={SearchScreen} />
            <AppTabs.Screen name="Save" component={SaveScreen} />
            <AppTabs.Screen name="Account" component={AccountScreen} />
          </AppTabs.Navigator>
        )}
      </NavigationContainer>
    </AuthContext.Provider>
  );
}

Auth Context

const AuthContext = React.createContext();
export default AuthContext;

Auth Reducer

export const authReducer = (state, action) => {
  switch (action.type) {
    case 'RESTORE_TOKEN':
      return {
        ...state,
        token: action.token,
        isLoading: false,
      };

    case 'SIGN_IN': {
      return {
        errorMessage: '',
        token: action.payload,
      };
    }

    case 'SIGN_OUT': {
      return {
        errorMessage: '',
        token: null,
      };
    }

    default:
      return state;
  }
};

As you can see the flow will be showing the welcome screen till the token is loaded from async storage and then based on that show the tabs or the login screen. Also the parameters are passed as props. I've moved the actions to app.js but it can be separated as well.

You can see a fully running sample here
https://snack.expo.io/@guruparan/navigation-sample-3

Hope this helps, Feel free to ask if there are any questions.