问:
How to configure apache httpclient 4.5+ SSLContext to use mutual TLS authentication with a self signed certificate?
I am trying to configure a CloseableHttpClient from httpclient 4.5 with mutual TLS authentication. The server certificate is self-signed.
Here is the code I use (inpired by various StackOverflow posts):

KeyStore trustStore = KeyStore.getInstance("pkcs12");
try (InputStream fis = new FileInputStream(ssl_cert_file)) {
    trustStore.load(fis, password.toCharArray());
}
SSLContext sslContext = SSLContexts.custom()
        .loadKeyMaterial(trustStore, password.toCharArray(), (map, socket) -> "client")
        .loadTrustMaterial(trustStore, new TrustSelfSignedStrategy())
        .build();
SSLConnectionSocketFactory socketFactory = new SSLConnectionSocketFactory(sslContext,  new DefaultHostnameVerifier());
Registry<ConnectionSocketFactory> registry = RegistryBuilder.<ConnectionSocketFactory>create()
        .register("https", socketFactory).build();
connectionManager = new PoolingHttpClientConnectionManager(registry);
```

<p>I Got </p>

```sun.security.provider.certpath.SunCertPathBuilderException:
unable to find valid certification path to requested target
```

<p>Which looks like that the CA certificate I provide is not used I think.
Any idea of what could be wrong ?</p>

<hr>

<p>The CA's certificate, client's certificate and client's private key are in a pkcs12 file
generated with</p>

```openssl pkcs12 -export -in client-cert.pem -inkey private/client-key.pem -certfile cacert.pem -name "client" -out client-cert.p12
```

<p>I tried <code>openssl s_client</code> to check the certificate return the expected output (it does). I suspect the problem is coming from my code rather than the <code>client-cert.p12</code> file.</p>

```openssl s_client -connect host:port -tls1 -cert client-cert.pem -key private/client-key.pem -CAfile cacert.pem
```

答:
As pointed by @Gimby The problem was my CA's certificate was not recognized as a TrustCertEntry in my keystore.

My workaround is to generate a jks file just for the CA's certificate:

```<code>keytool -import -alias client -file cacert.pem -storetype JKS -keystore cacert.jks
</code>```

Build a KeyStore object from it and use it for SSLContext.loadTrustMaterial.

Body:
I am trying to configure a CloseableHttpClient from httpclient 4.5 with mutual TLS authentication. The server certificate is self-signed.
Here is the code I use (inpired by various StackOverflow posts):

KeyStore trustStore = KeyStore.getInstance("pkcs12");
try (InputStream fis = new FileInputStream(ssl_cert_file)) {
    trustStore.load(fis, password.toCharArray());
}
SSLContext sslContext = SSLContexts.custom()
        .loadKeyMaterial(trustStore, password.toCharArray(), (map, socket) -> "client")
        .loadTrustMaterial(trustStore, new TrustSelfSignedStrategy())
        .build();
SSLConnectionSocketFactory socketFactory = new SSLConnectionSocketFactory(sslContext,  new DefaultHostnameVerifier());
Registry<ConnectionSocketFactory> registry = RegistryBuilder.<ConnectionSocketFactory>create()
        .register("https", socketFactory).build();
connectionManager = new PoolingHttpClientConnectionManager(registry);

I Got

sun.security.provider.certpath.SunCertPathBuilderException:
unable to find valid certification path to requested target

Which looks like that the CA certificate I provide is not used I think.
Any idea of what could be wrong ?

The CA's certificate, client's certificate and client's private key are in a pkcs12 file
generated with

openssl pkcs12 -export -in client-cert.pem -inkey private/client-key.pem -certfile cacert.pem -name "client" -out client-cert.p12

I tried openssl s_client to check the certificate return the expected output (it does). I suspect the problem is coming from my code rather than the client-cert.p12 file.

openssl s_client -connect host:port -tls1 -cert client-cert.pem -key private/client-key.pem -CAfile cacert.pem

Body:
As pointed by @Gimby The problem was my CA's certificate was not recognized as a TrustCertEntry in my keystore.

My workaround is to generate a jks file just for the CA's certificate:

keytool -import -alias client -file cacert.pem -storetype JKS -keystore cacert.jks

Build a KeyStore object from it and use it for SSLContext.loadTrustMaterial.


来源: stackexchange

问:
how to create nested object from an array with same key (javascript)
arr = ["sadik", "arif", "rahman"]

I want to create a nested object with the same key but different value like:

```<code>{
  subcategory: {
    name: sadik
    subcategory: {
      name: arif
      subcategory: {
        name: rahman
      }
    }
  }
}

</code>```

my code:

```<code>let arr = ['sadik', 'arif', 'babor']
let obj = {}
arr.forEach((elem) => {
  let a = {}
  a["subcategory"] = {name:elem}
  Object.assign(obj, a)
})
</code>```

i get only last value:

```<code>{
subcategory: {
   name:"babor"
  }
}
</code>```

答:
Your code did not work because you were replacing your "subcategory" key value in each iteration. You have change the reference object to the next nested level each time to get the expected output, see the working snippet below:

const arr = ['level-1', 'level-2', 'level-3']
const obj = {}

let refObj = obj;
arr.forEach( ele => {
  refObj = refObj['subcategory'] || refObj;
  refObj['subcategory'] = { 'name': ele};
})

console.log('output', obj);```

ContentLicense: CC BY-SA 4.0

Body:
arr = ["sadik", "arif", "rahman"]

I want to create a nested object with the same key but different value like:

{
  subcategory: {
    name: sadik
    subcategory: {
      name: arif
      subcategory: {
        name: rahman
      }
    }
  }
}

my code:

let arr = ['sadik', 'arif', 'babor']
let obj = {}
arr.forEach((elem) => {
  let a = {}
  a["subcategory"] = {name:elem}
  Object.assign(obj, a)
})

i get only last value:

{
subcategory: {
   name:"babor"
  }
}


问:
Set MDC properties only for rabbitmq events
I  want to apply filters to log only for  rabbit events using MDC properties.
And set trace Id  && correlation id from the event header.

I already have a RequestResponseLoggingFilter which is used for setting the tenant Id.

I am not sure how to trigger this filter only for async rabbit events.

答:
If you are using a MessageListener or @RabbitListener you can add a MessagePostProcessor to the listener container (or listener container factory, respectively) in the afterReceivePostProcessors property.

The post processor(s) are called after a message is received and before the listener is called.


问:
having trouble writing a dataframe to an sqlite database
sqlite3.InterfaceError: Error binding parameter 2 - probably unsupported type.

Here is the exact code to reproduce the error

go to this page with dev tools open

https://stats.nba.com/teams/boxscores-traditional/

on teamgamelogs endpoint right click -> Copy -> copy as curl(bash)

convert to python requests
https://curl.trillworks.com/

```<code>import pandas as pd
import requests
import sqlite3

response = requests.get('https://stats.nba.com/stats/teamgamelogs?DateFrom=&DateTo=&GameSegment=&LastNGames=0&LeagueID=00&Location=&MeasureType=Base&Month=0&OpponentTeamID=0&Outcome=&PORound=0&PaceAdjust=N&PerMode=Totals&Period=0&PlusMinus=N&Rank=N&Season=2019-20&SeasonSegment=&SeasonType=Regular+Season&ShotClockRange=&VsConference=&VsDivision=', headers=headers, cookies=cookies)

conn = sqlite3.connect('nbastats.db')
stats = response.json()

df = pd.DataFrame(stats["resultSets"])

df.to_sql('Team_stats', conn, if_exists='append')
</code>```

I dont understand why I am getting this error.

```<code>sqlite3.InterfaceError: Error binding parameter 2 - probably unsupported type.
</code>```

答:
Check the returned data types from the response. As we know SQLite 3 only supports below data types.

1)NULL. The value is a NULL value.

2)INTEGER. The value is a signed integer, stored in 1, 2, 3, 4, 6, or 8 bytes depending on the magnitude of the value.

3)REAL. The value is a floating-point value, stored as an 8-byte IEEE floating-point number.

4)TEXT. The value is a text string, stored using the database encoding (UTF-8, UTF-16BE or UTF-16LE).

5)BLOB. The value is a blob of data, stored exactly as it was input.

Reference : https://www.sqlite.org/datatype3.html

Body:
sqlite3.InterfaceError: Error binding parameter 2 - probably unsupported type.

Here is the exact code to reproduce the error

go to this page with dev tools open

https://stats.nba.com/teams/boxscores-traditional/

on teamgamelogs endpoint right click -> Copy -> copy as curl(bash)

convert to python requests
https://curl.trillworks.com/

import pandas as pd
import requests
import sqlite3

response = requests.get('https://stats.nba.com/stats/teamgamelogs?DateFrom=&DateTo=&GameSegment=&LastNGames=0&LeagueID=00&Location=&MeasureType=Base&Month=0&OpponentTeamID=0&Outcome=&PORound=0&PaceAdjust=N&PerMode=Totals&Period=0&PlusMinus=N&Rank=N&Season=2019-20&SeasonSegment=&SeasonType=Regular+Season&ShotClockRange=&VsConference=&VsDivision=', headers=headers, cookies=cookies)

conn = sqlite3.connect('nbastats.db')
stats = response.json()

df = pd.DataFrame(stats["resultSets"])

df.to_sql('Team_stats', conn, if_exists='append')

I dont understand why I am getting this error.

sqlite3.InterfaceError: Error binding parameter 2 - probably unsupported type.


问:
How do I add multiple Keys and Values into dictionary at the same time?
I am trying to add a column as the key and the column to the right of it as the value.

Can I do this without a loop?

I tried:

```<code>analystDict.Add Key:=refWS.Range("A2:A21"), Item:=refWS.Range("B2:B21")
</code>```

When I try to Debug.Print I get a Type mismatch error:

```<code>For Each x In analystDict.Keys
    Debug.Print x, analystDict(x)
Next x
</code>```

答:
You're taking a shortcut that's not allowed; Dictionary.Add is implemented such that it expects one key/value pair, and adds one item to the dictionary. If you need to add multiple items, you need multiple calls to Dictionary.Add - there's no way around it.

A shortcut that would be allowed though, would be to just grab the values in any 2-column Range and turn that into a dictionary, rather than taking any random two arrays that may or may not be the same size.

Make a function that takes a 2D array and turns it into a dictionary by treating the first column as unique keys, and the second column as values.

```<code>Public Function ToDictionary(ByVal keyValuePairs As Variant) As Scripting.Dictionary
    If Not IsArray(keyValuePairs) Then Err.Raise 5
    If GetDimensions(keyValuePairs) <> 2 Then Err.Raise 5 'see https://stackoverflow.com/q/6901991/1188513

    Dim result As Scripting.Dictionary
    Set result = New Scripting.Dictionary

    Const KEYCOL = 1, VALUECOL = 2

    Dim i As Long
    For i = LBound(keyValuePairs, KEYCOL) To UBound(keyValuePairs, KEYCOL)
        If result.Exists(keyValuePairs(i, KEYCOL)) Then Err.Raise 457
        result.Add Key:=keyValuePairs(i, KEYCOL), Item:=keyValuePairs(i, VALUECOL)
    Next

    Set ToDictionary = result
End Function
</code>```

Now you can turn any 2-column Range into a Dictionary like this:

```<code>Dim things As Scripting.Dictionary
Set things = ToDictionary(Sheet1.Range("A2:B21").Value)
</code>```

Note that Range.Value yields a 1-based, 2D Variant array whenever it refers to multiple cells.

Body:
I am trying to add a column as the key and the column to the right of it as the value.

Can I do this without a loop?

I tried:

analystDict.Add Key:=refWS.Range("A2:A21"), Item:=refWS.Range("B2:B21")

When I try to Debug.Print I get a Type mismatch error:

For Each x In analystDict.Keys
    Debug.Print x, analystDict(x)
Next x

Body:
You're taking a shortcut that's not allowed; Dictionary.Add is implemented such that it expects one key/value pair, and adds one item to the dictionary. If you need to add multiple items, you need multiple calls to Dictionary.Add - there's no way around it.

A shortcut that would be allowed though, would be to just grab the values in any 2-column Range and turn that into a dictionary, rather than taking any random two arrays that may or may not be the same size.

Make a function that takes a 2D array and turns it into a dictionary by treating the first column as unique keys, and the second column as values.

Public Function ToDictionary(ByVal keyValuePairs As Variant) As Scripting.Dictionary
    If Not IsArray(keyValuePairs) Then Err.Raise 5
    If GetDimensions(keyValuePairs) <> 2 Then Err.Raise 5 'see https://stackoverflow.com/q/6901991/1188513

    Dim result As Scripting.Dictionary
    Set result = New Scripting.Dictionary

    Const KEYCOL = 1, VALUECOL = 2

    Dim i As Long
    For i = LBound(keyValuePairs, KEYCOL) To UBound(keyValuePairs, KEYCOL)
        If result.Exists(keyValuePairs(i, KEYCOL)) Then Err.Raise 457
        result.Add Key:=keyValuePairs(i, KEYCOL), Item:=keyValuePairs(i, VALUECOL)
    Next

    Set ToDictionary = result
End Function

Now you can turn any 2-column Range into a Dictionary like this:

Dim things As Scripting.Dictionary
Set things = ToDictionary(Sheet1.Range("A2:B21").Value)

Note that Range.Value yields a 1-based, 2D Variant array whenever it refers to multiple cells.


问:
OpenStreetMap API - find & visualize gravel roads
Does the API allow you to retrieve all roads/trails of a given type (eg gravel/unpaved)?
If so, could you point me to some good tutorial that explain it's usage, if better than its own documentation?

I'd like to make a map where gravel/unpaved roads are highlighted, or live on its own layer.

Thanks in advance, and be merciful of the general question - I don't expect an in-depth answer, just a primer for my own research :)

答:
The most popular API for running read-only queries on OpenStreetMap data is Overpass API. The results of an Overpass query can be easily displayed in typical map libraries like Leaflet.js (as well as offline GIS and cartography tools). When you test your Overpass queries, I recommend using the convenient "Overpass Turbo" online editor.

To work with OSM data, it also helps a lot to know the OSM tags (attributes) related to the topic at hand. The OSM wiki documents this kind of information. For your use case, you are probably interested in the highway and surface keys.

As a starting point for your own experiments, this Overpass query looks for highway ways (which includes roads, tracks, paths, etc.) which are known to have a gravel surface.

Of course, there are alternative approaches – for example, you could render your own maps based on a custom map style where roads look different depending on their surface. This would involve very different tooling. But showing Overpass API results in front of a regular OSM map as the background gets you quite far, and may be entirely sufficient for your needs.


问:
How to ajax POST json data along with files?
I'm currently changing a form in a React project to allow uploading of files along with the form. Currently the ajax post looks something like the following:

```<code>return axios.post(url, {
      item: {
        prop: 'test',
        otherProp: 'test2'
        arrayOfObjects: [
          {
            prop: 'test'
          },
          {
            prop: 'test'
          },
        ]
      }
    });
</code>```

Now the problem is I need to send multiple pictures to be uploaded along with the other form data.

Everything I've seen says to use FormData, but the problem is I have lots of nested arrays and object literals, which makes using FormData rather complex.

Is there anyway I can submit multiple files along with my JSON data without increasing the complexity a ton?

I'm also using a rails backend if there's anything I can do on that end to make things easier.

答:
I've solved this by creating a method to convert json data to a FormData object.

```<code>function convertToFormData(data) {
  var fd = new FormData();
  convertToFormDataRecursive(fd, '', data);
  return fd;
}

function convertToFormDataRecursive(formData, key, data) {
  if (moment.isMoment(data)) {
    convertToFormData(formData, key, data.toString());
  }
  else if (data instanceof File) {
    formData.append(key, data, data.name);
  }
  else if (data instanceof Array) {
    for (let i = 0; i < data.length; i++) {
      convertToFormDataRecursive(formData, key + '[' + i + ']', data[i]);
    }
  }
  else if (data instanceof Object) {
    for (let prop in data ) {
      var newKey = (key != '') ? key + '[' + prop + ']' : prop;
      convertToFormDataRecursive(formData, newKey, data[prop]);
    }
  }
  else {
    formData.append(key, data);
  }
}

</code>```

Body:
I'm currently changing a form in a React project to allow uploading of files along with the form. Currently the ajax post looks something like the following:

return axios.post(url, {
      item: {
        prop: 'test',
        otherProp: 'test2'
        arrayOfObjects: [
          {
            prop: 'test'
          },
          {
            prop: 'test'
          },
        ]
      }
    });

Now the problem is I need to send multiple pictures to be uploaded along with the other form data.

Everything I've seen says to use FormData, but the problem is I have lots of nested arrays and object literals, which makes using FormData rather complex.

Is there anyway I can submit multiple files along with my JSON data without increasing the complexity a ton?

I'm also using a rails backend if there's anything I can do on that end to make things easier.

Body:
I've solved this by creating a method to convert json data to a FormData object.

function convertToFormData(data) {
  var fd = new FormData();
  convertToFormDataRecursive(fd, '', data);
  return fd;
}

function convertToFormDataRecursive(formData, key, data) {
  if (moment.isMoment(data)) {
    convertToFormData(formData, key, data.toString());
  }
  else if (data instanceof File) {
    formData.append(key, data, data.name);
  }
  else if (data instanceof Array) {
    for (let i = 0; i < data.length; i++) {
      convertToFormDataRecursive(formData, key + '[' + i + ']', data[i]);
    }
  }
  else if (data instanceof Object) {
    for (let prop in data ) {
      var newKey = (key != '') ? key + '[' + prop + ']' : prop;
      convertToFormDataRecursive(formData, newKey, data[prop]);
    }
  }
  else {
    formData.append(key, data);
  }
}


问:
NumPy - Selecting a submatrix from another 2d array
This is my original 2d array A

```<code>[[0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
 [0, 2, 2, 2, 2, 2, 2, 0, 0, 0],
 [0, 2, 2, 2, 2, 2, 2, 0, 0, 0],
 [0, 2, 2, 2, 2, 2, 2, 0, 0, 0],
 [0, 2, 2, 2, 2, 2, 2, 0, 0, 0],
 [0, 2, 2, 8, 8, 8, 2, 0, 0, 0],
 [0, 2, 2, 8, 8, 8, 2, 0, 0, 0],
 [0, 0, 0, 8, 8, 8, 0, 0, 0, 0],
 [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
 [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]]
</code>```

Let's say I want to return a 3x3 submatrix of 8's at the middle. I made a boolean mask with this expression A == 8 and it looks like this.

```<code>array([[False, False, False, False, False, False, False, False, False,
        False],
       [False, False, False, False, False, False, False, False, False,
        False],
       [False, False, False, False, False, False, False, False, False,
        False],
       [False, False, False, False, False, False, False, False, False,
        False],
       [False, False, False, False, False, False, False, False, False,
        False],
       [False, False, False,  True,  True,  True, False, False, False,
        False],
       [False, False, False,  True,  True,  True, False, False, False,
        False],
       [False, False, False,  True,  True,  True, False, False, False,
        False],
       [False, False, False, False, False, False, False, False, False,
        False],
       [False, False, False, False, False, False, False, False, False,
        False]])
</code>```

This is the point where I'm stuck. How can I return the submatrix with that boolean mask? If I do A[A == 8], it returns a flat array of 8s like this

```<code>array([8, 8, 8, 8, 8, 8, 8, 8, 8])
</code>```

Another way is getting the row and column numbers with np.where(A == 8) which returns (array([5, 5, 5, 6, 6, 6, 7, 7, 7]), array([3, 4, 5, 3, 4, 5, 3, 4, 5])). How can I return the matrix using them?

Is there any better approach for this problem?

答:
As mentioned in comments, A[A==some_value] correctly returns a rank-1 array of values some_value. That makes sense because it's not always the case that the values some_value are organized in a block (take for example some_value = 2) or the number of values some_value is such that they cannot be put into a 2D array.

However If you are sure that there is such a block, you can do the following to get it:

```<code>import numpy as np

inds = np.where(A==8)
slice_x = slice(inds[0].min(), inds[0].max() + 1)  # slice(6, 9, None)
slice_y = slice(inds[1].min(), inds[1].max() + 1)  # slice(3, 6, None)

A[slice_x, slice_y]

# array([[8, 8, 8],
#        [8, 8, 8],
#        [8, 8, 8]])
</code>```

As alternative, you can use scipy.ndimage.find_objects to obtain the slices:

```<code>from scipy import ndimage

slice_x, slice_y = ndimage.find_objects(A==8)[0]  # (slice(6, 9, None), slice(3, 6, None))
</code>```

Body:
This is my original 2d array A

[[0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
 [0, 2, 2, 2, 2, 2, 2, 0, 0, 0],
 [0, 2, 2, 2, 2, 2, 2, 0, 0, 0],
 [0, 2, 2, 2, 2, 2, 2, 0, 0, 0],
 [0, 2, 2, 2, 2, 2, 2, 0, 0, 0],
 [0, 2, 2, 8, 8, 8, 2, 0, 0, 0],
 [0, 2, 2, 8, 8, 8, 2, 0, 0, 0],
 [0, 0, 0, 8, 8, 8, 0, 0, 0, 0],
 [0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
 [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]]

Let's say I want to return a 3x3 submatrix of 8's at the middle. I made a boolean mask with this expression A == 8 and it looks like this.

array([[False, False, False, False, False, False, False, False, False,
        False],
       [False, False, False, False, False, False, False, False, False,
        False],
       [False, False, False, False, False, False, False, False, False,
        False],
       [False, False, False, False, False, False, False, False, False,
        False],
       [False, False, False, False, False, False, False, False, False,
        False],
       [False, False, False,  True,  True,  True, False, False, False,
        False],
       [False, False, False,  True,  True,  True, False, False, False,
        False],
       [False, False, False,  True,  True,  True, False, False, False,
        False],
       [False, False, False, False, False, False, False, False, False,
        False],
       [False, False, False, False, False, False, False, False, False,
        False]])

This is the point where I'm stuck. How can I return the submatrix with that boolean mask? If I do A[A == 8], it returns a flat array of 8s like this

array([8, 8, 8, 8, 8, 8, 8, 8, 8])

Another way is getting the row and column numbers with np.where(A == 8) which returns (array([5, 5, 5, 6, 6, 6, 7, 7, 7]), array([3, 4, 5, 3, 4, 5, 3, 4, 5])). How can I return the matrix using them?

Is there any better approach for this problem?

Body:
As mentioned in comments, A[A==some_value] correctly returns a rank-1 array of values some_value. That makes sense because it's not always the case that the values some_value are organized in a block (take for example some_value = 2) or the number of values some_value is such that they cannot be put into a 2D array.

However If you are sure that there is such a block, you can do the following to get it:

import numpy as np

inds = np.where(A==8)
slice_x = slice(inds[0].min(), inds[0].max() + 1)  # slice(6, 9, None)
slice_y = slice(inds[1].min(), inds[1].max() + 1)  # slice(3, 6, None)

A[slice_x, slice_y]

# array([[8, 8, 8],
#        [8, 8, 8],
#        [8, 8, 8]])

As alternative, you can use scipy.ndimage.find_objects to obtain the slices:

from scipy import ndimage

slice_x, slice_y = ndimage.find_objects(A==8)[0]  # (slice(6, 9, None), slice(3, 6, None))


问:
Broken pipe error when running Gradle test
I've got a problem regarding running tests in gradle. I know that in other machines my gradle config works but in mine unfortunately not. We have got junit test and testNG and both of them, when try to execute, produces stacktrace like this:

```<code>Could not write standard input into: Gradle Worker 1.
java.io.IOException: The pipe is being closed
    at java.io.FileOutputStream.writeBytes(Native Method)
    at java.io.FileOutputStream.write(FileOutputStream.java:318)
    at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
    at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
    at org.gradle.process.internal.streams.ExecOutputHandleRunner.run(ExecOutputHandleRunner.java:53)
    at org.gradle.internal.concurrent.DefaultExecutorFactory$StoppableExecutorImpl$1.run(DefaultExecutorFactory.java:66)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:722)
</code>```

Which continuously will produce this error with different number (Gradle Worker 2, Gradle Worker 3, etc). Has anybody ever faced similar problem?

I'm using gradle 1.6.

EDIT: I forgot to tell that I'm using gradle wrapper

EDIT: After changing to version 1.12 I'm receiving error:

```<code>Caused by: org.gradle.api.InvalidUserDataException: Could not create task '(custome taks name)': Unknown argument(s) in task definition: [mustRunAfter]
11:20:17.990 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.api.internal.project.taskfactory.TaskFactory.validateArgs(TaskFactory.java:147)
11:20:17.991 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.api.internal.project.taskfactory.TaskFactory.checkTaskArgsAndCreateDefaultValues(TaskFactory.java:134)
11:20:17.991 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.api.internal.project.taskfactory.TaskFactory.createTask(TaskFactory.java:72)
</code>```

EDIT: Ok I think I have found what cause this problem - it was Antivirus. When I disable it script moves a little bit forward but ended with:

```<code>org.gradle.messaging.remote.internal.ConnectException: Could not connect to server [e42b57ea-ced6-4bb6-9369-3186ab4983d6 port:63631, addresses:[/127.0.0.1, /0:0:0:0:0:0:0:1]]. Tried addresses: [/127.0.0.1, /0:0:0:0:0:0:0:1].
    at org.gradle.messaging.remote.internal.inet.TcpOutgoingConnector.connect(TcpOutgoingConnector.java:62)
    at org.gradle.messaging.remote.internal.hub.MessageHubBackedClient.getConnection(MessageHubBackedClient.java:35)
    at org.gradle.process.internal.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:54)
    at org.gradle.process.internal.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:35)
    at org.gradle.process.internal.child.ImplementationClassLoaderWorker.execute(ImplementationClassLoaderWorker.java:85)
    at org.gradle.process.internal.child.ImplementationClassLoaderWorker.execute(ImplementationClassLoaderWorker.java:41)
    at org.gradle.process.internal.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:43)
    at org.gradle.process.internal.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:32)
    at org.gradle.process.internal.launcher.BootstrapClassLoaderWorker.call(BootstrapClassLoaderWorker.java:46)
    at org.gradle.process.internal.launcher.BootstrapClassLoaderWorker.call(BootstrapClassLoaderWorker.java:32)
    at jarjar.org.gradle.process.internal.launcher.GradleWorkerMain.run(GradleWorkerMain.java:32)
    at jarjar.org.gradle.process.internal.launcher.GradleWorkerMain.main(GradleWorkerMain.java:37)
Caused by: java.net.ConnectException: Connection refused: connect
</code>```

Searching with google on this error gives me nothing (only some old gradle 1.1 errors info). Anybody faced this kind of problem?

答:
I faced the similar issue while running the test classes. I can able to build the project, but I couldn't able to run, so after some research I found one of the solution, which is to remove the some of the workerThread-jar from the .gradle/cache folder. I tried and it worked for me. You can use below commands to solve this.

In terminal I invoked gradle test --info command,

In response, I receive the logs, where I searched the for the

Gradle worker Daemon

key, to fetch the location of my gradle-worker.jar. Typically in windows you can find this under the following location C:\Users\.gradle\caches\\workerMain\gradle-worker.jar.

To fix this, one approach is to remove the the workerMain folder itself, and run the test classes.

ContentLicense: CC BY-SA 3.0

Body:
I've got a problem regarding running tests in gradle. I know that in other machines my gradle config works but in mine unfortunately not. We have got junit test and testNG and both of them, when try to execute, produces stacktrace like this:

Could not write standard input into: Gradle Worker 1.
java.io.IOException: The pipe is being closed
    at java.io.FileOutputStream.writeBytes(Native Method)
    at java.io.FileOutputStream.write(FileOutputStream.java:318)
    at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
    at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
    at org.gradle.process.internal.streams.ExecOutputHandleRunner.run(ExecOutputHandleRunner.java:53)
    at org.gradle.internal.concurrent.DefaultExecutorFactory$StoppableExecutorImpl$1.run(DefaultExecutorFactory.java:66)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:722)

Which continuously will produce this error with different number (Gradle Worker 2, Gradle Worker 3, etc). Has anybody ever faced similar problem?

I'm using gradle 1.6.

EDIT: I forgot to tell that I'm using gradle wrapper

EDIT: After changing to version 1.12 I'm receiving error:

Caused by: org.gradle.api.InvalidUserDataException: Could not create task '(custome taks name)': Unknown argument(s) in task definition: [mustRunAfter]
11:20:17.990 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.api.internal.project.taskfactory.TaskFactory.validateArgs(TaskFactory.java:147)
11:20:17.991 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.api.internal.project.taskfactory.TaskFactory.checkTaskArgsAndCreateDefaultValues(TaskFactory.java:134)
11:20:17.991 [ERROR] [org.gradle.BuildExceptionReporter]    at org.gradle.api.internal.project.taskfactory.TaskFactory.createTask(TaskFactory.java:72)

EDIT: Ok I think I have found what cause this problem - it was Antivirus. When I disable it script moves a little bit forward but ended with:

org.gradle.messaging.remote.internal.ConnectException: Could not connect to server [e42b57ea-ced6-4bb6-9369-3186ab4983d6 port:63631, addresses:[/127.0.0.1, /0:0:0:0:0:0:0:1]]. Tried addresses: [/127.0.0.1, /0:0:0:0:0:0:0:1].
    at org.gradle.messaging.remote.internal.inet.TcpOutgoingConnector.connect(TcpOutgoingConnector.java:62)
    at org.gradle.messaging.remote.internal.hub.MessageHubBackedClient.getConnection(MessageHubBackedClient.java:35)
    at org.gradle.process.internal.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:54)
    at org.gradle.process.internal.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:35)
    at org.gradle.process.internal.child.ImplementationClassLoaderWorker.execute(ImplementationClassLoaderWorker.java:85)
    at org.gradle.process.internal.child.ImplementationClassLoaderWorker.execute(ImplementationClassLoaderWorker.java:41)
    at org.gradle.process.internal.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:43)
    at org.gradle.process.internal.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:32)
    at org.gradle.process.internal.launcher.BootstrapClassLoaderWorker.call(BootstrapClassLoaderWorker.java:46)
    at org.gradle.process.internal.launcher.BootstrapClassLoaderWorker.call(BootstrapClassLoaderWorker.java:32)
    at jarjar.org.gradle.process.internal.launcher.GradleWorkerMain.run(GradleWorkerMain.java:32)
    at jarjar.org.gradle.process.internal.launcher.GradleWorkerMain.main(GradleWorkerMain.java:37)
Caused by: java.net.ConnectException: Connection refused: connect

Searching with google on this error gives me nothing (only some old gradle 1.1 errors info). Anybody faced this kind of problem?


问:
Error in UseMethod("group_by_") : no applicable method for 'group_by_' applied to an object of class "character"
This is the code:

```<code>PRICE <- as.vector(my.data[,"PRICE"])
ADDRESS_CITY_NAME <- as.vector(my.data[,"ADDRESS_CITY_NAME"])

my.data <- data.frame(ADDRESS_CITY_NAME,log(UNITS), log(PRICE))

my.data %>%
  +group_by(ADDRESS_CITY_NAME)%>%
  +do(tidy(lm(UNITS~PRICE, data=my.data))
</code>```

The data I used (part of it):

```<code>287      INDEPENDENCE  4.1108739  0.42526774
288      INDEPENDENCE  4.1431347  0.47000363
289      INDEPENDENCE  4.2766661  0.17395331
290      INDEPENDENCE  4.0604430  0.46373402
291          LOVELAND  4.3820266  0.38526240
292          LOVELAND  4.2626799  0.42526774
293          LOVELAND  4.1271344  0.51282363
294          LOVELAND  4.8040210  0.43178242
295          LOVELAND  4.1896547  0.37843644
296          LOVELAND  4.2766661  0.32930375
297          LOVELAND  4.4543473  0.39204209
298          LOVELAND  4.3820266  0.39877612
299          LOVELAND  3.7841896  0.43178242
</code>```

I keep getting the error:
Error in UseMethod("group_by_") :    no applicable method for 'group_by_' applied to an object of class "character"

答:
I used part of your data and played with the newer dplyr 0.8.3 syntax

```<code>pacman::p_load(tidyverse, broom)

my.data <- tibble::tribble(
  ~ADDRESS_CITY_NAME,    ~UNITS,     ~PRICE,
      "INDEPENDENCE", 4.1108739, 0.42526774,
      "INDEPENDENCE", 4.1431347, 0.47000363,
      "INDEPENDENCE", 4.2766661, 0.17395331,
      "INDEPENDENCE",  4.060443, 0.46373402,
          "LOVELAND", 4.3820266,  0.3852624,
          "LOVELAND", 4.2626799, 0.42526774,
          "LOVELAND", 4.1271344, 0.51282363,
          "LOVELAND",  4.804021, 0.43178242,
          "LOVELAND", 4.1896547, 0.37843644,
          "LOVELAND", 4.2766661, 0.32930375,
          "LOVELAND", 4.4543473, 0.39204209,
          "LOVELAND", 4.3820266, 0.39877612,
          "LOVELAND", 3.7841896, 0.43178242
  )
</code>```

Then I used syntax found here for new functions making it easier to apply a function for each group.

https://dplyr.tidyverse.org/reference/group_map.html

The following syntax will fit lm oneach group and return a table with coefficients and p values:

```<code>my.data %>%
  group_by(ADDRESS_CITY_NAME)%>%
  group_modify(~ tidy(lm(log(UNITS) ~ log(PRICE), data=.x)))
</code>```

Returns the desired result:

```<code># A tibble: 4 x 6
# Groups:   ADDRESS_CITY_NAME [2]
  ADDRESS_CITY_NAME term        estimate std.error statistic  p.value
  <chr>             <chr>          <dbl>     <dbl>     <dbl>    <dbl>
1 INDEPENDENCE      (Intercept)   1.38      0.0139    98.9   0.000102
2 INDEPENDENCE      log(PRICE)   -0.0426    0.0125    -3.40  0.0767
3 LOVELAND          (Intercept)   1.37      0.179      7.68  0.000118
4 LOVELAND          log(PRICE)   -0.0902    0.197     -0.457 0.662
</code>```

Another way to model on grouped data is nesting the groups. Here I would like to use all the three broom functions and not repeat the fitting:

Credit goes to this magnicent blog: https://drsimonj.svbtle.com/running-a-model-on-separate-groups

```<code>broomed <- my.data %>%
  nest(-ADDRESS_CITY_NAME)%>%
  mutate(fit=map(data, ~ lm(log(UNITS) ~ log(PRICE), data = .)),
         glanced = map(fit, glance),
         augmented = map(fit, augment),
         tidied = map(fit,tidy))

broomed %>% unnest(glanced)

broomed %>% unnest(augmented)

broomed %>% unnest(tidied)
</code>```

Body:
This is the code:

PRICE <- as.vector(my.data[,"PRICE"])
ADDRESS_CITY_NAME <- as.vector(my.data[,"ADDRESS_CITY_NAME"])

my.data <- data.frame(ADDRESS_CITY_NAME,log(UNITS), log(PRICE))

my.data %>%
  +group_by(ADDRESS_CITY_NAME)%>%
  +do(tidy(lm(UNITS~PRICE, data=my.data))

The data I used (part of it):

287      INDEPENDENCE  4.1108739  0.42526774
288      INDEPENDENCE  4.1431347  0.47000363
289      INDEPENDENCE  4.2766661  0.17395331
290      INDEPENDENCE  4.0604430  0.46373402
291          LOVELAND  4.3820266  0.38526240
292          LOVELAND  4.2626799  0.42526774
293          LOVELAND  4.1271344  0.51282363
294          LOVELAND  4.8040210  0.43178242
295          LOVELAND  4.1896547  0.37843644
296          LOVELAND  4.2766661  0.32930375
297          LOVELAND  4.4543473  0.39204209
298          LOVELAND  4.3820266  0.39877612
299          LOVELAND  3.7841896  0.43178242

I keep getting the error:
Error in UseMethod("group_by_") :    no applicable method for 'group_by_' applied to an object of class "character"

Body:
I used part of your data and played with the newer dplyr 0.8.3 syntax

pacman::p_load(tidyverse, broom)

my.data <- tibble::tribble(
  ~ADDRESS_CITY_NAME,    ~UNITS,     ~PRICE,
      "INDEPENDENCE", 4.1108739, 0.42526774,
      "INDEPENDENCE", 4.1431347, 0.47000363,
      "INDEPENDENCE", 4.2766661, 0.17395331,
      "INDEPENDENCE",  4.060443, 0.46373402,
          "LOVELAND", 4.3820266,  0.3852624,
          "LOVELAND", 4.2626799, 0.42526774,
          "LOVELAND", 4.1271344, 0.51282363,
          "LOVELAND",  4.804021, 0.43178242,
          "LOVELAND", 4.1896547, 0.37843644,
          "LOVELAND", 4.2766661, 0.32930375,
          "LOVELAND", 4.4543473, 0.39204209,
          "LOVELAND", 4.3820266, 0.39877612,
          "LOVELAND", 3.7841896, 0.43178242
  )

Then I used syntax found here for new functions making it easier to apply a function for each group.

https://dplyr.tidyverse.org/reference/group_map.html

The following syntax will fit lm oneach group and return a table with coefficients and p values:

my.data %>%
  group_by(ADDRESS_CITY_NAME)%>%
  group_modify(~ tidy(lm(log(UNITS) ~ log(PRICE), data=.x)))

Returns the desired result:

# A tibble: 4 x 6
# Groups:   ADDRESS_CITY_NAME [2]
  ADDRESS_CITY_NAME term        estimate std.error statistic  p.value
  <chr>             <chr>          <dbl>     <dbl>     <dbl>    <dbl>
1 INDEPENDENCE      (Intercept)   1.38      0.0139    98.9   0.000102
2 INDEPENDENCE      log(PRICE)   -0.0426    0.0125    -3.40  0.0767
3 LOVELAND          (Intercept)   1.37      0.179      7.68  0.000118
4 LOVELAND          log(PRICE)   -0.0902    0.197     -0.457 0.662

Another way to model on grouped data is nesting the groups. Here I would like to use all the three broom functions and not repeat the fitting:

Credit goes to this magnicent blog: https://drsimonj.svbtle.com/running-a-model-on-separate-groups

broomed <- my.data %>%
  nest(-ADDRESS_CITY_NAME)%>%
  mutate(fit=map(data, ~ lm(log(UNITS) ~ log(PRICE), data = .)),
         glanced = map(fit, glance),
         augmented = map(fit, augment),
         tidied = map(fit,tidy))

broomed %>% unnest(glanced)

broomed %>% unnest(augmented)

broomed %>% unnest(tidied)


问:
Sequelize condition on joined table doesn't work with limit condition
I have a Supplier model with associated Calendar models.

I want to fetch suppliers who either

have a calendar which is set to available

don't have a calendar

I can do this using the following:

```<code>Supplier.findAll({
  include: [
    {
      model: Calendar,
      as: 'calendars',
      required: false,
      where: {
        start_time: { [Op.lte]: date },
        end_time: { [Op.gte]: date },
      },
    },
  ],
  where: {
    '$calendars.state$': {
      [Op.or]: [
        { [Op.in]: ['available'] },
        { [Op.eq]: null },
      ],
    },
  },
});
</code>```

This generates the following SQL (irrelevant columns removed):

```<code>SELECT
  "suppliers"."uuid"
  ,"calendars"."uuid" AS "calendars.uuid"
  ,"calendars"."state" AS "calendars.state"
FROM "suppliers" AS "suppliers"
LEFT OUTER JOIN "suppliers_calendars" AS "calendars" ON
  "suppliers"."uuid" = "calendars"."supplier_id"
    AND "calendars"."start_time" <= '2019-05-27 23:00:00.000 +00:00'
    AND "calendars"."end_time" >= '2019-05-27 23:00:00.000 +00:00'
WHERE (
  ("calendars"."state" IN ('available')
    OR "calendars"."state" IS NULL
  )
)
ORDER BY "suppliers"."uuid"
;
</code>```

Cool, as expected. Now what happens if I add a limit? I.e.

```<code>Supplier.findAll({
  include: [
    {
      model: Calendar,
      as: 'calendars',
      required: false,
      where: {
        start_time: { [Op.lte]: date },
        end_time: { [Op.gte]: date },
      },
    },
  ],
  where: {
    '$calendars.state$': {
      [Op.or]: [
        { [Op.in]: ['available'] },
        { [Op.eq]: null },
      ],
    },
  },
  limit: 10,
});
</code>```

This produces the following:

```<code>SELECT
    "suppliers".*
    ,"calendars"."uuid" AS "calendars.uuid"
    ,"calendars"."state" AS "calendars.state"
FROM (
    SELECT "suppliers"."uuid"
    FROM "suppliers" AS "suppliers"
    WHERE (
        ("calendars"."state" IN ('available')
        OR "calendars"."state" IS NULL)
    )
    ORDER BY "suppliers"."uuid"
    LIMIT 10
) AS "suppliers"
LEFT OUTER JOIN "suppliers_calendars" AS "calendars" ON
    "suppliers"."uuid" = "calendars"."supplier_id"
    AND "calendars"."start_time" <= '2019-05-27 23:00:00.000 +00:00'
    AND "calendars"."end_time" >= '2019-05-27 23:00:00.000 +00:00'
    ORDER BY "suppliers"."uuid"
</code>```

This is a completely different query, with the main part put into a subquery and the join placed outside. But the where condition on the joined table is put inside the subquery, before the join has occurred, and so fails.

What is the correct approach here?

答:
After about a week of hell found acceptable workaround for my case. Believe it would be helpful as found a lot of unanswered topics/issues on github.

TL;DR; actual solution is at the end of post, just the last piece of code.

The main idea is that Sequelize builds correct SQL query, but when having left joins we produce carthesian product, so there will be a lot of rows as query result.

Example: A and B tables. Many to many relation. If we want to get all A joined with B we will receive A * B rows, so there will be a lot of rows for each record from A with different values from B.

CREATE TABLE IF NOT EXISTS a (
    id INTEGER PRIMARY KEY NOT NULL,
    title VARCHAR
)

CREATE TABLE IF NOT EXISTS b (
    id INTEGER PRIMARY KEY NOT NULL,
    age INTEGER
)

CREATE TABLE IF NOT EXISTS ab (
    id INTEGER PRIMARY KEY NOT NULL,
    aid INTEGER,
    bid INTEGER
)

SELECT *
FROM a
LEFT JOIN (ab JOIN b ON b.id = ab.bid) ON a.id = ab.aid
```

<p>In sequelize syntax:</p>

```class A extends Model {}
A.init({
    id: {
      type: Sequelize.INTEGER,
      autoIncrement: true,
      primaryKey: true,
    },
    title: {
      type: Sequelize.STRING,
    },
});

class B extends Model {}
B.init({
    id: {
      type: Sequelize.INTEGER,
      autoIncrement: true,
      primaryKey: true,
    },
    age: {
      type: Sequelize.INTEGER,
    },
});

A.belongsToMany(B, { foreignKey: ‘aid’, otherKey: ‘bid’, as: ‘ab’ });
B.belongsToMany(A, { foreignKey: ‘bid’, otherKey: ‘aid’, as: ‘ab’ });

A.findAll({
    distinct: true,
    include: [{ association: ‘ab’ }],
})
```

<p>Everything works ok.</p>

<p>So, imagine i want to receive 10 records from A with mapped to them records from B.
When we put LIMIT 10 on this query, Sequelize build correct query but LIMIT is applied to whole query and as result we receive only 10 rows , where all of them could be for only one record from A. Example:</p>

```A.findAll({
    distinct: true,
    include: [{ association: ‘ab’ }],
    limit: 10,
})
```

<p>Which will be converted into:</p>

<pre class="lang-sql prettyprint-override"><code>SELECT *
FROM a
LEFT JOIN (ab JOIN b ON b.id = ab.bid) ON a.id = ab.aid
LIMIT 10

id  |  title    |   id  |  aid  |  bid  |  id   |  age
--- |  -------- | ----- | ----- | ----- | ----- | -----
1   |   first   |   1   |   1   |   1   |   1   |   1
1   |   first   |   2   |   1   |   2   |   2   |   2
1   |   first   |   3   |   1   |   3   |   3   |   3
1   |   first   |   4   |   1   |   4   |   4   |   4
1   |   first   |   5   |   1   |   5   |   5   |   5
2   |   second  |   6   |   2   |   5   |   5   |   5
2   |   second  |   7   |   2   |   4   |   4   |   4
2   |   second  |   8   |   2   |   3   |   3   |   3
2   |   second  |   9   |   2   |   2   |   2   |   2
2   |   second  |   10  |   2   |   1   |   1   |   1
</code>```

After output is received, Seruqlize as ORM will make data mapping and over query result in code will be:

```<code>[
 {
  id: 1,
  title: 'first',
  ab: [
   { id: 1, age:1 },
   { id: 2, age:2 },
   { id: 3, age:3 },
   { id: 4, age:4 },
   { id: 5, age:5 },
  ],
 },
  {
  id: 2,
  title: 'second',
  ab: [
   { id: 5, age:5 },
   { id: 4, age:4 },
   { id: 3, age:3 },
   { id: 2, age:2 },
   { id: 1, age:1 },
  ],
 }
]
</code>```

Obviously NOT what we wanted. I wanted to receive 10 records for A, but received just 2, while i know that there are more in database.

So we have correct SQL query but still received incorrect result.

Ok, i had some ideas but the easiest and most logical is:
1. Make first request with joins, and group results by source table (table on which we are making query and to which making joins) 'id'  property. Seems easy.....

```<code>To make so we need to provide 'group' property to Sequelize query options. Here we have some problems. First - Sequelize makes aliases for each table while generating SQL query. Second - Sequelize puts all columns from JOINED table into SELECT statement of its query and passing __'attributes' = []__ won't help. In both cases we'll receive SQL error.

To solve first we need to convert Model.tableName to singluar form of this word (this logic is based on Sequelize). Just use [pluralize.singular()](https://www.npmjs.com/package/pluralize#usage). Then compose correct property to GROUP BY:
```ts
const tableAlias = pluralize.singular('Industries') // Industry

{
 ...,
 group: [`${tableAlias}.id`]
}
```

To solve second (it was the hardest and the most ... undocumented). We need to use undocumented property 'includeIgnoreAttributes' = false. This will remove all columns from SELECT statement unless we specify some manually. We should manually specify attributes = ['id'] on root query.
</code>```

Now we will receive correctly output with only necessary resources ids. Then we need to compose seconf query WITHOUT limit and offset, but specify additional 'where' clause:

```<code>{
 ...,
 where: {
  ...,
  id: Sequelize.Op.in: [array of ids],
 }
}
</code>```

With query about we can produce correct query with LEFT JOINS.

Solution
Method receives model and original query as arguments and returns correct query + additionally total amount of records in DB for pagination. It also correctly parse query order to provide ability to order by fields from joined tables:

```<code>/**
   *  Workaround for Sequelize illogical behavior when querying with LEFT JOINS and having LIMIT / OFFSET
   *
   *  Here we group by 'id' prop of main (source) model, abd using undocumented 'includeIgnoreAttributes'
   *  Sequelize prop (it is used in its static count() method) in order to get correct SQL request
   *  Witout usage of 'includeIgnoreAttributes' there are a lot of extra invalid columns in SELECT statement
   *
   *  Incorrect example without 'includeIgnoreAttributes'. Here we will get correct SQL query
   *  BUT useless according to business logic:
   *
   *  SELECT "Media"."id", "Solutions->MediaSolutions"."mediaId", "Industries->MediaIndustries"."mediaId",...,
   *  FROM "Medias" AS "Media"
   *  LEFT JOIN ...
   *  WHERE ...
   *  GROUP BY "Media"."id"
   *  ORDER BY ...
   *  LIMIT ...
   *  OFFSET ...
   *
   *  Correct example with 'includeIgnoreAttributes':
   *
   *  SELECT "Media"."id"
   *  FROM "Medias" AS "Media"
   *  LEFT JOIN ...
   *  WHERE ...
   *  GROUP BY "Media"."id"
   *  ORDER BY ...
   *  LIMIT ...
   *  OFFSET ...
   *
   *  @param model - Source model (necessary for getting its tableName for GROUP BY option)
   *  @param query - Parsed and ready to use query object
   */
  private async fixSequeliseQueryWithLeftJoins<C extends Model>(
    model: ModelCtor<C>, query: FindAndCountOptions,
  ): IMsgPromise<{ query: FindAndCountOptions; total?: number }> {
    const fixedQuery: FindAndCountOptions = { ...query };

    // If there is only Tenant data joined -> return original query
    if (query.include && query.include.length === 1 && (query.include[0] as IncludeOptions).model === Tenant) {
      return msg.ok({ query: fixedQuery });
    }

    // Here we need to put it to singular form,
    // because Sequelize gets singular form for models AS aliases in SQL query
    const modelAlias = singular(model.tableName);

    const firstQuery = {
      ...fixedQuery,
      group: [`${modelAlias}.id`],
      attributes: ['id'],
      raw: true,
      includeIgnoreAttributes: false,
      logging: true,
    };

    // Ordering by joined table column - when ordering by joined data need to add it into the group
    if (Array.isArray(firstQuery.order)) {
      firstQuery.order.forEach((item) => {
        if ((item as GenericObject).length === 2) {
          firstQuery.group.push(`${modelAlias}.${(item as GenericObject)[0]}`);
        } else if ((item as GenericObject).length === 3) {
          firstQuery.group.push(`${(item as GenericObject)[0]}.${(item as GenericObject)[1]}`);
        }
      });
    }

    return model.findAndCountAll<C>(firstQuery)
      .then((ids) => {
        if (ids && ids.rows && ids.rows.length) {
          fixedQuery.where = {
            ...fixedQuery.where,
            id: {
              [Op.in]: ids.rows.map((item: GenericObject) => item.id),
            },
          };
          delete fixedQuery.limit;
          delete fixedQuery.offset;
        }

        /* eslint-disable-next-line */
        const total = (ids.count as any).length || ids.count;

        return msg.ok({ query: fixedQuery, total });
      })
      .catch((err) => this.createCustomError(err));
  }
</code>```

Body:
I have a Supplier model with associated Calendar models.

I want to fetch suppliers who either

have a calendar which is set to available

don't have a calendar

I can do this using the following:

Supplier.findAll({
  include: [
    {
      model: Calendar,
      as: 'calendars',
      required: false,
      where: {
        start_time: { [Op.lte]: date },
        end_time: { [Op.gte]: date },
      },
    },
  ],
  where: {
    '$calendars.state$': {
      [Op.or]: [
        { [Op.in]: ['available'] },
        { [Op.eq]: null },
      ],
    },
  },
});

This generates the following SQL (irrelevant columns removed):

SELECT
  "suppliers"."uuid"
  ,"calendars"."uuid" AS "calendars.uuid"
  ,"calendars"."state" AS "calendars.state"
FROM "suppliers" AS "suppliers"
LEFT OUTER JOIN "suppliers_calendars" AS "calendars" ON
  "suppliers"."uuid" = "calendars"."supplier_id"
    AND "calendars"."start_time" <= '2019-05-27 23:00:00.000 +00:00'
    AND "calendars"."end_time" >= '2019-05-27 23:00:00.000 +00:00'
WHERE (
  ("calendars"."state" IN ('available')
    OR "calendars"."state" IS NULL
  )
)
ORDER BY "suppliers"."uuid"
;

Cool, as expected. Now what happens if I add a limit? I.e.

Supplier.findAll({
  include: [
    {
      model: Calendar,
      as: 'calendars',
      required: false,
      where: {
        start_time: { [Op.lte]: date },
        end_time: { [Op.gte]: date },
      },
    },
  ],
  where: {
    '$calendars.state$': {
      [Op.or]: [
        { [Op.in]: ['available'] },
        { [Op.eq]: null },
      ],
    },
  },
  limit: 10,
});

This produces the following:

SELECT
    "suppliers".*
    ,"calendars"."uuid" AS "calendars.uuid"
    ,"calendars"."state" AS "calendars.state"
FROM (
    SELECT "suppliers"."uuid"
    FROM "suppliers" AS "suppliers"
    WHERE (
        ("calendars"."state" IN ('available')
        OR "calendars"."state" IS NULL)
    )
    ORDER BY "suppliers"."uuid"
    LIMIT 10
) AS "suppliers"
LEFT OUTER JOIN "suppliers_calendars" AS "calendars" ON
    "suppliers"."uuid" = "calendars"."supplier_id"
    AND "calendars"."start_time" <= '2019-05-27 23:00:00.000 +00:00'
    AND "calendars"."end_time" >= '2019-05-27 23:00:00.000 +00:00'
    ORDER BY "suppliers"."uuid"

This is a completely different query, with the main part put into a subquery and the join placed outside. But the where condition on the joined table is put inside the subquery, before the join has occurred, and so fails.

What is the correct approach here?

Body:
After about a week of hell found acceptable workaround for my case. Believe it would be helpful as found a lot of unanswered topics/issues on github.

TL;DR; actual solution is at the end of post, just the last piece of code.

The main idea is that Sequelize builds correct SQL query, but when having left joins we produce carthesian product, so there will be a lot of rows as query result.

Example: A and B tables. Many to many relation. If we want to get all A joined with B we will receive A * B rows, so there will be a lot of rows for each record from A with different values from B.

CREATE TABLE IF NOT EXISTS a (
    id INTEGER PRIMARY KEY NOT NULL,
    title VARCHAR
)

CREATE TABLE IF NOT EXISTS b (
    id INTEGER PRIMARY KEY NOT NULL,
    age INTEGER
)

CREATE TABLE IF NOT EXISTS ab (
    id INTEGER PRIMARY KEY NOT NULL,
    aid INTEGER,
    bid INTEGER
)

SELECT *
FROM a
LEFT JOIN (ab JOIN b ON b.id = ab.bid) ON a.id = ab.aid

In sequelize syntax:

class A extends Model {}
A.init({
    id: {
      type: Sequelize.INTEGER,
      autoIncrement: true,
      primaryKey: true,
    },
    title: {
      type: Sequelize.STRING,
    },
});

class B extends Model {}
B.init({
    id: {
      type: Sequelize.INTEGER,
      autoIncrement: true,
      primaryKey: true,
    },
    age: {
      type: Sequelize.INTEGER,
    },
});

A.belongsToMany(B, { foreignKey: ‘aid’, otherKey: ‘bid’, as: ‘ab’ });
B.belongsToMany(A, { foreignKey: ‘bid’, otherKey: ‘aid’, as: ‘ab’ });

A.findAll({
    distinct: true,
    include: [{ association: ‘ab’ }],
})

Everything works ok.

So, imagine i want to receive 10 records from A with mapped to them records from B.
When we put LIMIT 10 on this query, Sequelize build correct query but LIMIT is applied to whole query and as result we receive only 10 rows , where all of them could be for only one record from A. Example:

A.findAll({
    distinct: true,
    include: [{ association: ‘ab’ }],
    limit: 10,
})

Which will be converted into:

SELECT *
FROM a
LEFT JOIN (ab JOIN b ON b.id = ab.bid) ON a.id = ab.aid
LIMIT 10

id  |  title    |   id  |  aid  |  bid  |  id   |  age
--- |  -------- | ----- | ----- | ----- | ----- | -----
1   |   first   |   1   |   1   |   1   |   1   |   1
1   |   first   |   2   |   1   |   2   |   2   |   2
1   |   first   |   3   |   1   |   3   |   3   |   3
1   |   first   |   4   |   1   |   4   |   4   |   4
1   |   first   |   5   |   1   |   5   |   5   |   5
2   |   second  |   6   |   2   |   5   |   5   |   5
2   |   second  |   7   |   2   |   4   |   4   |   4
2   |   second  |   8   |   2   |   3   |   3   |   3
2   |   second  |   9   |   2   |   2   |   2   |   2
2   |   second  |   10  |   2   |   1   |   1   |   1

After output is received, Seruqlize as ORM will make data mapping and over query result in code will be:

[
 {
  id: 1,
  title: 'first',
  ab: [
   { id: 1, age:1 },
   { id: 2, age:2 },
   { id: 3, age:3 },
   { id: 4, age:4 },
   { id: 5, age:5 },
  ],
 },
  {
  id: 2,
  title: 'second',
  ab: [
   { id: 5, age:5 },
   { id: 4, age:4 },
   { id: 3, age:3 },
   { id: 2, age:2 },
   { id: 1, age:1 },
  ],
 }
]

Obviously NOT what we wanted. I wanted to receive 10 records for A, but received just 2, while i know that there are more in database.

So we have correct SQL query but still received incorrect result.

Ok, i had some ideas but the easiest and most logical is:
1. Make first request with joins, and group results by source table (table on which we are making query and to which making joins) 'id'  property. Seems easy.....

To make so we need to provide 'group' property to Sequelize query options. Here we have some problems. First - Sequelize makes aliases for each table while generating SQL query. Second - Sequelize puts all columns from JOINED table into SELECT statement of its query and passing __'attributes' = []__ won't help. In both cases we'll receive SQL error.

To solve first we need to convert Model.tableName to singluar form of this word (this logic is based on Sequelize). Just use [pluralize.singular()](https://www.npmjs.com/package/pluralize#usage). Then compose correct property to GROUP BY:
```ts
const tableAlias = pluralize.singular('Industries') // Industry

{
 ...,
 group: [`${tableAlias}.id`]
}
```

To solve second (it was the hardest and the most ... undocumented). We need to use undocumented property 'includeIgnoreAttributes' = false. This will remove all columns from SELECT statement unless we specify some manually. We should manually specify attributes = ['id'] on root query.

Now we will receive correctly output with only necessary resources ids. Then we need to compose seconf query WITHOUT limit and offset, but specify additional 'where' clause:

{
 ...,
 where: {
  ...,
  id: Sequelize.Op.in: [array of ids],
 }
}

With query about we can produce correct query with LEFT JOINS.

Solution
Method receives model and original query as arguments and returns correct query + additionally total amount of records in DB for pagination. It also correctly parse query order to provide ability to order by fields from joined tables:

/**
   *  Workaround for Sequelize illogical behavior when querying with LEFT JOINS and having LIMIT / OFFSET
   *
   *  Here we group by 'id' prop of main (source) model, abd using undocumented 'includeIgnoreAttributes'
   *  Sequelize prop (it is used in its static count() method) in order to get correct SQL request
   *  Witout usage of 'includeIgnoreAttributes' there are a lot of extra invalid columns in SELECT statement
   *
   *  Incorrect example without 'includeIgnoreAttributes'. Here we will get correct SQL query
   *  BUT useless according to business logic:
   *
   *  SELECT "Media"."id", "Solutions->MediaSolutions"."mediaId", "Industries->MediaIndustries"."mediaId",...,
   *  FROM "Medias" AS "Media"
   *  LEFT JOIN ...
   *  WHERE ...
   *  GROUP BY "Media"."id"
   *  ORDER BY ...
   *  LIMIT ...
   *  OFFSET ...
   *
   *  Correct example with 'includeIgnoreAttributes':
   *
   *  SELECT "Media"."id"
   *  FROM "Medias" AS "Media"
   *  LEFT JOIN ...
   *  WHERE ...
   *  GROUP BY "Media"."id"
   *  ORDER BY ...
   *  LIMIT ...
   *  OFFSET ...
   *
   *  @param model - Source model (necessary for getting its tableName for GROUP BY option)
   *  @param query - Parsed and ready to use query object
   */
  private async fixSequeliseQueryWithLeftJoins<C extends Model>(
    model: ModelCtor<C>, query: FindAndCountOptions,
  ): IMsgPromise<{ query: FindAndCountOptions; total?: number }> {
    const fixedQuery: FindAndCountOptions = { ...query };

    // If there is only Tenant data joined -> return original query
    if (query.include && query.include.length === 1 && (query.include[0] as IncludeOptions).model === Tenant) {
      return msg.ok({ query: fixedQuery });
    }

    // Here we need to put it to singular form,
    // because Sequelize gets singular form for models AS aliases in SQL query
    const modelAlias = singular(model.tableName);

    const firstQuery = {
      ...fixedQuery,
      group: [`${modelAlias}.id`],
      attributes: ['id'],
      raw: true,
      includeIgnoreAttributes: false,
      logging: true,
    };

    // Ordering by joined table column - when ordering by joined data need to add it into the group
    if (Array.isArray(firstQuery.order)) {
      firstQuery.order.forEach((item) => {
        if ((item as GenericObject).length === 2) {
          firstQuery.group.push(`${modelAlias}.${(item as GenericObject)[0]}`);
        } else if ((item as GenericObject).length === 3) {
          firstQuery.group.push(`${(item as GenericObject)[0]}.${(item as GenericObject)[1]}`);
        }
      });
    }

    return model.findAndCountAll<C>(firstQuery)
      .then((ids) => {
        if (ids && ids.rows && ids.rows.length) {
          fixedQuery.where = {
            ...fixedQuery.where,
            id: {
              [Op.in]: ids.rows.map((item: GenericObject) => item.id),
            },
          };
          delete fixedQuery.limit;
          delete fixedQuery.offset;
        }

        /* eslint-disable-next-line */
        const total = (ids.count as any).length || ids.count;

        return msg.ok({ query: fixedQuery, total });
      })
      .catch((err) => this.createCustomError(err));
  }


问:
Webgl - Rendering depth to texture returns gl.INVALID_FRAMEBUFFER_OPERATION in Safari
If you look at the following snippet in Safari (taken from https://webglfundamentals.org/webgl/lessons/webgl-shadows.html), you will notice it looks different than in other browsers, because the depth texture is always completely black:

'use strict';

function main() {
  // Get A WebGL context
  /** @type {HTMLCanvasElement} */
  const canvas = document.getElementById('canvas');
  const gl = canvas.getContext('webgl');
  if (!gl) {
    return;
  }

  const ext = gl.getExtension('WEBGL_depth_texture');
  if (!ext) {
    return alert('need WEBGL_depth_texture');  // eslint-disable-line
  }

  // setup GLSL programs
  const textureProgramInfo = webglUtils.createProgramInfo(gl, ['3d-vertex-shader', '3d-fragment-shader']);
  const colorProgramInfo = webglUtils.createProgramInfo(gl, ['color-vertex-shader', 'color-fragment-shader']);

  const sphereBufferInfo = primitives.createSphereBufferInfo(
      gl,
      1,  // radius
      32, // subdivisions around
      24, // subdivisions down
  );
  const planeBufferInfo = primitives.createPlaneBufferInfo(
      gl,
      20,  // width
      20,  // height
      1,   // subdivisions across
      1,   // subdivisions down
  );
  const cubeBufferInfo = primitives.createCubeBufferInfo(
      gl,
      2,  // size
  );
  const cubeLinesBufferInfo = webglUtils.createBufferInfoFromArrays(gl, {
    position: [
      -1, -1, -1,
       1, -1, -1,
      -1,  1, -1,
       1,  1, -1,
      -1, -1,  1,
       1, -1,  1,
      -1,  1,  1,
       1,  1,  1,
    ],
    indices: [
      0, 1,
      1, 3,
      3, 2,
      2, 0,

      4, 5,
      5, 7,
      7, 6,
      6, 4,

      0, 4,
      1, 5,
      3, 7,
      2, 6,
    ],
  });

  // make a 8x8 checkerboard texture
  const checkerboardTexture = gl.createTexture();
  gl.bindTexture(gl.TEXTURE_2D, checkerboardTexture);
  gl.texImage2D(
      gl.TEXTURE_2D,
      0,                // mip level
      gl.LUMINANCE,     // internal format
      8,                // width
      8,                // height
      0,                // border
      gl.LUMINANCE,     // format
      gl.UNSIGNED_BYTE, // type
      new Uint8Array([  // data
        0xFF, 0xCC, 0xFF, 0xCC, 0xFF, 0xCC, 0xFF, 0xCC,
        0xCC, 0xFF, 0xCC, 0xFF, 0xCC, 0xFF, 0xCC, 0xFF,
        0xFF, 0xCC, 0xFF, 0xCC, 0xFF, 0xCC, 0xFF, 0xCC,
        0xCC, 0xFF, 0xCC, 0xFF, 0xCC, 0xFF, 0xCC, 0xFF,
        0xFF, 0xCC, 0xFF, 0xCC, 0xFF, 0xCC, 0xFF, 0xCC,
        0xCC, 0xFF, 0xCC, 0xFF, 0xCC, 0xFF, 0xCC, 0xFF,
        0xFF, 0xCC, 0xFF, 0xCC, 0xFF, 0xCC, 0xFF, 0xCC,
        0xCC, 0xFF, 0xCC, 0xFF, 0xCC, 0xFF, 0xCC, 0xFF,
      ]));
  gl.generateMipmap(gl.TEXTURE_2D);
  gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.NEAREST);

  const depthTexture = gl.createTexture();
  const depthTextureSize = 512;
  gl.bindTexture(gl.TEXTURE_2D, depthTexture);
  gl.texImage2D(
      gl.TEXTURE_2D,      // target
      0,                  // mip level
      gl.DEPTH_COMPONENT, // internal format
      depthTextureSize,   // width
      depthTextureSize,   // height
      0,                  // border
      gl.DEPTH_COMPONENT, // format
      gl.UNSIGNED_INT,    // type
      null);              // data
  gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.NEAREST);
  gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.NEAREST);
  gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
  gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);

  const depthFramebuffer = gl.createFramebuffer();
  gl.bindFramebuffer(gl.FRAMEBUFFER, depthFramebuffer);
  gl.framebufferTexture2D(
      gl.FRAMEBUFFER,       // target
      gl.DEPTH_ATTACHMENT,  // attachment point
      gl.TEXTURE_2D,        // texture target
      depthTexture,         // texture
      0);                   // mip level

  function degToRad(d) {
    return d * Math.PI / 180;
  }

  const settings = {
    cameraX: 6,
    cameraY: 5,
    posX: 2.5,
    posY: 4.8,
    posZ: 4.3,
    targetX: 2.5,
    targetY: 0,
    targetZ: 3.5,
    projWidth: 1,
    projHeight: 1,
    perspective: true,
    fieldOfView: 120,
  };
  webglLessonsUI.setupUI(document.querySelector('#ui'), settings, [
    { type: 'slider',   key: 'cameraX',    min: -10, max: 10, change: render, precision: 2, step: 0.001, },
    { type: 'slider',   key: 'cameraY',    min:   1, max: 20, change: render, precision: 2, step: 0.001, },
    { type: 'slider',   key: 'posX',       min: -10, max: 10, change: render, precision: 2, step: 0.001, },
    { type: 'slider',   key: 'posY',       min:   1, max: 20, change: render, precision: 2, step: 0.001, },
    { type: 'slider',   key: 'posZ',       min:   1, max: 20, change: render, precision: 2, step: 0.001, },
    { type: 'slider',   key: 'targetX',    min: -10, max: 10, change: render, precision: 2, step: 0.001, },
    { type: 'slider',   key: 'targetY',    min:   0, max: 20, change: render, precision: 2, step: 0.001, },
    { type: 'slider',   key: 'targetZ',    min: -10, max: 20, change: render, precision: 2, step: 0.001, },
    { type: 'slider',   key: 'projWidth',  min:   0, max:  2, change: render, precision: 2, step: 0.001, },
    { type: 'slider',   key: 'projHeight', min:   0, max:  2, change: render, precision: 2, step: 0.001, },
    { type: 'checkbox', key: 'perspective', change: render, },
    { type: 'slider',   key: 'fieldOfView', min:  1, max: 179, change: render, },
  ]);

  const fieldOfViewRadians = degToRad(60);

  // Uniforms for each object.
  const planeUniforms = {
    u_colorMult: [0.5, 0.5, 1, 1],  // lightblue
    u_color: [1, 0, 0, 1],
    u_texture: checkerboardTexture,
    u_world: m4.translation(0, 0, 0),
  };
  const sphereUniforms = {
    u_colorMult: [1, 0.5, 0.5, 1],  // pink
    u_color: [0, 0, 1, 1],
    u_texture: checkerboardTexture,
    u_world: m4.translation(2, 3, 4),
  };
  const cubeUniforms = {
    u_colorMult: [0.5, 1, 0.5, 1],  // lightgreen
    u_color: [0, 0, 1, 1],
    u_texture: checkerboardTexture,
    u_world: m4.translation(3, 1, 0),
  };

  function drawScene(projectionMatrix, cameraMatrix, textureMatrix, programInfo) {
    // Make a view matrix from the camera matrix.
    const viewMatrix = m4.inverse(cameraMatrix);

    gl.useProgram(programInfo.program);

    // set uniforms that are the same for both the sphere and plane
    // note: any values with no corresponding uniform in the shader
    // are ignored.
    webglUtils.setUniforms(programInfo, {
      u_view: viewMatrix,
      u_projection: projectionMatrix,
      u_textureMatrix: textureMatrix,
      u_projectedTexture: depthTexture,
    });

    // ------ Draw the sphere --------

    // Setup all the needed attributes.
    webglUtils.setBuffersAndAttributes(gl, programInfo, sphereBufferInfo);

    // Set the uniforms unique to the sphere
    webglUtils.setUniforms(programInfo, sphereUniforms);

    // calls gl.drawArrays or gl.drawElements
    webglUtils.drawBufferInfo(gl, sphereBufferInfo);

    // ------ Draw the cube --------

    // Setup all the needed attributes.
    webglUtils.setBuffersAndAttributes(gl, programInfo, cubeBufferInfo);

    // Set the uniforms unique to the cube
    webglUtils.setUniforms(programInfo, cubeUniforms);

    // calls gl.drawArrays or gl.drawElements
    webglUtils.drawBufferInfo(gl, cubeBufferInfo);

    // ------ Draw the plane --------

    // Setup all the needed attributes.
    webglUtils.setBuffersAndAttributes(gl, programInfo, planeBufferInfo);

    // Set the uniforms unique to the cube
    webglUtils.setUniforms(programInfo, planeUniforms);

    // calls gl.drawArrays or gl.drawElements
    webglUtils.drawBufferInfo(gl, planeBufferInfo);
  }

  // Draw the scene.
  function render() {
    webglUtils.resizeCanvasToDisplaySize(gl.canvas);

    gl.enable(gl.CULL_FACE);
    gl.enable(gl.DEPTH_TEST);

    // first draw from the POV of the light
    const lightWorldMatrix = m4.lookAt(
        [settings.posX, settings.posY, settings.posZ],          // position
        [settings.targetX, settings.targetY, settings.targetZ], // target
        [0, 1, 0],                                              // up
    );
    const lightProjectionMatrix = settings.perspective
        ? m4.perspective(
            degToRad(settings.fieldOfView),
            settings.projWidth / settings.projHeight,
            0.5,  // near
            10)   // far
        : m4.orthographic(
            -settings.projWidth / 2,   // left
             settings.projWidth / 2,   // right
            -settings.projHeight / 2,  // bottom
             settings.projHeight / 2,  // top
             0.5,                      // near
             10);                      // far

    // draw to the depth texture
    gl.bindFramebuffer(gl.FRAMEBUFFER, depthFramebuffer);
    gl.viewport(0, 0, depthTextureSize, depthTextureSize);
    gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT);

    drawScene(lightProjectionMatrix, lightWorldMatrix, m4.identity(), colorProgramInfo);

    // now draw scene to the canvas projecting the depth texture into the scene
    gl.bindFramebuffer(gl.FRAMEBUFFER, null);
    gl.viewport(0, 0, gl.canvas.width, gl.canvas.height);
    gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT);

    let textureMatrix = m4.identity();
    textureMatrix = m4.translate(textureMatrix, 0.5, 0.5, 0.5);
    textureMatrix = m4.scale(textureMatrix, 0.5, 0.5, 0.5);
    textureMatrix = m4.multiply(textureMatrix, lightProjectionMatrix);
    // use the inverse of this world matrix to make
    // a matrix that will transform other positions
    // to be relative this this world space.
    textureMatrix = m4.multiply(
        textureMatrix,
        m4.inverse(lightWorldMatrix));

    // Compute the projection matrix
    const aspect = gl.canvas.clientWidth / gl.canvas.clientHeight;
    const projectionMatrix =
        m4.perspective(fieldOfViewRadians, aspect, 1, 2000);

    // Compute the camera's matrix using look at.
    const cameraPosition = [settings.cameraX, settings.cameraY, 7];
    const target = [0, 0, 0];
    const up = [0, 1, 0];
    const cameraMatrix = m4.lookAt(cameraPosition, target, up);

    drawScene(projectionMatrix, cameraMatrix, textureMatrix, textureProgramInfo);

    // ------ Draw the frustum ------
    {
      const viewMatrix = m4.inverse(cameraMatrix);

      gl.useProgram(colorProgramInfo.program);

      // Setup all the needed attributes.
      webglUtils.setBuffersAndAttributes(gl, colorProgramInfo, cubeLinesBufferInfo);

      // scale the cube in Z so it's really long
      // to represent the texture is being projected to
      // infinity
      const mat = m4.multiply(
          lightWorldMatrix, m4.inverse(lightProjectionMatrix));

      // Set the uniforms we just computed
      webglUtils.setUniforms(colorProgramInfo, {
        u_color: [0, 0, 0, 1],
        u_view: viewMatrix,
        u_projection: projectionMatrix,
        u_world: mat,
      });

      // calls gl.drawArrays or gl.drawElements
      webglUtils.drawBufferInfo(gl, cubeLinesBufferInfo, gl.LINES);
    }
  }
  render();
}

main();```
<pre class="snippet-code-css lang-css prettyprint-override"><code>@import url("https://webglfundamentals.org/webgl/resources/webgl-tutorials.css");
body {
  margin: 0;
}
canvas {
  width: 100vw;
  height: 100vh;
  display: block;
}
.gman-widget-value {
  min-width: 5em;
}</code>```

<canvas id="canvas"></canvas>

```

I've based my own implementations on this, and run into the same issue. WebGL returns a gl.INVALID_FRAMEBUFFER_OPERATION when calling getError after a call to gl.clea or to gl.drawElements. I assume something is wrong with the setup of the framebuffer, but nothing I tried helps. Thanks in advance!

答:
For Safari pre version 15 you need to setup a color attachment even if though the sample doesn't use it

first create an RGBA/UNSIGNED_BYTE color attachment the same size as the depth texture

```<code>  const dummy = gl.createTexture();
  gl.bindTexture(gl.TEXTURE_2D, dummy);
  gl.texImage2D(
      gl.TEXTURE_2D,
      0,
      gl.RGBA,
      depthTextureSize,
      depthTextureSize,
      0,
      gl.RGBA,
      gl.UNSIGNED_BYTE,
      null,
  );
  gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.NEAREST);
  gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.NEAREST);
  gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
  gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
</code>```

Then attach it to the framebuffer

```<code>  const depthFramebuffer = gl.createFramebuffer();
  gl.bindFramebuffer(gl.FRAMEBUFFER, depthFramebuffer);

  gl.framebufferTexture2D(
      gl.FRAMEBUFFER,       // target
      gl.DEPTH_ATTACHMENT,  // attachment point
      gl.TEXTURE_2D,        // texture target
      depthTexture,         // texture
      0);                   // mip level

  // ---- added ----
  gl.framebufferTexture2D(
      gl.FRAMEBUFFER,
      gl.COLOR_ATTACHMENT0,
      gl.TEXTURE_2D,
      dummy,
      0,
  )
</code>```

Unfortunately this is not a bug in Safari. It's a reflection of unfortunate choices of the OpenGL ES 2.0 spec which says

4.4.5 Framebuffer Completeness

...

Framebuffer Completeness

...

The framebuffer object target is said to be framebuffer complete if it is the window-system-provided framebuffer, or if all the following conditons are true:

In this subsection, each rule is followed by an error enum in bold

...

The combination of internal formats of the attached images does not violate an implementation-dependent set of restrictions.

FRAMEBUFFER_UNSUPPORTED

In other words, in OpenGL ES 2.0 NO COMBINATIONS OF FRAMEBUFFER ATTACHMENTS ARE REQUIRED TO WORK!!!!!!!

The WebGL spec itself added the requirement that 3 combinations are required to work

6.8 Framebuffer Object Attachments

...

The following combinations of framebuffer object attachments, when all of the attachments are framebuffer attachment complete, non-zero, and have the same width and height, must result in the framebuffer being framebuffer complete:

COLOR_ATTACHMENT0 = RGBA/UNSIGNED_BYTE texture

COLOR_ATTACHMENT0 = RGBA/UNSIGNED_BYTE texture + DEPTH_ATTACHMENT = DEPTH_COMPONENT16 renderbuffer

COLOR_ATTACHMENT0 = RGBA/UNSIGNED_BYTE texture + DEPTH_STENCIL_ATTACHMENT = DEPTH_STENCIL renderbuffer

So, following the 2 specs, even though the WEBGL_depth_texture extension adds depth textures, none of them are required to work as attachments in framebuffers with or without any other attachments!!!

Yes, you read that right. The WEBGL_depth_texture extension spec says you can create them and attach them. The OpenGL ES 2.0 spec says they are not required to work as attachments or rather that it's up to the implementation whether any combinations work. WebGL lists 3 combinations that are required to work but those 3 combinations do not include depth textures, only depth renderbuffers.

Fortunately it appears they do work everywhere if you add an RGBA/UNSIGNED_BYTE color attachment.

On a positive note, OpenGL ES 3.0 and WebGL2 list many more combinations that are required to work.

Body:
If you look at the following snippet in Safari (taken from https://webglfundamentals.org/webgl/lessons/webgl-shadows.html), you will notice it looks different than in other browsers, because the depth texture is always completely black:

'use strict';

function main() {
  // Get A WebGL context
  /** @type {HTMLCanvasElement} */
  const canvas = document.getElementById('canvas');
  const gl = canvas.getContext('webgl');
  if (!gl) {
    return;
  }

  const ext = gl.getExtension('WEBGL_depth_texture');
  if (!ext) {
    return alert('need WEBGL_depth_texture');  // eslint-disable-line
  }

  // setup GLSL programs
  const textureProgramInfo = webglUtils.createProgramInfo(gl, ['3d-vertex-shader', '3d-fragment-shader']);
  const colorProgramInfo = webglUtils.createProgramInfo(gl, ['color-vertex-shader', 'color-fragment-shader']);

  const sphereBufferInfo = primitives.createSphereBufferInfo(
      gl,
      1,  // radius
      32, // subdivisions around
      24, // subdivisions down
  );
  const planeBufferInfo = primitives.createPlaneBufferInfo(
      gl,
      20,  // width
      20,  // height
      1,   // subdivisions across
      1,   // subdivisions down
  );
  const cubeBufferInfo = primitives.createCubeBufferInfo(
      gl,
      2,  // size
  );
  const cubeLinesBufferInfo = webglUtils.createBufferInfoFromArrays(gl, {
    position: [
      -1, -1, -1,
       1, -1, -1,
      -1,  1, -1,
       1,  1, -1,
      -1, -1,  1,
       1, -1,  1,
      -1,  1,  1,
       1,  1,  1,
    ],
    indices: [
      0, 1,
      1, 3,
      3, 2,
      2, 0,

      4, 5,
      5, 7,
      7, 6,
      6, 4,

      0, 4,
      1, 5,
      3, 7,
      2, 6,
    ],
  });

  // make a 8x8 checkerboard texture
  const checkerboardTexture = gl.createTexture();
  gl.bindTexture(gl.TEXTURE_2D, checkerboardTexture);
  gl.texImage2D(
      gl.TEXTURE_2D,
      0,                // mip level
      gl.LUMINANCE,     // internal format
      8,                // width
      8,                // height
      0,                // border
      gl.LUMINANCE,     // format
      gl.UNSIGNED_BYTE, // type
      new Uint8Array([  // data
        0xFF, 0xCC, 0xFF, 0xCC, 0xFF, 0xCC, 0xFF, 0xCC,
        0xCC, 0xFF, 0xCC, 0xFF, 0xCC, 0xFF, 0xCC, 0xFF,
        0xFF, 0xCC, 0xFF, 0xCC, 0xFF, 0xCC, 0xFF, 0xCC,
        0xCC, 0xFF, 0xCC, 0xFF, 0xCC, 0xFF, 0xCC, 0xFF,
        0xFF, 0xCC, 0xFF, 0xCC, 0xFF, 0xCC, 0xFF, 0xCC,
        0xCC, 0xFF, 0xCC, 0xFF, 0xCC, 0xFF, 0xCC, 0xFF,
        0xFF, 0xCC, 0xFF, 0xCC, 0xFF, 0xCC, 0xFF, 0xCC,
        0xCC, 0xFF, 0xCC, 0xFF, 0xCC, 0xFF, 0xCC, 0xFF,
      ]));
  gl.generateMipmap(gl.TEXTURE_2D);
  gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.NEAREST);

  const depthTexture = gl.createTexture();
  const depthTextureSize = 512;
  gl.bindTexture(gl.TEXTURE_2D, depthTexture);
  gl.texImage2D(
      gl.TEXTURE_2D,      // target
      0,                  // mip level
      gl.DEPTH_COMPONENT, // internal format
      depthTextureSize,   // width
      depthTextureSize,   // height
      0,                  // border
      gl.DEPTH_COMPONENT, // format
      gl.UNSIGNED_INT,    // type
      null);              // data
  gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.NEAREST);
  gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.NEAREST);
  gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
  gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);

  const depthFramebuffer = gl.createFramebuffer();
  gl.bindFramebuffer(gl.FRAMEBUFFER, depthFramebuffer);
  gl.framebufferTexture2D(
      gl.FRAMEBUFFER,       // target
      gl.DEPTH_ATTACHMENT,  // attachment point
      gl.TEXTURE_2D,        // texture target
      depthTexture,         // texture
      0);                   // mip level

  function degToRad(d) {
    return d * Math.PI / 180;
  }

  const settings = {
    cameraX: 6,
    cameraY: 5,
    posX: 2.5,
    posY: 4.8,
    posZ: 4.3,
    targetX: 2.5,
    targetY: 0,
    targetZ: 3.5,
    projWidth: 1,
    projHeight: 1,
    perspective: true,
    fieldOfView: 120,
  };
  webglLessonsUI.setupUI(document.querySelector('#ui'), settings, [
    { type: 'slider',   key: 'cameraX',    min: -10, max: 10, change: render, precision: 2, step: 0.001, },
    { type: 'slider',   key: 'cameraY',    min:   1, max: 20, change: render, precision: 2, step: 0.001, },
    { type: 'slider',   key: 'posX',       min: -10, max: 10, change: render, precision: 2, step: 0.001, },
    { type: 'slider',   key: 'posY',       min:   1, max: 20, change: render, precision: 2, step: 0.001, },
    { type: 'slider',   key: 'posZ',       min:   1, max: 20, change: render, precision: 2, step: 0.001, },
    { type: 'slider',   key: 'targetX',    min: -10, max: 10, change: render, precision: 2, step: 0.001, },
    { type: 'slider',   key: 'targetY',    min:   0, max: 20, change: render, precision: 2, step: 0.001, },
    { type: 'slider',   key: 'targetZ',    min: -10, max: 20, change: render, precision: 2, step: 0.001, },
    { type: 'slider',   key: 'projWidth',  min:   0, max:  2, change: render, precision: 2, step: 0.001, },
    { type: 'slider',   key: 'projHeight', min:   0, max:  2, change: render, precision: 2, step: 0.001, },
    { type: 'checkbox', key: 'perspective', change: render, },
    { type: 'slider',   key: 'fieldOfView', min:  1, max: 179, change: render, },
  ]);

  const fieldOfViewRadians = degToRad(60);

  // Uniforms for each object.
  const planeUniforms = {
    u_colorMult: [0.5, 0.5, 1, 1],  // lightblue
    u_color: [1, 0, 0, 1],
    u_texture: checkerboardTexture,
    u_world: m4.translation(0, 0, 0),
  };
  const sphereUniforms = {
    u_colorMult: [1, 0.5, 0.5, 1],  // pink
    u_color: [0, 0, 1, 1],
    u_texture: checkerboardTexture,
    u_world: m4.translation(2, 3, 4),
  };
  const cubeUniforms = {
    u_colorMult: [0.5, 1, 0.5, 1],  // lightgreen
    u_color: [0, 0, 1, 1],
    u_texture: checkerboardTexture,
    u_world: m4.translation(3, 1, 0),
  };

  function drawScene(projectionMatrix, cameraMatrix, textureMatrix, programInfo) {
    // Make a view matrix from the camera matrix.
    const viewMatrix = m4.inverse(cameraMatrix);

    gl.useProgram(programInfo.program);

    // set uniforms that are the same for both the sphere and plane
    // note: any values with no corresponding uniform in the shader
    // are ignored.
    webglUtils.setUniforms(programInfo, {
      u_view: viewMatrix,
      u_projection: projectionMatrix,
      u_textureMatrix: textureMatrix,
      u_projectedTexture: depthTexture,
    });

    // ------ Draw the sphere --------

    // Setup all the needed attributes.
    webglUtils.setBuffersAndAttributes(gl, programInfo, sphereBufferInfo);

    // Set the uniforms unique to the sphere
    webglUtils.setUniforms(programInfo, sphereUniforms);

    // calls gl.drawArrays or gl.drawElements
    webglUtils.drawBufferInfo(gl, sphereBufferInfo);

    // ------ Draw the cube --------

    // Setup all the needed attributes.
    webglUtils.setBuffersAndAttributes(gl, programInfo, cubeBufferInfo);

    // Set the uniforms unique to the cube
    webglUtils.setUniforms(programInfo, cubeUniforms);

    // calls gl.drawArrays or gl.drawElements
    webglUtils.drawBufferInfo(gl, cubeBufferInfo);

    // ------ Draw the plane --------

    // Setup all the needed attributes.
    webglUtils.setBuffersAndAttributes(gl, programInfo, planeBufferInfo);

    // Set the uniforms unique to the cube
    webglUtils.setUniforms(programInfo, planeUniforms);

    // calls gl.drawArrays or gl.drawElements
    webglUtils.drawBufferInfo(gl, planeBufferInfo);
  }

  // Draw the scene.
  function render() {
    webglUtils.resizeCanvasToDisplaySize(gl.canvas);

    gl.enable(gl.CULL_FACE);
    gl.enable(gl.DEPTH_TEST);

    // first draw from the POV of the light
    const lightWorldMatrix = m4.lookAt(
        [settings.posX, settings.posY, settings.posZ],          // position
        [settings.targetX, settings.targetY, settings.targetZ], // target
        [0, 1, 0],                                              // up
    );
    const lightProjectionMatrix = settings.perspective
        ? m4.perspective(
            degToRad(settings.fieldOfView),
            settings.projWidth / settings.projHeight,
            0.5,  // near
            10)   // far
        : m4.orthographic(
            -settings.projWidth / 2,   // left
             settings.projWidth / 2,   // right
            -settings.projHeight / 2,  // bottom
             settings.projHeight / 2,  // top
             0.5,                      // near
             10);                      // far

    // draw to the depth texture
    gl.bindFramebuffer(gl.FRAMEBUFFER, depthFramebuffer);
    gl.viewport(0, 0, depthTextureSize, depthTextureSize);
    gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT);

    drawScene(lightProjectionMatrix, lightWorldMatrix, m4.identity(), colorProgramInfo);

    // now draw scene to the canvas projecting the depth texture into the scene
    gl.bindFramebuffer(gl.FRAMEBUFFER, null);
    gl.viewport(0, 0, gl.canvas.width, gl.canvas.height);
    gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT);

    let textureMatrix = m4.identity();
    textureMatrix = m4.translate(textureMatrix, 0.5, 0.5, 0.5);
    textureMatrix = m4.scale(textureMatrix, 0.5, 0.5, 0.5);
    textureMatrix = m4.multiply(textureMatrix, lightProjectionMatrix);
    // use the inverse of this world matrix to make
    // a matrix that will transform other positions
    // to be relative this this world space.
    textureMatrix = m4.multiply(
        textureMatrix,
        m4.inverse(lightWorldMatrix));

    // Compute the projection matrix
    const aspect = gl.canvas.clientWidth / gl.canvas.clientHeight;
    const projectionMatrix =
        m4.perspective(fieldOfViewRadians, aspect, 1, 2000);

    // Compute the camera's matrix using look at.
    const cameraPosition = [settings.cameraX, settings.cameraY, 7];
    const target = [0, 0, 0];
    const up = [0, 1, 0];
    const cameraMatrix = m4.lookAt(cameraPosition, target, up);

    drawScene(projectionMatrix, cameraMatrix, textureMatrix, textureProgramInfo);

    // ------ Draw the frustum ------
    {
      const viewMatrix = m4.inverse(cameraMatrix);

      gl.useProgram(colorProgramInfo.program);

      // Setup all the needed attributes.
      webglUtils.setBuffersAndAttributes(gl, colorProgramInfo, cubeLinesBufferInfo);

      // scale the cube in Z so it's really long
      // to represent the texture is being projected to
      // infinity
      const mat = m4.multiply(
          lightWorldMatrix, m4.inverse(lightProjectionMatrix));

      // Set the uniforms we just computed
      webglUtils.setUniforms(colorProgramInfo, {
        u_color: [0, 0, 0, 1],
        u_view: viewMatrix,
        u_projection: projectionMatrix,
        u_world: mat,
      });

      // calls gl.drawArrays or gl.drawElements
      webglUtils.drawBufferInfo(gl, cubeLinesBufferInfo, gl.LINES);
    }
  }
  render();
}

main();

@import url("https://webglfundamentals.org/webgl/resources/webgl-tutorials.css");
body {
  margin: 0;
}
canvas {
  width: 100vw;
  height: 100vh;
  display: block;
}
.gman-widget-value {
  min-width: 5em;
}

<canvas id="canvas"></canvas>

I've based my own implementations on this, and run into the same issue. WebGL returns a gl.INVALID_FRAMEBUFFER_OPERATION when calling getError after a call to gl.clea or to gl.drawElements. I assume something is wrong with the setup of the framebuffer, but nothing I tried helps. Thanks in advance!

Body:
For Safari pre version 15 you need to setup a color attachment even if though the sample doesn't use it

first create an RGBA/UNSIGNED_BYTE color attachment the same size as the depth texture

  const dummy = gl.createTexture();
  gl.bindTexture(gl.TEXTURE_2D, dummy);
  gl.texImage2D(
      gl.TEXTURE_2D,
      0,
      gl.RGBA,
      depthTextureSize,
      depthTextureSize,
      0,
      gl.RGBA,
      gl.UNSIGNED_BYTE,
      null,
  );
  gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.NEAREST);
  gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.NEAREST);
  gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
  gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);

Then attach it to the framebuffer

  const depthFramebuffer = gl.createFramebuffer();
  gl.bindFramebuffer(gl.FRAMEBUFFER, depthFramebuffer);

  gl.framebufferTexture2D(
      gl.FRAMEBUFFER,       // target
      gl.DEPTH_ATTACHMENT,  // attachment point
      gl.TEXTURE_2D,        // texture target
      depthTexture,         // texture
      0);                   // mip level

  // ---- added ----
  gl.framebufferTexture2D(
      gl.FRAMEBUFFER,
      gl.COLOR_ATTACHMENT0,
      gl.TEXTURE_2D,
      dummy,
      0,
  )

Unfortunately this is not a bug in Safari. It's a reflection of unfortunate choices of the OpenGL ES 2.0 spec which says

4.4.5 Framebuffer Completeness

...

Framebuffer Completeness

...

The framebuffer object target is said to be framebuffer complete if it is the window-system-provided framebuffer, or if all the following conditons are true:

In this subsection, each rule is followed by an error enum in bold

...

The combination of internal formats of the attached images does not violate an implementation-dependent set of restrictions.

FRAMEBUFFER_UNSUPPORTED

In other words, in OpenGL ES 2.0 NO COMBINATIONS OF FRAMEBUFFER ATTACHMENTS ARE REQUIRED TO WORK!!!!!!!

The WebGL spec itself added the requirement that 3 combinations are required to work

6.8 Framebuffer Object Attachments

...

The following combinations of framebuffer object attachments, when all of the attachments are framebuffer attachment complete, non-zero, and have the same width and height, must result in the framebuffer being framebuffer complete:

COLOR_ATTACHMENT0 = RGBA/UNSIGNED_BYTE texture

COLOR_ATTACHMENT0 = RGBA/UNSIGNED_BYTE texture + DEPTH_ATTACHMENT = DEPTH_COMPONENT16 renderbuffer

COLOR_ATTACHMENT0 = RGBA/UNSIGNED_BYTE texture + DEPTH_STENCIL_ATTACHMENT = DEPTH_STENCIL renderbuffer

So, following the 2 specs, even though the WEBGL_depth_texture extension adds depth textures, none of them are required to work as attachments in framebuffers with or without any other attachments!!!

Yes, you read that right. The WEBGL_depth_texture extension spec says you can create them and attach them. The OpenGL ES 2.0 spec says they are not required to work as attachments or rather that it's up to the implementation whether any combinations work. WebGL lists 3 combinations that are required to work but those 3 combinations do not include depth textures, only depth renderbuffers.

Fortunately it appears they do work everywhere if you add an RGBA/UNSIGNED_BYTE color attachment.

On a positive note, OpenGL ES 3.0 and WebGL2 list many more combinations that are required to work.


问:
Pop-Up Canvas/Panel Not Blocking Raycast to Buttons/UI Below
I'm trying to create a pop-up window that will overlay on top of the current canvas and block all interaction with it. Everything I read says to use Canvas Group(s) and/or Graphic Raycaster, but I can't find the magic combination to make that work.

Canvas O is a Screen-Space overlay for typical buttons.
Canvas K is a World-Space canvas for my current UI stuff that I can pan around and whatnot.
Canvas P is my popup which is also an overlay and which I want to block all interaction with both O and K "behind" it. I've put a Canvas group on all 3, I've put a Graphic Raycaster on P, I've selected "Blocks Raycasts" in the Canvas Group of P as well as setting the "Blocking Objects" and "Blocking Mask" to All and Everything. I've put colliders on, and I've made the canvases different Sort Orders for layering. I'm out of ideas. Why won't it block??

答:
Canvas UI objects are drawn in descending order (so that the lowest object under the canvas in the inspector is drawn last and is in front of everything else).

Like this:

Canvas

Text1

Text2 (drawn after Text1)

OverlayImage (drawn after Text1 and Text2, blocking raycasts)

If you add an image UI object to a canvas, drag it to be the lowest object under that canvas in the inspector like in the example above, then expand it so that it takes up the entire canvas, and set the anchors (which control resizing) to expand based on the width and height of the window, that image should always cover the entire canvas. As long as it stays as the lowest GameObject in the inspector hierarchy, it should block events underneath that on that canvas.

I'm not sure about one canvas blocking events on other canvases in the scene, but I would add OverlayImages on all of the canvases, and control whether the OverlayImages are enabled or disabled from canvas P in your example.


问:
Will Python Daemon thread be blocked by busy parent thread given there is GIL?
I know Python daemon thread will auto exit if parent thread exit. The reason I ask if because my main thread is sometime working on CPU heavy calculation which blocks the heart beat message. Other module thinks this service is dead and planning for funeral.

I want to use some kind of daemon thread(or process?) that is the child of main thread, to send heart beat message.

My question is if my main thread is blocking, for example:

```<code>while True:
   a = a + 1
</code>```

Will my daemon child thread be blocked as well? any example or evidence? What role does GIL (Global Interpreter Lock) play here?

Edit

To @ShadowRanger's point, the following example shows, waiter can log out the the message, given the worker thread is blocking (taking 220% CPU resource)

```<code>import time
from threading import Thread

class worker(Thread):
    def run(self):
        t = 10
        x = 10
        while True:
            x = t + 1

class waiter(Thread):
    def run(self):
        for x in xrange(100,150):
            print x
            time.sleep(0.5)

def run():
    worker().start()
    waiter().start()

run()
</code>```

答: It is probably the IO loop callback inside a thread that blocks the heart beat. As @ShadowRanger points out, the thread is interleaving, which should not block another thread periodical callback (except the problematic python extension). In my case, it is probably the ioloop callback queue inside the thread is the culprit. No interleaving in ioloop callback queue. </p>

Body:
I know Python daemon thread will auto exit if parent thread exit. The reason I ask if because my main thread is sometime working on CPU heavy calculation which blocks the heart beat message. Other module thinks this service is dead and planning for funeral.

I want to use some kind of daemon thread(or process?) that is the child of main thread, to send heart beat message.

My question is if my main thread is blocking, for example:

while True:
   a = a + 1

Will my daemon child thread be blocked as well? any example or evidence? What role does GIL (Global Interpreter Lock) play here?

Edit

To @ShadowRanger's point, the following example shows, waiter can log out the the message, given the worker thread is blocking (taking 220% CPU resource)

import time
from threading import Thread

class worker(Thread):
    def run(self):
        t = 10
        x = 10
        while True:
            x = t + 1

class waiter(Thread):
    def run(self):
        for x in xrange(100,150):
            print x
            time.sleep(0.5)

def run():
    worker().start()
    waiter().start()

run()


问:
How to make your own scrollable Column like this?
I'm new in flutter and I'm doing an app from dribble, do not know how to do this kinda scrolling
https://cdn.dribbble.com/users/1837506/videos/32005/influenza_small.mp4

Please, can u show me how to make or share some links :)

答:
Check this Custom transitions applying on the PageViews

https://medium.com/flutter-community/a-deep-dive-into-pageview-in-flutter-with-custom-transitions-581d9ea6dded

May be it will solve your problem, please do let me know. Happy coding


问:
While making Plot in pandas python I am getting Error: EOL
df

```<code>import pandas as pd
df = pd.DataFrame({
    'SKU': ['BIKES','MATS', 'BLANKETS', 'CREAMS'],'Col1': [1,3,6,10],'Col2': [4,5, 6,16],'Month':[1,2,3,4],'Year':['2015','2010','2011','2012']})
</code>```

Code tried which doesnt work

```<code>    import matplotlib.pyplot as plt

#Visualisations

fig, ax = plt.subplots()
for col in ['Col1', 'Col2']:
    ax.plot(df[col], label=col)
ax.legend(loc="best")
ax.tick_params(axis="x", rotation=30)
</code>```

I need tickers for months and values for years. But before I can proceed with tickers etc., I am getting error:

```<code>File "<ipython-input-14-136f7818ccc7>", line 6
    for col in ['Col1', 'Col2']:
                                ^
SyntaxError: EOL while scanning string literal
</code>```

答:
You are opening a specific type of apostrophes and never closing it:

for col in ["Col1’, ‘Col2’]:

Needs to be replaced with:

```<code>for col in ["Col1","Col2"]:
</code>```

Body:
df

import pandas as pd
df = pd.DataFrame({
    'SKU': ['BIKES','MATS', 'BLANKETS', 'CREAMS'],'Col1': [1,3,6,10],'Col2': [4,5, 6,16],'Month':[1,2,3,4],'Year':['2015','2010','2011','2012']})

Code tried which doesnt work

    import matplotlib.pyplot as plt

#Visualisations

fig, ax = plt.subplots()
for col in ['Col1', 'Col2']:
    ax.plot(df[col], label=col)
ax.legend(loc="best")
ax.tick_params(axis="x", rotation=30)

I need tickers for months and values for years. But before I can proceed with tickers etc., I am getting error:

File "<ipython-input-14-136f7818ccc7>", line 6
    for col in ['Col1', 'Col2']:
                                ^
SyntaxError: EOL while scanning string literal

Body:
You are opening a specific type of apostrophes and never closing it:

for col in ["Col1’, ‘Col2’]:

Needs to be replaced with:

for col in ["Col1","Col2"]: