问:
Format plain text to date

Goal

Write a script that does the following:

get data from sheet

modify column to flip names from [Last, First] to [First Last]

modify 2 columns to abbreviate company names & statuses

write resulting data to another spreadsheet without changing the format

add timestamp for when data copied

Problem

I can get the data BUT it writes everything back as plain text. Thus instead of dates writing out as "yyyy-MM-dd", they write out as something like this:
 Mon Oct 19 2020 01:00:00 GMT-0400 (Eastern Daylight Time)

Expectation:
screenshot of dates as "yyyy-MM-dd"

Result:
screenshot of dates as whatever this garble is

I have googled extensively and can't seem to find a solution. I believe my problem is with using toString() in the Array.map. I'm not sure how to restrict the map method to only the columns that need modifying. Right now it affects the whole array.

(I used the code from Google Apps Script for Multiple Find and Replace in Google Sheets  to write this part)

```<code>//-----FIND AND REPLACE FOR COMPANY & STATUS ABBREVIATIONS
function replaceInSheet(initArray, to_replace, replace_with){
  //Loop over rows in array
  for(var row in initArray ){
    //Use Array.map to execute a replace call on each of the cells in the row.
    var replaced_values = initArray[row].map(function(originalValue){
        return originalValue.toString().replace(to_replace,replace_with);
      });

    //Replace the original row values with the replaced values
    initArray[row] = replaced_values;
  }
}
</code>```

Question--> How do I get the output of my script to format dates in two of my columns, correctly?

Attempted Solutions that didn't work

I tried swapping the name flip code to happen after the abbreviation code so I could add a setNumberFormat('yyyy-MM-dd') within the name flip for loop. I couldn't figure out how to apply this to columns within my array. Something like initarray[x][5].setNumberFormat("yyyy-MM-dd")  gave me an error saying "TypeError: initArrayx.setNumberFormat is not a function"

I tried adding code before (and then even after) .setValues() at the end to change the format. Some resources I referenced:

setNumberFormat('yyyy-MM-dd') from stackoverflow: Set cell format with google apps script

Utilities.formatDate(new Date(), "CST", "yyyy-MM-dd") from stackoverflow: Get today date in google appScript

website post by BLACKCJ: "Cell Number Formatting with Google Apps Script"

google's developer documentation

and all sorts of other articles, blogs, forums, etc.

I tried writing a completely separate function to change the format after my script runs. Nope. I can't get those 2 columns to format as dates

Code & Sample Spreadsheet

Here's the whole code I'm using, modified to work with a sample google sheet I created just for the purposes of this question.

Sample Google Sheet:

https://docs.google.com/spreadsheets/d/1Ys77hQHHajIo-Xaxyom0SVnyVMZ6bKOT8Smpadd2jv4/edit?usp=sharing

Script:

```<code>// ==================================================
// FUNCTION TO RUN
// ==================================================
function syncData(){
  //Ger Source Data
  var ss = SpreadsheetApp.getActiveSpreadsheet();
  var thisSheet = ss.getSheetByName("source");
  var thisData = thisSheet.getRange("A4:M11");
  var initArray = thisData.getValues();

  //Get Target Location
  var toSheet = ss.getSheetByName("target");
  var toRange = toSheet.getRange("A4:M11"); //Range starts at A4

  //CHANGE [LAST, FIRST] TO [FIRST LAST]
  for (var x = 0; x < initArray.length; x++){
    var indexOfFirstComma = initArray[x][0].indexOf(", ");
    if(indexOfFirstComma >= 0){
      //If comma found, split and update values in the values array
      var lastAndFirst = initArray[x][0];
      //Update name value in array
      initArray[x][0] = lastAndFirst.slice(indexOfFirstComma + 2).trim() + " " + lastAndFirst.slice(0, indexOfFirstComma).trim();
    }
  }

  //ABBREVIATE COMPANY
  replaceInSheet(initArray, 'Bluffington School','BLF HS');
  replaceInSheet(initArray, 'Honker Burger','HBGR');
  replaceInSheet(initArray, 'Funky Town','FT');

  //ABBRIVIATE STATUS
  replaceInSheet(initArray, 'Regular','Staff');
  replaceInSheet(initArray, 'Contractual','Part');
  replaceInSheet(initArray, 'Temporary','Temp');

  //Clear Target Location
  var toClear = toSheet.getRange("A4:M11")
  toClear.clearContent();

  //Write updated array to target location
  toRange.setValues(initArray);

  //Write timestamp of when code was last run
  setTimeStamp(toSheet);
}

//-----FIND AND REPLACE FOR COMPANY & STATUS ABBREVIATIONS
function replaceInSheet(initArray, to_replace, replace_with){
  //Loop over rows in array
  for(var row in initArray ){
    //Use Array.map to execute a replace call on each of the cells in the row.
    var replaced_values = initArray[row].map(function(originalValue){
        return originalValue.toString().replace(to_replace,replace_with);
      });

    //Replace the original row values with the replaced values
    initArray[row] = replaced_values;
  }
}

//-----ADD TIMESTAMP FOR WHEN THE SCRIPT LAST RAN
function setTimeStamp(toSheet) {
  var timestamp = Utilities.formatDate(new Date(), "CST", "yyyy-MM-dd @ h:mm a");
  toSheet.getRange('F1').setValue(timestamp);
}
</code>```

答:
setNumberFormat('yyyy-MM-dd') is a good solution but it's a method of a Range of the sheet. Not an array.

To apply the format you need to get a range first. Something like this:

```<code>toSheet.getRange('G4:G').setNumberFormat('yyyy-MM-dd');
</code>```

And there is one more thing ) Try to change this line:

```<code>var initArray = thisData.getValues();
</code>```

to:

```<code>var initArray = thisData.getDisplayValues();
</code>```

Title: Format plain text to date

Body:
Goal

Write a script that does the following:

get data from sheet

modify column to flip names from [Last, First] to [First Last]

modify 2 columns to abbreviate company names & statuses

write resulting data to another spreadsheet without changing the format

add timestamp for when data copied

Problem

I can get the data BUT it writes everything back as plain text. Thus instead of dates writing out as "yyyy-MM-dd", they write out as something like this:
 Mon Oct 19 2020 01:00:00 GMT-0400 (Eastern Daylight Time)

Expectation:
screenshot of dates as "yyyy-MM-dd"

Result:
screenshot of dates as whatever this garble is

I have googled extensively and can't seem to find a solution. I believe my problem is with using toString() in the Array.map. I'm not sure how to restrict the map method to only the columns that need modifying. Right now it affects the whole array.

(I used the code from Google Apps Script for Multiple Find and Replace in Google Sheets  to write this part)

//-----FIND AND REPLACE FOR COMPANY & STATUS ABBREVIATIONS
function replaceInSheet(initArray, to_replace, replace_with){
  //Loop over rows in array
  for(var row in initArray ){
    //Use Array.map to execute a replace call on each of the cells in the row.
    var replaced_values = initArray[row].map(function(originalValue){
        return originalValue.toString().replace(to_replace,replace_with);
      });

    //Replace the original row values with the replaced values
    initArray[row] = replaced_values;
  }
}

Question--> How do I get the output of my script to format dates in two of my columns, correctly?

Attempted Solutions that didn't work

I tried swapping the name flip code to happen after the abbreviation code so I could add a setNumberFormat('yyyy-MM-dd') within the name flip for loop. I couldn't figure out how to apply this to columns within my array. Something like initarray[x][5].setNumberFormat("yyyy-MM-dd")  gave me an error saying "TypeError: initArrayx.setNumberFormat is not a function"

I tried adding code before (and then even after) .setValues() at the end to change the format. Some resources I referenced:

setNumberFormat('yyyy-MM-dd') from stackoverflow: Set cell format with google apps script

Utilities.formatDate(new Date(), "CST", "yyyy-MM-dd") from stackoverflow: Get today date in google appScript

website post by BLACKCJ: "Cell Number Formatting with Google Apps Script"

google's developer documentation

and all sorts of other articles, blogs, forums, etc.

I tried writing a completely separate function to change the format after my script runs. Nope. I can't get those 2 columns to format as dates

Code & Sample Spreadsheet

Here's the whole code I'm using, modified to work with a sample google sheet I created just for the purposes of this question.

Sample Google Sheet:

https://docs.google.com/spreadsheets/d/1Ys77hQHHajIo-Xaxyom0SVnyVMZ6bKOT8Smpadd2jv4/edit?usp=sharing

Script:

// ==================================================
// FUNCTION TO RUN
// ==================================================
function syncData(){
  //Ger Source Data
  var ss = SpreadsheetApp.getActiveSpreadsheet();
  var thisSheet = ss.getSheetByName("source");
  var thisData = thisSheet.getRange("A4:M11");
  var initArray = thisData.getValues();

  //Get Target Location
  var toSheet = ss.getSheetByName("target");
  var toRange = toSheet.getRange("A4:M11"); //Range starts at A4

  //CHANGE [LAST, FIRST] TO [FIRST LAST]
  for (var x = 0; x < initArray.length; x++){
    var indexOfFirstComma = initArray[x][0].indexOf(", ");
    if(indexOfFirstComma >= 0){
      //If comma found, split and update values in the values array
      var lastAndFirst = initArray[x][0];
      //Update name value in array
      initArray[x][0] = lastAndFirst.slice(indexOfFirstComma + 2).trim() + " " + lastAndFirst.slice(0, indexOfFirstComma).trim();
    }
  }

  //ABBREVIATE COMPANY
  replaceInSheet(initArray, 'Bluffington School','BLF HS');
  replaceInSheet(initArray, 'Honker Burger','HBGR');
  replaceInSheet(initArray, 'Funky Town','FT');

  //ABBRIVIATE STATUS
  replaceInSheet(initArray, 'Regular','Staff');
  replaceInSheet(initArray, 'Contractual','Part');
  replaceInSheet(initArray, 'Temporary','Temp');

  //Clear Target Location
  var toClear = toSheet.getRange("A4:M11")
  toClear.clearContent();

  //Write updated array to target location
  toRange.setValues(initArray);

  //Write timestamp of when code was last run
  setTimeStamp(toSheet);
}

//-----FIND AND REPLACE FOR COMPANY & STATUS ABBREVIATIONS
function replaceInSheet(initArray, to_replace, replace_with){
  //Loop over rows in array
  for(var row in initArray ){
    //Use Array.map to execute a replace call on each of the cells in the row.
    var replaced_values = initArray[row].map(function(originalValue){
        return originalValue.toString().replace(to_replace,replace_with);
      });

    //Replace the original row values with the replaced values
    initArray[row] = replaced_values;
  }
}

//-----ADD TIMESTAMP FOR WHEN THE SCRIPT LAST RAN
function setTimeStamp(toSheet) {
  var timestamp = Utilities.formatDate(new Date(), "CST", "yyyy-MM-dd @ h:mm a");
  toSheet.getRange('F1').setValue(timestamp);
}

Body:
setNumberFormat('yyyy-MM-dd') is a good solution but it's a method of a Range of the sheet. Not an array.

To apply the format you need to get a range first. Something like this:

toSheet.getRange('G4:G').setNumberFormat('yyyy-MM-dd');

And there is one more thing ) Try to change this line:

var initArray = thisData.getValues();

to:

var initArray = thisData.getDisplayValues();


来源: stackexchange

问:
Search - Appscript - targetRange column start is not at the column I ask for
I am still learning a lot about Apps Script and needed to make a sheet for my work.
At the moment I am busy on a Search function.
The only problem is that the results that I want to get, do not go to that column.
Only to a column way further.

```<code>function Search() {
  var sourcesheet = SpreadsheetApp.getActiveSpreadsheet().getSheetByName("Source");
  //USER INPUT
  sourceRowStart = 2; //Row to start getting formulas from
  sourceColumnStart = 13; //Column to start getting formulas from
  numberOfSourceColumnsToGet = 9; //Number of columns to get formulas from
  targetRowStart = 4; //Row to start copying formulas to
  targetRange = 1; //Column to start copying formulas to
  //END OF USER INPUT
  activeSheet = SpreadsheetApp.getActiveSheet();
  sourceRange = activeSheet.getRange(sourceRowStart, sourceColumnStart, activeSheet.getLastRow(), numberOfSourceColumnsToGet);
  sourceFormulas = sourceRange.getFormulas();//Get only formulas from the source rang
  targetRange = activeSheet.getRange(targetRowStart, targetRange, sourceFormulas.length, sourceFormulas[0].length);
  targetRange.setFormulas(sourceFormulas);//Copy the formulas to the target range
}
</code>```

Following is the link of a test-sheet for this question:

https://docs.google.com/spreadsheets/d/1b5Xa1z6xdeqyFbAVtxwc-U76iDteC1Sn46Ll_v9WGPU/edit#gid=713972882

Thankyou in advance to whoever could help me out!

答:
Try this:</p>
```<code>function Search() {
  const ssr = 2;
  const ssc = 13;
  const sncols = 9;
  const tsr = 4;
  const tsc = 1;
  const ss = SpreadsheetApp.getActive();
  const sh = ss.getActiveSheet();
  const sfA = sh.getRange(ssr, ssc, sh.getLastRow() - ssr + 1, sncols).getFormulas();
  sh.getRange(tsr, tsc, sfA.length, sfA[0].length).setFormulas(sfA);
}
</code>```

ContentLicense: CC BY-SA 4.0

Body:
I am still learning a lot about Apps Script and needed to make a sheet for my work.
At the moment I am busy on a Search function.
The only problem is that the results that I want to get, do not go to that column.
Only to a column way further.

function Search() {
  var sourcesheet = SpreadsheetApp.getActiveSpreadsheet().getSheetByName("Source");
  //USER INPUT
  sourceRowStart = 2; //Row to start getting formulas from
  sourceColumnStart = 13; //Column to start getting formulas from
  numberOfSourceColumnsToGet = 9; //Number of columns to get formulas from
  targetRowStart = 4; //Row to start copying formulas to
  targetRange = 1; //Column to start copying formulas to
  //END OF USER INPUT
  activeSheet = SpreadsheetApp.getActiveSheet();
  sourceRange = activeSheet.getRange(sourceRowStart, sourceColumnStart, activeSheet.getLastRow(), numberOfSourceColumnsToGet);
  sourceFormulas = sourceRange.getFormulas();//Get only formulas from the source rang
  targetRange = activeSheet.getRange(targetRowStart, targetRange, sourceFormulas.length, sourceFormulas[0].length);
  targetRange.setFormulas(sourceFormulas);//Copy the formulas to the target range
}

Following is the link of a test-sheet for this question:

https://docs.google.com/spreadsheets/d/1b5Xa1z6xdeqyFbAVtxwc-U76iDteC1Sn46Ll_v9WGPU/edit#gid=713972882

Thankyou in advance to whoever could help me out!

Body:
Try this:

function Search() {
  const ssr = 2;
  const ssc = 13;
  const sncols = 9;
  const tsr = 4;
  const tsc = 1;
  const ss = SpreadsheetApp.getActive();
  const sh = ss.getActiveSheet();
  const sfA = sh.getRange(ssr, ssc, sh.getLastRow() - ssr + 1, sncols).getFormulas();
  sh.getRange(tsr, tsc, sfA.length, sfA[0].length).setFormulas(sfA);
}


问:
Excel - Filter data that exists in 2 columns
I am trying to Filter an excel column with data that exists both in Column A and Column B
For example Lets say that I have Column A with the following rows:
(A, B, C, D, E)
And Column B with:
(C,D,Z)

I would like to filter Column A that will show only:
(C,D)

This is because those are the 2 only rows with values that exists in both Columns.

Thanks

答:
It's an old post, so probably irrelevant now.

Another option would be to have conditional formatting on column A based on formula above and set format that could be used for filter e.g. interior color or font color

ContentLicense: CC BY-SA 3.0


问:
Numpy string arrays - Strange behavior calling tobytes() on numpy string array
I am trying to use Numpy to vectorize an operation to parse a text file containing lines of numbers and convert the data into a numpy array. The data in the text file looks like this:

```<code>*** .txt file ***

1 0 0 0 0
2 1 0 0 0
3 1 1 0 0
4 0 1 0 0
5 0 0 1 0
6 1 0 1 0
7 1 1 1 0
8 0 1 1 0
9 0.5 0.5 0 0
10 0.5 0.5 1 0
11 0.5 0 0.5 0
12 1 0.5 0.5 0
13 0.5 1 0.5 0
14 0 0.5 0.5 0

*** /.txt file ***
</code>```

My approach is to read the lines in using file.readlines(), then convert that list of line strings into a numpy array as follows - file.readlines() part omitted for testing.

```<code>short_list = ['1 0 0 0 0\n',
              '2 1 0 0 0\n',
              '3 1 1 0 0\n']

long_list = ['1 0 0 0 0\n',
             '2 1 0 0 0\n',
             '3 1 1 0 0\n',
             '4 0 1 0 0\n',
             '5 0 0 1 0\n',
             '6 1 0 1 0\n',
             '7 1 1 1 0\n',
             '8 0 1 1 0\n',
             '9 0.5 0.5 0 0\n',
             '10 0.5 0.5 1 0\n',
             '11 0.5 0 0.5 0\n',
             '12 1 0.5 0.5 0\n',
             '13 0.5 1 0.5 0\n',
             '14 0 0.5 0.5 0\n']

def lines_to_npy(lines):
    n_lines = len(lines)
    lines_array = np.array(lines).astype('S')
    tmp = lines_array.tobytes().decode('ascii')
    print(repr(tmp))
    print(lines_array.dtype)
    print(np.array(tmp.split(), dtype=np.int32).reshape(n_lines, -1))

lines_to_npy(short_list)
lines_to_npy(long_list)
</code>```

Calling the function with short_list produces the following output:

```<code>'1 0 0 0 0\n2 1 0 0 0\n3 1 1 0 0\n'
|S10
[[1 0 0 0 0]
 [2 1 0 0 0]
 [3 1 1 0 0]]
</code>```

Which is the desired result (from reading around I gather that "|S10" means that each element in the array is a 10 character string for which the endianness doesn't matter). However, calling with the long list inserts several null characters \x00 at the end of each string which makes it harder to parse.

```<code>'1 0 0 0 0\n\x00\x00\x00\x00\x002 1 0 0 0\n\x00\x00\x00\x00\x003 1 1 0 0\n\x00\x00\x00\x00\x004 0 1 0 0\n\x00\x00\x00\x00\x005 0 0 1 0\n\x00\x00\x00\x00\x006 1 0 1 0\n\x00\x00\x00\x00\x007 1 1 1 0\n\x00\x00\x00\x00\x008 0 1 1 0\n\x00\x00\x00\x00\x009 0.5 0.5 0 0\n\x0010 0.5 0.5 1 0\n11 0.5 0 0.5 0\n12 1 0.5 0.5 0\n13 0.5 1 0.5 0\n14 0 0.5 0.5 0\n'
|S15
</code>```

Note that an error was raised in my function when loading the null characters into an array, preventing a final result. I know that a "cheap and dirty" solution would be to just strip the null characters off the end. I also know that I could use Pandas to accomplish the main goal, too, but I'd like to understand why this behavior is occurring.

The \x00 are padded at the end of each string to make each string of length 15. This kind of makes sense, because the dtype of the short array was |S10, and each string just happened to be 10 characters long. The long array contains 14 strings, the dtype was |S15, and extra \x00 are appended to make the length of each item in the array 15 characters.

I am confused because the number of elements in the list of strings (3 vs 14) has no correlation to the length of each string, so I don't understand why the dtype changes to |S15 when adding more list elements.

Update: I did some more research on ways to efficiently read in data from a text file to a numpy array. I need a fast method for doing this because I am reading files with ~10M lines. numpy.loadfromtxt() and numpy.genfromtxt() are candidate solutions, but they are very slow because they are implemented in Python and basically do the same thing as manually looping through file.readlines(), stripping, and splitting the line strings (source). I noticed in my own testing that using numpy.loadtxt() was about twice as slow as the aforementioned manual method, which was also noted here.

I found that using pandas.from_csv().to_numpy(), I was able to get a speedup of ~10x that of looping through file.readlines(). See this answer here. Hopefully this helps anyone in the future with the same application.

答:
I am trying to use Numpy to vectorize an operation to parse a text file containing lines of numbers and convert the data into a numpy array.

Vectorization has nothing to do with reading your data. Doing, e.g. tmp.split() is still calling a plain Python function on a plain Python string object, creating lots of Python string objects as a result, and doing it within the main Python bytecode interpreter loop. No amount of surrounding Numpy code will change that.

That said, there is no meaningful performance gain to be had here anyway. Any halfway reasonable approach to reading and interpreting (i.e., parsing) the file is going to be lightning fast compared to fetching the contents from the hard drive, and much faster than even reading from an SSD.

My approach is to read the lines in using file.readlines(), then convert that list of line strings into a numpy array as follows - file.readlines() part omitted for testing.

Don't do that. This entire process is much more complex than necessary. Keep reading.

tmp = lines_array.tobytes().decode('ascii')

This just gives you the original contents of the file, which you could have gotten directly with .read() instead of .readlines().

from reading around I gather that "|S10" means that each element in the array is a 10 character string for which the endianness doesn't matter

Not quite; the elements are arrays (in the C sense) of 10 bytes each. They are not "strings"; they are raw data which is possibly interpreted as text.

The string '1 0 0 0 0\n', when encoded to bytes using the default encoding, uses 10 bytes. So do all the other strings in short_list. Thus, "array of 10 bytes" is a suitable dtype.

calling with the long list inserts several null characters \x00 at the end of each string which makes it harder to parse.

It does not insert "null characters"; it inserts null bytes (with a numeric value of 0). It does this because it takes 15 bytes to store the encoded representation of '14 0 0.5 0.5 0\n', and each element has to be the same size.

Keep in mind here that the symbol 0 in your text is translated into a single byte which does not have a numeric value of zero. It has a numeric value of 48.

Again: all these encoding and re-encoding steps are not useful - you could have just used the original data from the file, via .read() - all that .readlines() is helping you with is in determining the number of lines in the file.

But you do not either want or need to do any of that.

The logic you want is built directly into Numpy. You should have found this out for yourself by using a search engine.

You can directly ask Numpy to load the file for you, and you should do it that way: numpy.loadtxt('myfile.txt').

Body:
I am trying to use Numpy to vectorize an operation to parse a text file containing lines of numbers and convert the data into a numpy array. The data in the text file looks like this:

*** .txt file ***

1 0 0 0 0
2 1 0 0 0
3 1 1 0 0
4 0 1 0 0
5 0 0 1 0
6 1 0 1 0
7 1 1 1 0
8 0 1 1 0
9 0.5 0.5 0 0
10 0.5 0.5 1 0
11 0.5 0 0.5 0
12 1 0.5 0.5 0
13 0.5 1 0.5 0
14 0 0.5 0.5 0

*** /.txt file ***

My approach is to read the lines in using file.readlines(), then convert that list of line strings into a numpy array as follows - file.readlines() part omitted for testing.

short_list = ['1 0 0 0 0\n',
              '2 1 0 0 0\n',
              '3 1 1 0 0\n']

long_list = ['1 0 0 0 0\n',
             '2 1 0 0 0\n',
             '3 1 1 0 0\n',
             '4 0 1 0 0\n',
             '5 0 0 1 0\n',
             '6 1 0 1 0\n',
             '7 1 1 1 0\n',
             '8 0 1 1 0\n',
             '9 0.5 0.5 0 0\n',
             '10 0.5 0.5 1 0\n',
             '11 0.5 0 0.5 0\n',
             '12 1 0.5 0.5 0\n',
             '13 0.5 1 0.5 0\n',
             '14 0 0.5 0.5 0\n']

def lines_to_npy(lines):
    n_lines = len(lines)
    lines_array = np.array(lines).astype('S')
    tmp = lines_array.tobytes().decode('ascii')
    print(repr(tmp))
    print(lines_array.dtype)
    print(np.array(tmp.split(), dtype=np.int32).reshape(n_lines, -1))

lines_to_npy(short_list)
lines_to_npy(long_list)

Calling the function with short_list produces the following output:

'1 0 0 0 0\n2 1 0 0 0\n3 1 1 0 0\n'
|S10
[[1 0 0 0 0]
 [2 1 0 0 0]
 [3 1 1 0 0]]

Which is the desired result (from reading around I gather that "|S10" means that each element in the array is a 10 character string for which the endianness doesn't matter). However, calling with the long list inserts several null characters \x00 at the end of each string which makes it harder to parse.

'1 0 0 0 0\n\x00\x00\x00\x00\x002 1 0 0 0\n\x00\x00\x00\x00\x003 1 1 0 0\n\x00\x00\x00\x00\x004 0 1 0 0\n\x00\x00\x00\x00\x005 0 0 1 0\n\x00\x00\x00\x00\x006 1 0 1 0\n\x00\x00\x00\x00\x007 1 1 1 0\n\x00\x00\x00\x00\x008 0 1 1 0\n\x00\x00\x00\x00\x009 0.5 0.5 0 0\n\x0010 0.5 0.5 1 0\n11 0.5 0 0.5 0\n12 1 0.5 0.5 0\n13 0.5 1 0.5 0\n14 0 0.5 0.5 0\n'
|S15

Note that an error was raised in my function when loading the null characters into an array, preventing a final result. I know that a "cheap and dirty" solution would be to just strip the null characters off the end. I also know that I could use Pandas to accomplish the main goal, too, but I'd like to understand why this behavior is occurring.

The \x00 are padded at the end of each string to make each string of length 15. This kind of makes sense, because the dtype of the short array was |S10, and each string just happened to be 10 characters long. The long array contains 14 strings, the dtype was |S15, and extra \x00 are appended to make the length of each item in the array 15 characters.

I am confused because the number of elements in the list of strings (3 vs 14) has no correlation to the length of each string, so I don't understand why the dtype changes to |S15 when adding more list elements.

Update: I did some more research on ways to efficiently read in data from a text file to a numpy array. I need a fast method for doing this because I am reading files with ~10M lines. numpy.loadfromtxt() and numpy.genfromtxt() are candidate solutions, but they are very slow because they are implemented in Python and basically do the same thing as manually looping through file.readlines(), stripping, and splitting the line strings (source). I noticed in my own testing that using numpy.loadtxt() was about twice as slow as the aforementioned manual method, which was also noted here.

I found that using pandas.from_csv().to_numpy(), I was able to get a speedup of ~10x that of looping through file.readlines(). See this answer here. Hopefully this helps anyone in the future with the same application.


问:
Serial communication with Uponor Smatrix Base X-245
I'm trying to communicate with a Uponor Smatrix Base X-245 using a use a Raspberry Pi 3B+ over the serial port (UART), without much luck.

Here is a sample output from the UART (/dev/ttyS0) using pyserial:

import serial
ser = serial.Serial("/dev/ttyS0", baudrate=19200)
while True:
    received_data = ser.read()
    sleep(0.025)
    data_left = ser.inWaiting()
    received_data += ser.read(data_left)
    print (received_data.hex())
```
<p>Data without any thermostats (8) connected:</p>
```00  00  00  00  00  00  00  00  00  00  00  00  00  00  c0
00  00  00  00  00  00  00  00  00  00  00  00  00  00  c0
00  00  00  00  00  00  00  00  00  00  00  00  00  00  c0
dd  e7  c9  6b  01  0a  2e  fe
dd  e7  af  83  01  56  89  ff
dd  e7  f5  33  01  bf  36  fe
dd  e7  f7  6d  01  ce  f6  fe
dd  e7  f7  2b  01  aa  36  fe
dd  e7  f7  75  01  da  f6  fe
dd  e7  f7  49  01  fe  76  ff
dd  e7  f5  45  01  99  f6  fe
```
<p>Data with all thermostats (8) connected:</p>
```00  00  00  00  00  00  00  00  00  00  00  00  00  00  c0
00  00  00  00  00  00  00  00  00  00  00  00  00  00  c0
00  00  00  00  00  00  00  00  00  00  00  00  00  00  c0
dd  e7  c9  6b  01  0a  2e  fe  77  5e  6e  e3  ed  db  83  ff  ff  81  e7  fb  87  ff  6f  89  fb  3d  7b  99  ff  75  53  fe
dd  e7  c9  6b  a5  01  01  85  ff  ff  e7  ff  b7  91  fd  cb  8f  f9  93  89  fb  3d  87  ff  6f  95  ff  ff  8d  2f  ff
dd  e7  af  83  01  56  89  ff  77  de  6d  e4  ed  cb  83  ff  ff  81  e7  fb  87  ff  6f  89  fb  85  7b  99  ff  0b  07  ff
dd  e7  af  83  a5  01  01  85  ff  ff  e7  ff  b7  91  fd  cb  8f  f9  93  89  fb  85  87  ff  6f  95  ff  ff  25  3b  fe
dd  e7  f5  33  01  bf  36  fe  77  5e  df  d3  ed  eb  83  ff  ff  81  e7  fb  87  ff  6f  89  fb  85  7b  99  ff  71  0f  ff
dd  e7  f5  33  a5  01  01  85  ff  ff  e7  ff  b7  91  fd  cb  8f  f9  93  89  fb  85  87  ff  6f  95  ff  ff  d3  fb  ff
dd  e7  f7  6d  01  ce  f6  fe  f7  fe  fc  fe
dd  e7  f7  6d  a5  01  01  85  ff  ff  e7  ff  b7  91  fd  cb  8f  f9  93  89  fb  79  87  ff  6f  95  ff  ff  15  d3  fe
dd  e7  f7  2b  01  aa  36  fe
dd  e7  f7  2b  a5  01  01  85  ff  ff  e7  ff  b7  91  fd  cb  8f  f9  93  89  fb  67  87  ff  6f  95  ff  ff  43  91  ff
dd  e7  f7  75  01  da  f6  fe  77  de  5f  c7  ed  97  83  ff  ff  81  e7  fb  87  ff  6f  89  fb  07  7b  99  ff  cf  8f  fe
dd  e7  f7  75  a5  01  01  85  ff  ff  e7  ff  b7  91  fd  cb  8f  f9  93  89  fb  07  87  ff  6f  95  ff  ff  77  9b  ff
dd  e7  f7  49  01  fe  76  ff  77  de  5f  e2  ed  7b  83  ff  ff  81  e7  fb  87  ff  6f  89  fb  79  7b  99  ff  95  ff  ff
dd  e7  f7  49  a5  01  01  85  ff  7f  e7  ff  b7  91  fd  cb  8f  f9  93  89  fb  79  87  ff  6f  95  ff  ff  c1  3   ff
dd  e7  f5  45  01  99  f6  fe  77  5e  5f  d4  ed  bb  83  ff  ff  81  e7  fb  87  ff  6f  89  fb  3d  7b  99  ff  bb  d1  ff
dd  e7  f5  45  a5  01  01  85  ff  ff  e7  ff  b7  91  fd  cb  8f  f9  93  89  fb  3d  87  ff  6f  95  ff  ff  bf  0d  ff
```
<p>Included here the same data, but color coded to recognize patterns.</p>
<p>Also the raw bytes that were captured:</p>
```00 00 00 00 00 00 00 00 00 00 00 00 00 00 c0
00 00 00 00 00 00 00 00 00 00 00 00 00 00 c0
00 00 00 00 00 00 00 00 00 00 00 00 00 00 c0
dd e7 c9 k  01 \n .  fe w  ^  n  e3 ed f7 83 ff ff 81 e7 fb {  99 ff -  1f fe
dd e7 c9 k  a5 01 01 85 ff ff e7 ff b7 91 fd cb 8f f9 93 89 fb =  87 ff o  95 ff ff 8d /  ff

dd e7 af 83 01 V  89 ff w  de m  e4 ed cb 83 ff ff 81 e7 fb {  99 ff e9 d1 ff
dd e7 af 83 a5 01 01 85 ff ff e7 ff b7 91 fd cb 8f f9 93 89 fb 85 87 ff o  95 ff ff %  ;  fe

dd e7 f5 3  01 bf 6  fe w  ^  df d3 ed e5 83 ff ff 81 e7 fb {  99 ff 93 a7 fe
dd e7 f5 3  a5 01 01 85 ff ff e7 ff b7 91 fd cb 8f f9 93 89 fb 85 87 ff o  95 ff ff d3 fb ff

dd e7 f7 m  01 ce f6 fe ff ff ff ff `  ff ff
dd e7 f7 m  a5 01 01 85 ff ff e7 ff b7 91 fd cb 8f f9 93 89 fb y  87 ff o  95 ff ff 15 d3 fe

dd e7 f7 +  01 aa 6  fe
dd e7 f7 +  a5 01 01 85 ff ff e7 ff b7 91 fd cb 8f f9 93 89 fb w  87 ff o  95 ff ff Q  11 ff

dd e7 f7 u  01 da f6 fe w  de _  c7 ed ?  87 ff ff 81 e7 fb {  99 ff bd i  ff
dd e7 f7 u  a5 01 01 85 ff ff e7 ff b7 91 fd cb 8f f9 93 89 fb 07 87 ff o  95 ff ff w  9b ff

dd e7 f7 I  01 fe v  ff w  de _  e2 ed u  83 ff ff 81 e7 fb {  99 ff cf a1 ff
dd e7 f7 I  a5 01 01 85 ff ff e7 ff b7 91 fd cb 8f f9 93 89 fb y  87 ff o  95 ff ff eb 9d ff

dd e7 f5 E  01 99 f6 fe w  ^  _  d4 ed a7 83 ff ff 81 e7 fb {  99 ff 85 0f ff
dd e7 f5 E  a5 01 01 85 ff ff e7 ff b7 91 fd cb 8f f9 93 89 fb =  87 ff o  95 ff ff bf \r ff
```
<h2>Base station</h2>
<p>Uponor Smatrix Base X-245. Some sites claim it communicates over RS485. Do not confuse with the PRO version that supports Modbus.</p>
<h2>Thermostat</h2>
<p>The thermostats are Uponor Smatrix Base T-146 Bus, which have a Renesas RL78/L13 chipset which is used for serial communication (I think). From the datasheet:</p>
<blockquote>
<p>[80-pin]<br />
• CSI: 1 channel/UART (UART supporting LIN-bus): 1 channel/simplified I2C: 1 channel<br />
• CSI: 1 channel/UART: 1 channel/simplified I2C: 1 channel<br />
• UART: 2 channels</p>
</blockquote>
<h2>Setup</h2>
```Raspberry Pi 3B+         X-245        T-146
GPIO 14 (UART TX)  <-->    A    <-->    A
GPIO 15 (UART RX)  <-->    B    <-->    B
                           +    <-->    +
                           -    <-->    -
```

Who recognizes a protocol or can help me decode it? I've tried:

RS485/Modbus: Can't get a response, probably there is no modbus master/slave

LIN-bus: Can't see the 0x55 byte anywhere

I2C: i2cdetect -y 1 doesn't return an address

答:
So like Marcos suggested, the problem with sending was me not realising that for RX the RE should have been low and for TX the DE should have been high. Now I can read/write at the same time using this small class:

```<code>import time
import serial
import serial.rs485
import RPi.GPIO as GPIO
from crc import crc16

TX_DE = 11
RX_RE = 13

baud = 19200
T1_5 = 16500000 / baud / 1000
T3_5 = 38500000 / baud / 1000

GPIO.setwarnings(False)
GPIO.setmode(GPIO.BOARD)
GPIO.setup(TX_DE, GPIO.OUT, initial=GPIO.HIGH)  # HIGH enables TX
GPIO.setup(RX_RE, GPIO.OUT, initial=GPIO.LOW)  # LOW enables RX

ser = serial.rs485.RS485(port="/dev/serial0", baudrate=baud, timeout=T3_5)
ser.rs485_mode = serial.rs485.RS485Settings(
    rts_level_for_tx=False,
    rts_level_for_rx=False,
    loopback=True,
    delay_before_tx=T3_5,
    delay_before_rx=T3_5,
)

def write(command, direct=False):
    command_bytes = bytes.fromhex(command)
    if not direct:
        crc = crc16(command_bytes.decode("latin-1"))
        ser.write(command_bytes + bytes.fromhex(crc[-2:]) + bytes.fromhex(crc[:2]))
    else:
        ser.write(command_bytes)

def read():
    while True:
        x = ser.read(4095)
        print(x.hex())
</code>```

How ever now I can see the commands on the serial bus e.g. write(b'\x01\x04\x00\x01\x00\x0c\xA1\xCF'), but nothing happens (i.e. no Modbus response), what am I missing?

Body:
I'm trying to communicate with a Uponor Smatrix Base X-245 using a use a Raspberry Pi 3B+ over the serial port (UART), without much luck.

Here is a sample output from the UART (/dev/ttyS0) using pyserial:

import serial
ser = serial.Serial("/dev/ttyS0", baudrate=19200)
while True:
    received_data = ser.read()
    sleep(0.025)
    data_left = ser.inWaiting()
    received_data += ser.read(data_left)
    print (received_data.hex())

Data without any thermostats (8) connected:

00  00  00  00  00  00  00  00  00  00  00  00  00  00  c0
00  00  00  00  00  00  00  00  00  00  00  00  00  00  c0
00  00  00  00  00  00  00  00  00  00  00  00  00  00  c0
dd  e7  c9  6b  01  0a  2e  fe
dd  e7  af  83  01  56  89  ff
dd  e7  f5  33  01  bf  36  fe
dd  e7  f7  6d  01  ce  f6  fe
dd  e7  f7  2b  01  aa  36  fe
dd  e7  f7  75  01  da  f6  fe
dd  e7  f7  49  01  fe  76  ff
dd  e7  f5  45  01  99  f6  fe

Data with all thermostats (8) connected:

00  00  00  00  00  00  00  00  00  00  00  00  00  00  c0
00  00  00  00  00  00  00  00  00  00  00  00  00  00  c0
00  00  00  00  00  00  00  00  00  00  00  00  00  00  c0
dd  e7  c9  6b  01  0a  2e  fe  77  5e  6e  e3  ed  db  83  ff  ff  81  e7  fb  87  ff  6f  89  fb  3d  7b  99  ff  75  53  fe
dd  e7  c9  6b  a5  01  01  85  ff  ff  e7  ff  b7  91  fd  cb  8f  f9  93  89  fb  3d  87  ff  6f  95  ff  ff  8d  2f  ff
dd  e7  af  83  01  56  89  ff  77  de  6d  e4  ed  cb  83  ff  ff  81  e7  fb  87  ff  6f  89  fb  85  7b  99  ff  0b  07  ff
dd  e7  af  83  a5  01  01  85  ff  ff  e7  ff  b7  91  fd  cb  8f  f9  93  89  fb  85  87  ff  6f  95  ff  ff  25  3b  fe
dd  e7  f5  33  01  bf  36  fe  77  5e  df  d3  ed  eb  83  ff  ff  81  e7  fb  87  ff  6f  89  fb  85  7b  99  ff  71  0f  ff
dd  e7  f5  33  a5  01  01  85  ff  ff  e7  ff  b7  91  fd  cb  8f  f9  93  89  fb  85  87  ff  6f  95  ff  ff  d3  fb  ff
dd  e7  f7  6d  01  ce  f6  fe  f7  fe  fc  fe
dd  e7  f7  6d  a5  01  01  85  ff  ff  e7  ff  b7  91  fd  cb  8f  f9  93  89  fb  79  87  ff  6f  95  ff  ff  15  d3  fe
dd  e7  f7  2b  01  aa  36  fe
dd  e7  f7  2b  a5  01  01  85  ff  ff  e7  ff  b7  91  fd  cb  8f  f9  93  89  fb  67  87  ff  6f  95  ff  ff  43  91  ff
dd  e7  f7  75  01  da  f6  fe  77  de  5f  c7  ed  97  83  ff  ff  81  e7  fb  87  ff  6f  89  fb  07  7b  99  ff  cf  8f  fe
dd  e7  f7  75  a5  01  01  85  ff  ff  e7  ff  b7  91  fd  cb  8f  f9  93  89  fb  07  87  ff  6f  95  ff  ff  77  9b  ff
dd  e7  f7  49  01  fe  76  ff  77  de  5f  e2  ed  7b  83  ff  ff  81  e7  fb  87  ff  6f  89  fb  79  7b  99  ff  95  ff  ff
dd  e7  f7  49  a5  01  01  85  ff  7f  e7  ff  b7  91  fd  cb  8f  f9  93  89  fb  79  87  ff  6f  95  ff  ff  c1  3   ff
dd  e7  f5  45  01  99  f6  fe  77  5e  5f  d4  ed  bb  83  ff  ff  81  e7  fb  87  ff  6f  89  fb  3d  7b  99  ff  bb  d1  ff
dd  e7  f5  45  a5  01  01  85  ff  ff  e7  ff  b7  91  fd  cb  8f  f9  93  89  fb  3d  87  ff  6f  95  ff  ff  bf  0d  ff

Included here the same data, but color coded to recognize patterns.

Also the raw bytes that were captured:

00 00 00 00 00 00 00 00 00 00 00 00 00 00 c0
00 00 00 00 00 00 00 00 00 00 00 00 00 00 c0
00 00 00 00 00 00 00 00 00 00 00 00 00 00 c0
dd e7 c9 k  01 \n .  fe w  ^  n  e3 ed f7 83 ff ff 81 e7 fb {  99 ff -  1f fe
dd e7 c9 k  a5 01 01 85 ff ff e7 ff b7 91 fd cb 8f f9 93 89 fb =  87 ff o  95 ff ff 8d /  ff

dd e7 af 83 01 V  89 ff w  de m  e4 ed cb 83 ff ff 81 e7 fb {  99 ff e9 d1 ff
dd e7 af 83 a5 01 01 85 ff ff e7 ff b7 91 fd cb 8f f9 93 89 fb 85 87 ff o  95 ff ff %  ;  fe

dd e7 f5 3  01 bf 6  fe w  ^  df d3 ed e5 83 ff ff 81 e7 fb {  99 ff 93 a7 fe
dd e7 f5 3  a5 01 01 85 ff ff e7 ff b7 91 fd cb 8f f9 93 89 fb 85 87 ff o  95 ff ff d3 fb ff

dd e7 f7 m  01 ce f6 fe ff ff ff ff `  ff ff
dd e7 f7 m  a5 01 01 85 ff ff e7 ff b7 91 fd cb 8f f9 93 89 fb y  87 ff o  95 ff ff 15 d3 fe

dd e7 f7 +  01 aa 6  fe
dd e7 f7 +  a5 01 01 85 ff ff e7 ff b7 91 fd cb 8f f9 93 89 fb w  87 ff o  95 ff ff Q  11 ff

dd e7 f7 u  01 da f6 fe w  de _  c7 ed ?  87 ff ff 81 e7 fb {  99 ff bd i  ff
dd e7 f7 u  a5 01 01 85 ff ff e7 ff b7 91 fd cb 8f f9 93 89 fb 07 87 ff o  95 ff ff w  9b ff

dd e7 f7 I  01 fe v  ff w  de _  e2 ed u  83 ff ff 81 e7 fb {  99 ff cf a1 ff
dd e7 f7 I  a5 01 01 85 ff ff e7 ff b7 91 fd cb 8f f9 93 89 fb y  87 ff o  95 ff ff eb 9d ff

dd e7 f5 E  01 99 f6 fe w  ^  _  d4 ed a7 83 ff ff 81 e7 fb {  99 ff 85 0f ff
dd e7 f5 E  a5 01 01 85 ff ff e7 ff b7 91 fd cb 8f f9 93 89 fb =  87 ff o  95 ff ff bf \r ff

Base station

Uponor Smatrix Base X-245. Some sites claim it communicates over RS485. Do not confuse with the PRO version that supports Modbus.

Thermostat

The thermostats are Uponor Smatrix Base T-146 Bus, which have a Renesas RL78/L13 chipset which is used for serial communication (I think). From the datasheet:

[80-pin]

• CSI: 1 channel/UART (UART supporting LIN-bus): 1 channel/simplified I2C: 1 channel

• CSI: 1 channel/UART: 1 channel/simplified I2C: 1 channel

• UART: 2 channels

Setup

Raspberry Pi 3B+         X-245        T-146
GPIO 14 (UART TX)  <-->    A    <-->    A
GPIO 15 (UART RX)  <-->    B    <-->    B
                           +    <-->    +
                           -    <-->    -

Who recognizes a protocol or can help me decode it? I've tried:

RS485/Modbus: Can't get a response, probably there is no modbus master/slave

LIN-bus: Can't see the 0x55 byte anywhere

I2C: i2cdetect -y 1 doesn't return an address

Body:
So like Marcos suggested, the problem with sending was me not realising that for RX the RE should have been low and for TX the DE should have been high. Now I can read/write at the same time using this small class:

import time
import serial
import serial.rs485
import RPi.GPIO as GPIO
from crc import crc16

TX_DE = 11
RX_RE = 13

baud = 19200
T1_5 = 16500000 / baud / 1000
T3_5 = 38500000 / baud / 1000

GPIO.setwarnings(False)
GPIO.setmode(GPIO.BOARD)
GPIO.setup(TX_DE, GPIO.OUT, initial=GPIO.HIGH)  # HIGH enables TX
GPIO.setup(RX_RE, GPIO.OUT, initial=GPIO.LOW)  # LOW enables RX

ser = serial.rs485.RS485(port="/dev/serial0", baudrate=baud, timeout=T3_5)
ser.rs485_mode = serial.rs485.RS485Settings(
    rts_level_for_tx=False,
    rts_level_for_rx=False,
    loopback=True,
    delay_before_tx=T3_5,
    delay_before_rx=T3_5,
)

def write(command, direct=False):
    command_bytes = bytes.fromhex(command)
    if not direct:
        crc = crc16(command_bytes.decode("latin-1"))
        ser.write(command_bytes + bytes.fromhex(crc[-2:]) + bytes.fromhex(crc[:2]))
    else:
        ser.write(command_bytes)

def read():
    while True:
        x = ser.read(4095)
        print(x.hex())

How ever now I can see the commands on the serial bus e.g. write(b'\x01\x04\x00\x01\x00\x0c\xA1\xCF'), but nothing happens (i.e. no Modbus response), what am I missing?


问:
Detect if Facebook user likes a page without app authorization
I'm building an iframe Facebook application and I want to set up a 'like gate' where a user has to like my page to continue past the opening of the application. (After they click like I allow them to click a 'continue' button using JavaScript).

Is there any way to detect if a visiting Facebook user has liked a page without first forcing them to authorize my application? I know I can query their likes using the graph api or fql but I'd like to avoid forcing them through the auth dialog for this one part of the app. The reason I ask is that I've seen some applications on Facebook that appear to allow this functionality.

For example, this page:
http://www.facebook.com/velcrobrand?sk=app_156218351098324

I don't have to authorize the facebook application but the page knows if I have liked it or not.

答:
I've made some progress figuring this out.

http://www.chilipepperdesign.com/2011/02/15/reveal-fan-gate-like-gate-facebook-iframe-tab-tutorial-with-php

The method described in the above link only works if the page is loaded inside a tab of an application fan page and NOT the applications primary iframe page.

Here: 'pages' object is present
http://www.facebook.com/apps/application.php?id=249709505050393&sk=app_249709505050393

Here: 'pages' object is not present
http://apps.facebook.com/flipkey/

Not sure why this is but I think I have enough to move forwards and create my fan/like-gate.


问:
JavaFx msal4j NoClassDefFoundError
I am running into an exception trying to get the example code here working with a JavaFx desktop app.

Java Version - OpenJDK 17
OS - Linux Mint 20.1 Ulyssa

My Azure app reg has been setup,

```<code>private String PUBLIC_CLIENT_ID = "my client id from azure";
private String AUTHORITY = "https://login.microsoftonline.com/organizations/";
</code>```

My code is setup inside my Controller's constructor

```<code>    public HelloController() throws MalformedURLException, URISyntaxException {
        PublicClientApplication app = PublicClientApplication
                .builder(PUBLIC_CLIENT_ID)
                .authority(AUTHORITY)
                .build();

        InteractiveRequestParameters parameters = InteractiveRequestParameters
                .builder(new URI("http://localhost"))
                .scopes(Collections.singleton("user.read"))
                .build();

        IAuthenticationResult result = app.acquireToken(parameters).join();

    }
</code>```

And gets this stack trace upon startup complaining about not being able to find com/sun/net/httpserver/HttpHandler

```<code>Exception in Application start method
java.lang.reflect.InvocationTargetException
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:568)
    at javafx.graphics/com.sun.javafx.application.LauncherImpl.launchApplicationWithArgs(LauncherImpl.java:465)
    at javafx.graphics/com.sun.javafx.application.LauncherImpl.launchApplication(LauncherImpl.java:364)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:568)
    at java.base/sun.launcher.LauncherHelper$FXHelper.main(LauncherHelper.java:1071)
Caused by: java.lang.RuntimeException: Exception in Application start method
    at javafx.graphics/com.sun.javafx.application.LauncherImpl.launchApplication1(LauncherImpl.java:901)
    at javafx.graphics/com.sun.javafx.application.LauncherImpl.lambda$launchApplication$2(LauncherImpl.java:196)
    at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: javafx.fxml.LoadException:
/home/chewy/IdeaProjects/FxTimer/target/classes/com/example/fxtimer/hello-view.fxml:23

    at javafx.fxml/javafx.fxml.FXMLLoader.constructLoadException(FXMLLoader.java:2707)
    at javafx.fxml/javafx.fxml.FXMLLoader$ValueElement.processAttribute(FXMLLoader.java:944)
    at javafx.fxml/javafx.fxml.FXMLLoader$InstanceDeclarationElement.processAttribute(FXMLLoader.java:981)
    at javafx.fxml/javafx.fxml.FXMLLoader$Element.processStartElement(FXMLLoader.java:230)
    at javafx.fxml/javafx.fxml.FXMLLoader$ValueElement.processStartElement(FXMLLoader.java:755)
    at javafx.fxml/javafx.fxml.FXMLLoader.processStartElement(FXMLLoader.java:2808)
    at javafx.fxml/javafx.fxml.FXMLLoader.loadImpl(FXMLLoader.java:2634)
    at javafx.fxml/javafx.fxml.FXMLLoader.loadImpl(FXMLLoader.java:2548)
    at javafx.fxml/javafx.fxml.FXMLLoader.load(FXMLLoader.java:2516)
    at com.example.fxtimer/com.example.fxtimer.HelloApplication.start(HelloApplication.java:16)
    at javafx.graphics/com.sun.javafx.application.LauncherImpl.lambda$launchApplication1$9(LauncherImpl.java:847)
    at javafx.graphics/com.sun.javafx.application.PlatformImpl.lambda$runAndWait$12(PlatformImpl.java:484)
    at javafx.graphics/com.sun.javafx.application.PlatformImpl.lambda$runLater$10(PlatformImpl.java:457)
    at java.base/java.security.AccessController.doPrivileged(AccessController.java:399)
    at javafx.graphics/com.sun.javafx.application.PlatformImpl.lambda$runLater$11(PlatformImpl.java:456)
    at javafx.graphics/com.sun.glass.ui.InvokeLaterDispatcher$Future.run(InvokeLaterDispatcher.java:96)
    at javafx.graphics/com.sun.glass.ui.gtk.GtkApplication._runLoop(Native Method)
    at javafx.graphics/com.sun.glass.ui.gtk.GtkApplication.lambda$runLoop$11(GtkApplication.java:290)
    ... 1 more
Caused by: java.lang.reflect.InvocationTargetException
    at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77)
    at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
    at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:480)
    at javafx.fxml/javafx.fxml.FXMLLoader$ValueElement.processAttribute(FXMLLoader.java:939)
    ... 17 more
Caused by: java.lang.NoClassDefFoundError: com/sun/net/httpserver/HttpHandler
    at msal4j@1.11.2/com.microsoft.aad.msal4j.AbstractClientApplicationBase.getAuthenticationResultSupplier(AbstractClientApplicationBase.java:145)
    at msal4j@1.11.2/com.microsoft.aad.msal4j.AbstractClientApplicationBase.executeRequest(AbstractClientApplicationBase.java:117)
    at msal4j@1.11.2/com.microsoft.aad.msal4j.PublicClientApplication.acquireToken(PublicClientApplication.java:115)
    at com.example.fxtimer/com.example.fxtimer.HelloController.<init>(HelloController.java:112)
    ... 23 more
Caused by: java.lang.ClassNotFoundException: com.sun.net.httpserver.HttpHandler
    at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
    at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520)
    ... 27 more
Exception running application com.example.fxtimer.HelloApplication

Process finished with exit code 1
</code>```

If I do add that httpserver as a dependency

```<code> <dependency>
            <groupId>com.sun.net.httpserver</groupId>
            <artifactId>http</artifactId>
            <version>20070405</version>
        </dependency>
</code>```

I get a little further until something else is undefined. No browser window ever launches.

```<code>Exception in Application start method
java.lang.reflect.InvocationTargetException
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:568)
    at javafx.graphics/com.sun.javafx.application.LauncherImpl.launchApplicationWithArgs(LauncherImpl.java:465)
    at javafx.graphics/com.sun.javafx.application.LauncherImpl.launchApplication(LauncherImpl.java:364)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:568)
    at java.base/sun.launcher.LauncherHelper$FXHelper.main(LauncherHelper.java:1071)
Caused by: java.lang.RuntimeException: Exception in Application start method
    at javafx.graphics/com.sun.javafx.application.LauncherImpl.launchApplication1(LauncherImpl.java:901)
    at javafx.graphics/com.sun.javafx.application.LauncherImpl.lambda$launchApplication$2(LauncherImpl.java:196)
    at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: javafx.fxml.LoadException:
/home/chewy/IdeaProjects/FxTimer/target/classes/com/example/fxtimer/hello-view.fxml:23

    at javafx.fxml/javafx.fxml.FXMLLoader.constructLoadException(FXMLLoader.java:2707)
    at javafx.fxml/javafx.fxml.FXMLLoader$ValueElement.processAttribute(FXMLLoader.java:944)
    at javafx.fxml/javafx.fxml.FXMLLoader$InstanceDeclarationElement.processAttribute(FXMLLoader.java:981)
    at javafx.fxml/javafx.fxml.FXMLLoader$Element.processStartElement(FXMLLoader.java:230)
    at javafx.fxml/javafx.fxml.FXMLLoader$ValueElement.processStartElement(FXMLLoader.java:755)
    at javafx.fxml/javafx.fxml.FXMLLoader.processStartElement(FXMLLoader.java:2808)
    at javafx.fxml/javafx.fxml.FXMLLoader.loadImpl(FXMLLoader.java:2634)
    at javafx.fxml/javafx.fxml.FXMLLoader.loadImpl(FXMLLoader.java:2548)
    at javafx.fxml/javafx.fxml.FXMLLoader.load(FXMLLoader.java:2516)
    at com.example.fxtimer/com.example.fxtimer.HelloApplication.start(HelloApplication.java:16)
    at javafx.graphics/com.sun.javafx.application.LauncherImpl.lambda$launchApplication1$9(LauncherImpl.java:847)
    at javafx.graphics/com.sun.javafx.application.PlatformImpl.lambda$runAndWait$12(PlatformImpl.java:484)
    at javafx.graphics/com.sun.javafx.application.PlatformImpl.lambda$runLater$10(PlatformImpl.java:457)
    at java.base/java.security.AccessController.doPrivileged(AccessController.java:399)
    at javafx.graphics/com.sun.javafx.application.PlatformImpl.lambda$runLater$11(PlatformImpl.java:456)
    at javafx.graphics/com.sun.glass.ui.InvokeLaterDispatcher$Future.run(InvokeLaterDispatcher.java:96)
    at javafx.graphics/com.sun.glass.ui.gtk.GtkApplication._runLoop(Native Method)
    at javafx.graphics/com.sun.glass.ui.gtk.GtkApplication.lambda$runLoop$11(GtkApplication.java:290)
    ... 1 more
Caused by: java.lang.reflect.InvocationTargetException
    at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77)
    at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
    at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:480)
    at javafx.fxml/javafx.fxml.FXMLLoader$ValueElement.processAttribute(FXMLLoader.java:939)
    ... 17 more
Caused by: java.util.concurrent.CompletionException: java.lang.NoClassDefFoundError: sun/misc/ServiceConfigurationError
    at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:315)
    at java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:320)
    at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1770)
    at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.exec(CompletableFuture.java:1760)
    at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373)
    at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182)
    at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655)
    at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622)
    at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165)
Caused by: java.lang.NoClassDefFoundError: sun/misc/ServiceConfigurationError
    at com.sun.net.httpserver.HttpServer.create(HttpServer.java:110)
    at msal4j@1.11.2/com.microsoft.aad.msal4j.HttpListener.startListener(HttpListener.java:19)
    at msal4j@1.11.2/com.microsoft.aad.msal4j.AcquireTokenByInteractiveFlowSupplier.startHttpListener(AcquireTokenByInteractiveFlowSupplier.java:90)
    at msal4j@1.11.2/com.microsoft.aad.msal4j.AcquireTokenByInteractiveFlowSupplier.getAuthorizationResult(AcquireTokenByInteractiveFlowSupplier.java:55)
    at msal4j@1.11.2/com.microsoft.aad.msal4j.AcquireTokenByInteractiveFlowSupplier.execute(AcquireTokenByInteractiveFlowSupplier.java:37)
    at msal4j@1.11.2/com.microsoft.aad.msal4j.AuthenticationResultSupplier.get(AuthenticationResultSupplier.java:71)
    at msal4j@1.11.2/com.microsoft.aad.msal4j.AuthenticationResultSupplier.get(AuthenticationResultSupplier.java:20)
    at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1768)
    ... 6 more
Caused by: java.lang.ClassNotFoundException: sun.misc.ServiceConfigurationError
    at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
    at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520)
    ... 14 more
Exception running application com.example.fxtimer.HelloApplication

Process finished with exit code 1

</code>```

Does anyone know what is causing this Class definitions to be missing/what I need to do in order to get everything working nicely?

Thanks for any help!

答:
You don't need to add a dependency for the http server, it is in the jdk.

Perhaps you need to require it as a module.

Asker confirmed this worked in comments:

my javafx app does have a module-info.java file and by adding requires jdk.httpserver I am able to get past the errors

The Microsoft library should define a module-info.java file which appropriately defines the module and its requirements; however, it does not do so. Log a bug report in their issue tracker. You might be able to get it to work in a modular environment by some hacks or VM arguments or running stuff off the class path instead.

Try to replicate this behavior in a modular Java app (app with module-info.java), which does not use JavaFX, to create a minimal reproducible example, then edit the question to include your example and also put the example in your issue report to Microsoft.

Asker also did this, as confirmed in comments, see the issue report at:

https://github.com/AzureAD/microsoft-authentication-library-for-java/issues/473

The second exception you posted,

```<code>java.lang.ClassNotFoundException: sun.misc.ServiceConfigurationError
</code>```

is because sun.misc APIs were never part of the public JRE API and several sun.misc APIs (including this one) were deprecated in JDK 9 and removed from the JDK as part of the modularity work.

The exception occurred because you tried to bring in a really old dependency on a 2007 version of the com.sun.net.httpserver API which is not compatible with Java 9+.

Title: JavaFx msal4j NoClassDefFoundError

Body:
I am running into an exception trying to get the example code here working with a JavaFx desktop app.

Java Version - OpenJDK 17
OS - Linux Mint 20.1 Ulyssa

My Azure app reg has been setup,

private String PUBLIC_CLIENT_ID = "my client id from azure";
private String AUTHORITY = "https://login.microsoftonline.com/organizations/";

My code is setup inside my Controller's constructor

    public HelloController() throws MalformedURLException, URISyntaxException {
        PublicClientApplication app = PublicClientApplication
                .builder(PUBLIC_CLIENT_ID)
                .authority(AUTHORITY)
                .build();

        InteractiveRequestParameters parameters = InteractiveRequestParameters
                .builder(new URI("http://localhost"))
                .scopes(Collections.singleton("user.read"))
                .build();

        IAuthenticationResult result = app.acquireToken(parameters).join();

    }

And gets this stack trace upon startup complaining about not being able to find com/sun/net/httpserver/HttpHandler

Exception in Application start method
java.lang.reflect.InvocationTargetException
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:568)
    at javafx.graphics/com.sun.javafx.application.LauncherImpl.launchApplicationWithArgs(LauncherImpl.java:465)
    at javafx.graphics/com.sun.javafx.application.LauncherImpl.launchApplication(LauncherImpl.java:364)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:568)
    at java.base/sun.launcher.LauncherHelper$FXHelper.main(LauncherHelper.java:1071)
Caused by: java.lang.RuntimeException: Exception in Application start method
    at javafx.graphics/com.sun.javafx.application.LauncherImpl.launchApplication1(LauncherImpl.java:901)
    at javafx.graphics/com.sun.javafx.application.LauncherImpl.lambda$launchApplication$2(LauncherImpl.java:196)
    at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: javafx.fxml.LoadException:
/home/chewy/IdeaProjects/FxTimer/target/classes/com/example/fxtimer/hello-view.fxml:23

    at javafx.fxml/javafx.fxml.FXMLLoader.constructLoadException(FXMLLoader.java:2707)
    at javafx.fxml/javafx.fxml.FXMLLoader$ValueElement.processAttribute(FXMLLoader.java:944)
    at javafx.fxml/javafx.fxml.FXMLLoader$InstanceDeclarationElement.processAttribute(FXMLLoader.java:981)
    at javafx.fxml/javafx.fxml.FXMLLoader$Element.processStartElement(FXMLLoader.java:230)
    at javafx.fxml/javafx.fxml.FXMLLoader$ValueElement.processStartElement(FXMLLoader.java:755)
    at javafx.fxml/javafx.fxml.FXMLLoader.processStartElement(FXMLLoader.java:2808)
    at javafx.fxml/javafx.fxml.FXMLLoader.loadImpl(FXMLLoader.java:2634)
    at javafx.fxml/javafx.fxml.FXMLLoader.loadImpl(FXMLLoader.java:2548)
    at javafx.fxml/javafx.fxml.FXMLLoader.load(FXMLLoader.java:2516)
    at com.example.fxtimer/com.example.fxtimer.HelloApplication.start(HelloApplication.java:16)
    at javafx.graphics/com.sun.javafx.application.LauncherImpl.lambda$launchApplication1$9(LauncherImpl.java:847)
    at javafx.graphics/com.sun.javafx.application.PlatformImpl.lambda$runAndWait$12(PlatformImpl.java:484)
    at javafx.graphics/com.sun.javafx.application.PlatformImpl.lambda$runLater$10(PlatformImpl.java:457)
    at java.base/java.security.AccessController.doPrivileged(AccessController.java:399)
    at javafx.graphics/com.sun.javafx.application.PlatformImpl.lambda$runLater$11(PlatformImpl.java:456)
    at javafx.graphics/com.sun.glass.ui.InvokeLaterDispatcher$Future.run(InvokeLaterDispatcher.java:96)
    at javafx.graphics/com.sun.glass.ui.gtk.GtkApplication._runLoop(Native Method)
    at javafx.graphics/com.sun.glass.ui.gtk.GtkApplication.lambda$runLoop$11(GtkApplication.java:290)
    ... 1 more
Caused by: java.lang.reflect.InvocationTargetException
    at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77)
    at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
    at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:480)
    at javafx.fxml/javafx.fxml.FXMLLoader$ValueElement.processAttribute(FXMLLoader.java:939)
    ... 17 more
Caused by: java.lang.NoClassDefFoundError: com/sun/net/httpserver/HttpHandler
    at msal4j@1.11.2/com.microsoft.aad.msal4j.AbstractClientApplicationBase.getAuthenticationResultSupplier(AbstractClientApplicationBase.java:145)
    at msal4j@1.11.2/com.microsoft.aad.msal4j.AbstractClientApplicationBase.executeRequest(AbstractClientApplicationBase.java:117)
    at msal4j@1.11.2/com.microsoft.aad.msal4j.PublicClientApplication.acquireToken(PublicClientApplication.java:115)
    at com.example.fxtimer/com.example.fxtimer.HelloController.<init>(HelloController.java:112)
    ... 23 more
Caused by: java.lang.ClassNotFoundException: com.sun.net.httpserver.HttpHandler
    at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
    at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520)
    ... 27 more
Exception running application com.example.fxtimer.HelloApplication

Process finished with exit code 1

If I do add that httpserver as a dependency

 <dependency>
            <groupId>com.sun.net.httpserver</groupId>
            <artifactId>http</artifactId>
            <version>20070405</version>
        </dependency>

I get a little further until something else is undefined. No browser window ever launches.

Exception in Application start method
java.lang.reflect.InvocationTargetException
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:568)
    at javafx.graphics/com.sun.javafx.application.LauncherImpl.launchApplicationWithArgs(LauncherImpl.java:465)
    at javafx.graphics/com.sun.javafx.application.LauncherImpl.launchApplication(LauncherImpl.java:364)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:568)
    at java.base/sun.launcher.LauncherHelper$FXHelper.main(LauncherHelper.java:1071)
Caused by: java.lang.RuntimeException: Exception in Application start method
    at javafx.graphics/com.sun.javafx.application.LauncherImpl.launchApplication1(LauncherImpl.java:901)
    at javafx.graphics/com.sun.javafx.application.LauncherImpl.lambda$launchApplication$2(LauncherImpl.java:196)
    at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: javafx.fxml.LoadException:
/home/chewy/IdeaProjects/FxTimer/target/classes/com/example/fxtimer/hello-view.fxml:23

    at javafx.fxml/javafx.fxml.FXMLLoader.constructLoadException(FXMLLoader.java:2707)
    at javafx.fxml/javafx.fxml.FXMLLoader$ValueElement.processAttribute(FXMLLoader.java:944)
    at javafx.fxml/javafx.fxml.FXMLLoader$InstanceDeclarationElement.processAttribute(FXMLLoader.java:981)
    at javafx.fxml/javafx.fxml.FXMLLoader$Element.processStartElement(FXMLLoader.java:230)
    at javafx.fxml/javafx.fxml.FXMLLoader$ValueElement.processStartElement(FXMLLoader.java:755)
    at javafx.fxml/javafx.fxml.FXMLLoader.processStartElement(FXMLLoader.java:2808)
    at javafx.fxml/javafx.fxml.FXMLLoader.loadImpl(FXMLLoader.java:2634)
    at javafx.fxml/javafx.fxml.FXMLLoader.loadImpl(FXMLLoader.java:2548)
    at javafx.fxml/javafx.fxml.FXMLLoader.load(FXMLLoader.java:2516)
    at com.example.fxtimer/com.example.fxtimer.HelloApplication.start(HelloApplication.java:16)
    at javafx.graphics/com.sun.javafx.application.LauncherImpl.lambda$launchApplication1$9(LauncherImpl.java:847)
    at javafx.graphics/com.sun.javafx.application.PlatformImpl.lambda$runAndWait$12(PlatformImpl.java:484)
    at javafx.graphics/com.sun.javafx.application.PlatformImpl.lambda$runLater$10(PlatformImpl.java:457)
    at java.base/java.security.AccessController.doPrivileged(AccessController.java:399)
    at javafx.graphics/com.sun.javafx.application.PlatformImpl.lambda$runLater$11(PlatformImpl.java:456)
    at javafx.graphics/com.sun.glass.ui.InvokeLaterDispatcher$Future.run(InvokeLaterDispatcher.java:96)
    at javafx.graphics/com.sun.glass.ui.gtk.GtkApplication._runLoop(Native Method)
    at javafx.graphics/com.sun.glass.ui.gtk.GtkApplication.lambda$runLoop$11(GtkApplication.java:290)
    ... 1 more
Caused by: java.lang.reflect.InvocationTargetException
    at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77)
    at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
    at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:480)
    at javafx.fxml/javafx.fxml.FXMLLoader$ValueElement.processAttribute(FXMLLoader.java:939)
    ... 17 more
Caused by: java.util.concurrent.CompletionException: java.lang.NoClassDefFoundError: sun/misc/ServiceConfigurationError
    at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:315)
    at java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:320)
    at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1770)
    at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.exec(CompletableFuture.java:1760)
    at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373)
    at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182)
    at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655)
    at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622)
    at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165)
Caused by: java.lang.NoClassDefFoundError: sun/misc/ServiceConfigurationError
    at com.sun.net.httpserver.HttpServer.create(HttpServer.java:110)
    at msal4j@1.11.2/com.microsoft.aad.msal4j.HttpListener.startListener(HttpListener.java:19)
    at msal4j@1.11.2/com.microsoft.aad.msal4j.AcquireTokenByInteractiveFlowSupplier.startHttpListener(AcquireTokenByInteractiveFlowSupplier.java:90)
    at msal4j@1.11.2/com.microsoft.aad.msal4j.AcquireTokenByInteractiveFlowSupplier.getAuthorizationResult(AcquireTokenByInteractiveFlowSupplier.java:55)
    at msal4j@1.11.2/com.microsoft.aad.msal4j.AcquireTokenByInteractiveFlowSupplier.execute(AcquireTokenByInteractiveFlowSupplier.java:37)
    at msal4j@1.11.2/com.microsoft.aad.msal4j.AuthenticationResultSupplier.get(AuthenticationResultSupplier.java:71)
    at msal4j@1.11.2/com.microsoft.aad.msal4j.AuthenticationResultSupplier.get(AuthenticationResultSupplier.java:20)
    at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1768)
    ... 6 more
Caused by: java.lang.ClassNotFoundException: sun.misc.ServiceConfigurationError
    at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
    at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520)
    ... 14 more
Exception running application com.example.fxtimer.HelloApplication

Process finished with exit code 1

Does anyone know what is causing this Class definitions to be missing/what I need to do in order to get everything working nicely?

Thanks for any help!

Body:
You don't need to add a dependency for the http server, it is in the jdk.

Perhaps you need to require it as a module.

Asker confirmed this worked in comments:

my javafx app does have a module-info.java file and by adding requires jdk.httpserver I am able to get past the errors

The Microsoft library should define a module-info.java file which appropriately defines the module and its requirements; however, it does not do so. Log a bug report in their issue tracker. You might be able to get it to work in a modular environment by some hacks or VM arguments or running stuff off the class path instead.

Try to replicate this behavior in a modular Java app (app with module-info.java), which does not use JavaFX, to create a minimal reproducible example, then edit the question to include your example and also put the example in your issue report to Microsoft.

Asker also did this, as confirmed in comments, see the issue report at:

https://github.com/AzureAD/microsoft-authentication-library-for-java/issues/473

The second exception you posted,

java.lang.ClassNotFoundException: sun.misc.ServiceConfigurationError

is because sun.misc APIs were never part of the public JRE API and several sun.misc APIs (including this one) were deprecated in JDK 9 and removed from the JDK as part of the modularity work.

The exception occurred because you tried to bring in a really old dependency on a 2007 version of the com.sun.net.httpserver API which is not compatible with Java 9+.


问:
Getting permission denied even as root inside the docker container
Connecting to running docker container as a root still gets Operation not permitted error when trying to apt-get update, yet I can still see sensitive file like /etc/passwd. Below are my configurations and also the error message from apt-get update. My host operating system is Ubuntu 18.04.3. My docker version is Docker version 19.03.5, build 633a0ea838

I create a container with the following Dockerfile

```<code>FROM python:3.8-slim-buster
RUN useradd -ms /bin/bash andrej
WORKDIR /home/andrej
COPY . /home/andrej/

RUN apt-get update && \
    apt-get install -y gcc && \
    pip install -r requirements.txt && \
    apt-get remove -y gcc && apt-get -y autoremove

RUN chown andrej:andrej pycurl && \
    chmod 0744 pycurl

USER andrej
ENTRYPOINT ["uwsgi"]
CMD ["--ini", "uwsgi.ini"]

</code>```

starting it with docker compose looking like this:

```<code>version: "3.3"

services:

  andrej-cv:
    build: ./andrej_cv
    container_name: andrej-cv
    restart: always
    security_opt:
      - no-new-privileges
    expose:
      - 5000
    healthcheck:
      test: ./pycurl --host=127.0.0.1 --port=5050 --uri=/health_check
      interval: 1m30s
      timeout: 10s
      retries: 3
</code>```

My docker daemon config:

```<code>{
    "icc": false,
    "userns-remap": "default",
    "log-driver": "syslog",
    "live-restore": true,
    "userland-proxy": false,
    "no-new-privileges": true
}

</code>```

I connect to the container with following command (as root):
docker exec -it -u root <container_hash> /bin/bash but when I try to update I got the following:

```<code>root@ed984abff684:/home/andrej# apt-get update
E: setgroups 65534 failed - setgroups (1: Operation not permitted)
E: setegid 65534 failed - setegid (1: Operation not permitted)
E: seteuid 100 failed - seteuid (1: Operation not permitted)
E: setgroups 0 failed - setgroups (1: Operation not permitted)
Hit:1 http://deb.debian.org/debian buster InRelease
Ign:2 http://deb.debian.org/debian buster-updates InRelease
Err:4 http://deb.debian.org/debian buster-updates Release
  Could not open file /var/lib/apt/lists/partial/deb.debian.org_debian_dists_buster-updates_Release - open (13: Permission denied) [IP: 151.101.36.204 80]
Hit:3 http://security-cdn.debian.org/debian-security buster/updates InRelease
rm: cannot remove '/var/cache/apt/archives/partial/*.deb': Permission denied
Reading package lists... Done
W: chown to _apt:root of directory /var/lib/apt/lists/partial failed - SetupAPTPartialDirectory (1: Operation not permitted)
W: chmod 0700 of directory /var/lib/apt/lists/partial failed - SetupAPTPartialDirectory (1: Operation not permitted)
W: chown to _apt:root of directory /var/lib/apt/lists/auxfiles failed - SetupAPTPartialDirectory (1: Operation not permitted)
W: chmod 0700 of directory /var/lib/apt/lists/auxfiles failed - SetupAPTPartialDirectory (1: Operation not permitted)
E: setgroups 65534 failed - setgroups (1: Operation not permitted)
E: setegid 65534 failed - setegid (1: Operation not permitted)
E: seteuid 100 failed - seteuid (1: Operation not permitted)
W: Download is performed unsandboxed as root as file '/var/lib/apt/lists/partial/deb.debian.org_debian_dists_buster_InRelease' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied)
E: setgroups 0 failed - setgroups (1: Operation not permitted)
W: Problem unlinking the file /var/lib/apt/lists/partial/deb.debian.org_debian_dists_buster_InRelease - PrepareFiles (13: Permission denied)
W: Problem unlinking the file /var/lib/apt/lists/partial/deb.debian.org_debian_dists_buster-updates_InRelease - PrepareFiles (13: Permission denied)
W: Problem unlinking the file /var/lib/apt/lists/partial/deb.debian.org_debian_dists_buster-updates_Release - PrepareFiles (13: Permission denied)
E: The repository 'http://deb.debian.org/debian buster-updates Release' no longer has a Release file.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
W: Problem unlinking the file /var/lib/apt/lists/partial/security.debian.org_debian-security_dists_buster_updates_InRelease - PrepareFiles (13: Permission denied)
</code>```

In the container /etc/subuid and /etc/subgid look like this (both):

```<code>andrej:100000:65536
</code>```

On the host /etc/subuid and /etc/subgid look like this (both):

```<code>andrej:100000:65536
dockremap:165536:65536
</code>```

Apparmor is running on Ubuntu host with following status (only docker-default profile):

```<code>andrej@machine:/etc/apparmor.d$ sudo aa-status
apparmor module is loaded.
38 profiles are loaded.
36 profiles are in enforce mode.
   /sbin/dhclient
   /snap/core/8268/usr/lib/snapd/snap-confine
   /snap/core/8268/usr/lib/snapd/snap-confine//mount-namespace-capture-helper
   /usr/bin/evince
   /usr/bin/evince-previewer
   /usr/bin/evince-previewer//sanitized_helper
   /usr/bin/evince-thumbnailer
   /usr/bin/evince//sanitized_helper
   /usr/bin/man
   /usr/lib/NetworkManager/nm-dhcp-client.action
   /usr/lib/NetworkManager/nm-dhcp-helper
   /usr/lib/connman/scripts/dhclient-script
   /usr/lib/cups/backend/cups-pdf
   /usr/lib/snapd/snap-confine
   /usr/lib/snapd/snap-confine//mount-namespace-capture-helper
   /usr/sbin/cups-browsed
   /usr/sbin/cupsd
   /usr/sbin/cupsd//third_party
   /usr/sbin/ippusbxd
   /usr/sbin/tcpdump
   docker-default
   libreoffice-senddoc
   libreoffice-soffice//gpg
   libreoffice-xpdfimport
   man_filter
   man_groff
   snap-update-ns.core
   snap-update-ns.gnome-calculator
   snap-update-ns.gnome-characters
   snap-update-ns.gnome-logs
   snap-update-ns.gnome-system-monitor
   snap.core.hook.configure
   snap.gnome-calculator.gnome-calculator
   snap.gnome-characters.gnome-characters
   snap.gnome-logs.gnome-logs
   snap.gnome-system-monitor.gnome-system-monitor
2 profiles are in complain mode.
   libreoffice-oopslash
   libreoffice-soffice
17 processes have profiles defined.
14 processes are in enforce mode.
   docker-default (1101)
   docker-default (1102)
   docker-default (1111)
   docker-default (1600)
   docker-default (1728)
   docker-default (1729)
   docker-default (1730)
   docker-default (1731)
   docker-default (1732)
   docker-default (1798)
   docker-default (1799)
   docker-default (1800)
   docker-default (1801)
   docker-default (1802)
0 processes are in complain mode.
3 processes are unconfined but have a profile defined.
   /sbin/dhclient (491)
   /usr/sbin/cups-browsed (431)
   /usr/sbin/cupsd (402)
</code>```

Selinux seems to be disabled as there is not /etc/selinux/config file and getenfoce and sestatus command are not available.

Also su andrej command run as root (where andrej is unprivileged user in the container) errors out with su: cannot set groups: Operation not permitted

答:
Not sure about Docker, but in kubernetes in runc container for me helps:

Get root access to container
List all containers

```<code>minikube ssh docker container ls
</code>```

Connect to your container (use your container id from previous command instead of 44a7ad70d45b):

```<code>minikube ssh "docker container exec -it -u 0 44a7ad70d45b /bin/bash"
</code>```

As root inside container:

```<code>root@mycontainer:/# apt-config dump | grep Sandbox::User
APT::Sandbox::User "_apt";

root@mycontainer:/# cat <<EOF > /etc/apt/apt.conf.d/sandbox-disable
APT::Sandbox::User "root";
EOF
</code>```

Be sure, that result is valid:

```<code>root@mycontainer:/# apt-config dump | grep Sandbox::User
APT::Sandbox::User "root";
</code>```

apt update and see

```<code>Get:1 http://archive.ubuntu.com/ubuntu hirsute InRelease [269 kB]
Get:2 http://apt.postgresql.org/pub/repos/apt hirsute-pgdg InRelease [16.7 kB]
...
</code>```

Body:
Connecting to running docker container as a root still gets Operation not permitted error when trying to apt-get update, yet I can still see sensitive file like /etc/passwd. Below are my configurations and also the error message from apt-get update. My host operating system is Ubuntu 18.04.3. My docker version is Docker version 19.03.5, build 633a0ea838

I create a container with the following Dockerfile

FROM python:3.8-slim-buster
RUN useradd -ms /bin/bash andrej
WORKDIR /home/andrej
COPY . /home/andrej/

RUN apt-get update && \
    apt-get install -y gcc && \
    pip install -r requirements.txt && \
    apt-get remove -y gcc && apt-get -y autoremove

RUN chown andrej:andrej pycurl && \
    chmod 0744 pycurl

USER andrej
ENTRYPOINT ["uwsgi"]
CMD ["--ini", "uwsgi.ini"]

starting it with docker compose looking like this:

version: "3.3"

services:

  andrej-cv:
    build: ./andrej_cv
    container_name: andrej-cv
    restart: always
    security_opt:
      - no-new-privileges
    expose:
      - 5000
    healthcheck:
      test: ./pycurl --host=127.0.0.1 --port=5050 --uri=/health_check
      interval: 1m30s
      timeout: 10s
      retries: 3

My docker daemon config:

{
    "icc": false,
    "userns-remap": "default",
    "log-driver": "syslog",
    "live-restore": true,
    "userland-proxy": false,
    "no-new-privileges": true
}

I connect to the container with following command (as root):
docker exec -it -u root <container_hash> /bin/bash but when I try to update I got the following:

root@ed984abff684:/home/andrej# apt-get update
E: setgroups 65534 failed - setgroups (1: Operation not permitted)
E: setegid 65534 failed - setegid (1: Operation not permitted)
E: seteuid 100 failed - seteuid (1: Operation not permitted)
E: setgroups 0 failed - setgroups (1: Operation not permitted)
Hit:1 http://deb.debian.org/debian buster InRelease
Ign:2 http://deb.debian.org/debian buster-updates InRelease
Err:4 http://deb.debian.org/debian buster-updates Release
  Could not open file /var/lib/apt/lists/partial/deb.debian.org_debian_dists_buster-updates_Release - open (13: Permission denied) [IP: 151.101.36.204 80]
Hit:3 http://security-cdn.debian.org/debian-security buster/updates InRelease
rm: cannot remove '/var/cache/apt/archives/partial/*.deb': Permission denied
Reading package lists... Done
W: chown to _apt:root of directory /var/lib/apt/lists/partial failed - SetupAPTPartialDirectory (1: Operation not permitted)
W: chmod 0700 of directory /var/lib/apt/lists/partial failed - SetupAPTPartialDirectory (1: Operation not permitted)
W: chown to _apt:root of directory /var/lib/apt/lists/auxfiles failed - SetupAPTPartialDirectory (1: Operation not permitted)
W: chmod 0700 of directory /var/lib/apt/lists/auxfiles failed - SetupAPTPartialDirectory (1: Operation not permitted)
E: setgroups 65534 failed - setgroups (1: Operation not permitted)
E: setegid 65534 failed - setegid (1: Operation not permitted)
E: seteuid 100 failed - seteuid (1: Operation not permitted)
W: Download is performed unsandboxed as root as file '/var/lib/apt/lists/partial/deb.debian.org_debian_dists_buster_InRelease' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied)
E: setgroups 0 failed - setgroups (1: Operation not permitted)
W: Problem unlinking the file /var/lib/apt/lists/partial/deb.debian.org_debian_dists_buster_InRelease - PrepareFiles (13: Permission denied)
W: Problem unlinking the file /var/lib/apt/lists/partial/deb.debian.org_debian_dists_buster-updates_InRelease - PrepareFiles (13: Permission denied)
W: Problem unlinking the file /var/lib/apt/lists/partial/deb.debian.org_debian_dists_buster-updates_Release - PrepareFiles (13: Permission denied)
E: The repository 'http://deb.debian.org/debian buster-updates Release' no longer has a Release file.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
W: Problem unlinking the file /var/lib/apt/lists/partial/security.debian.org_debian-security_dists_buster_updates_InRelease - PrepareFiles (13: Permission denied)

In the container /etc/subuid and /etc/subgid look like this (both):

andrej:100000:65536

On the host /etc/subuid and /etc/subgid look like this (both):

andrej:100000:65536
dockremap:165536:65536

Apparmor is running on Ubuntu host with following status (only docker-default profile):

andrej@machine:/etc/apparmor.d$ sudo aa-status
apparmor module is loaded.
38 profiles are loaded.
36 profiles are in enforce mode.
   /sbin/dhclient
   /snap/core/8268/usr/lib/snapd/snap-confine
   /snap/core/8268/usr/lib/snapd/snap-confine//mount-namespace-capture-helper
   /usr/bin/evince
   /usr/bin/evince-previewer
   /usr/bin/evince-previewer//sanitized_helper
   /usr/bin/evince-thumbnailer
   /usr/bin/evince//sanitized_helper
   /usr/bin/man
   /usr/lib/NetworkManager/nm-dhcp-client.action
   /usr/lib/NetworkManager/nm-dhcp-helper
   /usr/lib/connman/scripts/dhclient-script
   /usr/lib/cups/backend/cups-pdf
   /usr/lib/snapd/snap-confine
   /usr/lib/snapd/snap-confine//mount-namespace-capture-helper
   /usr/sbin/cups-browsed
   /usr/sbin/cupsd
   /usr/sbin/cupsd//third_party
   /usr/sbin/ippusbxd
   /usr/sbin/tcpdump
   docker-default
   libreoffice-senddoc
   libreoffice-soffice//gpg
   libreoffice-xpdfimport
   man_filter
   man_groff
   snap-update-ns.core
   snap-update-ns.gnome-calculator
   snap-update-ns.gnome-characters
   snap-update-ns.gnome-logs
   snap-update-ns.gnome-system-monitor
   snap.core.hook.configure
   snap.gnome-calculator.gnome-calculator
   snap.gnome-characters.gnome-characters
   snap.gnome-logs.gnome-logs
   snap.gnome-system-monitor.gnome-system-monitor
2 profiles are in complain mode.
   libreoffice-oopslash
   libreoffice-soffice
17 processes have profiles defined.
14 processes are in enforce mode.
   docker-default (1101)
   docker-default (1102)
   docker-default (1111)
   docker-default (1600)
   docker-default (1728)
   docker-default (1729)
   docker-default (1730)
   docker-default (1731)
   docker-default (1732)
   docker-default (1798)
   docker-default (1799)
   docker-default (1800)
   docker-default (1801)
   docker-default (1802)
0 processes are in complain mode.
3 processes are unconfined but have a profile defined.
   /sbin/dhclient (491)
   /usr/sbin/cups-browsed (431)
   /usr/sbin/cupsd (402)

Selinux seems to be disabled as there is not /etc/selinux/config file and getenfoce and sestatus command are not available.

Also su andrej command run as root (where andrej is unprivileged user in the container) errors out with su: cannot set groups: Operation not permitted

Body:
Not sure about Docker, but in kubernetes in runc container for me helps:

Get root access to container
List all containers

minikube ssh docker container ls

Connect to your container (use your container id from previous command instead of 44a7ad70d45b):

minikube ssh "docker container exec -it -u 0 44a7ad70d45b /bin/bash"

As root inside container:

root@mycontainer:/# apt-config dump | grep Sandbox::User
APT::Sandbox::User "_apt";

root@mycontainer:/# cat <<EOF > /etc/apt/apt.conf.d/sandbox-disable
APT::Sandbox::User "root";
EOF

Be sure, that result is valid:

root@mycontainer:/# apt-config dump | grep Sandbox::User
APT::Sandbox::User "root";

apt update and see

Get:1 http://archive.ubuntu.com/ubuntu hirsute InRelease [269 kB]
Get:2 http://apt.postgresql.org/pub/repos/apt hirsute-pgdg InRelease [16.7 kB]
...


问:
Getting this SQL Statement right with aliases
what i basically want to do is: compute the difference of to attributes and then get the row with the biggest difference.

So, here is what i tried:

```<code>SELECT caller_aor, (end_time - start_time) as duration
    FROM cdrs
    GROUP BY caller_aor
    HAVING duration = (SELECT MAX(end_time - start_time) FROM cdrs);
</code>```

duration in the HAVING clause gives an error but i cant figure out what i have to do there.

Help is appreciated.

答:
Another way of looking at this is to use a limit.

```<code>SELECT caller_aor, (end_time-start_time) AS duration
FROM cdrs
ORDER BY duration DESC
LIMIT 1
</code>```

Body:
what i basically want to do is: compute the difference of to attributes and then get the row with the biggest difference.

So, here is what i tried:

SELECT caller_aor, (end_time - start_time) as duration
    FROM cdrs
    GROUP BY caller_aor
    HAVING duration = (SELECT MAX(end_time - start_time) FROM cdrs);

duration in the HAVING clause gives an error but i cant figure out what i have to do there.

Help is appreciated.

Body:
Another way of looking at this is to use a limit.

SELECT caller_aor, (end_time-start_time) AS duration
FROM cdrs
ORDER BY duration DESC
LIMIT 1


问:
reload an specific div without refresh whole page
I want to reload just an specific div. I searched and read some similar questions too. What I made is below. What is wrong with my code that does not refresh the div ?</p>
```<code><html>
    <body>
        <div id="reloadable">
            <?php include 'rand.php'; ?>
        </div>
        <button class="ref" onclick="refreshDiv();"> refresh </button>

        <script type="text/javacript">
            function refreshDiv()
            {
                $('#reloadable').load(location.href + ' #reloadable');
            }
        </script>
    </body>
</html>
</code>```
```<code><?php

$a= rand(0,10);
echo $a;

?>
</code>```

答:
With the pure respect to all friends that helped me specially Barmar and Ken lee, I can not accept Barmar's answer as correct answer. Because Barmar said $('#reloadable').load(location.href + ' #reloadable'); can not run properly and mentioned some reasons. But I added

```<code><script
  src="https://code.jquery.com/jquery-3.6.0.js"
  integrity="sha256-H+K7U5CnXl1h5ywQfKtSj8PCmoN9aaq30gDh27Xc0jk="
  crossorigin="anonymous"></script>
</code>```

from Ken lee's answer and also added onclick="javascript:refreshDiv() to my original code attached to main question and it worked. So the following code is working

```<code><script
  src="https://code.jquery.com/jquery-3.6.0.js"
  integrity="sha256-H+K7U5CnXl1h5ywQfKtSj8PCmoN9aaq30gDh27Xc0jk="
  crossorigin="anonymous"></script>

<html>
    <body>

        <div id="reloadable"><?php include "rand.php"; ?></div>
        <button class="ref" onclick="javascript:refreshDiv();"> refresh </button>

        <script>
            function refreshDiv() {
            $('#reloadable').load(location.href + ' #reloadable');
            }
        </script>

    </body>
</html>
</code>```

Body:
I want to reload just an specific div. I searched and read some similar questions too. What I made is below. What is wrong with my code that does not refresh the div ?

            <?php include 'rand.php'; ?>

        <button class="ref" onclick="refreshDiv();"> refresh </button>

<?php

$a= rand(0,10);
echo $a;

?>

Body:
With the pure respect to all friends that helped me specially Barmar and Ken lee, I can not accept Barmar's answer as correct answer. Because Barmar said $('#reloadable').load(location.href + ' #reloadable'); can not run properly and mentioned some reasons. But I added

<script
  src="https://code.jquery.com/jquery-3.6.0.js"
  integrity="sha256-H+K7U5CnXl1h5ywQfKtSj8PCmoN9aaq30gDh27Xc0jk="
  crossorigin="anonymous"></script>

from Ken lee's answer and also added onclick="javascript:refreshDiv() to my original code attached to main question and it worked. So the following code is working

<script
  src="https://code.jquery.com/jquery-3.6.0.js"
  integrity="sha256-H+K7U5CnXl1h5ywQfKtSj8PCmoN9aaq30gDh27Xc0jk="
  crossorigin="anonymous"></script>

<html>
    <body>

        <div id="reloadable"><?php include "rand.php"; ?></div>
        <button class="ref" onclick="javascript:refreshDiv();"> refresh </button>

        <script>
            function refreshDiv() {
            $('#reloadable').load(location.href + ' #reloadable');
            }
        </script>

    </body>
</html>


问:
Query in jsonb (JSON array) column with JPA native query
I am using JPA where one of the entity holds user_details (jsonb) column.

Below is user_details json array data on which I want to query.

```<code>[{
    "user": "test1",
    "email": "test1@gmail.com"
},
{
    "user": "test2",
    "email": "test2@gmail.com"
}]
</code>```

In postgres client, Below query is working absolutely fine.

```<code>SELECT * FROM table1 WHERE "user_details" @> '[{"email": "test1@gmail.com"}]';
</code>```

The same in @Repository I want to achieve through native query where email value(test1@gmail.com) would be dynamic. Here is my code :

```<code>1. @Query(value ="select * from table1 WHERE user_details @> '[{\"email\": :email}]'", nativeQuery = true)
List<Entity> findByEmail(@Param("email") String email);
</code>```

ERROR: invalid input syntax for type json

Detail: Expected JSON value, but found ":".

Where: JSON data, line 1: [{"email": :...

```<code>2. @Query(value ="select * from table1 WHERE user_details @> '[{\"email\": ?1}]'", nativeQuery = true)
List<Entity> findByEmail(String email);
</code>```

ERROR: invalid input syntax for type json

Detail: Token "?" is invalid.

Where: JSON data, line 1: [{"email": ?...

```<code>3. @Query(value ="select * from table1 WHERE user_details @> :param", nativeQuery = true)
List<Entity> findByEmail(@Param("param") String param); (i.e. param= "'[{ \"email\" : \"test1@gmail.com\"}]'")
</code>```

ERROR: operator does not exist: jsonb @> character varying

Hint: No operator matches the given name and argument types. You might need to add explicit type casts.

Edited:

```<code>4. @Query(nativeQuery = true, value = "select * from table1 where jsonb_contains(user_details , :param )") (i.e. param= "'[{ \"email\" : \"test1@gmail.com\"}]'")
List<Entity> findByEmail(@Param("param") String email);
</code>```

ERROR: function jsonb_contains(jsonb, character varying) does not exist
Hint: No function matches the given name and argument types. You might
need to add explicit type casts.

```<code>5. @Query(nativeQuery = true, value = "select * from table1 where user_details @> jsonb_build_object('email', :param )")
List<Entity> findByEmail(@Param("param") String email);
</code>```

NO ERROR but not giving the result.

```<code>6. @Query(nativeQuery = true, value = "select * from table1 where user_details ->>'email' = :param")
List<Entity> findByEmail(@Param("param") String email);
</code>```

NO ERROR but not giving the result.

```<code>7. @Query(nativeQuery = true, value = "select * from table1 WHERE jsonb_extract_path_text(user_details , 'email') = :param")
List<Entity> findByEmail(@Param("param") String email);
</code>```

NO ERROR but not giving the result.

These native queries are not giving me the expected result.

Please help me with the native query where I can bind the parameter with query.
Thanks in advance.

答:
Before you call this function, create a jsonarray:

```<code>JSONObject obj = new JSONObject();
obj.put("email", email);
JSONArray a = new JSONArray();
a.add(0, obj);
</code>```

Call the function by passing jsonArray string

```<code>findByEmail(a.toJSONString());
</code>```

Your query should be :

```<code>@Query(nativeQuery = true, value = "select * from table1 where user_details @> cast(:param as jsonb)")
List<Entity> findByEmail(@Param("param") String email);
</code>```

Body:
I am using JPA where one of the entity holds user_details (jsonb) column.

Below is user_details json array data on which I want to query.

[{
    "user": "test1",
    "email": "test1@gmail.com"
},
{
    "user": "test2",
    "email": "test2@gmail.com"
}]

In postgres client, Below query is working absolutely fine.

SELECT * FROM table1 WHERE "user_details" @> '[{"email": "test1@gmail.com"}]';

The same in @Repository I want to achieve through native query where email value(test1@gmail.com) would be dynamic. Here is my code :

1. @Query(value ="select * from table1 WHERE user_details @> '[{\"email\": :email}]'", nativeQuery = true)
List<Entity> findByEmail(@Param("email") String email);

ERROR: invalid input syntax for type json

Detail: Expected JSON value, but found ":".

Where: JSON data, line 1: [{"email": :...

2. @Query(value ="select * from table1 WHERE user_details @> '[{\"email\": ?1}]'", nativeQuery = true)
List<Entity> findByEmail(String email);

ERROR: invalid input syntax for type json

Detail: Token "?" is invalid.

Where: JSON data, line 1: [{"email": ?...

3. @Query(value ="select * from table1 WHERE user_details @> :param", nativeQuery = true)
List<Entity> findByEmail(@Param("param") String param); (i.e. param= "'[{ \"email\" : \"test1@gmail.com\"}]'")

ERROR: operator does not exist: jsonb @> character varying

Hint: No operator matches the given name and argument types. You might need to add explicit type casts.

Edited:

4. @Query(nativeQuery = true, value = "select * from table1 where jsonb_contains(user_details , :param )") (i.e. param= "'[{ \"email\" : \"test1@gmail.com\"}]'")
List<Entity> findByEmail(@Param("param") String email);

ERROR: function jsonb_contains(jsonb, character varying) does not exist
Hint: No function matches the given name and argument types. You might
need to add explicit type casts.

5. @Query(nativeQuery = true, value = "select * from table1 where user_details @> jsonb_build_object('email', :param )")
List<Entity> findByEmail(@Param("param") String email);

NO ERROR but not giving the result.

6. @Query(nativeQuery = true, value = "select * from table1 where user_details ->>'email' = :param")
List<Entity> findByEmail(@Param("param") String email);

NO ERROR but not giving the result.

7. @Query(nativeQuery = true, value = "select * from table1 WHERE jsonb_extract_path_text(user_details , 'email') = :param")
List<Entity> findByEmail(@Param("param") String email);

NO ERROR but not giving the result.

These native queries are not giving me the expected result.

Please help me with the native query where I can bind the parameter with query.
Thanks in advance.

Body:
Before you call this function, create a jsonarray:

JSONObject obj = new JSONObject();
obj.put("email", email);
JSONArray a = new JSONArray();
a.add(0, obj);

Call the function by passing jsonArray string

findByEmail(a.toJSONString());

Your query should be :

@Query(nativeQuery = true, value = "select * from table1 where user_details @> cast(:param as jsonb)")
List<Entity> findByEmail(@Param("param") String email);