问:
Recyclerview DiffUtil Item Update
I have endless scroll in my recyclerview, so, it will update when there is new data. and i am using DiffUtil to update data in the recyclerview. DiffUtil does updates the data but whenever there is update data, recyclerview scroll to top and what it looks like is "using the notifydatasetchanged()". here is my DiffUtil and my adapter to update data.

```<code>class ProductDiffUtil(
    val oldProductList: List<ProductModel>, val newProductList: List<ProductModel>
) : DiffUtil.Callback() {

    override fun areItemsTheSame(oldItemPosition: Int, newItemPosition: Int): Boolean {
        return oldProductList[oldItemPosition].id == newProductList[newItemPosition].id
    }

    override fun getOldListSize(): Int = oldProductList.size

    override fun getNewListSize(): Int = newProductList.size

    override fun areContentsTheSame(oldItemPosition: Int, newItemPosition: Int): Boolean {
        return oldProductList[oldItemPosition] == newProductList[newItemPosition]
    }

    override fun getChangePayload(oldItemPosition: Int, newItemPosition: Int): Any? {
        return super.getChangePayload(oldItemPosition, newItemPosition)
    }
}
</code>```

Here is my adapter to update data

```<code>fun addProductList(productList: List<ProductModel>?) {
        val diffResult = DiffUtil.calculateDiff(ProductDiffUtil(this.productList, productList!!))
        this.productList.addAll(productList)
        diffResult.dispatchUpdatesTo(this)
    }
</code>```

please help me with this. it is working fine when i am using notifyItemRangeChanged()... so what should i use to update data in recyclerview for best practice.

https://drive.google.com/open?id=1SapXW2wusmTpyGCRA9fa0aSLCYNL1fzN

答:
Consider making a generic diffUtil class instead of creating it for each adapter.

```<code>fun <T>diffList(oldList: List<T>, newList: List<T>, sameItem: (a: T, b: T) -> Boolean): DiffUtil.DiffResult {
val callback: DiffUtil.Callback = object : DiffUtil.Callback() {
    override fun getOldListSize(): Int {
        return oldList.size
    }

    override fun getNewListSize(): Int {
        return newList.size
    }

    override fun areItemsTheSame(oldItemPosition: Int, newItemPosition: Int): Boolean {
        return sameItem(oldList[oldItemPosition], newList[newItemPosition])
    }

    override fun areContentsTheSame(oldItemPosition: Int, newItemPosition: Int): Boolean {
        return oldList[oldItemPosition] == newList[newItemPosition]
    }
}

return DiffUtil.calculateDiff(callback) }
</code>```

You can use it in your adapter like this:

```<code> fun setItems(products: List<Product>) {
    val oldList = productList
    productList = products
    diffList(oldList, products, sameItem = { a, b -> a.id == b.id }).dispatchUpdatesTo(this)
}
</code>```

Title: Recyclerview DiffUtil Item Update

Body:
I have endless scroll in my recyclerview, so, it will update when there is new data. and i am using DiffUtil to update data in the recyclerview. DiffUtil does updates the data but whenever there is update data, recyclerview scroll to top and what it looks like is "using the notifydatasetchanged()". here is my DiffUtil and my adapter to update data.

class ProductDiffUtil(
    val oldProductList: List<ProductModel>, val newProductList: List<ProductModel>
) : DiffUtil.Callback() {

    override fun areItemsTheSame(oldItemPosition: Int, newItemPosition: Int): Boolean {
        return oldProductList[oldItemPosition].id == newProductList[newItemPosition].id
    }

    override fun getOldListSize(): Int = oldProductList.size

    override fun getNewListSize(): Int = newProductList.size

    override fun areContentsTheSame(oldItemPosition: Int, newItemPosition: Int): Boolean {
        return oldProductList[oldItemPosition] == newProductList[newItemPosition]
    }

    override fun getChangePayload(oldItemPosition: Int, newItemPosition: Int): Any? {
        return super.getChangePayload(oldItemPosition, newItemPosition)
    }
}

Here is my adapter to update data

fun addProductList(productList: List<ProductModel>?) {
        val diffResult = DiffUtil.calculateDiff(ProductDiffUtil(this.productList, productList!!))
        this.productList.addAll(productList)
        diffResult.dispatchUpdatesTo(this)
    }

please help me with this. it is working fine when i am using notifyItemRangeChanged()... so what should i use to update data in recyclerview for best practice.

https://drive.google.com/open?id=1SapXW2wusmTpyGCRA9fa0aSLCYNL1fzN

Body:
Consider making a generic diffUtil class instead of creating it for each adapter.

fun <T>diffList(oldList: List<T>, newList: List<T>, sameItem: (a: T, b: T) -> Boolean): DiffUtil.DiffResult {
val callback: DiffUtil.Callback = object : DiffUtil.Callback() {
    override fun getOldListSize(): Int {
        return oldList.size
    }

    override fun getNewListSize(): Int {
        return newList.size
    }

    override fun areItemsTheSame(oldItemPosition: Int, newItemPosition: Int): Boolean {
        return sameItem(oldList[oldItemPosition], newList[newItemPosition])
    }

    override fun areContentsTheSame(oldItemPosition: Int, newItemPosition: Int): Boolean {
        return oldList[oldItemPosition] == newList[newItemPosition]
    }
}

return DiffUtil.calculateDiff(callback) }

You can use it in your adapter like this:

 fun setItems(products: List<Product>) {
    val oldList = productList
    productList = products
    diffList(oldList, products, sameItem = { a, b -> a.id == b.id }).dispatchUpdatesTo(this)
}


来源: stackexchange

问:
Can't combine dataframes in list
I am working on a function that extract the contents of several CSV files and put in one data frame and then get the mean of a specific component of that data frame specified by the user

```<code>> airpollut <- function(pathway, pollutent, id = 1:322){
+   airpollutionfiles <- list.files(path = pathway, full.names = TRUE, pattern = ".csv")
+   totalvalues <- list()
+   for(i in id){
+    filereading <- read.csv(airpollutionfiles[i])
+    totalvalues[[i]] <- filereading
+   }
+   finaldata1 <- data.frame(rbind(totalvalues))
+   mean(finaldata1[[pollutent]])
+   }
</code>```

but when I run the function I will get the error message:

```<code>> airpollut("finaldata", "sulfate")
[1] NA
Warning message:
In mean.default(finaldata1[[pollutent]]) :
  argument is not numeric or logical: returning NA
</code>```

so I tried to check out the output of that dataframe binding and I removed the mean function from the function I am creating and put head() instead:

```<code>airpollut <- function(pathway, pollutent, id = 1:322){
  airpollutionfiles <- list.files(path = pathway, full.names = TRUE, pattern = ".csv")
  totalvalues <- list()
  for(i in id){
   filereading <- read.csv(airpollutionfiles[i])
   totalvalues[[i]] <- filereading
  }
  finaldata1 <- data.frame(rbind(totalvalues))
  head(finaldata1)
}
</code>```

the output is just the data stacked next to each other like a vector and my computer crushes.
can you please tell me how to combine the rows from all the files? without using functions that are not in r base.

答:
A minimal reproducible example to facilitate understanding of the problem:

```<code>l <-list()
l[[1]] <- mtcars
l[[2]] <- mtcars
mean(as.data.frame(rbind(l))[[1]])
[1] NA
Warning message:
In mean.default(as.data.frame(rbind(l))[[1]]) :
  argument is not numeric or logical: returning NA
</code>```

You could use do.call:

```<code>mean(do.call(rbind,l)[[1]])
[1] 20.09062
</code>```

ContentLicense: CC BY-SA 4.0

Title: Can't combine dataframes in list

Body:
I am working on a function that extract the contents of several CSV files and put in one data frame and then get the mean of a specific component of that data frame specified by the user

> airpollut <- function(pathway, pollutent, id = 1:322){
+   airpollutionfiles <- list.files(path = pathway, full.names = TRUE, pattern = ".csv")
+   totalvalues <- list()
+   for(i in id){
+    filereading <- read.csv(airpollutionfiles[i])
+    totalvalues[[i]] <- filereading
+   }
+   finaldata1 <- data.frame(rbind(totalvalues))
+   mean(finaldata1[[pollutent]])
+   }

but when I run the function I will get the error message:

> airpollut("finaldata", "sulfate")
[1] NA
Warning message:
In mean.default(finaldata1[[pollutent]]) :
  argument is not numeric or logical: returning NA

so I tried to check out the output of that dataframe binding and I removed the mean function from the function I am creating and put head() instead:

airpollut <- function(pathway, pollutent, id = 1:322){
  airpollutionfiles <- list.files(path = pathway, full.names = TRUE, pattern = ".csv")
  totalvalues <- list()
  for(i in id){
   filereading <- read.csv(airpollutionfiles[i])
   totalvalues[[i]] <- filereading
  }
  finaldata1 <- data.frame(rbind(totalvalues))
  head(finaldata1)
}

the output is just the data stacked next to each other like a vector and my computer crushes.
can you please tell me how to combine the rows from all the files? without using functions that are not in r base.

Body:
A minimal reproducible example to facilitate understanding of the problem:

l <-list()
l[[1]] <- mtcars
l[[2]] <- mtcars
mean(as.data.frame(rbind(l))[[1]])
[1] NA
Warning message:
In mean.default(as.data.frame(rbind(l))[[1]]) :
  argument is not numeric or logical: returning NA

You could use do.call:

mean(do.call(rbind,l)[[1]])
[1] 20.09062


问:
protecting a ShadowRoot element
I am working on a web extension that adds a custom overlay element to any website. naturally I don't want the websites CSS and JavaScript to interfere with my custom element, nor do I want to influence the appearance of any websites in any way other than with my overlay.

So naturally I turn to Shadow DOM. I soon learn that I can't simply attach my shadow DOM to document.body since this will displace the whole page.

It turns out I need one regular element in the body to act as a shadow root:

    const shadowHost = document.createElement('div');
    document.body.appendChild(shadowHost);
    const shadow = shadowHost.attachShadow({mode: 'closed'});
```

But how can I protect this shadow root from being influenced by the css or js of the website?

答:
What have you tried?

shadowRoots are not styled by global CSS, unless they are inheritable styles

https://lamplightdev.com/blog/2019/03/26/why-is-my-web-component-inheriting-styles/

mode:"closed" has nothing to do with CSS, it just doesn't publish the shadowRoot in element.shadowRoot, making it impossible to access the shadow DOM from the outside.

When the mode of a shadow root is "closed", the shadow root’s implementation internals are inaccessible and unchangeable from JavaScript—in the same way the implementation internals of, for example, the <video> element are inaccessible and unchangeable from JavaScript.

document.body
  .appendChild(document.createElement('div'))
  .attachShadow({mode:'closed'})
  .innerHTML = `<style>h1{color:green}</style><h1>Hello</h1><h2>Component</h2>`;```
<pre class="snippet-code-html lang-html prettyprint-override"><code><style>
  body {
    /* inheritable styles do style shadowRoots */
    font: 10px Arial;
    color: red;
  }
</style></code>```

Title: protecting a ShadowRoot element

Body:
I am working on a web extension that adds a custom overlay element to any website. naturally I don't want the websites CSS and JavaScript to interfere with my custom element, nor do I want to influence the appearance of any websites in any way other than with my overlay.

So naturally I turn to Shadow DOM. I soon learn that I can't simply attach my shadow DOM to document.body since this will displace the whole page.

It turns out I need one regular element in the body to act as a shadow root:

    const shadowHost = document.createElement('div');
    document.body.appendChild(shadowHost);
    const shadow = shadowHost.attachShadow({mode: 'closed'});

But how can I protect this shadow root from being influenced by the css or js of the website?

Body:
What have you tried?

shadowRoots are not styled by global CSS, unless they are inheritable styles

https://lamplightdev.com/blog/2019/03/26/why-is-my-web-component-inheriting-styles/

mode:"closed" has nothing to do with CSS, it just doesn't publish the shadowRoot in element.shadowRoot, making it impossible to access the shadow DOM from the outside.

When the mode of a shadow root is "closed", the shadow root’s implementation internals are inaccessible and unchangeable from JavaScript—in the same way the implementation internals of, for example, the <video> element are inaccessible and unchangeable from JavaScript.

document.body
  .appendChild(document.createElement('div'))
  .attachShadow({mode:'closed'})
  .innerHTML = `<style>h1{color:green}</style><h1>Hello</h1><h2>Component</h2>`;

<style>
  body {
    /* inheritable styles do style shadowRoots */
    font: 10px Arial;
    color: red;
  }
</style>


问:
Maximum String Mismatches in Java
I was recently going through a question in codehub and I was unable to solve this query. Can anyone help me how can this be solved?

You are given a string S of length N. You can select and reverse any substring of S of any length. You are allowed to perform this operation many number of times.

Determine maximum number of mismatches by performing operation.

Mismatch(S) is defined as number of corresponding positions were characters are different in S and reverse(S). For example : S = abab, reverse(S) = baba. Number of mismatches = 4. S= abca. Number of mismatches = 2.

Pseudo code :

```<code>static int solve( String S, int n)
{
//To do
}
</code>```

Will be helpful if some one can explain once the code is done how this can be interpreted more easily and approached to solve.

答:
```<code>#include <bits/stdc++.h>
using namespace std;
int max_mismatch(string s)
{
  unordered_map<char,int> fmap;
  for(auto &ch:s)
    fmap[ch]++;
  vector<int> tempV;
  for(auto &[_,f]:fmap)
     tempV.push_back(f);
  sort(tempV.begin(),tempV.end(),greater<int>());
  vector<int> freq(tempV.size());
  int diff=0,left=0,right=freq.size()-1;
  bool leftinsert=true;
  for(auto f:tempV)
  {
      if(leftinsert)
          freq[left++]=f;
      else
         freq[right--]=f;
      leftinsert^=1;
  }
  /*for(auto &val:freq)
     cout<<val<<" ";
  cout<<endl;  */
  left=0;right=freq.size()-1;
  for(int i=0;i<s.size()/2;i++)
  {
     if(left==right)
       break;
     if(!(--freq[left])) left++;
     if(!(--freq[right])) right--;
     diff+=2;
  }
  return diff;
}
int main()
{
    string s="aatacanaa";
    cout<<max_mismatch(s);
}
</code>```

Title: Maximum String Mismatches in Java

Body:
I was recently going through a question in codehub and I was unable to solve this query. Can anyone help me how can this be solved?

You are given a string S of length N. You can select and reverse any substring of S of any length. You are allowed to perform this operation many number of times.

Determine maximum number of mismatches by performing operation.

Mismatch(S) is defined as number of corresponding positions were characters are different in S and reverse(S). For example : S = abab, reverse(S) = baba. Number of mismatches = 4. S= abca. Number of mismatches = 2.

Pseudo code :

static int solve( String S, int n)
{
//To do
}

Will be helpful if some one can explain once the code is done how this can be interpreted more easily and approached to solve.


问:
npm faker issue in angular
In angular project I am creating two files to generate fake data.

File structure as below
https://i.stack.imgur.com/UBSFQ.png

Writing generate.js to create data into database.json

And creating command in package.json as below.
https://i.stack.imgur.com/hIj6t.png

But am getting errors while running code as "npm run generate"
https://i.stack.imgur.com/70c2M.png

FYI : I tried deleting node modules and regerating them but not able to get solution

```<code>
var database = { products: []};

for (var i = 1; i<= 300; i++) {
  database.products.push({
    id: i,
    name: faker.commerce.productName(),
    description: faker.lorem.sentences(),
    price: faker.commerce.price(),
    imageUrl: "https://source.unsplash.com/1600x900/?product",
    quantity: faker.random.number()
  });
}

console.log(JSON.stringify(database)); ```
```

答:
if you are using faker@6.6.6 latest version then it's not going to be work because developer closed this project with last commit of endgame but if you want to use fakerjs then downgrade your package by faker@5.5.3 then its going to work like</p>
```<code>npm i faker@5.5.3
</code>```

Title: npm faker issue in angular

Body:
In angular project I am creating two files to generate fake data.

File structure as below
https://i.stack.imgur.com/UBSFQ.png

Writing generate.js to create data into database.json

And creating command in package.json as below.
https://i.stack.imgur.com/hIj6t.png

But am getting errors while running code as "npm run generate"
https://i.stack.imgur.com/70c2M.png

FYI : I tried deleting node modules and regerating them but not able to get solution

var database = { products: []};

for (var i = 1; i<= 300; i++) {
  database.products.push({
    id: i,
    name: faker.commerce.productName(),
    description: faker.lorem.sentences(),
    price: faker.commerce.price(),
    imageUrl: "https://source.unsplash.com/1600x900/?product",
    quantity: faker.random.number()
  });
}

console.log(JSON.stringify(database)); ```

Body:
if you are using faker@6.6.6 latest version then it's not going to be work because developer closed this project with last commit of endgame but if you want to use fakerjs then downgrade your package by faker@5.5.3 then its going to work like

npm i faker@5.5.3


问:
Uploading files by AJAX and geting error in PHP handling
I'm banging my head on the walls with this thing for 4 hours now.

I've created an input (files multiple) as show below, that is not inside a form.

```<code><label for="adaugaFisiere" class="btn btn-success tip" title="Poți adăuga fișiere JPG,PDF,DOC,XLS" data-original-title="Adauga imagini"><span>Adaugă fișiere</span></label>
                        <input type="file" multiple="" id="adaugaFisiere" name="docImgs[]" class="filestyle invisible" required="" accept="image/*,.pdf,.doc,.docx,.xls,.xlsx" onChange="updateList(<?= $nextNrDoc ?>)">
</code>```

Next , I have the following JavaScript code, that uploads the files to an PHP file

```<code>function updateList(nrDoc){
var inputFiles = $('#adaugaFisiere').prop('files');
var formData = new FormData();
formData.append('listFiles', inputFiles);
$.ajax({
    url: 'exec/files-upload.php',
    type: 'POST',
    cache: false,
    data: formData,
    contentType: false,
    processData: false,
    //Ajax events
    success: function(html){
        $('#fileList').html(html);
    },
    fail: function(res){
        alert('Eroare upload');
    }
});
</code>```

}

And finally, the PHP file

```<code>var_dump($_POST);

$name_array = $_FILES['listFiles']['name'];
$tmp_name_array = $_FILES['listFiles']['tmp_name'];
// Number of files
$count_tmp_name_array = count($tmp_name_array);
</code>```

The PHP file is bigger, but it's not important, because I get an error at the top of the file.

The dump_var returns this:

```<code>array(1) { ["listFiles"]=> string(17) "[object FileList]" }
</code>```

And right after comes the error:

```<code>    Warning: Undefined array key "listFiles" in D:\xampp\htdocs\cityadmin\exec\files-upload.php on line 4

Warning: Trying to access array offset on value of type null in D:\xampp\htdocs\cityadmin\exec\files-upload.php on line 4

Warning: Undefined array key "listFiles" in D:\xampp\htdocs\cityadmin\exec\files-upload.php on line 5

Warning: Trying to access array offset on value of type null in D:\xampp\htdocs\cityadmin\exec\files-upload.php on line 5
</code>```

I ran in circles but couldn't figure it out. Why I get this error, if dump_var shows the data arriving at the PHP file ? Anyone, any ideeas ?

答:
Please don't vote on this, anyone, I've just posted for those who are in the same situation as I am.

I got my answer from this thread

PHP/Javascript/AJAX file upload via drag and drop. Files get converted to strings

The point is when you have multiple files from an input, you need to loop through each one and add it to the formData as an array. After that, you can retrieve it in PHP as $_FILES[][].
It is possible to append another $_POST[] data to the formData, with the formData.append() and get both a $_POST[] and a $_FILES[][] array in PHP handler.

So, instead of doing this formData.append("fileList", fileList);, I've done this

```<code>for (let i = 0; i < fileList.length; i++) {
    formData.append("fileList[]", fileList[i]);
}
formData.append('nextId', nrDoc);
formData.append('post', 'files-upload');
</code>```

After that change, I could access the files in PHP handler, and my other variables sent along.

It used to work some time ago, but I've been busy with other type of projects and I didn't realize that something has definitely changed in the way AJAX process requests.

Now, everything work as designed, without any warnings or errors. Hope somebody else will find a solution in this post !

Body:
I'm banging my head on the walls with this thing for 4 hours now.

I've created an input (files multiple) as show below, that is not inside a form.

Adaugă fișiere
                        <input type="file" multiple="" id="adaugaFisiere" name="docImgs[]" class="filestyle invisible" required="" accept="image/*,.pdf,.doc,.docx,.xls,.xlsx" onChange="updateList(<?= $nextNrDoc ?>)">

Next , I have the following JavaScript code, that uploads the files to an PHP file

function updateList(nrDoc){
var inputFiles = $('#adaugaFisiere').prop('files');
var formData = new FormData();
formData.append('listFiles', inputFiles);
$.ajax({
    url: 'exec/files-upload.php',
    type: 'POST',
    cache: false,
    data: formData,
    contentType: false,
    processData: false,
    //Ajax events
    success: function(html){
        $('#fileList').html(html);
    },
    fail: function(res){
        alert('Eroare upload');
    }
});

}

And finally, the PHP file

var_dump($_POST);

$name_array = $_FILES['listFiles']['name'];
$tmp_name_array = $_FILES['listFiles']['tmp_name'];
// Number of files
$count_tmp_name_array = count($tmp_name_array);

The PHP file is bigger, but it's not important, because I get an error at the top of the file.

The dump_var returns this:

array(1) { ["listFiles"]=> string(17) "[object FileList]" }

And right after comes the error:

    Warning: Undefined array key "listFiles" in D:\xampp\htdocs\cityadmin\exec\files-upload.php on line 4

Warning: Trying to access array offset on value of type null in D:\xampp\htdocs\cityadmin\exec\files-upload.php on line 4

Warning: Undefined array key "listFiles" in D:\xampp\htdocs\cityadmin\exec\files-upload.php on line 5

Warning: Trying to access array offset on value of type null in D:\xampp\htdocs\cityadmin\exec\files-upload.php on line 5

I ran in circles but couldn't figure it out. Why I get this error, if dump_var shows the data arriving at the PHP file ? Anyone, any ideeas ?

Body:
Please don't vote on this, anyone, I've just posted for those who are in the same situation as I am.

I got my answer from this thread

PHP/Javascript/AJAX file upload via drag and drop. Files get converted to strings

The point is when you have multiple files from an input, you need to loop through each one and add it to the formData as an array. After that, you can retrieve it in PHP as $_FILES[][].
It is possible to append another $_POST[] data to the formData, with the formData.append() and get both a $_POST[] and a $_FILES[][] array in PHP handler.

So, instead of doing this formData.append("fileList", fileList);, I've done this

for (let i = 0; i < fileList.length; i++) {
    formData.append("fileList[]", fileList[i]);
}
formData.append('nextId', nrDoc);
formData.append('post', 'files-upload');

After that change, I could access the files in PHP handler, and my other variables sent along.

It used to work some time ago, but I've been busy with other type of projects and I didn't realize that something has definitely changed in the way AJAX process requests.

Now, everything work as designed, without any warnings or errors. Hope somebody else will find a solution in this post !


问:
Merging PHP Arrays
I have multiple associative arrays that I want to merge if all values except startdate are the same. If two arrays are indeed the same, I want to merge them and create a new element enddate so that startdate and enddate show the date range. All dates within the range must be represented in the original arrays, i.e. if a date is missing, the dates on either side of it must not be merged.

```<code>Array('color'=>'red','size'=>'large','shape'=>'circle','startdate'=>'2011-08-17')
Array('color'=>'red','size'=>'large','shape'=>'circle','startdate'=>'2011-08-18')
Array('color'=>'red','size'=>'large','shape'=>'square','startdate'=>'2011-08-20')
</code>```

should become:

```<code>Array('color'=>'red','size'=>'large','shape'=>'circle','startdate'=>'2011-08-17','enddate'=>'2011-08-18')
Array('color'=>'red','size'=>'large','shape'=>'square','startdate'=>'2011-08-20')
</code>```

So far I have tried looping through each array and creating a multidimensional array:

```<code>foreach($arrays as $id => $array){
    $mergearray[$array['red']][$array['large']][$array['circle']] = $id;
};
</code>```

in order to check whether another array has the same values. I'm trying to use those arrays to reconstruct arrays in the original structure.

答:
I can't see the actual implementation of such a merge, but this should work. It may look long winded, but it takes care of a few things as it goes.</p>

```<code>/**
 * @example array_merge_allbutstartdate($array1, $array2, $array3, $array4, $arr...)
 * @param array $array1
 * @param array $array2, $arr...
 * @return array
 */
function array_merge_allbutstartdate(array $array1, array $array2)
{
    $out  = array();
    $date = array();
    $startKey = 'startdate';
    $endKey   = 'enddate';

    foreach (func_get_args() as $item) {
        if (!is_array($item)) {
            trigger_error('All arguments should be an array.', E_USER_ERROR);
        }

        $temp = null;
        if (isset($item[$startKey])) {
            $temp = $item[$startKey];
            unset($item[$startKey]);
        }

        if (!in_array($item, $out)) {
            $i = count($out);
            $out[] = $item;
        } else {
            $i = array_search($item, $out);
        }

        if (null !== $temp) {
            $date[$i][] = $temp;
        }
    }

    foreach ($date as $j => $row) {
        array_map('strtotime', $row);
        $start = array_search(min($row), $row);
        $end = array_search(max($row), $row);

        // Add start date
        $out[$j][$startKey] = $date[$j][$start];

        // Only add end date if it is not equal to start date
        if ($date[$j][$start] !== $date[$j][$end]) {
            $out[$j][$endKey] = $date[$j][$end];
        }
   }

    return $out;
}
</code>```

ContentLicense: CC BY-SA 3.0

Title: Merging PHP Arrays

Body:
I have multiple associative arrays that I want to merge if all values except startdate are the same. If two arrays are indeed the same, I want to merge them and create a new element enddate so that startdate and enddate show the date range. All dates within the range must be represented in the original arrays, i.e. if a date is missing, the dates on either side of it must not be merged.

Array('color'=>'red','size'=>'large','shape'=>'circle','startdate'=>'2011-08-17')
Array('color'=>'red','size'=>'large','shape'=>'circle','startdate'=>'2011-08-18')
Array('color'=>'red','size'=>'large','shape'=>'square','startdate'=>'2011-08-20')

should become:

Array('color'=>'red','size'=>'large','shape'=>'circle','startdate'=>'2011-08-17','enddate'=>'2011-08-18')
Array('color'=>'red','size'=>'large','shape'=>'square','startdate'=>'2011-08-20')

So far I have tried looping through each array and creating a multidimensional array:

foreach($arrays as $id => $array){
    $mergearray[$array['red']][$array['large']][$array['circle']] = $id;
};

in order to check whether another array has the same values. I'm trying to use those arrays to reconstruct arrays in the original structure.

Body:
I can't see the actual implementation of such a merge, but this should work. It may look long winded, but it takes care of a few things as it goes.

/**
 * @example array_merge_allbutstartdate($array1, $array2, $array3, $array4, $arr...)
 * @param array $array1
 * @param array $array2, $arr...
 * @return array
 */
function array_merge_allbutstartdate(array $array1, array $array2)
{
    $out  = array();
    $date = array();
    $startKey = 'startdate';
    $endKey   = 'enddate';

    foreach (func_get_args() as $item) {
        if (!is_array($item)) {
            trigger_error('All arguments should be an array.', E_USER_ERROR);
        }

        $temp = null;
        if (isset($item[$startKey])) {
            $temp = $item[$startKey];
            unset($item[$startKey]);
        }

        if (!in_array($item, $out)) {
            $i = count($out);
            $out[] = $item;
        } else {
            $i = array_search($item, $out);
        }

        if (null !== $temp) {
            $date[$i][] = $temp;
        }
    }

    foreach ($date as $j => $row) {
        array_map('strtotime', $row);
        $start = array_search(min($row), $row);
        $end = array_search(max($row), $row);

        // Add start date
        $out[$j][$startKey] = $date[$j][$start];

        // Only add end date if it is not equal to start date
        if ($date[$j][$start] !== $date[$j][$end]) {
            $out[$j][$endKey] = $date[$j][$end];
        }
   }

    return $out;
}


问:
How would I convert this PCRE regex to the ECMAScript (JS) Regx for parsing street number and address?
I have been looking for the best regex that would parse the street number and name from an address. I found one (https://regex101.com/r/lU7gY7/1), but it is in PCRE instead of JavaScript. I have been playing around with it for quite some time now, but I can't seem to get the same (or any) output with JavaScript. There are comments in the link above explaining the code a little more, but below is the version with no comments.

\A\s*(?:(?:(?P<A_Addition_to_address_1>.*?),\s*)?(?:No\.\s*)?(?P<A_House_number_1>\pN+[a-zA-Z]?(?:\s*[-\/\pP]\s*\pN+[a-zA-Z]?)*)\s*,?\s*(?P<A_Street_name_1>(?:[a-zA-Z]\s*|\pN\pL{2,}\s\pL)\S[^,#]*?(?<!\s))\s*(?:(?:[,\/]|(?=\#))\s*(?!\s*No\.)(?P<A_Addition_to_address_2>(?!\s).*?))?
|
(?:(?P<B_Addition_to_address_1>.*?),\s*(?=.*[,\/]))?
(?!\s*No\.)(?P<B_Street_name>\S\s*\S(?:[^,#](?!\b\pN+\s))*?(?<!\s))\s*[\/,]?\s*(?:\sNo\.)?\s+(?P<B_House_number>\pN+\s*-?[a-zA-Z]?(?:\s*[-\/\pP]?\s*\pN+(?:\s*[\-a-zA-Z])?)*|[IVXLCDM]+(?!.*\b\pN+\b))(?<!\s)\s*(?:(?:[,\/]|(?=\#)|\s)\s*(?!\s*No\.)\s*(?P<B_Addition_to_address_2>(?!\s).*?))?)\s*\Z
```
<p>These addresses are the set of addresses that I'm trying to parse:</p>
```100 Baker Street
109 - 111 Wharfside Street
40-42 Parkway
25b-26 Sun Street
43a Garden Walk
6/7 Marine Road
10 - 12 Acacia Ave
4513 3RD STREET CIRCLE WEST
0 1/2 Fifth Avenue
194-03 1/2 50th Avenue
```
<p>I have tried to enter them into the regex sites and switched to JavaScript instead of PCRE and have corrected the issues that the sites highlight, but that doesn't seem to work. I have also tried the code snippets to convert PCRE to JS, but that hasn't worked either. I'm thinking there are some fundamental differences that I am missing. Could someone help me out here with the conversion?</p>
<p>Update: The goal is to have this as the end result:</p>
```{A_House_number_1: "109 - 111", A_Street_name_1: "Wharfside Street"}
{A_House_number_1: "40-42", A_Street_name_1: "Parkway"}
{A_House_number_1: "25b-26", A_Street_name_1: "Sun Street"}
{A_House_number_1: "43a",  A_Street_name_1: "Garden Walk"}
{A_House_number_1: "6/7", A_Street_name_1: "Marine Road"}
{A_House_number_1: "10 - 12", A_Street_name_1: "Acacia Ave"}
{A_House_number_1: "4513", A_Street_name_1: "3RD STREET CIRCLE WEST"}
{A_House_number_1: "0 1/2", A_Street_name_1: "Fifth Avenue"}
{A_House_number_1: "194-03 1/2", A_Street_name_1: "50th Avenue"}
```

答:
In your regex, there are Unicode property classes whose syntax is not compliant with ECMAScript 2018+ standard, and the named capturing group syntax is different across the two engines. More, since you have to use Unicode property classes in the ECMAScript regex, you need to use the /u flag, and it requires a stricter approach to escaping special chars, so you need to make sure you only escape what you must escape. Besides, \A and \Z / \z anchors are not supported in ECMAScript regex flavor, just use ^ and $.

Here are examples of what is changed:

Removed the comments as the COMMENT / FREESPACING mode (usually enabled with /x or (?x) flags/options) is not supported in ECMAScript regex

(?P<A_Addition_to_address_1>.*?) => (?<A_Addition_to_address_1>.*?) (the P after ? is not supported)

\pN => \p{N} (the Unicode category name/alias must appear inside curly braces)

[\-a-zA-Z] => [-a-zA-Z] and \# => # (unnecessary escapes, mandated by the use of /u flag)

\A => ^ (unsupported anchor)

\Z => $ (unsupported anchor)

You can use

const addresses = ['109 - 111 Wharfside Street', '40-42 Parkway', '25b-26 Sun Street', '43a Garden Walk', '6/7 Marine Road', '10 - 12 Acacia Ave', '4513 3RD STREET CIRCLE WEST', '0 1/2 Fifth Avenue', '194-03 1/2 50th Avenue'];
const regex = /^\s*(?:(?:(?<A_Addition_to_address_1>.*?),\s*)?(?:No\.\s*)?(?<A_House_number_1>\p{N}+[a-zA-Z]?(?:\s*[-\/\p{P}]\s*\p{N}+[a-zA-Z]?)*)\s*,?\s*(?<A_Street_name_1>(?:[a-zA-Z]\s*|\p{N}\p{L}{2,}\s\p{L})\S[^,#]*?(?<!\s))\s*(?:(?:[,\/]|(?=#))\s*(?!\s*No\.)(?<A_Addition_to_address_2>(?!\s).*?))?|(?:(?<B_Addition_to_address_1>.*?),\s*(?=.*[,\/]))?(?!\s*No\.)(?<B_Street_name>\S\s*\S(?:[^,#](?!\b\p{N}+\s))*?(?<!\s))\s*[\/,]?\s*(?:\sNo\.)?\s+(?<B_House_number>\p{N}+\s*-?[a-zA-Z]?(?:\s*[-\/\p{P}]?\s*\p{N}+(?:\s*[-a-zA-Z])?)*|[IVXLCDM]+(?!.*\b\p{N}+\b))(?<!\s)\s*(?:(?:[,\/]|(?=#)|\s)\s*(?!\s*No\.)\s*(?<B_Addition_to_address_2>(?!\s).*?))?)\s*$/u;
for (const address of addresses) {
  const m = regex.exec(address);
  if (m) {
      console.log( Object.fromEntries(Object.entries(m.groups).filter(([k,v]) => v!==undefined)) )
  } else {
      console.log( `No match found in "${address}"` )
  }
}```

See the regex demo.

Inside the code, Object.fromEntries(Object.entries(m.groups).filter(([k,v]) => v!==undefined)) is used to remove all the named capturing groups where the value is undefined.

Body:
I have been looking for the best regex that would parse the street number and name from an address. I found one (https://regex101.com/r/lU7gY7/1), but it is in PCRE instead of JavaScript. I have been playing around with it for quite some time now, but I can't seem to get the same (or any) output with JavaScript. There are comments in the link above explaining the code a little more, but below is the version with no comments.

\A\s*(?:(?:(?P<A_Addition_to_address_1>.*?),\s*)?(?:No\.\s*)?(?P<A_House_number_1>\pN+[a-zA-Z]?(?:\s*[-\/\pP]\s*\pN+[a-zA-Z]?)*)\s*,?\s*(?P<A_Street_name_1>(?:[a-zA-Z]\s*|\pN\pL{2,}\s\pL)\S[^,#]*?(?<!\s))\s*(?:(?:[,\/]|(?=\#))\s*(?!\s*No\.)(?P<A_Addition_to_address_2>(?!\s).*?))?
|
(?:(?P<B_Addition_to_address_1>.*?),\s*(?=.*[,\/]))?
(?!\s*No\.)(?P<B_Street_name>\S\s*\S(?:[^,#](?!\b\pN+\s))*?(?<!\s))\s*[\/,]?\s*(?:\sNo\.)?\s+(?P<B_House_number>\pN+\s*-?[a-zA-Z]?(?:\s*[-\/\pP]?\s*\pN+(?:\s*[\-a-zA-Z])?)*|[IVXLCDM]+(?!.*\b\pN+\b))(?<!\s)\s*(?:(?:[,\/]|(?=\#)|\s)\s*(?!\s*No\.)\s*(?P<B_Addition_to_address_2>(?!\s).*?))?)\s*\Z

These addresses are the set of addresses that I'm trying to parse:

100 Baker Street
109 - 111 Wharfside Street
40-42 Parkway
25b-26 Sun Street
43a Garden Walk
6/7 Marine Road
10 - 12 Acacia Ave
4513 3RD STREET CIRCLE WEST
0 1/2 Fifth Avenue
194-03 1/2 50th Avenue

I have tried to enter them into the regex sites and switched to JavaScript instead of PCRE and have corrected the issues that the sites highlight, but that doesn't seem to work. I have also tried the code snippets to convert PCRE to JS, but that hasn't worked either. I'm thinking there are some fundamental differences that I am missing. Could someone help me out here with the conversion?

Update: The goal is to have this as the end result:

{A_House_number_1: "109 - 111", A_Street_name_1: "Wharfside Street"}
{A_House_number_1: "40-42", A_Street_name_1: "Parkway"}
{A_House_number_1: "25b-26", A_Street_name_1: "Sun Street"}
{A_House_number_1: "43a",  A_Street_name_1: "Garden Walk"}
{A_House_number_1: "6/7", A_Street_name_1: "Marine Road"}
{A_House_number_1: "10 - 12", A_Street_name_1: "Acacia Ave"}
{A_House_number_1: "4513", A_Street_name_1: "3RD STREET CIRCLE WEST"}
{A_House_number_1: "0 1/2", A_Street_name_1: "Fifth Avenue"}
{A_House_number_1: "194-03 1/2", A_Street_name_1: "50th Avenue"}

Body:
In your regex, there are Unicode property classes whose syntax is not compliant with ECMAScript 2018+ standard, and the named capturing group syntax is different across the two engines. More, since you have to use Unicode property classes in the ECMAScript regex, you need to use the /u flag, and it requires a stricter approach to escaping special chars, so you need to make sure you only escape what you must escape. Besides, \A and \Z / \z anchors are not supported in ECMAScript regex flavor, just use ^ and $.

Here are examples of what is changed:

Removed the comments as the COMMENT / FREESPACING mode (usually enabled with /x or (?x) flags/options) is not supported in ECMAScript regex

(?P<A_Addition_to_address_1>.*?) => (?<A_Addition_to_address_1>.*?) (the P after ? is not supported)

\pN => \p{N} (the Unicode category name/alias must appear inside curly braces)

[\-a-zA-Z] => [-a-zA-Z] and \# => # (unnecessary escapes, mandated by the use of /u flag)

\A => ^ (unsupported anchor)

\Z => $ (unsupported anchor)

You can use

const addresses = ['109 - 111 Wharfside Street', '40-42 Parkway', '25b-26 Sun Street', '43a Garden Walk', '6/7 Marine Road', '10 - 12 Acacia Ave', '4513 3RD STREET CIRCLE WEST', '0 1/2 Fifth Avenue', '194-03 1/2 50th Avenue'];
const regex = /^\s*(?:(?:(?<A_Addition_to_address_1>.*?),\s*)?(?:No\.\s*)?(?<A_House_number_1>\p{N}+[a-zA-Z]?(?:\s*[-\/\p{P}]\s*\p{N}+[a-zA-Z]?)*)\s*,?\s*(?<A_Street_name_1>(?:[a-zA-Z]\s*|\p{N}\p{L}{2,}\s\p{L})\S[^,#]*?(?<!\s))\s*(?:(?:[,\/]|(?=#))\s*(?!\s*No\.)(?<A_Addition_to_address_2>(?!\s).*?))?|(?:(?<B_Addition_to_address_1>.*?),\s*(?=.*[,\/]))?(?!\s*No\.)(?<B_Street_name>\S\s*\S(?:[^,#](?!\b\p{N}+\s))*?(?<!\s))\s*[\/,]?\s*(?:\sNo\.)?\s+(?<B_House_number>\p{N}+\s*-?[a-zA-Z]?(?:\s*[-\/\p{P}]?\s*\p{N}+(?:\s*[-a-zA-Z])?)*|[IVXLCDM]+(?!.*\b\p{N}+\b))(?<!\s)\s*(?:(?:[,\/]|(?=#)|\s)\s*(?!\s*No\.)\s*(?<B_Addition_to_address_2>(?!\s).*?))?)\s*$/u;
for (const address of addresses) {
  const m = regex.exec(address);
  if (m) {
      console.log( Object.fromEntries(Object.entries(m.groups).filter(([k,v]) => v!==undefined)) )
  } else {
      console.log( `No match found in "${address}"` )
  }
}

See the regex demo.

Inside the code, Object.fromEntries(Object.entries(m.groups).filter(([k,v]) => v!==undefined)) is used to remove all the named capturing groups where the value is undefined.


问:
BigQuery convert Month Year column to timestamp or date
I have a table

```<code>Year_month     amount
08-2021         100
09-2021         200
10-2021         300
</code>```

I want to add a column Month to be

```<code>Year_month   Month         amount
08-2021    01-08-2021         100
09-2021    01-09-2021         200
10-2021    01-10-2021         300
</code>```

答:
It depends of the type of data you have. check below code:

```<code>with sodata as (
  select '08-2021' as Year_month_string,  PARSE_DATE("%m-%Y", "08-2021") as Year_month_date, 100 amount,
  union all
  select '09-2021',PARSE_DATE("%m-%Y", "09-2021"), 200
  union all
  select '10-2021',PARSE_DATE("%m-%Y", "10-2021"), 300

)
SELECT Year_month_string,FORMAT('01-%s',Year_month_string) as using_format,FORMAT_DATE("%d-%m-%Y", Year_month_date) as using_format_date,amount from sodata
</code>```

output

Row
Year_month
using_format
using_format_date
amount

1
08-2021
01-08-2021
01-08-2021
100

2
09-2021
01-09-2021
01-09-2021
200

3
10-2021
01-10-2021
01-10-2021
300

To see more details about the functions used please go to this links:

Format (string)

Format Date

Body:
I have a table

Year_month     amount
08-2021         100
09-2021         200
10-2021         300

I want to add a column Month to be

Year_month   Month         amount
08-2021    01-08-2021         100
09-2021    01-09-2021         200
10-2021    01-10-2021         300

Body:
It depends of the type of data you have. check below code:

with sodata as (
  select '08-2021' as Year_month_string,  PARSE_DATE("%m-%Y", "08-2021") as Year_month_date, 100 amount,
  union all
  select '09-2021',PARSE_DATE("%m-%Y", "09-2021"), 200
  union all
  select '10-2021',PARSE_DATE("%m-%Y", "10-2021"), 300

)
SELECT Year_month_string,FORMAT('01-%s',Year_month_string) as using_format,FORMAT_DATE("%d-%m-%Y", Year_month_date) as using_format_date,amount from sodata

output

Row
Year_month
using_format
using_format_date
amount

1
08-2021
01-08-2021
01-08-2021
100

2
09-2021
01-09-2021
01-09-2021
200

3
10-2021
01-10-2021
01-10-2021
300

To see more details about the functions used please go to this links:

Format (string)

Format Date


问:
PyQT Remove the programs Title Bar?
hi does anybody knows how to remove the programs title bar? (In PyQT or in Designer)
my goal is i need to replace it with my own title bar.

thanks to anyone who can help me.

答:
The title bar is managed by the window manager and not by the application.
Therefore you can not modify the title bar with the Qt Designer.

You can hide the title bar with widget.setWindowFlags(QtCore.Qt.CustomizeWindowHint) or widget.setWindowFlags(QtCore.Qt.FramelessWindowHint).
These and other options are described in the Qt documentation (Qt Namespace).

The Qt Window Flags Example makes them easier to understand.

This problem looks similar to yours: Qt4 custom window frames like in office 2007?

See also: How can I hide/delete the "?" help button on the "title bar" of a QT Dialog?

Title: PyQT Remove the programs Title Bar?


问:
snakemake parameter exploration: how to pass options to a command in shell directive?
I'm performing parameter exploration using Paramspace utility as described here.  I've read parameters in a pandas dataframe and next I wish to pass these as values of options of a shell command but can't figure out how.

In the below minimal example, I wish to pass parameter s (read in dataframe df) as the value of option -n for head command in the shell directive.

from snakemake.utils import Paramspace
import pandas as pd

df = pd.DataFrame(
    {'s' : [1, 2, 3]},
    index = [1, 2, 3]
    )

paramspace_empty = Paramspace(df)

rule all:
    input:
        expand("results/{params}.tsv", params=paramspace_empty.instance_patterns)

rule simulate_empty:
    output:
        f"results/{paramspace_empty.wildcard_pattern}.tsv"
    params:
        simulation=paramspace_empty.instance
    shell: """
        head input.txt > {output}
    """
```
<p>I tried the below and similar variations but nothing worked.</p>
<pre class="lang-py prettyprint-override"><code>shell: """
    head -n {params.simulation['s']} input.txt > {output}
"""
</code>```

The above example is extracted (and modified a bit) from the Snakefile here which tests paramspace utility.

I seem to be missing something fundamental or trivial. Thank you in advance for any help.

答:
You're almost there, you do not need to quote the dictionary key. Here's a slightly modified working version:

import pandas as pd
from snakemake.utils import Paramspace

df = pd.DataFrame({"s": [1, 2, 3]}, index=[1, 2, 3])

paramspace_empty = Paramspace(df, filename_params="*")

rule all:
    input:
        expand("{params}.tsv", params=paramspace_empty.instance_patterns),

rule simulate_empty:
    output:
        f"{paramspace_empty.wildcard_pattern}.tsv",
    params:
        simulation=paramspace_empty.instance,
    shell:
        """
        seq 10 | head -n {params.simulation[s]} > {output}
        """
```

Body:
I'm performing parameter exploration using Paramspace utility as described here.  I've read parameters in a pandas dataframe and next I wish to pass these as values of options of a shell command but can't figure out how.

In the below minimal example, I wish to pass parameter s (read in dataframe df) as the value of option -n for head command in the shell directive.

from snakemake.utils import Paramspace
import pandas as pd

df = pd.DataFrame(
    {'s' : [1, 2, 3]},
    index = [1, 2, 3]
    )

paramspace_empty = Paramspace(df)

rule all:
    input:
        expand("results/{params}.tsv", params=paramspace_empty.instance_patterns)

rule simulate_empty:
    output:
        f"results/{paramspace_empty.wildcard_pattern}.tsv"
    params:
        simulation=paramspace_empty.instance
    shell: """
        head input.txt > {output}
    """

I tried the below and similar variations but nothing worked.

shell: """
    head -n {params.simulation['s']} input.txt > {output}
"""

The above example is extracted (and modified a bit) from the Snakefile here which tests paramspace utility.

I seem to be missing something fundamental or trivial. Thank you in advance for any help.


问:
Find a specific value/characters in a string field (If statement)
I have these string statments:

10 - Holiday Booking

20 - Summer term

34 - Nighttime

Autumn time

I wanted to create a IF statement which says:

If the start of the string contains 'X - ' (a number and a hyphen with spaces, although I think it will recognise the number as a string type), then keep it as it is, otherwise add a random number and hyphen to the start

10 - Holiday Booking

20 - Summer term

34 - Nighttime

55 - Autumn time

I am using Google App Scripts/Java.

答:
You are looking for something like this:

```<code>yourArrayOfStrings.map(e => /^\d*\s-\s/.test(e) ? e : `${Math.floor(Math.random()*100)} - ${e}`);
</code>```

Array.prototoype.map(e => ...) loop over array and return modified element

Regex: /^\d*\s-\s/ - checks if string starts with number - ...

const strings = ["10 - Holiday Booking", "20 - Summer term", "34 - Nighttime", "Autumn time"];

const res = strings.map(e => /^\d*\s-\s/.test(e) ? e : `${Math.floor(Math.random()*100)} - ${e}`);
console.log(res);```

Body:
You are looking for something like this:

yourArrayOfStrings.map(e => /^\d*\s-\s/.test(e) ? e : `${Math.floor(Math.random()*100)} - ${e}`);

Array.prototoype.map(e => ...) loop over array and return modified element

Regex: /^\d*\s-\s/ - checks if string starts with number - ...

const strings = ["10 - Holiday Booking", "20 - Summer term", "34 - Nighttime", "Autumn time"];

const res = strings.map(e => /^\d*\s-\s/.test(e) ? e : `${Math.floor(Math.random()*100)} - ${e}`);
console.log(res);


问:
SQL to build a list using either EXIST (or NOT EXIST) or a combination of both
I want to find the list of COMPANYID/USERID combos on the USER table that have at least 1 Account found on the ACCOUNT table, but all Accounts are closed.  (If there are no Accounts (on ACCOUNT) for the CompanyID/UserID (on USER), it should not be in the results). If just one of the Accounts for a CompanyID/UserID (on USER) is OPEN (on the ACCOUNT table), the CompanyID/UserID should not be on the list.

In the example below, ABC/USER3 is the only combo that fits this.  It has 2 accounts (333 & 666) and both are closed.  The rest of the CompanyID/UserID combinations have least 1 ACCOUNT that is OPEN. DDD/USER5 is excluded because 888 is not on the ACCOUNT table.

TABLE: USERS

COMPANYID
USERID
ACCOUNT

AAA
USER1
111

AAA
USER1
333

ABB
USER2
333

ABB
USER2
444

ABC
USER3
333

ABC
USER3
666

BBB
USER4
555

CCC
USER4
666

DDD
USER5
888

TABLE: ACCOUNT

ACCOUNT
STATUS

111
OPEN

222
OPEN

333
CLOSED

444
OPEN

555
OPEN

666
CLOSED

777
OPEN

I wrote the original question with table names that I thought would make it easier to follow (instead of the actual DB2 tables I am using)  NB002.NBTBAUTH (is USERS)    NB002.NBTBMF (is ACCOUNT)

This is the SQL I tried to use:

```<code>SELECT NB002.NBTBAUTH.*, Q.USERID, Q.CNTOPEN
FROM NB002.NBTBAUTH LEFT JOIN (SELECT NB002.NBTBAUTH.USERID,
COUNT(IF(NB002.NBTBMF.ACCOUNTSTATUS = 'OPEN',1,NULL)) AS CNTOPEN
FROM USERS INNER JOIN NB002.NBTBMF ON NB002.NBTBAUTH.ACCOUNTID
     = NB002.NBTBMF.ACCOUNTID
GROUP BY USERID) AS Q ON NB002.NBTBAUTH.USERID = Q.USERID
WHERE NOT Q.USERID IS NULL AND Q.CNTOPEN = 0;
</code>```

Error message:

SQLCODE -104    Illegal Symbol "=".

It is the first = that is causing it. I changed it to a < and got the same message for the <.

答:
```<code>/*
WITH
  USERS (COMPANYID, USERID, ACCOUNT) AS
(
          SELECT 'AAA', 'USER1', '111' FROM SYSIBM.SYSDUMMY1
UNION ALL SELECT 'AAA', 'USER1', '333' FROM SYSIBM.SYSDUMMY1
UNION ALL SELECT 'ABB', 'USER2', '333' FROM SYSIBM.SYSDUMMY1
UNION ALL SELECT 'ABB', 'USER2', '444' FROM SYSIBM.SYSDUMMY1
UNION ALL SELECT 'ABC', 'USER3', '333' FROM SYSIBM.SYSDUMMY1
UNION ALL SELECT 'ABC', 'USER3', '666' FROM SYSIBM.SYSDUMMY1
UNION ALL SELECT 'BBB', 'USER4', '555' FROM SYSIBM.SYSDUMMY1
UNION ALL SELECT 'CCC', 'USER4', '666' FROM SYSIBM.SYSDUMMY1
UNION ALL SELECT 'DDD', 'USER5', '888' FROM SYSIBM.SYSDUMMY1
)
, ACCOUNTS (ACCOUNT, STATUS) AS
(
          SELECT '111', 'OPEN'   FROM SYSIBM.SYSDUMMY1
UNION ALL SELECT '222', 'OPEN'   FROM SYSIBM.SYSDUMMY1
UNION ALL SELECT '333', 'CLOSED' FROM SYSIBM.SYSDUMMY1
UNION ALL SELECT '444', 'OPEN'   FROM SYSIBM.SYSDUMMY1
UNION ALL SELECT '555', 'OPEN'   FROM SYSIBM.SYSDUMMY1
UNION ALL SELECT '666', 'CLOSED' FROM SYSIBM.SYSDUMMY1
UNION ALL SELECT '777', 'OPEN'   FROM SYSIBM.SYSDUMMY1
)
*/
SELECT U.USERID
FROM USERS U
JOIN ACCOUNTS A ON A.ACCOUNT = U.ACCOUNT
GROUP BY U.USERID
HAVING COUNT (1) = COUNT (CASE A.STATUS WHEN 'CLOSED' THEN 1 END)
</code>```

USERID

USER3

It's not clear what the USER identifier is - if it's USERID or (COMPANYID, USERID). If latter, then add U.COMPANYID into GROUP BY and SELECT.

Body:
I want to find the list of COMPANYID/USERID combos on the USER table that have at least 1 Account found on the ACCOUNT table, but all Accounts are closed.  (If there are no Accounts (on ACCOUNT) for the CompanyID/UserID (on USER), it should not be in the results). If just one of the Accounts for a CompanyID/UserID (on USER) is OPEN (on the ACCOUNT table), the CompanyID/UserID should not be on the list.

In the example below, ABC/USER3 is the only combo that fits this.  It has 2 accounts (333 & 666) and both are closed.  The rest of the CompanyID/UserID combinations have least 1 ACCOUNT that is OPEN. DDD/USER5 is excluded because 888 is not on the ACCOUNT table.

TABLE: USERS

COMPANYID
USERID
ACCOUNT

AAA
USER1
111

AAA
USER1
333

ABB
USER2
333

ABB
USER2
444

ABC
USER3
333

ABC
USER3
666

BBB
USER4
555

CCC
USER4
666

DDD
USER5
888

TABLE: ACCOUNT

ACCOUNT
STATUS

111
OPEN

222
OPEN

333
CLOSED

444
OPEN

555
OPEN

666
CLOSED

777
OPEN

I wrote the original question with table names that I thought would make it easier to follow (instead of the actual DB2 tables I am using)  NB002.NBTBAUTH (is USERS)    NB002.NBTBMF (is ACCOUNT)

This is the SQL I tried to use:

SELECT NB002.NBTBAUTH.*, Q.USERID, Q.CNTOPEN
FROM NB002.NBTBAUTH LEFT JOIN (SELECT NB002.NBTBAUTH.USERID,
COUNT(IF(NB002.NBTBMF.ACCOUNTSTATUS = 'OPEN',1,NULL)) AS CNTOPEN
FROM USERS INNER JOIN NB002.NBTBMF ON NB002.NBTBAUTH.ACCOUNTID
     = NB002.NBTBMF.ACCOUNTID
GROUP BY USERID) AS Q ON NB002.NBTBAUTH.USERID = Q.USERID
WHERE NOT Q.USERID IS NULL AND Q.CNTOPEN = 0;

Error message:

SQLCODE -104    Illegal Symbol "=".

It is the first = that is causing it. I changed it to a < and got the same message for the <.

Body:
/*
WITH
  USERS (COMPANYID, USERID, ACCOUNT) AS
(
          SELECT 'AAA', 'USER1', '111' FROM SYSIBM.SYSDUMMY1
UNION ALL SELECT 'AAA', 'USER1', '333' FROM SYSIBM.SYSDUMMY1
UNION ALL SELECT 'ABB', 'USER2', '333' FROM SYSIBM.SYSDUMMY1
UNION ALL SELECT 'ABB', 'USER2', '444' FROM SYSIBM.SYSDUMMY1
UNION ALL SELECT 'ABC', 'USER3', '333' FROM SYSIBM.SYSDUMMY1
UNION ALL SELECT 'ABC', 'USER3', '666' FROM SYSIBM.SYSDUMMY1
UNION ALL SELECT 'BBB', 'USER4', '555' FROM SYSIBM.SYSDUMMY1
UNION ALL SELECT 'CCC', 'USER4', '666' FROM SYSIBM.SYSDUMMY1
UNION ALL SELECT 'DDD', 'USER5', '888' FROM SYSIBM.SYSDUMMY1
)
, ACCOUNTS (ACCOUNT, STATUS) AS
(
          SELECT '111', 'OPEN'   FROM SYSIBM.SYSDUMMY1
UNION ALL SELECT '222', 'OPEN'   FROM SYSIBM.SYSDUMMY1
UNION ALL SELECT '333', 'CLOSED' FROM SYSIBM.SYSDUMMY1
UNION ALL SELECT '444', 'OPEN'   FROM SYSIBM.SYSDUMMY1
UNION ALL SELECT '555', 'OPEN'   FROM SYSIBM.SYSDUMMY1
UNION ALL SELECT '666', 'CLOSED' FROM SYSIBM.SYSDUMMY1
UNION ALL SELECT '777', 'OPEN'   FROM SYSIBM.SYSDUMMY1
)
*/
SELECT U.USERID
FROM USERS U
JOIN ACCOUNTS A ON A.ACCOUNT = U.ACCOUNT
GROUP BY U.USERID
HAVING COUNT (1) = COUNT (CASE A.STATUS WHEN 'CLOSED' THEN 1 END)

USERID

USER3

It's not clear what the USER identifier is - if it's USERID or (COMPANYID, USERID). If latter, then add U.COMPANYID into GROUP BY and SELECT.


问:
How to Install Matplotlib Basemap Module on Windows 7 with WinPython (or any Python stack install)?
I've found that the Basemap (module for matplotlib and Python) binary installer for Windows cannot detect Python on the system when Python is installed as part of a stack install, like Anaconda or WinPython.  The installer exits, rather than allow you to point to the directory of installation.

I have to therefore install via source.  However, I don't know the method to do this.  The Readme.txt and install instructions don't seem to help.  They simply instruct to "install geos-3.3.3 first" - well, how exactly?  I don't see a setup.py in that directory.

Any help would be great.  (Python 2.7, Win 7, 64b)

答:
The following worked for me using Python 3.10.1.

Setup environment using terminal in Visual Studio Code (https://code.visualstudio.com/docs/python/python-tutorial):

```<code>py -3 -m venv .venv
Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope Process
.venv\scripts\activate
</code>```

Install basemap using pip:

```<code>pip install matplotlib
pip install basemap-data
pip install basemap-data-hires
pip install basemap
</code>```

Create file with code from: https://matplotlib.org/basemap/users/examples.html

Remember to select the correct interpreter (In VSCode use Ctrl+Shift+P to select interpreter).

That's it.

Body:
The following worked for me using Python 3.10.1.

Setup environment using terminal in Visual Studio Code (https://code.visualstudio.com/docs/python/python-tutorial):

py -3 -m venv .venv
Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope Process
.venv\scripts\activate

Install basemap using pip:

pip install matplotlib
pip install basemap-data
pip install basemap-data-hires
pip install basemap

Create file with code from: https://matplotlib.org/basemap/users/examples.html

Remember to select the correct interpreter (In VSCode use Ctrl+Shift+P to select interpreter).

That's it.


问:
Throwing exception on missing property in Jackson upon deserialization
I am using jackson 2.12 and lombok 1.18.22.

I have a class

```<code>@Data
@With
@AllArgsConstructor
@Builder
Class A {
    int i;
    int j;
}
</code>```

String value = "{\"i\"=1}"

I want new ObjectMapper().readValue(i, A.class)  to throw an error for missing the field j instead it sets the field to 0.

答:
It is possible if you annotate your all args constructor with the JsonCreator annotation and the same time annotate your required j field with the JsonProperty annotation with required set to true:

```<code>@Data
@With
@Builder
@NoArgsConstructor
public class A {

    int i;
    int j;

    @JsonCreator
    public A(@JsonProperty(value = "i") int i,
             @JsonProperty(value = "j", required = true) int j) {
        this.i = i;
        this.j = j;
    }

}
</code>```

Body:
I am using jackson 2.12 and lombok 1.18.22.

I have a class

@Data
@With
@AllArgsConstructor
@Builder
Class A {
    int i;
    int j;
}

String value = "{\"i\"=1}"

I want new ObjectMapper().readValue(i, A.class)  to throw an error for missing the field j instead it sets the field to 0.

Body:
It is possible if you annotate your all args constructor with the JsonCreator annotation and the same time annotate your required j field with the JsonProperty annotation with required set to true:

@Data
@With
@Builder
@NoArgsConstructor
public class A {

    int i;
    int j;

    @JsonCreator
    public A(@JsonProperty(value = "i") int i,
             @JsonProperty(value = "j", required = true) int j) {
        this.i = i;
        this.j = j;
    }

}


问:
How to get output of a code script as a python exception message?
I have the code below. When i run it get an error :

*

```<code>STDOUT:
[librpc/rpc/dcerpc_connect.c:337:dcerpc_pipe_connect_ncacn_ip_tcp_recv()] failed NT status (c00000b5) in dcerpc_pipe_connect_ncacn_ip_tcp_recv
[librpc/rpc/dcerpc_connect.c:828:dcerpc_pipe_connect_b_recv()] failed NT status (c00000b5) in dcerpc_pipe_connect_b_recv
[wmi/wmic.c:196:main()] ERROR: Login to remote object.

  STDERR:
NTSTATUS: NT_STATUS_IO_TIMEOUT - NT_STATUS_IO_TIMEOUT
</code>```

I want to get this error message in my exception. And then, i use linux logger and save it into my db. How can i get this output as an exception message?

```<code>#!/usr/bin/python3
#!/usr/bin/python

import wmi_client_wrapper as wmi

try:
  wmic = wmi.WmiClientWrapper(username="Administrator",password="Abc",host="192.168.1.2",)
  output = wmic.query("SELECT * FROM Win32_Processor")
  print(output)
except:
  print("An exception occurred")
</code>```

答:
To redirect standard error into a variable, one easy approach is to assign sys.stderr to an io.StringIO object, and later retrieve the error message with the object's getvalue method:

```<code>import sys
from io import StringIO

error_stream = StringIO()
sys.stderr = error_stream
wmic = wmi.WmiClientWrapper(username="Administrator",password="Abc",host="192.168.1.2",)
output = wmic.query("SELECT * FROM Win32_Processor")
error = error_stream.getvalue()
print('OUTPUT:', output)
print('ERROR:', error)
</code>```

Body:
I have the code below. When i run it get an error :

*

STDOUT:
[librpc/rpc/dcerpc_connect.c:337:dcerpc_pipe_connect_ncacn_ip_tcp_recv()] failed NT status (c00000b5) in dcerpc_pipe_connect_ncacn_ip_tcp_recv
[librpc/rpc/dcerpc_connect.c:828:dcerpc_pipe_connect_b_recv()] failed NT status (c00000b5) in dcerpc_pipe_connect_b_recv
[wmi/wmic.c:196:main()] ERROR: Login to remote object.

  STDERR:
NTSTATUS: NT_STATUS_IO_TIMEOUT - NT_STATUS_IO_TIMEOUT

I want to get this error message in my exception. And then, i use linux logger and save it into my db. How can i get this output as an exception message?

#!/usr/bin/python3
#!/usr/bin/python

import wmi_client_wrapper as wmi

try:
  wmic = wmi.WmiClientWrapper(username="Administrator",password="Abc",host="192.168.1.2",)
  output = wmic.query("SELECT * FROM Win32_Processor")
  print(output)
except:
  print("An exception occurred")

Body:
To redirect standard error into a variable, one easy approach is to assign sys.stderr to an io.StringIO object, and later retrieve the error message with the object's getvalue method:

import sys
from io import StringIO

error_stream = StringIO()
sys.stderr = error_stream
wmic = wmi.WmiClientWrapper(username="Administrator",password="Abc",host="192.168.1.2",)
output = wmic.query("SELECT * FROM Win32_Processor")
error = error_stream.getvalue()
print('OUTPUT:', output)
print('ERROR:', error)


问:
Applying Azure Metric Alerts to multiple Event Grid Topics
I'm attempting to monitor dead-lettered events among multiple Event Grid Topics within an ecosystem, but can't seem to find a query within Azure Monitor that would output these types of events for alert rule creation at scale - I'm thinking my only option is to set the metric alerts within the Event Grid Topics themselves rather than creating a single alert rule that monitors all EG topics. I'd like to complete this task using IaC, but am worried that the only way I could create that is with a PowerShell script that finds all the possible Event Grid Topics within a subscription, and executes a new metric alert command within a loop (so that each Event Grid Topic has it's own alert).

Am I losing my mind here? Any ideas for a better way to get about this task?

答:
As mentioned in the documentation, we tried passing multiple resourceId's of event grid  to -target resource scope flag in `add-azmetricalertrulev2' cmdlet it failed with below error.

We have tested this in our local environment it is working fine , Below statements are based on the analysis.

You can use the below PowerShell script to create an alert for each event grid topic in the subscription &  to monitor Dead Letter Events. The Script will create the following resources

create a single action group with email Receivers as notification.

creates an alert for all the Event Grid Topics in the subscription with a condition of "DeadLetteredCount>1".

Here is the Powershell Script:

$rg="<RGGroup>" ## resource group to for creating action group & alert

$eventgridlist=Get-AzEventGridTopic
$eventgridid=$eventgridlist.PsTopicsList.ToArray().id
$windowSize = New-TimeSpan -Minutes 5
$frequency = New-TimeSpan -Minutes 1

$condition = New-AzMetricAlertRuleV2Criteria -MetricName "DeadLetteredCount" -TimeAggregation Total -Operator GreaterThan -Threshold 1

$email = New-AzActionGroupReceiver -Name "AGgroupactions" -EmailReceiver -EmailAddress "<emailid>"
Set-AzActionGroup -Name "EGTopic_actionGroup" -ResourceGroup $rg -Receiver $email -ShortName "egalert"
$name=1

foreach( $i in $eventgridid){
Add-AzMetricAlertRuleV2 -Name "eventgridalert_$name" -ResourceGroupName $rg -WindowSize $windowSize -Frequency $frequency -TargetResourceScope $i -TargetResourceType "Microsoft.EventGrid/topics" -TargetResourceRegion " " -Description "This is description" -Severity 3 -ActionGroup $actionGroupId -Condition $condition
$name++
}
```

Here is the Sample Output screenshot for reference:

Body:
As mentioned in the documentation, we tried passing multiple resourceId's of event grid  to -target resource scope flag in `add-azmetricalertrulev2' cmdlet it failed with below error.

We have tested this in our local environment it is working fine , Below statements are based on the analysis.

You can use the below PowerShell script to create an alert for each event grid topic in the subscription &  to monitor Dead Letter Events. The Script will create the following resources

create a single action group with email Receivers as notification.

creates an alert for all the Event Grid Topics in the subscription with a condition of "DeadLetteredCount>1".

Here is the Powershell Script:

$rg="<RGGroup>" ## resource group to for creating action group & alert

$eventgridlist=Get-AzEventGridTopic
$eventgridid=$eventgridlist.PsTopicsList.ToArray().id
$windowSize = New-TimeSpan -Minutes 5
$frequency = New-TimeSpan -Minutes 1

$condition = New-AzMetricAlertRuleV2Criteria -MetricName "DeadLetteredCount" -TimeAggregation Total -Operator GreaterThan -Threshold 1

$email = New-AzActionGroupReceiver -Name "AGgroupactions" -EmailReceiver -EmailAddress "<emailid>"
Set-AzActionGroup -Name "EGTopic_actionGroup" -ResourceGroup $rg -Receiver $email -ShortName "egalert"
$name=1

foreach( $i in $eventgridid){
Add-AzMetricAlertRuleV2 -Name "eventgridalert_$name" -ResourceGroupName $rg -WindowSize $windowSize -Frequency $frequency -TargetResourceScope $i -TargetResourceType "Microsoft.EventGrid/topics" -TargetResourceRegion " " -Description "This is description" -Severity 3 -ActionGroup $actionGroupId -Condition $condition
$name++
}

Here is the Sample Output screenshot for reference:


问:
Java's AES/GCM/NoPadding equivalent in JS
We are using the services of a third party in our organization in which we have to send some data to them in an encrypted manner. Recently, they updated the encryption algorithm to AES/GCM/NoPadding.

They have their code in java whereas we use javascript. they have shared with us their implementation of the algorithm in Java which we have to replicate and implement in JS because that is what we use.

I am facing challenges in converting this code. Attaching both Java implementation which works like a charm and the JS code which is not working as expected. Although I have tried multiple things but none of them worked for me. So, I am sharing only the latest code that I tried.

I have no knowledge of Java or cryptography so any help in that direction will be highly appreciated.

JAVA Code -

import java.io.UnsupportedEncodingException;
import java.nio.ByteBuffer;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.security.SecureRandom;
import java.security.spec.InvalidKeySpecException;
import java.security.spec.KeySpec;

import javax.crypto.Cipher;
import javax.crypto.SecretKey;
import javax.crypto.SecretKeyFactory;
import javax.crypto.spec.GCMParameterSpec;
import javax.crypto.spec.IvParameterSpec;
import javax.crypto.spec.PBEKeySpec;
import javax.crypto.spec.SecretKeySpec;

/**
 * Encryption class for managing all types of AES encryptions
 */
public class EncryptionUtil {

    private final Builder mBuilder;
    private final static String HEX = "0123456789ABCDEF";

    private EncryptionUtil(Builder builder) {
        mBuilder = builder;
    }

    public static EncryptionUtil getDefault(String key, String salt, byte[] iv) {
        try {
            return Builder.getDefaultBuilder(key, salt, iv).build();
        } catch (NoSuchAlgorithmException e) {

            return null;
        }
    }

    public String encryptOrNull(String data) {
        try {
            return encrypt(data);
        } catch (Exception e) {

            return "";
        }
    }

    private String encrypt(String data) throws Exception {
        if (data == null) return null;
        SecretKey secretKey = getSecretKey(hashTheKey(mBuilder.getKey()));
        return doEncryptAES(data, secretKey, mBuilder.getAlgorithm(), mBuilder.getCharsetName());
    }

    private String decrypt(String data) throws Exception {
        if (data == null) return null;
        SecretKey secretKey = getSecretKey(hashTheKey(mBuilder.getKey()));
        return doDecryptAES(data, secretKey, mBuilder.getAlgorithm());
    }

    private String doEncryptAES(String inputString,
                                SecretKey key, String xForm, String charset) throws Exception {
        byte inpBytes[] = inputString.getBytes(charset);
        Cipher cipher = Cipher.getInstance(xForm);
        switch (xForm) {
            case "AES/ECB/PKCS5Padding":
            case "AES/ECB/NoPadding":
                cipher.init(Cipher.ENCRYPT_MODE, key);
                break;
            case "AES/CBC/PKCS5Padding":
            case "AES/CBC/NoPadding":
                cipher.init(Cipher.ENCRYPT_MODE, key, mBuilder.getIvParameterSpec(), mBuilder.getSecureRandom());
                break;
            case "AES/GCM/NoPadding":

                    cipher.init(Cipher.ENCRYPT_MODE, key, new GCMParameterSpec(128, mBuilder.getIv()));
                    byte[] encryptedData = cipher.doFinal(inpBytes);

                    ByteBuffer byteBuffer = ByteBuffer.allocate(4 + mBuilder.getIv().length + encryptedData.length);
                    byteBuffer.putInt(mBuilder.getIv().length);
                    byteBuffer.put(mBuilder.getIv());
                    byteBuffer.put(encryptedData);
                    return toHex(byteBuffer.array());

        }
        return toHex(cipher.doFinal(inpBytes));
    }

    /**
     * for AES in GCM mode kitkat version is required
     *
     * @param inputString is String we want to decrypt
     * @param key         is symmetric key use for decryption and it similar to key used for encryption (128,192,256)
     * @param xForm       is the transformation form in which form we want to transform
     *                    (AES/ECB/PKCS5Padding,AES/ECB/NoPadding,AES/CBC/PKCS5Padding,AES/CBC/NoPadding,AES/GCM/NoPadding)
     * @return it reurn decrypted string
     * @throws Exception NOSuchAlgorithmEXception,NoSuchPaddingEXception
     */
    private String doDecryptAES(String inputString,
                                SecretKey key, String xForm) throws Exception {
        byte[] inpBytes = toByte(inputString);
        Cipher cipher = Cipher.getInstance(xForm);
        switch (xForm) {
            case "AES/ECB/PKCS5Padding":
            case "AES/ECB/NoPadding":
                cipher.init(Cipher.DECRYPT_MODE, key);
                break;
            case "AES/CBC/PKCS5Padding":
            case "AES/CBC/NoPadding":
                cipher.init(Cipher.DECRYPT_MODE, key, mBuilder.getIvParameterSpec(), mBuilder.getSecureRandom());
                break;
            case "AES/GCM/NoPadding":

                    ByteBuffer byteBuffer = ByteBuffer.wrap(inpBytes);
                    int noonceSize = byteBuffer.getInt();

                    if (noonceSize < 12 || noonceSize >= 16)
                        throw new IllegalArgumentException("Nonce size is incorrect. Make sure that the incoming data is an AES encrypted file.");

                    byte[] iv = new byte[noonceSize];
                    byteBuffer.get(iv);

                    byte[] cipherBytes = new byte[byteBuffer.remaining()];
                    byteBuffer.get(cipherBytes);
                    cipher.init(Cipher.DECRYPT_MODE, key, new GCMParameterSpec(128, iv));
                    return new String(cipher.doFinal(cipherBytes), mBuilder.getCharsetName());

        }
        return new String(cipher.doFinal(inpBytes), mBuilder.getCharsetName());
    }

    public String decryptOrNull(String data) {
        try {
            return decrypt(data);
        } catch (Exception e) {
            return "";
        }
    }

    private SecretKey getSecretKey(char[] key) throws NoSuchAlgorithmException, UnsupportedEncodingException, InvalidKeySpecException {
        SecretKeyFactory factory = SecretKeyFactory.getInstance(mBuilder.getSecretKeyType());
        KeySpec spec = new PBEKeySpec(key, mBuilder.getSalt().getBytes(mBuilder.getCharsetName()), mBuilder.getIterationCount(), mBuilder.getKeyLength());
        SecretKey tmp = factory.generateSecret(spec);
        return new SecretKeySpec(tmp.getEncoded(), mBuilder.getKeyAlgorithm());
    }

    private char[] hashTheKey(String key) throws UnsupportedEncodingException, NoSuchAlgorithmException {
        MessageDigest messageDigest = MessageDigest.getInstance(mBuilder.getDigestAlgorithm());
        messageDigest.update(key.getBytes(mBuilder.getCharsetName()));
        return toHex(messageDigest.digest()).toCharArray();
    }

    private byte[] toByte(String hexString) {
        int len = hexString.length() / 2;

        byte[] result = new byte[len];

        for (int i = 0; i < len; i++)
            result[i] = Integer.valueOf(hexString.substring(2 * i, 2 * i + 2), 16).byteValue();
        return result;
    }

    public String toHex(byte[] stringBytes) {
        StringBuffer result = new StringBuffer(2 * stringBytes.length);

        for (int i = 0; i < stringBytes.length; i++) {
            result.append(HEX.charAt((stringBytes[i] >> 4) & 0x0f)).append(HEX.charAt(stringBytes[i] & 0x0f));
        }

        return result.toString();
    }

    private static class Builder {

        private byte[] mIv;
        private int mKeyLength;
        private int mIterationCount;
        private String mSalt;
        private String mKey;
        private String mAlgorithm;
        private String mKeyAlgorithm;
        private String mCharsetName;
        private String mSecretKeyType;
        private String mDigestAlgorithm;
        private String mSecureRandomAlgorithm;
        private SecureRandom mSecureRandom;
        private IvParameterSpec mIvParameterSpec;

        static Builder getDefaultBuilder(String key, String salt, byte[] iv) {
            return new Builder()
                    .setIv(iv)
                    .setKey(key)
                    .setSalt(salt)
                    .setKeyLength(128)
                    .setKeyAlgorithm("AES")
                    .setCharsetName("UTF8")
                    .setIterationCount(1)
                    .setDigestAlgorithm("SHA-256")
                    .setAlgorithm("AES/GCM/NoPadding")
                    .setSecureRandomAlgorithm("SHA1PRNG")
                    .setSecretKeyType("PBKDF2WithHmacSHA1");
        }

        private EncryptionUtil build() throws NoSuchAlgorithmException {
            setSecureRandom(SecureRandom.getInstance(getSecureRandomAlgorithm()));
            SecureRandom secureRandom = new SecureRandom();
            byte[] iv = getIv();
            secureRandom.nextBytes(iv);
            setIvParameterSpec(new IvParameterSpec(iv));
            return new EncryptionUtil(this);
        }

        private String getCharsetName() {
            return mCharsetName;
        }

        private Builder setCharsetName(String charsetName) {
            mCharsetName = charsetName;
            return this;
        }

        private String getAlgorithm() {
            return mAlgorithm;
        }

        private Builder setAlgorithm(String algorithm) {
            mAlgorithm = algorithm;
            return this;
        }

        private String getKeyAlgorithm() {
            return mKeyAlgorithm;
        }

        private Builder setKeyAlgorithm(String keyAlgorithm) {
            mKeyAlgorithm = keyAlgorithm;
            return this;
        }

        private String getSecretKeyType() {
            return mSecretKeyType;
        }

        private Builder setSecretKeyType(String secretKeyType) {
            mSecretKeyType = secretKeyType;
            return this;
        }

        private String getSalt() {
            return mSalt;
        }

        private Builder setSalt(String salt) {
            mSalt = salt;
            return this;
        }

        private String getKey() {
            return mKey;
        }

        private Builder setKey(String key) {
            mKey = key;
            return this;
        }

        private int getKeyLength() {
            return mKeyLength;
        }

        Builder setKeyLength(int keyLength) {
            mKeyLength = keyLength;
            return this;
        }

        private int getIterationCount() {
            return mIterationCount;
        }

        Builder setIterationCount(int iterationCount) {
            mIterationCount = iterationCount;
            return this;
        }

        private String getSecureRandomAlgorithm() {
            return mSecureRandomAlgorithm;
        }

        Builder setSecureRandomAlgorithm(String secureRandomAlgorithm) {
            mSecureRandomAlgorithm = secureRandomAlgorithm;
            return this;
        }

        private byte[] getIv() {
            return mIv;
        }

        Builder setIv(byte[] iv) {
            mIv = iv;
            return this;
        }

        private SecureRandom getSecureRandom() {
            return mSecureRandom;
        }

        Builder setSecureRandom(SecureRandom secureRandom) {
            mSecureRandom = secureRandom;
            return this;
        }

        private IvParameterSpec getIvParameterSpec() {
            return mIvParameterSpec;
        }

        Builder setIvParameterSpec(IvParameterSpec ivParameterSpec) {
            mIvParameterSpec = ivParameterSpec;
            return this;
        }

        private String getDigestAlgorithm() {
            return mDigestAlgorithm;
        }

        Builder setDigestAlgorithm(String digestAlgorithm) {
            mDigestAlgorithm = digestAlgorithm;
            return this;
        }

    }

    public static void main(String[] args) {
        String secretKey = "some_secret_key";
        String salt = "some_secret_salt";
        EncryptionUtil encryptionUtil = EncryptionUtil.getDefault(secretKey, salt, new byte[12]);
        String data = "Data to encrypt";
        System.out.println("Encrypted:");
        String encrypted = encryptionUtil.encryptOrNull(data);
        System.out.println(encrypted);
        System.out.println("Decrypted:");
        System.out.println(encryptionUtil.decryptOrNull(encrypted));
    }
}
```
<p>Please note I need help only to encrypt the data</p>
<p>JS Code -</p>
<pre class="lang-js prettyprint-override"><code>import * as crypto from 'crypto';

export const encData = () => {
    const data = 'Data to encrypt';
    const secretKey = 'some_secret_key';
    const salt = 'some_secret_salt';
    let key = '';

    const keyHash = key => {
        const hash = crypto.createHash('sha256');
        const hashedKey = hash.update(key, 'utf-8');
        return hashedKey.digest('hex').toUpperCase();
    };

    const getSecretKey = key => {
        return crypto.pbkdf2Sync(key, salt, 1, 16, 'sha1');
    };

    key = getSecretKey(keyHash(secretKey));

    const iv = crypto.randomBytes(12);
    const cipher = crypto.createCipheriv('aes-128-gcm', key, iv);
    const buffer = Buffer.from(_.isPlainObject(data) ? JSON.stringify(data) : data);

    // Updating text
    let encrypted = cipher.update(buffer);

    // Using concatenation
    encrypted = Buffer.concat([encrypted, cipher.final()]);

    return encrypted.toString('base64');
};

console.log(encData());
</code>```

To make sure my code is working fine I am decrypting my encoded string generated with JS function by passing it to Java decrypt function.

答:
In the Java code, the result of the encryption is composed as follows:

iv-length (4 bytes, BE) | IV | ciphertext | authentication tag
```
<p>In contrast, in the NodeJS code the result consists only of the ciphertext, i.e. IV length, IV and tag are missing and must be added.<br>
Here it must be taken into account that Java's SunJCE provider automatically concatenates ciphertext and tag, while this must happen explicitly in the NodeJS code.</p>
<p>Also, the ciphertext is returned hex encoded in the Java code, while it is Base64 encoded in the NodeJS code. This also needs to be changed in the NodeJS code.</p>
<p>The fix is to replace in the NodeJS code the lines:</p>
<pre class="lang-js prettyprint-override"><code>// Using concatenation
encrypted = Buffer.concat([encrypted, cipher.final()]);

return encrypted.toString('base64');
</code>```

with:

const length = Buffer.allocUnsafe(4);
length.writeUInt32BE(iv.length);

// Using concatenation
encrypted = Buffer.concat([length, iv, encrypted, cipher.final(), cipher.getAuthTag()]);

return encrypted.toString('hex');
```

With this, the NodeJS code returns a result that can be decrypted by the Java code.

Note that a static salt is insecure. Instead, the salt should be randomly generated like the IV for each encryption and passed along with the ciphertext.

Also, an iteration count of 1 is not secure, the value should be as high as possible with acceptable performance.

Hashing the key with SHA256 before the PBKDF2 derivation is actually not necessary (at least if PBKDF2 is applied correctly).

Body:
We are using the services of a third party in our organization in which we have to send some data to them in an encrypted manner. Recently, they updated the encryption algorithm to AES/GCM/NoPadding.

They have their code in java whereas we use javascript. they have shared with us their implementation of the algorithm in Java which we have to replicate and implement in JS because that is what we use.

I am facing challenges in converting this code. Attaching both Java implementation which works like a charm and the JS code which is not working as expected. Although I have tried multiple things but none of them worked for me. So, I am sharing only the latest code that I tried.

I have no knowledge of Java or cryptography so any help in that direction will be highly appreciated.

JAVA Code -

import java.io.UnsupportedEncodingException;
import java.nio.ByteBuffer;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.security.SecureRandom;
import java.security.spec.InvalidKeySpecException;
import java.security.spec.KeySpec;

import javax.crypto.Cipher;
import javax.crypto.SecretKey;
import javax.crypto.SecretKeyFactory;
import javax.crypto.spec.GCMParameterSpec;
import javax.crypto.spec.IvParameterSpec;
import javax.crypto.spec.PBEKeySpec;
import javax.crypto.spec.SecretKeySpec;

/**
 * Encryption class for managing all types of AES encryptions
 */
public class EncryptionUtil {

    private final Builder mBuilder;
    private final static String HEX = "0123456789ABCDEF";

    private EncryptionUtil(Builder builder) {
        mBuilder = builder;
    }

    public static EncryptionUtil getDefault(String key, String salt, byte[] iv) {
        try {
            return Builder.getDefaultBuilder(key, salt, iv).build();
        } catch (NoSuchAlgorithmException e) {

            return null;
        }
    }

    public String encryptOrNull(String data) {
        try {
            return encrypt(data);
        } catch (Exception e) {

            return "";
        }
    }

    private String encrypt(String data) throws Exception {
        if (data == null) return null;
        SecretKey secretKey = getSecretKey(hashTheKey(mBuilder.getKey()));
        return doEncryptAES(data, secretKey, mBuilder.getAlgorithm(), mBuilder.getCharsetName());
    }

    private String decrypt(String data) throws Exception {
        if (data == null) return null;
        SecretKey secretKey = getSecretKey(hashTheKey(mBuilder.getKey()));
        return doDecryptAES(data, secretKey, mBuilder.getAlgorithm());
    }

    private String doEncryptAES(String inputString,
                                SecretKey key, String xForm, String charset) throws Exception {
        byte inpBytes[] = inputString.getBytes(charset);
        Cipher cipher = Cipher.getInstance(xForm);
        switch (xForm) {
            case "AES/ECB/PKCS5Padding":
            case "AES/ECB/NoPadding":
                cipher.init(Cipher.ENCRYPT_MODE, key);
                break;
            case "AES/CBC/PKCS5Padding":
            case "AES/CBC/NoPadding":
                cipher.init(Cipher.ENCRYPT_MODE, key, mBuilder.getIvParameterSpec(), mBuilder.getSecureRandom());
                break;
            case "AES/GCM/NoPadding":

                    cipher.init(Cipher.ENCRYPT_MODE, key, new GCMParameterSpec(128, mBuilder.getIv()));
                    byte[] encryptedData = cipher.doFinal(inpBytes);

                    ByteBuffer byteBuffer = ByteBuffer.allocate(4 + mBuilder.getIv().length + encryptedData.length);
                    byteBuffer.putInt(mBuilder.getIv().length);
                    byteBuffer.put(mBuilder.getIv());
                    byteBuffer.put(encryptedData);
                    return toHex(byteBuffer.array());

        }
        return toHex(cipher.doFinal(inpBytes));
    }

    /**
     * for AES in GCM mode kitkat version is required
     *
     * @param inputString is String we want to decrypt
     * @param key         is symmetric key use for decryption and it similar to key used for encryption (128,192,256)
     * @param xForm       is the transformation form in which form we want to transform
     *                    (AES/ECB/PKCS5Padding,AES/ECB/NoPadding,AES/CBC/PKCS5Padding,AES/CBC/NoPadding,AES/GCM/NoPadding)
     * @return it reurn decrypted string
     * @throws Exception NOSuchAlgorithmEXception,NoSuchPaddingEXception
     */
    private String doDecryptAES(String inputString,
                                SecretKey key, String xForm) throws Exception {
        byte[] inpBytes = toByte(inputString);
        Cipher cipher = Cipher.getInstance(xForm);
        switch (xForm) {
            case "AES/ECB/PKCS5Padding":
            case "AES/ECB/NoPadding":
                cipher.init(Cipher.DECRYPT_MODE, key);
                break;
            case "AES/CBC/PKCS5Padding":
            case "AES/CBC/NoPadding":
                cipher.init(Cipher.DECRYPT_MODE, key, mBuilder.getIvParameterSpec(), mBuilder.getSecureRandom());
                break;
            case "AES/GCM/NoPadding":

                    ByteBuffer byteBuffer = ByteBuffer.wrap(inpBytes);
                    int noonceSize = byteBuffer.getInt();

                    if (noonceSize < 12 || noonceSize >= 16)
                        throw new IllegalArgumentException("Nonce size is incorrect. Make sure that the incoming data is an AES encrypted file.");

                    byte[] iv = new byte[noonceSize];
                    byteBuffer.get(iv);

                    byte[] cipherBytes = new byte[byteBuffer.remaining()];
                    byteBuffer.get(cipherBytes);
                    cipher.init(Cipher.DECRYPT_MODE, key, new GCMParameterSpec(128, iv));
                    return new String(cipher.doFinal(cipherBytes), mBuilder.getCharsetName());

        }
        return new String(cipher.doFinal(inpBytes), mBuilder.getCharsetName());
    }

    public String decryptOrNull(String data) {
        try {
            return decrypt(data);
        } catch (Exception e) {
            return "";
        }
    }

    private SecretKey getSecretKey(char[] key) throws NoSuchAlgorithmException, UnsupportedEncodingException, InvalidKeySpecException {
        SecretKeyFactory factory = SecretKeyFactory.getInstance(mBuilder.getSecretKeyType());
        KeySpec spec = new PBEKeySpec(key, mBuilder.getSalt().getBytes(mBuilder.getCharsetName()), mBuilder.getIterationCount(), mBuilder.getKeyLength());
        SecretKey tmp = factory.generateSecret(spec);
        return new SecretKeySpec(tmp.getEncoded(), mBuilder.getKeyAlgorithm());
    }

    private char[] hashTheKey(String key) throws UnsupportedEncodingException, NoSuchAlgorithmException {
        MessageDigest messageDigest = MessageDigest.getInstance(mBuilder.getDigestAlgorithm());
        messageDigest.update(key.getBytes(mBuilder.getCharsetName()));
        return toHex(messageDigest.digest()).toCharArray();
    }

    private byte[] toByte(String hexString) {
        int len = hexString.length() / 2;

        byte[] result = new byte[len];

        for (int i = 0; i < len; i++)
            result[i] = Integer.valueOf(hexString.substring(2 * i, 2 * i + 2), 16).byteValue();
        return result;
    }

    public String toHex(byte[] stringBytes) {
        StringBuffer result = new StringBuffer(2 * stringBytes.length);

        for (int i = 0; i < stringBytes.length; i++) {
            result.append(HEX.charAt((stringBytes[i] >> 4) & 0x0f)).append(HEX.charAt(stringBytes[i] & 0x0f));
        }

        return result.toString();
    }

    private static class Builder {

        private byte[] mIv;
        private int mKeyLength;
        private int mIterationCount;
        private String mSalt;
        private String mKey;
        private String mAlgorithm;
        private String mKeyAlgorithm;
        private String mCharsetName;
        private String mSecretKeyType;
        private String mDigestAlgorithm;
        private String mSecureRandomAlgorithm;
        private SecureRandom mSecureRandom;
        private IvParameterSpec mIvParameterSpec;

        static Builder getDefaultBuilder(String key, String salt, byte[] iv) {
            return new Builder()
                    .setIv(iv)
                    .setKey(key)
                    .setSalt(salt)
                    .setKeyLength(128)
                    .setKeyAlgorithm("AES")
                    .setCharsetName("UTF8")
                    .setIterationCount(1)
                    .setDigestAlgorithm("SHA-256")
                    .setAlgorithm("AES/GCM/NoPadding")
                    .setSecureRandomAlgorithm("SHA1PRNG")
                    .setSecretKeyType("PBKDF2WithHmacSHA1");
        }

        private EncryptionUtil build() throws NoSuchAlgorithmException {
            setSecureRandom(SecureRandom.getInstance(getSecureRandomAlgorithm()));
            SecureRandom secureRandom = new SecureRandom();
            byte[] iv = getIv();
            secureRandom.nextBytes(iv);
            setIvParameterSpec(new IvParameterSpec(iv));
            return new EncryptionUtil(this);
        }

        private String getCharsetName() {
            return mCharsetName;
        }

        private Builder setCharsetName(String charsetName) {
            mCharsetName = charsetName;
            return this;
        }

        private String getAlgorithm() {
            return mAlgorithm;
        }

        private Builder setAlgorithm(String algorithm) {
            mAlgorithm = algorithm;
            return this;
        }

        private String getKeyAlgorithm() {
            return mKeyAlgorithm;
        }

        private Builder setKeyAlgorithm(String keyAlgorithm) {
            mKeyAlgorithm = keyAlgorithm;
            return this;
        }

        private String getSecretKeyType() {
            return mSecretKeyType;
        }

        private Builder setSecretKeyType(String secretKeyType) {
            mSecretKeyType = secretKeyType;
            return this;
        }

        private String getSalt() {
            return mSalt;
        }

        private Builder setSalt(String salt) {
            mSalt = salt;
            return this;
        }

        private String getKey() {
            return mKey;
        }

        private Builder setKey(String key) {
            mKey = key;
            return this;
        }

        private int getKeyLength() {
            return mKeyLength;
        }

        Builder setKeyLength(int keyLength) {
            mKeyLength = keyLength;
            return this;
        }

        private int getIterationCount() {
            return mIterationCount;
        }

        Builder setIterationCount(int iterationCount) {
            mIterationCount = iterationCount;
            return this;
        }

        private String getSecureRandomAlgorithm() {
            return mSecureRandomAlgorithm;
        }

        Builder setSecureRandomAlgorithm(String secureRandomAlgorithm) {
            mSecureRandomAlgorithm = secureRandomAlgorithm;
            return this;
        }

        private byte[] getIv() {
            return mIv;
        }

        Builder setIv(byte[] iv) {
            mIv = iv;
            return this;
        }

        private SecureRandom getSecureRandom() {
            return mSecureRandom;
        }

        Builder setSecureRandom(SecureRandom secureRandom) {
            mSecureRandom = secureRandom;
            return this;
        }

        private IvParameterSpec getIvParameterSpec() {
            return mIvParameterSpec;
        }

        Builder setIvParameterSpec(IvParameterSpec ivParameterSpec) {
            mIvParameterSpec = ivParameterSpec;
            return this;
        }

        private String getDigestAlgorithm() {
            return mDigestAlgorithm;
        }

        Builder setDigestAlgorithm(String digestAlgorithm) {
            mDigestAlgorithm = digestAlgorithm;
            return this;
        }

    }

    public static void main(String[] args) {
        String secretKey = "some_secret_key";
        String salt = "some_secret_salt";
        EncryptionUtil encryptionUtil = EncryptionUtil.getDefault(secretKey, salt, new byte[12]);
        String data = "Data to encrypt";
        System.out.println("Encrypted:");
        String encrypted = encryptionUtil.encryptOrNull(data);
        System.out.println(encrypted);
        System.out.println("Decrypted:");
        System.out.println(encryptionUtil.decryptOrNull(encrypted));
    }
}

Please note I need help only to encrypt the data

JS Code -

import * as crypto from 'crypto';

export const encData = () => {
    const data = 'Data to encrypt';
    const secretKey = 'some_secret_key';
    const salt = 'some_secret_salt';
    let key = '';

    const keyHash = key => {
        const hash = crypto.createHash('sha256');
        const hashedKey = hash.update(key, 'utf-8');
        return hashedKey.digest('hex').toUpperCase();
    };

    const getSecretKey = key => {
        return crypto.pbkdf2Sync(key, salt, 1, 16, 'sha1');
    };

    key = getSecretKey(keyHash(secretKey));

    const iv = crypto.randomBytes(12);
    const cipher = crypto.createCipheriv('aes-128-gcm', key, iv);
    const buffer = Buffer.from(_.isPlainObject(data) ? JSON.stringify(data) : data);

    // Updating text
    let encrypted = cipher.update(buffer);

    // Using concatenation
    encrypted = Buffer.concat([encrypted, cipher.final()]);

    return encrypted.toString('base64');
};

console.log(encData());

To make sure my code is working fine I am decrypting my encoded string generated with JS function by passing it to Java decrypt function.

Body:
In the Java code, the result of the encryption is composed as follows:

iv-length (4 bytes, BE) | IV | ciphertext | authentication tag

In contrast, in the NodeJS code the result consists only of the ciphertext, i.e. IV length, IV and tag are missing and must be added.

Here it must be taken into account that Java's SunJCE provider automatically concatenates ciphertext and tag, while this must happen explicitly in the NodeJS code.

Also, the ciphertext is returned hex encoded in the Java code, while it is Base64 encoded in the NodeJS code. This also needs to be changed in the NodeJS code.

The fix is to replace in the NodeJS code the lines:

// Using concatenation
encrypted = Buffer.concat([encrypted, cipher.final()]);

return encrypted.toString('base64');

with:

const length = Buffer.allocUnsafe(4);
length.writeUInt32BE(iv.length);

// Using concatenation
encrypted = Buffer.concat([length, iv, encrypted, cipher.final(), cipher.getAuthTag()]);

return encrypted.toString('hex');

With this, the NodeJS code returns a result that can be decrypted by the Java code.

Note that a static salt is insecure. Instead, the salt should be randomly generated like the IV for each encryption and passed along with the ciphertext.

Also, an iteration count of 1 is not secure, the value should be as high as possible with acceptable performance.

Hashing the key with SHA256 before the PBKDF2 derivation is actually not necessary (at least if PBKDF2 is applied correctly).