问:
Excel keeps crashing after saving VBA
I have a spreadsheet that is managing staff caseloads and managed to find some VBA on here that helped to sort that data effectively, but now it keeps crashing whenever I save. I'm a bit of a beginner so the code is hashed together from various question I have found on this site (thanks by the way! amazing resource!)

Is anyone able to look at my code and see anything glaringly obvious please? My worksheet only crashes after I click the button I have attached this to and then try to save immediately after

```<code>Private Sub CommandButton2_Click()
 Dim LRow As Long
    Dim rngDB As Range
    Dim v As Integer
    Dim Ws As Worksheet

    Application.AddCustomList ListArray:=Array("TBC", "Waiting", "Active", "Closed")
    v = Application.CustomListCount

    Set Ws = ActiveWorkbook.Worksheets("Cases")

    With Ws
        LRow = .Cells(.Rows.Count, "A").End(xlUp).Row
        Set rngDB = .Range("A5", .Range("T" & LRow))
        .Sort.SortFields.Clear
        .Sort.SortFields.Add Key:=.Range("J3"), Order:=xlAscending, CustomOrder:=v
        .Sort.SortFields.Add Key:=.Range("G1"), Order:=xlDescending
          With .Sort
            .SetRange rngDB
            .Header = xlNo
            .Orientation = xlTopToBottom
            .Apply
        End With
    End With
    Application.DeleteCustomList v
End Sub
</code>```

Title: Excel keeps crashing after saving VBA

Body:
I have a spreadsheet that is managing staff caseloads and managed to find some VBA on here that helped to sort that data effectively, but now it keeps crashing whenever I save. I'm a bit of a beginner so the code is hashed together from various question I have found on this site (thanks by the way! amazing resource!)

Is anyone able to look at my code and see anything glaringly obvious please? My worksheet only crashes after I click the button I have attached this to and then try to save immediately after

Private Sub CommandButton2_Click()
 Dim LRow As Long
    Dim rngDB As Range
    Dim v As Integer
    Dim Ws As Worksheet

    Application.AddCustomList ListArray:=Array("TBC", "Waiting", "Active", "Closed")
    v = Application.CustomListCount

    Set Ws = ActiveWorkbook.Worksheets("Cases")

    With Ws
        LRow = .Cells(.Rows.Count, "A").End(xlUp).Row
        Set rngDB = .Range("A5", .Range("T" & LRow))
        .Sort.SortFields.Clear
        .Sort.SortFields.Add Key:=.Range("J3"), Order:=xlAscending, CustomOrder:=v
        .Sort.SortFields.Add Key:=.Range("G1"), Order:=xlDescending
          With .Sort
            .SetRange rngDB
            .Header = xlNo
            .Orientation = xlTopToBottom
            .Apply
        End With
    End With
    Application.DeleteCustomList v
End Sub


来源: stackexchange

问:
How to set parameters for included T4 templates?
If I have a reusable T4 template, how can I set parameters when recalling it?

Consider these two example files...

Hello.t4:

```<code><#@ parameter type="System.String" name="Name" #>
Hello <#= this.Name #>!
</code>```

HelloWorld.tt:

```<code>Testing text templates,
<#@ include file="Hello.t4" #>
</code>```

How to pass the string "World" as the "Name" parameter of the Hello.t4 template?

ContentLicense: CC BY-SA 4.0

Body:
If I have a reusable T4 template, how can I set parameters when recalling it?

Consider these two example files...

Hello.t4:

<#@ parameter type="System.String" name="Name" #>
Hello <#= this.Name #>!

HelloWorld.tt:

Testing text templates,
<#@ include file="Hello.t4" #>

How to pass the string "World" as the "Name" parameter of the Hello.t4 template?


问:
Unauthorized access. 'Send' claim(s) are required to perform this operation - Azure Queue Send Message
We have been trying to send a test message to Azure Service Bus Queue from SPFx webpart, however, every time we are seeing --> Unauthorized access. 'Send' claim(s) are required to perform this operation. Resource: 'sb://.servicebus.windows.net/'.

Steps we have followed -

Created a sample SPFx webpart.

Used
{
"resource": "Microsoft.ServiceBus",
"scope": "user_impersonation"
}
inside webApiPermissionRequests of package-solution.json.

Deployed the webpart and approved scopes from Admin center so that an Azure AD App service principal gets created with API Permissions.

Now migrated the Redirect Uris to SPA for PKCE flow (MSAL 2.0).

Set Allow Public Client Flows to "Yes".

Then got the client id and tenant id from AD App and used in our SPFx solution to connect Service Bus Queue through javascript SDK.

Used InteractiveBrowserCredential from @azure/identity and ServiceBusClient from @azure/service-bus to send a message to the Queue. Snippet below -

```<code>try{
      const credential = new InteractiveBrowserCredential({
          tenantId: "<tenant-id>",
          clientId: "<SharePoint Online Client Extensibility Web Application Principal client-id>"
      });
      const fullyQualifiedNamespace = "<servicebus-namespace>.servicebus.windows.net";
      const serviceBusClient = new ServiceBusClient(fullyQualifiedNamespace, credential);
      const sender = serviceBusClient.createSender("queue-name");
      await sender.sendMessages({body: "test"});
      console.log("message sent");
}
catch(error){
      console.log(error);
}
</code>```

Provided Owner and Azure Service Bus Data Owner access from IAM to the Service Principal. Both queue and service bus level.

However, every time code block is throwing exception and the message is - Unauthorized access. 'Send' claim(s) are required to perform this operation. Resource: 'sb://.servicebus.windows.net/'.

Any help would be appreciated. Thanks !!

Body:
We have been trying to send a test message to Azure Service Bus Queue from SPFx webpart, however, every time we are seeing --> Unauthorized access. 'Send' claim(s) are required to perform this operation. Resource: 'sb://.servicebus.windows.net/'.

Steps we have followed -

Created a sample SPFx webpart.

Used
{
"resource": "Microsoft.ServiceBus",
"scope": "user_impersonation"
}
inside webApiPermissionRequests of package-solution.json.

Deployed the webpart and approved scopes from Admin center so that an Azure AD App service principal gets created with API Permissions.

Now migrated the Redirect Uris to SPA for PKCE flow (MSAL 2.0).

Set Allow Public Client Flows to "Yes".

Then got the client id and tenant id from AD App and used in our SPFx solution to connect Service Bus Queue through javascript SDK.

Used InteractiveBrowserCredential from @azure/identity and ServiceBusClient from @azure/service-bus to send a message to the Queue. Snippet below -

try{
      const credential = new InteractiveBrowserCredential({
          tenantId: "<tenant-id>",
          clientId: "<SharePoint Online Client Extensibility Web Application Principal client-id>"
      });
      const fullyQualifiedNamespace = "<servicebus-namespace>.servicebus.windows.net";
      const serviceBusClient = new ServiceBusClient(fullyQualifiedNamespace, credential);
      const sender = serviceBusClient.createSender("queue-name");
      await sender.sendMessages({body: "test"});
      console.log("message sent");
}
catch(error){
      console.log(error);
}

Provided Owner and Azure Service Bus Data Owner access from IAM to the Service Principal. Both queue and service bus level.

However, every time code block is throwing exception and the message is - Unauthorized access. 'Send' claim(s) are required to perform this operation. Resource: 'sb://.servicebus.windows.net/'.

Any help would be appreciated. Thanks !!


问:
Woocommerce - Hide a product category based on the stock status of the products that are under this category?
Is there any way to automatically hide a woocommerce category if all the products in that category are out of stock?

Same question was asked here, but there was no answer at the time.

Woocommerce - Is it possible to hide a product category based on the stock status of the products that are under this category?

I actually thought this would be something that would happen by default and just noticed a bunch of categories with no products in stock haha!

As sold out products aren't displayed, it means a customer sees the category and clicks on it, only to be greated by a "No products were found matching your selection" message.

Ideally once all products in any given category sell out, that category will then be automatically hidden.
Then if a new product is added to the category, or an existing one is restocked, the category would be visible again.

Any help or recommendations would be very much appreciated!


问:
Google Mail API Error: 'Account has been deleted error'
When we try to get a accessToken for a user for the Google Mail API via the https://www.googleapis.com/oauth2/v3/token endpoint we get an error

'Account has been deleted error'.

This works fine for all users but one.  Before last week also for this user everything worked find.  When I check the Users in de Google Workspace Admin Console the user and email address still exists and is active.

What can be the cause of this error?

We tried the same with other users in the organisation and these work just fine.


问:
SSH connection reset error when running a task
I'm running an Ansible playbook on my host and get this error when executing a task on an Ubuntu server:

TASK [Get externalId] ***********************************************************************************************************************************************************************************************
task path: /home/ankhas/Work/ansible/playbooks/09-start-ubuntu-application.yaml:76
<178.20.1.160> ESTABLISH SSH CONNECTION FOR USER: ubuntu
<178.20.1.160> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o ServerAliveInterval=50 -o 'IdentityFile="/home/ankhas/.ssh/id_ed25519"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="ubuntu"' -o ConnectTimeout=10 -o StrictHostKeyChecking=no -o 'UserKnownHostsFile=~/.ssh/known_hosts' -o 'ControlPath="/home/ankhas/.ansible/cp/c66e8da40c"' 178.20.1.160 '/bin/sh -c '"'"'echo ~ubuntu && sleep 0'"'"''
<178.20.1.160> (0, b'/home/ubuntu\n', b'')
<178.20.1.160> ESTABLISH SSH CONNECTION FOR USER: ubuntu
<178.20.1.160> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o ServerAliveInterval=50 -o 'IdentityFile="/home/ankhas/.ssh/id_ed25519"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="ubuntu"' -o ConnectTimeout=10 -o StrictHostKeyChecking=no -o 'UserKnownHostsFile=~/.ssh/known_hosts' -o 'ControlPath="/home/ankhas/.ansible/cp/c66e8da40c"' 178.20.1.160 '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo /home/ubuntu/.ansible/tmp `"&& mkdir "` echo /home/ubuntu/.ansible/tmp/ansible-tmp-1673361715.6199737-72504-90630208728260 `" && echo ansible-tmp-1673361715.6199737-72504-90630208728260="` echo /home/ubuntu/.ansible/tmp/ansible-tmp-1673361715.6199737-72504-90630208728260 `" ) && sleep 0'"'"''
<178.20.1.160> (0, b'ansible-tmp-1673361715.6199737-72504-90630208728260=/home/ubuntu/.ansible/tmp/ansible-tmp-1673361715.6199737-72504-90630208728260\n', b'')
Using module file /home/ankhas/.local/lib/python3.10/site-packages/ansible/modules/uri.py
<178.20.1.160> PUT /home/ankhas/.ansible/tmp/ansible-local-72407qnz3utfp/tmpzfjmd8o8 TO /home/ubuntu/.ansible/tmp/ansible-tmp-1673361715.6199737-72504-90630208728260/AnsiballZ_uri.py
<178.20.1.160> SSH: EXEC sftp -b - -C -o ControlMaster=auto -o ControlPersist=60s -o ServerAliveInterval=50 -o 'IdentityFile="/home/ankhas/.ssh/id_ed25519"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="ubuntu"' -o ConnectTimeout=10 -o StrictHostKeyChecking=no -o 'UserKnownHostsFile=~/.ssh/known_hosts' -o 'ControlPath="/home/ankhas/.ansible/cp/c66e8da40c"' '[178.20.1.160]'
<178.20.1.160> (0, b'sftp> put /home/ankhas/.ansible/tmp/ansible-local-72407qnz3utfp/tmpzfjmd8o8 /home/ubuntu/.ansible/tmp/ansible-tmp-1673361715.6199737-72504-90630208728260/AnsiballZ_uri.py\n', b'')
<178.20.1.160> ESTABLISH SSH CONNECTION FOR USER: ubuntu
<178.20.1.160> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o ServerAliveInterval=50 -o 'IdentityFile="/home/ankhas/.ssh/id_ed25519"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="ubuntu"' -o ConnectTimeout=10 -o StrictHostKeyChecking=no -o 'UserKnownHostsFile=~/.ssh/known_hosts' -o 'ControlPath="/home/ankhas/.ansible/cp/c66e8da40c"' 178.20.1.160 '/bin/sh -c '"'"'chmod u+x /home/ubuntu/.ansible/tmp/ansible-tmp-1673361715.6199737-72504-90630208728260/ /home/ubuntu/.ansible/tmp/ansible-tmp-1673361715.6199737-72504-90630208728260/AnsiballZ_uri.py && sleep 0'"'"''
<178.20.1.160> (0, b'', b'')
<178.20.1.160> ESTABLISH SSH CONNECTION FOR USER: ubuntu
<178.20.1.160> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o ServerAliveInterval=50 -o 'IdentityFile="/home/ankhas/.ssh/id_ed25519"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="ubuntu"' -o ConnectTimeout=10 -o StrictHostKeyChecking=no -o 'UserKnownHostsFile=~/.ssh/known_hosts' -o 'ControlPath="/home/ankhas/.ansible/cp/c66e8da40c"' -tt 178.20.1.160 '/bin/sh -c '"'"'/usr/bin/python3 /home/ubuntu/.ansible/tmp/ansible-tmp-1673361715.6199737-72504-90630208728260/AnsiballZ_uri.py && sleep 0'"'"''
<178.20.1.160> (1, b'Traceback (most recent call last):\r\n  File "/home/ubuntu/.ansible/tmp/ansible-tmp-1673361715.6199737-72504-90630208728260/AnsiballZ_uri.py", line 107, in <module>\r\n    _ansiballz_main()\r\n  File "/home/ubuntu/.ansible/tmp/ansible-tmp-1673361715.6199737-72504-90630208728260/AnsiballZ_uri.py", line 99, in _ansiballz_main\r\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\r\n  File "/home/ubuntu/.ansible/tmp/ansible-tmp-1673361715.6199737-72504-90630208728260/AnsiballZ_uri.py", line 47, in invoke_module\r\n    runpy.run_module(mod_name=\'ansible.modules.uri\', init_globals=dict(_module_fqn=\'ansible.modules.uri\', _modlib_path=modlib_path),\r\n  File "/usr/lib/python3.8/runpy.py", line 207, in run_module\r\n    return _run_module_code(code, init_globals, run_name, mod_spec)\r\n  File "/usr/lib/python3.8/runpy.py", line 97, in _run_module_code\r\n    _run_code(code, mod_globals, init_globals,\r\n  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code\r\n    exec(code, run_globals)\r\n  File "/tmp/ansible_ansible.legacy.uri_payload_0o98qj0h/ansible_ansible.legacy.uri_payload.zip/ansible/modules/uri.py", line 788, in <module>\r\n  File "/tmp/ansible_ansible.legacy.uri_payload_0o98qj0h/ansible_ansible.legacy.uri_payload.zip/ansible/modules/uri.py", line 718, in main\r\n  File "/usr/lib/python3.8/http/client.py", line 466, in read\r\n    return self._readall_chunked()\r\n  File "/usr/lib/python3.8/http/client.py", line 576, in _readall_chunked\r\n    value.append(self._safe_read(chunk_left))\r\n  File "/usr/lib/python3.8/http/client.py", line 613, in _safe_read\r\n    data = self.fp.read(amt)\r\n  File "/usr/lib/python3.8/socket.py", line 669, in readinto\r\n    return self._sock.recv_into(b)\r\n  File "/usr/lib/python3.8/ssl.py", line 1241, in recv_into\r\n    return self.read(nbytes, buffer)\r\n  File "/usr/lib/python3.8/ssl.py", line 1099, in read\r\n    return self._sslobj.read(len, buffer)\r\nConnectionResetError: [Errno 104] Connection reset by peer\r\n', b'Shared connection to 178.20.1.160 closed.\r\n')
<178.20.1.160> Failed to connect to the host via ssh: Shared connection to 178.20.1.160 closed.
<178.20.1.160> ESTABLISH SSH CONNECTION FOR USER: ubuntu
<178.20.1.160> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o ServerAliveInterval=50 -o 'IdentityFile="/home/ankhas/.ssh/id_ed25519"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="ubuntu"' -o ConnectTimeout=10 -o StrictHostKeyChecking=no -o 'UserKnownHostsFile=~/.ssh/known_hosts' -o 'ControlPath="/home/ankhas/.ansible/cp/c66e8da40c"' 178.20.1.160 '/bin/sh -c '"'"'rm -f -r /home/ubuntu/.ansible/tmp/ansible-tmp-1673361715.6199737-72504-90630208728260/ > /dev/null 2>&1 && sleep 0'"'"''
<178.20.1.160> (0, b'', b'')
The full traceback is:
Traceback (most recent call last):
  File "/home/ubuntu/.ansible/tmp/ansible-tmp-1673361715.6199737-72504-90630208728260/AnsiballZ_uri.py", line 107, in <module>
    _ansiballz_main()
  File "/home/ubuntu/.ansible/tmp/ansible-tmp-1673361715.6199737-72504-90630208728260/AnsiballZ_uri.py", line 99, in _ansiballz_main
    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
  File "/home/ubuntu/.ansible/tmp/ansible-tmp-1673361715.6199737-72504-90630208728260/AnsiballZ_uri.py", line 47, in invoke_module
    runpy.run_module(mod_name='ansible.modules.uri', init_globals=dict(_module_fqn='ansible.modules.uri', _modlib_path=modlib_path),
  File "/usr/lib/python3.8/runpy.py", line 207, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/usr/lib/python3.8/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/tmp/ansible_ansible.legacy.uri_payload_0o98qj0h/ansible_ansible.legacy.uri_payload.zip/ansible/modules/uri.py", line 788, in <module>
  File "/tmp/ansible_ansible.legacy.uri_payload_0o98qj0h/ansible_ansible.legacy.uri_payload.zip/ansible/modules/uri.py", line 718, in main
  File "/usr/lib/python3.8/http/client.py", line 466, in read
    return self._readall_chunked()
  File "/usr/lib/python3.8/http/client.py", line 576, in _readall_chunked
    value.append(self._safe_read(chunk_left))
  File "/usr/lib/python3.8/http/client.py", line 613, in _safe_read
    data = self.fp.read(amt)
  File "/usr/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
  File "/usr/lib/python3.8/ssl.py", line 1241, in recv_into
    return self.read(nbytes, buffer)
  File "/usr/lib/python3.8/ssl.py", line 1099, in read
    return self._sslobj.read(len, buffer)
ConnectionResetError: [Errno 104] Connection reset by peer
fatal: [ubuntu]: FAILED! => {
    "changed": false,
    "module_stderr": "Shared connection to 178.20.1.160 closed.\r\n",
    "module_stdout": "Traceback (most recent call last):\r\n  File \"/home/ubuntu/.ansible/tmp/ansible-tmp-1673361715.6199737-72504-90630208728260/AnsiballZ_uri.py\", line 107, in <module>\r\n    _ansiballz_main()\r\n  File \"/home/ubuntu/.ansible/tmp/ansible-tmp-1673361715.6199737-72504-90630208728260/AnsiballZ_uri.py\", line 99, in _ansiballz_main\r\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\r\n  File \"/home/ubuntu/.ansible/tmp/ansible-tmp-1673361715.6199737-72504-90630208728260/AnsiballZ_uri.py\", line 47, in invoke_module\r\n    runpy.run_module(mod_name='ansible.modules.uri', init_globals=dict(_module_fqn='ansible.modules.uri', _modlib_path=modlib_path),\r\n  File \"/usr/lib/python3.8/runpy.py\", line 207, in run_module\r\n    return _run_module_code(code, init_globals, run_name, mod_spec)\r\n  File \"/usr/lib/python3.8/runpy.py\", line 97, in _run_module_code\r\n    _run_code(code, mod_globals, init_globals,\r\n  File \"/usr/lib/python3.8/runpy.py\", line 87, in _run_code\r\n    exec(code, run_globals)\r\n  File \"/tmp/ansible_ansible.legacy.uri_payload_0o98qj0h/ansible_ansible.legacy.uri_payload.zip/ansible/modules/uri.py\", line 788, in <module>\r\n  File \"/tmp/ansible_ansible.legacy.uri_payload_0o98qj0h/ansible_ansible.legacy.uri_payload.zip/ansible/modules/uri.py\", line 718, in main\r\n  File \"/usr/lib/python3.8/http/client.py\", line 466, in read\r\n    return self._readall_chunked()\r\n  File \"/usr/lib/python3.8/http/client.py\", line 576, in _readall_chunked\r\n    value.append(self._safe_read(chunk_left))\r\n  File \"/usr/lib/python3.8/http/client.py\", line 613, in _safe_read\r\n    data = self.fp.read(amt)\r\n  File \"/usr/lib/python3.8/socket.py\", line 669, in readinto\r\n    return self._sock.recv_into(b)\r\n  File \"/usr/lib/python3.8/ssl.py\", line 1241, in recv_into\r\n    return self.read(nbytes, buffer)\r\n  File \"/usr/lib/python3.8/ssl.py\", line 1099, in read\r\n    return self._sslobj.read(len, buffer)\r\nConnectionResetError: [Errno 104] Connection reset by peer\r\n",
    "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
    "rc": 1
}
```
<p>My task is:</p>
<pre class="lang-yaml prettyprint-override"><code>- name: Get externalId
  ansible.builtin.uri:
    url: "{{ link }}"
    method: GET
    validate_certs: no
    body_format: json
    headers:
      Authorization: "Bearer {{ token }}"
    return_content: yes
    status_code:
      - 200
  register: external
</code>```

My ansible.cfg file is :

```<code>[defaults]
inventory = ./inventory/test.ini
host_key_checking = no

[ssh_connection]
ssh_args = "-C -o ControlMaster=auto -o ControlPersist=60s -o ServerAliveInterval=50"
</code>```

My inventory is something like this:

```<code>...

[all:vars]
ansible_user="ubuntu"
ansible_become_method="sudo"
ansible_become_password="{{ vault_become_password }}"

ansible_ssh_private_key_file="~/.ssh/id_ed25519"
ansible_ssh_common_args='-o StrictHostKeyChecking=no -o UserKnownHostsFile=~/.ssh/known_hosts'
ansible_python_interpreter='/usr/bin/python3'
</code>```

The SSH connection seems to drop and the playbook stop suddenly. Moreover, several tasks are defined and works well before this one. Some of them are also using ansible.builtin.uri module.
But when I ran the playbook with debug mode (ie. -vvvv argument`), I noticed these logs in every previous task:

```<code><178.20.1.160> ESTABLISH SSH CONNECTION FOR USER: ubuntu
<178.20.1.160> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o ServerAliveInterval=50 -o StrictHostKeyChecking=no -o 'IdentityFile="/home/ankhas/.ssh/id_ed25519"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="ubuntu"' -o ConnectTimeout=10 -o StrictHostKeyChecking=no -o 'UserKnownHostsFile=~/.ssh/known_hosts' -o 'ControlPath="/home/ankhas/.ansible/cp/c66e8da40c"' 178.20.1.160 '/bin/sh -c '"'"'rm -f -r /home/ubuntu/.ansible/tmp/ansible-tmp-1673363204.2430837-75125-79554162758890/ > /dev/null 2>&1 && sleep 0'"'"''
<178.20.1.160> (0, b'', b"OpenSSH_9.0p1 Ubuntu-1ubuntu7.1, OpenSSL 3.0.5 5 Jul 2022\r\ndebug1: Reading configuration data /home/ankhas/.ssh/config\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files\r\ndebug1: /etc/ssh/ssh_config line 21: Applying options for *\r\ndebug2: resolve_canonicalize: hostname 178.20.1.160 is address\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/home/ankhas/.ssh/known_hosts'\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 75060\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n")
</code>```

Nevertheless, sometimes the task works with no issue and the playbook can continue.

By the way, I'm also wondering why there is not one but several SSH connection attempts in the above logs.

I tried to change [ssh_connection] configuration in my ansible.cfg by setting:

pipelining to True

timeout to 3000

ssh_args

I'm expecting this playbook to works every time by not having SSH Connection reset by peer error.

Do you have any idea how to solve this issue?

Body:
I'm running an Ansible playbook on my host and get this error when executing a task on an Ubuntu server:

TASK [Get externalId] ***********************************************************************************************************************************************************************************************
task path: /home/ankhas/Work/ansible/playbooks/09-start-ubuntu-application.yaml:76
<178.20.1.160> ESTABLISH SSH CONNECTION FOR USER: ubuntu
<178.20.1.160> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o ServerAliveInterval=50 -o 'IdentityFile="/home/ankhas/.ssh/id_ed25519"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="ubuntu"' -o ConnectTimeout=10 -o StrictHostKeyChecking=no -o 'UserKnownHostsFile=~/.ssh/known_hosts' -o 'ControlPath="/home/ankhas/.ansible/cp/c66e8da40c"' 178.20.1.160 '/bin/sh -c '"'"'echo ~ubuntu && sleep 0'"'"''
<178.20.1.160> (0, b'/home/ubuntu\n', b'')
<178.20.1.160> ESTABLISH SSH CONNECTION FOR USER: ubuntu
<178.20.1.160> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o ServerAliveInterval=50 -o 'IdentityFile="/home/ankhas/.ssh/id_ed25519"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="ubuntu"' -o ConnectTimeout=10 -o StrictHostKeyChecking=no -o 'UserKnownHostsFile=~/.ssh/known_hosts' -o 'ControlPath="/home/ankhas/.ansible/cp/c66e8da40c"' 178.20.1.160 '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo /home/ubuntu/.ansible/tmp `"&& mkdir "` echo /home/ubuntu/.ansible/tmp/ansible-tmp-1673361715.6199737-72504-90630208728260 `" && echo ansible-tmp-1673361715.6199737-72504-90630208728260="` echo /home/ubuntu/.ansible/tmp/ansible-tmp-1673361715.6199737-72504-90630208728260 `" ) && sleep 0'"'"''
<178.20.1.160> (0, b'ansible-tmp-1673361715.6199737-72504-90630208728260=/home/ubuntu/.ansible/tmp/ansible-tmp-1673361715.6199737-72504-90630208728260\n', b'')
Using module file /home/ankhas/.local/lib/python3.10/site-packages/ansible/modules/uri.py
<178.20.1.160> PUT /home/ankhas/.ansible/tmp/ansible-local-72407qnz3utfp/tmpzfjmd8o8 TO /home/ubuntu/.ansible/tmp/ansible-tmp-1673361715.6199737-72504-90630208728260/AnsiballZ_uri.py
<178.20.1.160> SSH: EXEC sftp -b - -C -o ControlMaster=auto -o ControlPersist=60s -o ServerAliveInterval=50 -o 'IdentityFile="/home/ankhas/.ssh/id_ed25519"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="ubuntu"' -o ConnectTimeout=10 -o StrictHostKeyChecking=no -o 'UserKnownHostsFile=~/.ssh/known_hosts' -o 'ControlPath="/home/ankhas/.ansible/cp/c66e8da40c"' '[178.20.1.160]'
<178.20.1.160> (0, b'sftp> put /home/ankhas/.ansible/tmp/ansible-local-72407qnz3utfp/tmpzfjmd8o8 /home/ubuntu/.ansible/tmp/ansible-tmp-1673361715.6199737-72504-90630208728260/AnsiballZ_uri.py\n', b'')
<178.20.1.160> ESTABLISH SSH CONNECTION FOR USER: ubuntu
<178.20.1.160> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o ServerAliveInterval=50 -o 'IdentityFile="/home/ankhas/.ssh/id_ed25519"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="ubuntu"' -o ConnectTimeout=10 -o StrictHostKeyChecking=no -o 'UserKnownHostsFile=~/.ssh/known_hosts' -o 'ControlPath="/home/ankhas/.ansible/cp/c66e8da40c"' 178.20.1.160 '/bin/sh -c '"'"'chmod u+x /home/ubuntu/.ansible/tmp/ansible-tmp-1673361715.6199737-72504-90630208728260/ /home/ubuntu/.ansible/tmp/ansible-tmp-1673361715.6199737-72504-90630208728260/AnsiballZ_uri.py && sleep 0'"'"''
<178.20.1.160> (0, b'', b'')
<178.20.1.160> ESTABLISH SSH CONNECTION FOR USER: ubuntu
<178.20.1.160> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o ServerAliveInterval=50 -o 'IdentityFile="/home/ankhas/.ssh/id_ed25519"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="ubuntu"' -o ConnectTimeout=10 -o StrictHostKeyChecking=no -o 'UserKnownHostsFile=~/.ssh/known_hosts' -o 'ControlPath="/home/ankhas/.ansible/cp/c66e8da40c"' -tt 178.20.1.160 '/bin/sh -c '"'"'/usr/bin/python3 /home/ubuntu/.ansible/tmp/ansible-tmp-1673361715.6199737-72504-90630208728260/AnsiballZ_uri.py && sleep 0'"'"''
<178.20.1.160> (1, b'Traceback (most recent call last):\r\n  File "/home/ubuntu/.ansible/tmp/ansible-tmp-1673361715.6199737-72504-90630208728260/AnsiballZ_uri.py", line 107, in <module>\r\n    _ansiballz_main()\r\n  File "/home/ubuntu/.ansible/tmp/ansible-tmp-1673361715.6199737-72504-90630208728260/AnsiballZ_uri.py", line 99, in _ansiballz_main\r\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\r\n  File "/home/ubuntu/.ansible/tmp/ansible-tmp-1673361715.6199737-72504-90630208728260/AnsiballZ_uri.py", line 47, in invoke_module\r\n    runpy.run_module(mod_name=\'ansible.modules.uri\', init_globals=dict(_module_fqn=\'ansible.modules.uri\', _modlib_path=modlib_path),\r\n  File "/usr/lib/python3.8/runpy.py", line 207, in run_module\r\n    return _run_module_code(code, init_globals, run_name, mod_spec)\r\n  File "/usr/lib/python3.8/runpy.py", line 97, in _run_module_code\r\n    _run_code(code, mod_globals, init_globals,\r\n  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code\r\n    exec(code, run_globals)\r\n  File "/tmp/ansible_ansible.legacy.uri_payload_0o98qj0h/ansible_ansible.legacy.uri_payload.zip/ansible/modules/uri.py", line 788, in <module>\r\n  File "/tmp/ansible_ansible.legacy.uri_payload_0o98qj0h/ansible_ansible.legacy.uri_payload.zip/ansible/modules/uri.py", line 718, in main\r\n  File "/usr/lib/python3.8/http/client.py", line 466, in read\r\n    return self._readall_chunked()\r\n  File "/usr/lib/python3.8/http/client.py", line 576, in _readall_chunked\r\n    value.append(self._safe_read(chunk_left))\r\n  File "/usr/lib/python3.8/http/client.py", line 613, in _safe_read\r\n    data = self.fp.read(amt)\r\n  File "/usr/lib/python3.8/socket.py", line 669, in readinto\r\n    return self._sock.recv_into(b)\r\n  File "/usr/lib/python3.8/ssl.py", line 1241, in recv_into\r\n    return self.read(nbytes, buffer)\r\n  File "/usr/lib/python3.8/ssl.py", line 1099, in read\r\n    return self._sslobj.read(len, buffer)\r\nConnectionResetError: [Errno 104] Connection reset by peer\r\n', b'Shared connection to 178.20.1.160 closed.\r\n')
<178.20.1.160> Failed to connect to the host via ssh: Shared connection to 178.20.1.160 closed.
<178.20.1.160> ESTABLISH SSH CONNECTION FOR USER: ubuntu
<178.20.1.160> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o ServerAliveInterval=50 -o 'IdentityFile="/home/ankhas/.ssh/id_ed25519"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="ubuntu"' -o ConnectTimeout=10 -o StrictHostKeyChecking=no -o 'UserKnownHostsFile=~/.ssh/known_hosts' -o 'ControlPath="/home/ankhas/.ansible/cp/c66e8da40c"' 178.20.1.160 '/bin/sh -c '"'"'rm -f -r /home/ubuntu/.ansible/tmp/ansible-tmp-1673361715.6199737-72504-90630208728260/ > /dev/null 2>&1 && sleep 0'"'"''
<178.20.1.160> (0, b'', b'')
The full traceback is:
Traceback (most recent call last):
  File "/home/ubuntu/.ansible/tmp/ansible-tmp-1673361715.6199737-72504-90630208728260/AnsiballZ_uri.py", line 107, in <module>
    _ansiballz_main()
  File "/home/ubuntu/.ansible/tmp/ansible-tmp-1673361715.6199737-72504-90630208728260/AnsiballZ_uri.py", line 99, in _ansiballz_main
    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
  File "/home/ubuntu/.ansible/tmp/ansible-tmp-1673361715.6199737-72504-90630208728260/AnsiballZ_uri.py", line 47, in invoke_module
    runpy.run_module(mod_name='ansible.modules.uri', init_globals=dict(_module_fqn='ansible.modules.uri', _modlib_path=modlib_path),
  File "/usr/lib/python3.8/runpy.py", line 207, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/usr/lib/python3.8/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/tmp/ansible_ansible.legacy.uri_payload_0o98qj0h/ansible_ansible.legacy.uri_payload.zip/ansible/modules/uri.py", line 788, in <module>
  File "/tmp/ansible_ansible.legacy.uri_payload_0o98qj0h/ansible_ansible.legacy.uri_payload.zip/ansible/modules/uri.py", line 718, in main
  File "/usr/lib/python3.8/http/client.py", line 466, in read
    return self._readall_chunked()
  File "/usr/lib/python3.8/http/client.py", line 576, in _readall_chunked
    value.append(self._safe_read(chunk_left))
  File "/usr/lib/python3.8/http/client.py", line 613, in _safe_read
    data = self.fp.read(amt)
  File "/usr/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
  File "/usr/lib/python3.8/ssl.py", line 1241, in recv_into
    return self.read(nbytes, buffer)
  File "/usr/lib/python3.8/ssl.py", line 1099, in read
    return self._sslobj.read(len, buffer)
ConnectionResetError: [Errno 104] Connection reset by peer
fatal: [ubuntu]: FAILED! => {
    "changed": false,
    "module_stderr": "Shared connection to 178.20.1.160 closed.\r\n",
    "module_stdout": "Traceback (most recent call last):\r\n  File \"/home/ubuntu/.ansible/tmp/ansible-tmp-1673361715.6199737-72504-90630208728260/AnsiballZ_uri.py\", line 107, in <module>\r\n    _ansiballz_main()\r\n  File \"/home/ubuntu/.ansible/tmp/ansible-tmp-1673361715.6199737-72504-90630208728260/AnsiballZ_uri.py\", line 99, in _ansiballz_main\r\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\r\n  File \"/home/ubuntu/.ansible/tmp/ansible-tmp-1673361715.6199737-72504-90630208728260/AnsiballZ_uri.py\", line 47, in invoke_module\r\n    runpy.run_module(mod_name='ansible.modules.uri', init_globals=dict(_module_fqn='ansible.modules.uri', _modlib_path=modlib_path),\r\n  File \"/usr/lib/python3.8/runpy.py\", line 207, in run_module\r\n    return _run_module_code(code, init_globals, run_name, mod_spec)\r\n  File \"/usr/lib/python3.8/runpy.py\", line 97, in _run_module_code\r\n    _run_code(code, mod_globals, init_globals,\r\n  File \"/usr/lib/python3.8/runpy.py\", line 87, in _run_code\r\n    exec(code, run_globals)\r\n  File \"/tmp/ansible_ansible.legacy.uri_payload_0o98qj0h/ansible_ansible.legacy.uri_payload.zip/ansible/modules/uri.py\", line 788, in <module>\r\n  File \"/tmp/ansible_ansible.legacy.uri_payload_0o98qj0h/ansible_ansible.legacy.uri_payload.zip/ansible/modules/uri.py\", line 718, in main\r\n  File \"/usr/lib/python3.8/http/client.py\", line 466, in read\r\n    return self._readall_chunked()\r\n  File \"/usr/lib/python3.8/http/client.py\", line 576, in _readall_chunked\r\n    value.append(self._safe_read(chunk_left))\r\n  File \"/usr/lib/python3.8/http/client.py\", line 613, in _safe_read\r\n    data = self.fp.read(amt)\r\n  File \"/usr/lib/python3.8/socket.py\", line 669, in readinto\r\n    return self._sock.recv_into(b)\r\n  File \"/usr/lib/python3.8/ssl.py\", line 1241, in recv_into\r\n    return self.read(nbytes, buffer)\r\n  File \"/usr/lib/python3.8/ssl.py\", line 1099, in read\r\n    return self._sslobj.read(len, buffer)\r\nConnectionResetError: [Errno 104] Connection reset by peer\r\n",
    "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
    "rc": 1
}

My task is:

- name: Get externalId
  ansible.builtin.uri:
    url: "{{ link }}"
    method: GET
    validate_certs: no
    body_format: json
    headers:
      Authorization: "Bearer {{ token }}"
    return_content: yes
    status_code:
      - 200
  register: external

My ansible.cfg file is :

[defaults]
inventory = ./inventory/test.ini
host_key_checking = no

[ssh_connection]
ssh_args = "-C -o ControlMaster=auto -o ControlPersist=60s -o ServerAliveInterval=50"

My inventory is something like this:

...

[all:vars]
ansible_user="ubuntu"
ansible_become_method="sudo"
ansible_become_password="{{ vault_become_password }}"

ansible_ssh_private_key_file="~/.ssh/id_ed25519"
ansible_ssh_common_args='-o StrictHostKeyChecking=no -o UserKnownHostsFile=~/.ssh/known_hosts'
ansible_python_interpreter='/usr/bin/python3'

The SSH connection seems to drop and the playbook stop suddenly. Moreover, several tasks are defined and works well before this one. Some of them are also using ansible.builtin.uri module.
But when I ran the playbook with debug mode (ie. -vvvv argument`), I noticed these logs in every previous task:

<178.20.1.160> ESTABLISH SSH CONNECTION FOR USER: ubuntu
<178.20.1.160> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o ServerAliveInterval=50 -o StrictHostKeyChecking=no -o 'IdentityFile="/home/ankhas/.ssh/id_ed25519"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="ubuntu"' -o ConnectTimeout=10 -o StrictHostKeyChecking=no -o 'UserKnownHostsFile=~/.ssh/known_hosts' -o 'ControlPath="/home/ankhas/.ansible/cp/c66e8da40c"' 178.20.1.160 '/bin/sh -c '"'"'rm -f -r /home/ubuntu/.ansible/tmp/ansible-tmp-1673363204.2430837-75125-79554162758890/ > /dev/null 2>&1 && sleep 0'"'"''
<178.20.1.160> (0, b'', b"OpenSSH_9.0p1 Ubuntu-1ubuntu7.1, OpenSSL 3.0.5 5 Jul 2022\r\ndebug1: Reading configuration data /home/ankhas/.ssh/config\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files\r\ndebug1: /etc/ssh/ssh_config line 21: Applying options for *\r\ndebug2: resolve_canonicalize: hostname 178.20.1.160 is address\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/home/ankhas/.ssh/known_hosts'\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 75060\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n")

Nevertheless, sometimes the task works with no issue and the playbook can continue.

By the way, I'm also wondering why there is not one but several SSH connection attempts in the above logs.

I tried to change [ssh_connection] configuration in my ansible.cfg by setting:

pipelining to True

timeout to 3000

ssh_args

I'm expecting this playbook to works every time by not having SSH Connection reset by peer error.

Do you have any idea how to solve this issue?


问:
Unable to retrieve a one to one polymorphic relationship
I have a pretty complex Laravel application who creates a one to one polymorphic relationship in a middleware if it does not exists. The fact is that I am unable to retrieve the relationship in the same request (create in middleware, then pass it and retrieve it), I have an error 500 when I try to do this. BUT, if I a make an other request I can retrieve it... When I look into my database, I have the userable_id and type defined, I have no idea where this can probably occur.

My middleware look like this:

```<code>public function handle(Request $request, Closure $next): mixed
{
    if (!Storage::disk('cecurity')->exists(config('coffre_fort.ssl_certificate.name')) ||
        !config('coffre_fort.reverse_proxy.username') ||
        !config('coffre_fort.reverse_proxy.password') ||
        !config('coffre_fort.encryption_key')) {
        return response()->json([
            'error' => 'Cecurity is not configured correctly.',
        ], 500);
    } elseif (!Auth::user()) {
        return response()->json([
            'error' => 'You are not authenticated.',
        ], 401);
    }
    if (!Auth::user()->cecurityAccount) {
        try {
            $userType = Auth::user()::class;
            /** @phpstan-ignore-next-line  */
            if ($userType == "App\Models\Admin") {
                $this->cecurityRepository->createAdminUser(Auth::user());
            } else {
                // It enter in this function for creating relationship
                $this->cecurityRepository->createFullCustomer(Auth::user());
            }
        } catch (Exception $e) {
            return response()->json([
                'error' => $e->getMessage(),
            ], 500);
        }
    } elseif (!$this->cecurityRepository->checkConnection()) {
        $this->cecurityRepository->connect(
            Auth::user()->cecurityAccount->cecurity_user_id,
            openssl_decrypt(
                Auth::user()->cecurityAccount->password,
                'AES-256-CBC',
                config('coffre_fort.encryption_key'),
                0,
                (int) Auth::user()->cecurityAccount->encryption_iv
            ),
            Auth::user()->cecurityAccount->coffre_id
        );
    }
    return $next($request);
}
</code>```

Then it creates the relationship as this (in the createFulCustomer() function):

```<code>$user = new Cecurity;
$user->userable_id = $customer->id_customer;
$user->userable_type = Customer::class;
</code>```

And then pass the middleware to go to listFiles() function in a controller:

```<code>public function listFiles(ListFilesRequest $request): mixed
{
    try {
        return $this->cecurityRepository->listFiles(
            $request->get('nbRowByPage'),
            $request->get('pageIndex'),
        );
    } catch (\Exception $e) {
        return response()->json([
            'message' => $e->getMessage()
        ], 500);
    }
}
</code>```

Just after the middleware has passed, my database is completed (Cecurity table related):

Body:
I have a pretty complex Laravel application who creates a one to one polymorphic relationship in a middleware if it does not exists. The fact is that I am unable to retrieve the relationship in the same request (create in middleware, then pass it and retrieve it), I have an error 500 when I try to do this. BUT, if I a make an other request I can retrieve it... When I look into my database, I have the userable_id and type defined, I have no idea where this can probably occur.

My middleware look like this:

public function handle(Request $request, Closure $next): mixed
{
    if (!Storage::disk('cecurity')->exists(config('coffre_fort.ssl_certificate.name')) ||
        !config('coffre_fort.reverse_proxy.username') ||
        !config('coffre_fort.reverse_proxy.password') ||
        !config('coffre_fort.encryption_key')) {
        return response()->json([
            'error' => 'Cecurity is not configured correctly.',
        ], 500);
    } elseif (!Auth::user()) {
        return response()->json([
            'error' => 'You are not authenticated.',
        ], 401);
    }
    if (!Auth::user()->cecurityAccount) {
        try {
            $userType = Auth::user()::class;
            /** @phpstan-ignore-next-line  */
            if ($userType == "App\Models\Admin") {
                $this->cecurityRepository->createAdminUser(Auth::user());
            } else {
                // It enter in this function for creating relationship
                $this->cecurityRepository->createFullCustomer(Auth::user());
            }
        } catch (Exception $e) {
            return response()->json([
                'error' => $e->getMessage(),
            ], 500);
        }
    } elseif (!$this->cecurityRepository->checkConnection()) {
        $this->cecurityRepository->connect(
            Auth::user()->cecurityAccount->cecurity_user_id,
            openssl_decrypt(
                Auth::user()->cecurityAccount->password,
                'AES-256-CBC',
                config('coffre_fort.encryption_key'),
                0,
                (int) Auth::user()->cecurityAccount->encryption_iv
            ),
            Auth::user()->cecurityAccount->coffre_id
        );
    }
    return $next($request);
}

Then it creates the relationship as this (in the createFulCustomer() function):

$user = new Cecurity;
$user->userable_id = $customer->id_customer;
$user->userable_type = Customer::class;

And then pass the middleware to go to listFiles() function in a controller:

public function listFiles(ListFilesRequest $request): mixed
{
    try {
        return $this->cecurityRepository->listFiles(
            $request->get('nbRowByPage'),
            $request->get('pageIndex'),
        );
    } catch (\Exception $e) {
        return response()->json([
            'message' => $e->getMessage()
        ], 500);
    }
}

Just after the middleware has passed, my database is completed (Cecurity table related):


问:
TeamCity Rest API "Failed to initialize Spring Context" Error
I installed TC on my new pc but ı have some failures. I am getting Failed to "initialize Spring Context" Error after TC installation and TC WebUI is not working properly. I connected db from PostgreSQL and JRE 1.8.0_351 installed.

Here is the screenshot:

Error Message:

```<code>org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'graphQLEndpoint' defined in URL [jar:file:/C:/TeamCity/webapps/ROOT/WEB-INF/plugins/.unpacked/rest-api/server/rest-api.jar!/jetbrains/buildServer/server/graphql/GraphQLEndpoint.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'graphQLProvider': Invocation of init method failed; nested exception is graphql.kickstart.tools.resolver.FieldResolverError: No method or field found as defined in schema <unknown>:282 with any of the following signatures (with or without one of [interface graphql.schema.DataFetchingEnvironment] as the last argument), in priority order:

jetbrains.buildServer.server.graphql.resolver.AgentResolver.id(jetbrains.buildServer.server.graphql.model.Agent)
jetbrains.buildServer.server.graphql.resolver.AgentResolver.getİd(jetbrains.buildServer.server.graphql.model.Agent)
jetbrains.buildServer.server.graphql.resolver.AgentResolver.id
jetbrains.buildServer.server.graphql.model.Agent.id()
jetbrains.buildServer.server.graphql.model.Agent.getİd()
jetbrains.buildServer.server.graphql.model.Agent.id
at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:800)
at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:229)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1372)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1222)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:582)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542)
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:953)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:918)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:583)
at jetbrains.buildServer.plugins.spring.SpringPluginLoader.pluginClassesLoaded(SpringPluginLoader.java:126)
at jdk.internal.reflect.GeneratedMethodAccessor109.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at jetbrains.buildServer.util.EventDispatcher.invokeListeners(EventDispatcher.java:157)
at jetbrains.buildServer.util.EventDispatcher.dispatch(EventDispatcher.java:136)
at jetbrains.buildServer.util.EventDispatcher$2.invoke(EventDispatcher.java:82)
at com.sun.proxy.$Proxy48.pluginClassesLoaded(Unknown Source)
at jetbrains.buildServer.plugins.PluginManagerImpl$5.visitPlugin(PluginManagerImpl.java:441)
at jetbrains.buildServer.plugins.PluginsCollection$1.run(PluginsCollection.java:98)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'graphQLProvider': Invocation of init method failed; nested exception is graphql.kickstart.tools.resolver.FieldResolverError: No method or field found as defined in schema <unknown>:282 with any of the following signatures (with or without one of [interface graphql.schema.DataFetchingEnvironment] as the last argument), in priority order:

jetbrains.buildServer.server.graphql.resolver.AgentResolver.id(jetbrains.buildServer.server.graphql.model.Agent)...
</code>```

Body:
I installed TC on my new pc but ı have some failures. I am getting Failed to "initialize Spring Context" Error after TC installation and TC WebUI is not working properly. I connected db from PostgreSQL and JRE 1.8.0_351 installed.

Here is the screenshot:

Error Message:

org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'graphQLEndpoint' defined in URL [jar:file:/C:/TeamCity/webapps/ROOT/WEB-INF/plugins/.unpacked/rest-api/server/rest-api.jar!/jetbrains/buildServer/server/graphql/GraphQLEndpoint.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'graphQLProvider': Invocation of init method failed; nested exception is graphql.kickstart.tools.resolver.FieldResolverError: No method or field found as defined in schema <unknown>:282 with any of the following signatures (with or without one of [interface graphql.schema.DataFetchingEnvironment] as the last argument), in priority order:

jetbrains.buildServer.server.graphql.resolver.AgentResolver.id(jetbrains.buildServer.server.graphql.model.Agent)
jetbrains.buildServer.server.graphql.resolver.AgentResolver.getİd(jetbrains.buildServer.server.graphql.model.Agent)
jetbrains.buildServer.server.graphql.resolver.AgentResolver.id
jetbrains.buildServer.server.graphql.model.Agent.id()
jetbrains.buildServer.server.graphql.model.Agent.getİd()
jetbrains.buildServer.server.graphql.model.Agent.id
at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:800)
at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:229)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1372)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1222)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:582)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542)
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:953)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:918)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:583)
at jetbrains.buildServer.plugins.spring.SpringPluginLoader.pluginClassesLoaded(SpringPluginLoader.java:126)
at jdk.internal.reflect.GeneratedMethodAccessor109.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at jetbrains.buildServer.util.EventDispatcher.invokeListeners(EventDispatcher.java:157)
at jetbrains.buildServer.util.EventDispatcher.dispatch(EventDispatcher.java:136)
at jetbrains.buildServer.util.EventDispatcher$2.invoke(EventDispatcher.java:82)
at com.sun.proxy.$Proxy48.pluginClassesLoaded(Unknown Source)
at jetbrains.buildServer.plugins.PluginManagerImpl$5.visitPlugin(PluginManagerImpl.java:441)
at jetbrains.buildServer.plugins.PluginsCollection$1.run(PluginsCollection.java:98)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'graphQLProvider': Invocation of init method failed; nested exception is graphql.kickstart.tools.resolver.FieldResolverError: No method or field found as defined in schema <unknown>:282 with any of the following signatures (with or without one of [interface graphql.schema.DataFetchingEnvironment] as the last argument), in priority order:

jetbrains.buildServer.server.graphql.resolver.AgentResolver.id(jetbrains.buildServer.server.graphql.model.Agent)...


问:
How do I get the data from the database randomly for a quiz game
with (Paginator(QuesModel.objects.filter(kategorie__exact=globalCategory).order_by("?"), 1)) I get some data displayed twice, but I want each question to be displayed randomly once. How can I do that? Thanks?

```<code>views.py
globalCategory = request.POST.get("Categorie")
        first= False
        count_of_selected_category = Paginator(QuesModel.objects.filter(kategorie__exact=globalCategory).order_by("?"), 1)
        page = request.GET.get("page")
        x = count_of_selected_category.get_page(page)
        global_counter = 1
        questions = x
        globalQuestions = questions

        context = {
                'questions': globalQuestions,
                'category': request.POST.get("categorie"),

            }
</code>```

Maybe somebody can help me :(

Body:
with (Paginator(QuesModel.objects.filter(kategorie__exact=globalCategory).order_by("?"), 1)) I get some data displayed twice, but I want each question to be displayed randomly once. How can I do that? Thanks?

views.py
globalCategory = request.POST.get("Categorie")
        first= False
        count_of_selected_category = Paginator(QuesModel.objects.filter(kategorie__exact=globalCategory).order_by("?"), 1)
        page = request.GET.get("page")
        x = count_of_selected_category.get_page(page)
        global_counter = 1
        questions = x
        globalQuestions = questions

        context = {
                'questions': globalQuestions,
                'category': request.POST.get("categorie"),

            }

Maybe somebody can help me :(


问:
Getting error while writing the spark dataframe as CSV file
```<code>23/01/10 14:51:56 ERROR FileFormatWriter: Aborting job 18d74180-1f1e-44ea-80dc-caa5a2fe0525.
java.io.IOException: Failed to rename DeprecatedRawLocalFileStatus{path=file:/d/myproject/FCD/Sparck_updated_code/first-class-data-backend/first_class/test/\_temporary/0/task_2023011020
21531054031999327673221_0005_m_000000/part-00000-250cb7ce-e146-4cfd-b9f1-f810af4630f2-c000.csv; isDirectory=false; length=13520; replication=1; blocksize=33554432; modification_time=16
73362315593; access_time=1673362315593; owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false} to file:/d/myproject/FCD/Sparck_updated_code/first-class-data-backend/first_class/test/part-00000-250cb7ce-e146-4cfd-b9f1-f810af4630f2-c000.csv
  at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.mergePaths(FileOutputCommitter.java:477)
  at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.mergePaths(FileOutputCommitter.java:490)
  at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.commitJobInternal(FileOutputCommitter.java:405)
  at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.commitJob(FileOutputCommitter.java:377)
  at org.apache.spark.internal.io.HadoopMapReduceCommitProtocol.commitJob(HadoopMapReduceCommitProtocol.scala:192)
  at org.apache.spark.sql.execution.datasources.FileFormatWriter$.$anonfun$write$25(FileFormatWriter.scala:267)
  at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
  at org.apache.spark.util.Utils$.timeTakenMs(Utils.scala:642)
  at org.apache.spark.sql.execution.datasources.FileFormatWriter$.write(FileFormatWriter.scala:267)
  at org.apache.spark.sql.execution.datasources.InsertIntoHadoopFsRelationCommand.run(InsertIntoHadoopFsRelationCommand.scala:186)
  at org.apache.spark.sql.execution.command.DataWritingCommandExec.sideEffectResult$lzycompute(commands.scala:113)
  at org.apache.spark.sql.execution.command.DataWritingCommandExec.sideEffectResult(commands.scala:111)
  at org.apache.spark.sql.execution.command.DataWritingCommandExec.executeCollect(commands.scala:125)
  at org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$1.$anonfun$applyOrElse$1(QueryExecution.scala:98)
  at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$6(SQLExecution.scala:109)
  at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:169)
  at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$1(SQLExecution.scala:95)
  at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:779)
  at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:64)
  at org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$1.applyOrElse(QueryExecution.scala:98)
  at org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$1.applyOrElse(QueryExecution.scala:94)
  at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformDownWithPruning$1(TreeNode.scala:584)
  at org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(TreeNode.scala:176)
  at org.apache.spark.sql.catalyst.trees.TreeNode.transformDownWithPruning(TreeNode.scala:584)
  at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.org$apache$spark$sql$catalyst$plans$logical$AnalysisHelper$$super$transformDownWithPruning(LogicalPlan.scala:30)
  at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.transformDownWithPruning(AnalysisHelper.scala:267)
  at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.transformDownWithPruning$(AnalysisHelper.scala:263)
  at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.transformDownWithPruning(LogicalPlan.scala:30)
  at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.transformDownWithPruning(LogicalPlan.scala:30)
  at org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:560)
  at org.apache.spark.sql.execution.QueryExecution.eagerlyExecuteCommands(QueryExecution.scala:94)
  at org.apache.spark.sql.execution.QueryExecution.commandExecuted$lzycompute(QueryExecution.scala:81)
  at org.apache.spark.sql.execution.QueryExecution.commandExecuted(QueryExecution.scala:79)
  at org.apache.spark.sql.execution.QueryExecution.assertCommandExecuted(QueryExecution.scala:116)
  at org.apache.spark.sql.DataFrameWriter.runCommand(DataFrameWriter.scala:860)
  at org.apache.spark.sql.DataFrameWriter.saveToV1Source(DataFrameWriter.scala:390)
  at org.apache.spark.sql.DataFrameWriter.saveInternal(DataFrameWriter.scala:363)
  at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:239)
  at org.apache.spark.sql.DataFrameWriter.csv(DataFrameWriter.scala:851)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:498)
  at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
  at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
  at py4j.Gateway.invoke(Gateway.java:282)
  at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
  at py4j.commands.CallCommand.execute(CallCommand.java:79)
  at py4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:182)
  at py4j.ClientServerConnection.run(ClientServerConnection.java:106)
  at java.lang.Thread.run(Thread.java:750)
23/01/10 14:51:56 WARN FileUtil: Failed to delete file or dir \[/d/myproject/FCD/Sparck_updated_code/first-class-data-backend/first_class/test/\_temporary/0/task_202301102021531054031999327673221_0005_m_000000/.part-00000-250cb7ce-e146-4cfd-b9f1-f810af4630f2-c000.csv.crc\]: it still exists.
23/01/10 14:51:56 WARN FileUtil: Failed to delete file or dir \[/d/myproject/FCD/Sparck_updated_code/first-class-data-backend/first_class/test/\_temporary/0/task_202301102021531054031999327673221_0005_m_000000/part-00000-250cb7ce-e146-4cfd-b9f1-f810af4630f2-c000.csv\]: it still exists.
Traceback (most recent call last):
File "/d/myproject/FCD/Sparck_updated_code/first-class-data-backend/first_class/manage.py", line 21, in \<module\>
main()
File "/d/myproject/FCD/Sparck_updated_code/first-class-data-backend/first_class/manage.py", line 17, in main
execute_from_command_line(sys.argv)
File "/home/admin123/.virtualenvs/fcd/lib/python3.10/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
utility.execute()
File "/home/admin123/.virtualenvs/fcd/lib/python3.10/site-packages/django/core/management/__init__.py", line 413, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/home/admin123/.virtualenvs/fcd/lib/python3.10/site-packages/django/core/management/base.py", line 354, in run_from_argv
self.execute(\*args, \*\*cmd_options)
File "/home/admin123/.virtualenvs/fcd/lib/python3.10/site-packages/django/core/management/base.py", line 398, in execute
output = self.handle(\*args, \*\*options)
File "/d/myproject/FCD/Sparck_updated_code/first-class-data-backend/first_class/core/management/commands/prepare_ncoa_sp.py", line 26, in handle
step.start()
File "/d/myproject/FCD/Sparck_updated_code/first-class-data-backend/first_class/core/management/commands/prepare_ncoa_sp.py", line 45, in start
self.prepare_agent_address_updates()
File "/d/myproject/FCD/Sparck_updated_code/first-class-data-backend/first_class/core/management/commands/prepare_ncoa_sp.py", line 141, in prepare_agent_address_updates
self.\_load_and_normalize(file_glob, {
File "/d/myproject/FCD/Sparck_updated_code/first-class-data-backend/first_class/core/management/commands/prepare_ncoa_sp.py", line 127, in \_load_and_normalize
df.write.option("header", True).csv('test')
File "/home/admin123/.virtualenvs/fcd/lib/python3.10/site-packages/pyspark/sql/readwriter.py", line 1240, in csv
self.\_jwrite.csv(path)
File "/home/admin123/.virtualenvs/fcd/lib/python3.10/site-packages/py4j/java_gateway.py", line 1321, in __call__
return_value = get_return_value(
File "/home/admin123/.virtualenvs/fcd/lib/python3.10/site-packages/pyspark/sql/utils.py", line 190, in deco
return f(\*a, \*\*kw)
File "/home/admin123/.virtualenvs/fcd/lib/python3.10/site-packages/py4j/protocol.py", line 326, in get_return_value
raise Py4JJavaError(
py4j.protocol.Py4JJavaError: An error occurred while calling o298.csv.
: org.apache.spark.SparkException: Job aborted.
  at org.apache.spark.sql.errors.QueryExecutionErrors$.jobAbortedError(QueryExecutionErrors.scala:651)
  at org.apache.spark.sql.execution.datasources.FileFormatWriter$.write(FileFormatWriter.scala:278)
  at org.apache.spark.sql.execution.datasources.InsertIntoHadoopFsRelationCommand.run(InsertIntoHadoopFsRelationCommand.scala:186)
  at org.apache.spark.sql.execution.command.DataWritingCommandExec.sideEffectResult$lzycompute(commands.scala:113)
  at org.apache.spark.sql.execution.command.DataWritingCommandExec.sideEffectResult(commands.scala:111)
  at org.apache.spark.sql.execution.command.DataWritingCommandExec.executeCollect(commands.scala:125)
  at org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$1.$anonfun$applyOrElse$1(QueryExecution.scala:98)
  at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$6(SQLExecution.scala:109)
  at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:169)
  at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$1(SQLExecution.scala:95)
  at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:779)
  at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:64)
  at org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$1.applyOrElse(QueryExecution.scala:98)
  at org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$1.applyOrElse(QueryExecution.scala:94)
  at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformDownWithPruning$1(TreeNode.scala:584)
  at org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(TreeNode.scala:176)
  at org.apache.spark.sql.catalyst.trees.TreeNode.transformDownWithPruning(TreeNode.scala:584)
  at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.org$apache$spark$sql$catalyst$plans$logical$AnalysisHelper$$super$transformDownWithPruning(LogicalPlan.scala:30)
  at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.transformDownWithPruning(AnalysisHelper.scala:267)
  at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.transformDownWithPruning$(AnalysisHelper.scala:263)
  at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.transformDownWithPruning(LogicalPlan.scala:30)
  at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.transformDownWithPruning(LogicalPlan.scala:30)
  at org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:560)
  at org.apache.spark.sql.execution.QueryExecution.eagerlyExecuteCommands(QueryExecution.scala:94)
  at org.apache.spark.sql.execution.QueryExecution.commandExecuted$lzycompute(QueryExecution.scala:81)
  at org.apache.spark.sql.execution.QueryExecution.commandExecuted(QueryExecution.scala:79)
  at org.apache.spark.sql.execution.QueryExecution.assertCommandExecuted(QueryExecution.scala:116)
  at org.apache.spark.sql.DataFrameWriter.runCommand(DataFrameWriter.scala:860)
  at org.apache.spark.sql.DataFrameWriter.saveToV1Source(DataFrameWriter.scala:390)
  at org.apache.spark.sql.DataFrameWriter.saveInternal(DataFrameWriter.scala:363)
  at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:239)
  at org.apache.spark.sql.DataFrameWriter.csv(DataFrameWriter.scala:851)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:498)
  at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
  at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
  at py4j.Gateway.invoke(Gateway.java:282)
  at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
  at py4j.commands.CallCommand.execute(CallCommand.java:79)
  at py4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:182)
  at py4j.ClientServerConnection.run(ClientServerConnection.java:106)
  at java.lang.Thread.run(Thread.java:750)
Caused by: java.io.IOException: Failed to rename DeprecatedRawLocalFileStatus{path=file:/d/myproject/FCD/Sparck_updated_code/first-class-data-backend/first_class/test/\_temporary/0/task
  at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.mergePaths(FileOutputCommitter.java:490)
  at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.commitJobInternal(FileOutputCommitter.java:405)
  at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.commitJob(FileOutputCommitter.java:377)
  at org.apache.spark.internal.io.HadoopMapReduceCommitProtocol.commitJob(HadoopMapReduceCommitProtocol.scala:192)
  at org.apache.spark.sql.execution.datasources.FileFormatWriter$.$anonfun$write$25(FileFormatWriter.scala:267)
  at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
  at org.apache.spark.util.Utils$.timeTakenMs(Utils.scala:642)
  at org.apache.spark.sql.execution.datasources.FileFormatWriter$.write(FileFormatWriter.scala:267)
... 42 more
</code>```

As you can see above error occurred while try to write the spark dataframe as csv file

```<code> sparkContext = SparkContext("spark://DESKTOP-1L1BM8L.localdomain:7077", "fcd_spark_session")
        spark_configuration = sparkContext._conf.setAll(
            [("spark.shuffle.service.enabled", "false"), ("spark.dynamicAllocation.enabled", "false"),
             ("spark.executor.memory", "2g"), ("spark.executor.instances", 2)])
        sparkContext.stop()
        self.spark_session = SparkSession.builder.appName("fcd_spark_session").config(
            conf=spark_configuration) \
            .master('spark://DESKTOP-1L1BM8L.localdomain:7077').getOrCreate()

    def _load_and_normalize(self, glob_paths, renames=None, columns=[], processed_columns=[],
                            remove_duplicates=[], ):
        renames = renames or {}
        files = sorted(glob.glob(glob_paths))
        for filepath in files:
            file_name = basename(filepath)
            logger.info(f'adding {basename(filepath)}')
            file_write_path = self.csv_fullpath(self.cleaned_folder, "NCOA_address", file_name)
            print(file_write_path)
            if not os.path.exists(file_write_path):
                df = self.read_csv(filepath)
                df = df[columns].copy()
                df = df[:100]
                df = df.fillna('').astype('str')
                df = df.apply(tuple, axis=1).tolist()
                df = self.spark_session.createDataFrame(df, columns)
                df = self.add_procuredate(df, file_name)
                df = self.uppercase_and_trim_all_columns(df)
                for rename_columns in renames:
                    df = df.withColumnRenamed(rename_columns, renames[rename_columns])
                all_cols_except_procure = [col for col in df.schema.names if col != 'procure_date']
                df = df.dropDuplicates(all_cols_except_procure)
                df = self.get_normalized_address(df)
                df = self.get_normalized_address(df, col_name='orig_normalized_address',
                                                 full_address_col='orig_address', city_col='orig_city',
                                                 state_col='orig_state',
                                                 zip_col='orig_zip')
                df = df.where((df.full_address != '') & (df.normalized_address != ''))
                df = df.select(processed_columns)
                df = df.dropDuplicates(remove_duplicates)
                df.write.option("header", True).format("csv").csv('test')
                gc.collect()
            else:
                logger.info(f'{basename(filepath)} file is already available in cleaned folder')
</code>```

Any suggestions and please comment if you need any additional info regarding the code and config

Body:
23/01/10 14:51:56 ERROR FileFormatWriter: Aborting job 18d74180-1f1e-44ea-80dc-caa5a2fe0525.
java.io.IOException: Failed to rename DeprecatedRawLocalFileStatus{path=file:/d/myproject/FCD/Sparck_updated_code/first-class-data-backend/first_class/test/\_temporary/0/task_2023011020
21531054031999327673221_0005_m_000000/part-00000-250cb7ce-e146-4cfd-b9f1-f810af4630f2-c000.csv; isDirectory=false; length=13520; replication=1; blocksize=33554432; modification_time=16
73362315593; access_time=1673362315593; owner=; group=; permission=rw-rw-rw-; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false} to file:/d/myproject/FCD/Sparck_updated_code/first-class-data-backend/first_class/test/part-00000-250cb7ce-e146-4cfd-b9f1-f810af4630f2-c000.csv
  at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.mergePaths(FileOutputCommitter.java:477)
  at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.mergePaths(FileOutputCommitter.java:490)
  at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.commitJobInternal(FileOutputCommitter.java:405)
  at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.commitJob(FileOutputCommitter.java:377)
  at org.apache.spark.internal.io.HadoopMapReduceCommitProtocol.commitJob(HadoopMapReduceCommitProtocol.scala:192)
  at org.apache.spark.sql.execution.datasources.FileFormatWriter$.$anonfun$write$25(FileFormatWriter.scala:267)
  at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
  at org.apache.spark.util.Utils$.timeTakenMs(Utils.scala:642)
  at org.apache.spark.sql.execution.datasources.FileFormatWriter$.write(FileFormatWriter.scala:267)
  at org.apache.spark.sql.execution.datasources.InsertIntoHadoopFsRelationCommand.run(InsertIntoHadoopFsRelationCommand.scala:186)
  at org.apache.spark.sql.execution.command.DataWritingCommandExec.sideEffectResult$lzycompute(commands.scala:113)
  at org.apache.spark.sql.execution.command.DataWritingCommandExec.sideEffectResult(commands.scala:111)
  at org.apache.spark.sql.execution.command.DataWritingCommandExec.executeCollect(commands.scala:125)
  at org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$1.$anonfun$applyOrElse$1(QueryExecution.scala:98)
  at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$6(SQLExecution.scala:109)
  at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:169)
  at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$1(SQLExecution.scala:95)
  at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:779)
  at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:64)
  at org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$1.applyOrElse(QueryExecution.scala:98)
  at org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$1.applyOrElse(QueryExecution.scala:94)
  at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformDownWithPruning$1(TreeNode.scala:584)
  at org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(TreeNode.scala:176)
  at org.apache.spark.sql.catalyst.trees.TreeNode.transformDownWithPruning(TreeNode.scala:584)
  at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.org$apache$spark$sql$catalyst$plans$logical$AnalysisHelper$$super$transformDownWithPruning(LogicalPlan.scala:30)
  at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.transformDownWithPruning(AnalysisHelper.scala:267)
  at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.transformDownWithPruning$(AnalysisHelper.scala:263)
  at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.transformDownWithPruning(LogicalPlan.scala:30)
  at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.transformDownWithPruning(LogicalPlan.scala:30)
  at org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:560)
  at org.apache.spark.sql.execution.QueryExecution.eagerlyExecuteCommands(QueryExecution.scala:94)
  at org.apache.spark.sql.execution.QueryExecution.commandExecuted$lzycompute(QueryExecution.scala:81)
  at org.apache.spark.sql.execution.QueryExecution.commandExecuted(QueryExecution.scala:79)
  at org.apache.spark.sql.execution.QueryExecution.assertCommandExecuted(QueryExecution.scala:116)
  at org.apache.spark.sql.DataFrameWriter.runCommand(DataFrameWriter.scala:860)
  at org.apache.spark.sql.DataFrameWriter.saveToV1Source(DataFrameWriter.scala:390)
  at org.apache.spark.sql.DataFrameWriter.saveInternal(DataFrameWriter.scala:363)
  at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:239)
  at org.apache.spark.sql.DataFrameWriter.csv(DataFrameWriter.scala:851)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:498)
  at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
  at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
  at py4j.Gateway.invoke(Gateway.java:282)
  at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
  at py4j.commands.CallCommand.execute(CallCommand.java:79)
  at py4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:182)
  at py4j.ClientServerConnection.run(ClientServerConnection.java:106)
  at java.lang.Thread.run(Thread.java:750)
23/01/10 14:51:56 WARN FileUtil: Failed to delete file or dir \[/d/myproject/FCD/Sparck_updated_code/first-class-data-backend/first_class/test/\_temporary/0/task_202301102021531054031999327673221_0005_m_000000/.part-00000-250cb7ce-e146-4cfd-b9f1-f810af4630f2-c000.csv.crc\]: it still exists.
23/01/10 14:51:56 WARN FileUtil: Failed to delete file or dir \[/d/myproject/FCD/Sparck_updated_code/first-class-data-backend/first_class/test/\_temporary/0/task_202301102021531054031999327673221_0005_m_000000/part-00000-250cb7ce-e146-4cfd-b9f1-f810af4630f2-c000.csv\]: it still exists.
Traceback (most recent call last):
File "/d/myproject/FCD/Sparck_updated_code/first-class-data-backend/first_class/manage.py", line 21, in \<module\>
main()
File "/d/myproject/FCD/Sparck_updated_code/first-class-data-backend/first_class/manage.py", line 17, in main
execute_from_command_line(sys.argv)
File "/home/admin123/.virtualenvs/fcd/lib/python3.10/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
utility.execute()
File "/home/admin123/.virtualenvs/fcd/lib/python3.10/site-packages/django/core/management/__init__.py", line 413, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/home/admin123/.virtualenvs/fcd/lib/python3.10/site-packages/django/core/management/base.py", line 354, in run_from_argv
self.execute(\*args, \*\*cmd_options)
File "/home/admin123/.virtualenvs/fcd/lib/python3.10/site-packages/django/core/management/base.py", line 398, in execute
output = self.handle(\*args, \*\*options)
File "/d/myproject/FCD/Sparck_updated_code/first-class-data-backend/first_class/core/management/commands/prepare_ncoa_sp.py", line 26, in handle
step.start()
File "/d/myproject/FCD/Sparck_updated_code/first-class-data-backend/first_class/core/management/commands/prepare_ncoa_sp.py", line 45, in start
self.prepare_agent_address_updates()
File "/d/myproject/FCD/Sparck_updated_code/first-class-data-backend/first_class/core/management/commands/prepare_ncoa_sp.py", line 141, in prepare_agent_address_updates
self.\_load_and_normalize(file_glob, {
File "/d/myproject/FCD/Sparck_updated_code/first-class-data-backend/first_class/core/management/commands/prepare_ncoa_sp.py", line 127, in \_load_and_normalize
df.write.option("header", True).csv('test')
File "/home/admin123/.virtualenvs/fcd/lib/python3.10/site-packages/pyspark/sql/readwriter.py", line 1240, in csv
self.\_jwrite.csv(path)
File "/home/admin123/.virtualenvs/fcd/lib/python3.10/site-packages/py4j/java_gateway.py", line 1321, in __call__
return_value = get_return_value(
File "/home/admin123/.virtualenvs/fcd/lib/python3.10/site-packages/pyspark/sql/utils.py", line 190, in deco
return f(\*a, \*\*kw)
File "/home/admin123/.virtualenvs/fcd/lib/python3.10/site-packages/py4j/protocol.py", line 326, in get_return_value
raise Py4JJavaError(
py4j.protocol.Py4JJavaError: An error occurred while calling o298.csv.
: org.apache.spark.SparkException: Job aborted.
  at org.apache.spark.sql.errors.QueryExecutionErrors$.jobAbortedError(QueryExecutionErrors.scala:651)
  at org.apache.spark.sql.execution.datasources.FileFormatWriter$.write(FileFormatWriter.scala:278)
  at org.apache.spark.sql.execution.datasources.InsertIntoHadoopFsRelationCommand.run(InsertIntoHadoopFsRelationCommand.scala:186)
  at org.apache.spark.sql.execution.command.DataWritingCommandExec.sideEffectResult$lzycompute(commands.scala:113)
  at org.apache.spark.sql.execution.command.DataWritingCommandExec.sideEffectResult(commands.scala:111)
  at org.apache.spark.sql.execution.command.DataWritingCommandExec.executeCollect(commands.scala:125)
  at org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$1.$anonfun$applyOrElse$1(QueryExecution.scala:98)
  at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$6(SQLExecution.scala:109)
  at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:169)
  at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$1(SQLExecution.scala:95)
  at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:779)
  at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:64)
  at org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$1.applyOrElse(QueryExecution.scala:98)
  at org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$1.applyOrElse(QueryExecution.scala:94)
  at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformDownWithPruning$1(TreeNode.scala:584)
  at org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(TreeNode.scala:176)
  at org.apache.spark.sql.catalyst.trees.TreeNode.transformDownWithPruning(TreeNode.scala:584)
  at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.org$apache$spark$sql$catalyst$plans$logical$AnalysisHelper$$super$transformDownWithPruning(LogicalPlan.scala:30)
  at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.transformDownWithPruning(AnalysisHelper.scala:267)
  at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.transformDownWithPruning$(AnalysisHelper.scala:263)
  at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.transformDownWithPruning(LogicalPlan.scala:30)
  at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.transformDownWithPruning(LogicalPlan.scala:30)
  at org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:560)
  at org.apache.spark.sql.execution.QueryExecution.eagerlyExecuteCommands(QueryExecution.scala:94)
  at org.apache.spark.sql.execution.QueryExecution.commandExecuted$lzycompute(QueryExecution.scala:81)
  at org.apache.spark.sql.execution.QueryExecution.commandExecuted(QueryExecution.scala:79)
  at org.apache.spark.sql.execution.QueryExecution.assertCommandExecuted(QueryExecution.scala:116)
  at org.apache.spark.sql.DataFrameWriter.runCommand(DataFrameWriter.scala:860)
  at org.apache.spark.sql.DataFrameWriter.saveToV1Source(DataFrameWriter.scala:390)
  at org.apache.spark.sql.DataFrameWriter.saveInternal(DataFrameWriter.scala:363)
  at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:239)
  at org.apache.spark.sql.DataFrameWriter.csv(DataFrameWriter.scala:851)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:498)
  at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
  at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
  at py4j.Gateway.invoke(Gateway.java:282)
  at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
  at py4j.commands.CallCommand.execute(CallCommand.java:79)
  at py4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:182)
  at py4j.ClientServerConnection.run(ClientServerConnection.java:106)
  at java.lang.Thread.run(Thread.java:750)
Caused by: java.io.IOException: Failed to rename DeprecatedRawLocalFileStatus{path=file:/d/myproject/FCD/Sparck_updated_code/first-class-data-backend/first_class/test/\_temporary/0/task
  at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.mergePaths(FileOutputCommitter.java:490)
  at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.commitJobInternal(FileOutputCommitter.java:405)
  at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.commitJob(FileOutputCommitter.java:377)
  at org.apache.spark.internal.io.HadoopMapReduceCommitProtocol.commitJob(HadoopMapReduceCommitProtocol.scala:192)
  at org.apache.spark.sql.execution.datasources.FileFormatWriter$.$anonfun$write$25(FileFormatWriter.scala:267)
  at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
  at org.apache.spark.util.Utils$.timeTakenMs(Utils.scala:642)
  at org.apache.spark.sql.execution.datasources.FileFormatWriter$.write(FileFormatWriter.scala:267)
... 42 more

As you can see above error occurred while try to write the spark dataframe as csv file

 sparkContext = SparkContext("spark://DESKTOP-1L1BM8L.localdomain:7077", "fcd_spark_session")
        spark_configuration = sparkContext._conf.setAll(
            [("spark.shuffle.service.enabled", "false"), ("spark.dynamicAllocation.enabled", "false"),
             ("spark.executor.memory", "2g"), ("spark.executor.instances", 2)])
        sparkContext.stop()
        self.spark_session = SparkSession.builder.appName("fcd_spark_session").config(
            conf=spark_configuration) \
            .master('spark://DESKTOP-1L1BM8L.localdomain:7077').getOrCreate()

    def _load_and_normalize(self, glob_paths, renames=None, columns=[], processed_columns=[],
                            remove_duplicates=[], ):
        renames = renames or {}
        files = sorted(glob.glob(glob_paths))
        for filepath in files:
            file_name = basename(filepath)
            logger.info(f'adding {basename(filepath)}')
            file_write_path = self.csv_fullpath(self.cleaned_folder, "NCOA_address", file_name)
            print(file_write_path)
            if not os.path.exists(file_write_path):
                df = self.read_csv(filepath)
                df = df[columns].copy()
                df = df[:100]
                df = df.fillna('').astype('str')
                df = df.apply(tuple, axis=1).tolist()
                df = self.spark_session.createDataFrame(df, columns)
                df = self.add_procuredate(df, file_name)
                df = self.uppercase_and_trim_all_columns(df)
                for rename_columns in renames:
                    df = df.withColumnRenamed(rename_columns, renames[rename_columns])
                all_cols_except_procure = [col for col in df.schema.names if col != 'procure_date']
                df = df.dropDuplicates(all_cols_except_procure)
                df = self.get_normalized_address(df)
                df = self.get_normalized_address(df, col_name='orig_normalized_address',
                                                 full_address_col='orig_address', city_col='orig_city',
                                                 state_col='orig_state',
                                                 zip_col='orig_zip')
                df = df.where((df.full_address != '') & (df.normalized_address != ''))
                df = df.select(processed_columns)
                df = df.dropDuplicates(remove_duplicates)
                df.write.option("header", True).format("csv").csv('test')
                gc.collect()
            else:
                logger.info(f'{basename(filepath)} file is already available in cleaned folder')

Any suggestions and please comment if you need any additional info regarding the code and config


问:
Webclient OnStatus call stubbing
I need to mock the call to onStatus. The probelem is that I am getting  the below error.

org.mockito.exceptions.misusing.PotentialStubbingProblem:
Strict stubbing argument mismatch. Please check:

```<code>when(responseSpec.onStatus(HttpStatus::isError,(clientResponse) -> {
  if (clientResponse.statusCode() == HttpStatus.resolve(402)) {
    return Mono.error(new Exception("402"));
  }
  if (clientResponse.statusCode() == HttpStatus.resolve(500)) {
    return Mono.error(new Exception("500"));
  }
  if (clientResponse.statusCode() == HttpStatus.resolve(512)) {
    return Mono.error(new Exception("512")));
  }
  return Mono.error(new Exception("Error while processing request"));
}))
        .thenReturn(responseSpec);
</code>```

}

Title: Webclient OnStatus call stubbing

Body:
I need to mock the call to onStatus. The probelem is that I am getting  the below error.

org.mockito.exceptions.misusing.PotentialStubbingProblem:
Strict stubbing argument mismatch. Please check:

when(responseSpec.onStatus(HttpStatus::isError,(clientResponse) -> {
  if (clientResponse.statusCode() == HttpStatus.resolve(402)) {
    return Mono.error(new Exception("402"));
  }
  if (clientResponse.statusCode() == HttpStatus.resolve(500)) {
    return Mono.error(new Exception("500"));
  }
  if (clientResponse.statusCode() == HttpStatus.resolve(512)) {
    return Mono.error(new Exception("512")));
  }
  return Mono.error(new Exception("Error while processing request"));
}))
        .thenReturn(responseSpec);

}


问:
Nesting sequelize.query results in a array of objects for INNER JOIN results
I'm trying to nest the results of a query with sequelize, I'm passing the raw query string in the sequelize.query function and organizing the results with the dotted notation with the option nest set to true. It is working and returning the results as intended, but it is not nesting the results in an array format, leaving the "root" object duplicated.

```<code>const result = await db.sequelize.query(s.toString(),
  {
    nest: true,
    type: db.sequelize.QueryTypes.SELECT,
    raw: true,
  })
return result
</code>```

This is the query generated by sequelize:

SELECT [UserCertifications].id AS [UserCertifications.id],[UserCertifications].campaign_id AS [UserCertifications.campaign_id],[UserCertifications].user_id AS [UserCertifications.user_id],[UserCertifications].certification AS [UserCertifications.certification],[Users].name AS [UserCertifications.Items.Users.name],[Items].title AS [UserCertifications.Items.title] FROM [UserCertifications] INNER JOIN [Items] ON (UserCertifications.item_id = Items.id) INNER JOIN [Users_Items] ON (Users_Items.item_id = Items.id) INNER JOIN [Users] ON (Users_Items.user_id = [Users].id) WHERE (campaign_id = 78)

It returns like this:

```<code>[
    {
        "UserCertifications": {
            "id": 324,
            "campaign_id": 78,
            "user_id": 1,
            "certification": null,
            "Items": {
                "Users": {
                    "name": "Admin"
                },
                "title": "Item 1"
            }
        }
    },
    {
        "UserCertifications": {
            "id": 324,
            "campaign_id": 78,
            "user_id": 1,
            "certification": null,
            "Items": {
                "Users": {
                    "name": "Services"
                },
                "title": "Item 1"
            }
        }
    },
    {
        "UserCertifications": {
            "id": 324,
            "campaign_id": 78,
            "user_id": 1,
            "certification": null,
            "Items": {
                "Users": {
                    "name": "Lucas"
                },
                "title": "Item 1"
            }
        }
    },
    {
        "UserCertifications": {
            "id": 324,
            "campaign_id": 78,
            "user_id": 1,
            "certification": null,
            "Items": {
                "Users": {
                    "name": "Wally"
                },
                "title": "Item 1"
            }
        }
    },
    {
        "UserCertifications": {
            "id": 325,
            "campaign_id": 78,
            "user_id": 1,
            "certification": null,
            "Items": {
                "Users": {
                    "name": "Admin"
                },
                "title": "Item 1"
            }
        }
    },
        ...this is not the entire result, it is too big, but you can get the idea of what im alking about through this example.
</code>```

The problem is, the result is correct, with "Items" and "Users" nested as intended, but i would like the result to be nested in a way that all the UserCertifications with "id: 324" are a single object with a single "Item" ("Item 1") and the users as an Array of users. Is there a way to nest the results like this using the dotted notation with Sequelize?
Aditional context: I'm passing the alias of the table names "Items" and "Users" like this to get that result, "UserCertifications.Items.title" and "UserCertifications.Items.Users.name", respectively
Disclaimer: It needs to be a raw query because I am building a query string from scratch using a "query builder", so i can't use the "Include" notation from sequelize.

Body:
I'm trying to nest the results of a query with sequelize, I'm passing the raw query string in the sequelize.query function and organizing the results with the dotted notation with the option nest set to true. It is working and returning the results as intended, but it is not nesting the results in an array format, leaving the "root" object duplicated.

const result = await db.sequelize.query(s.toString(),
  {
    nest: true,
    type: db.sequelize.QueryTypes.SELECT,
    raw: true,
  })
return result

This is the query generated by sequelize:

SELECT [UserCertifications].id AS [UserCertifications.id],[UserCertifications].campaign_id AS [UserCertifications.campaign_id],[UserCertifications].user_id AS [UserCertifications.user_id],[UserCertifications].certification AS [UserCertifications.certification],[Users].name AS [UserCertifications.Items.Users.name],[Items].title AS [UserCertifications.Items.title] FROM [UserCertifications] INNER JOIN [Items] ON (UserCertifications.item_id = Items.id) INNER JOIN [Users_Items] ON (Users_Items.item_id = Items.id) INNER JOIN [Users] ON (Users_Items.user_id = [Users].id) WHERE (campaign_id = 78)

It returns like this:

[
    {
        "UserCertifications": {
            "id": 324,
            "campaign_id": 78,
            "user_id": 1,
            "certification": null,
            "Items": {
                "Users": {
                    "name": "Admin"
                },
                "title": "Item 1"
            }
        }
    },
    {
        "UserCertifications": {
            "id": 324,
            "campaign_id": 78,
            "user_id": 1,
            "certification": null,
            "Items": {
                "Users": {
                    "name": "Services"
                },
                "title": "Item 1"
            }
        }
    },
    {
        "UserCertifications": {
            "id": 324,
            "campaign_id": 78,
            "user_id": 1,
            "certification": null,
            "Items": {
                "Users": {
                    "name": "Lucas"
                },
                "title": "Item 1"
            }
        }
    },
    {
        "UserCertifications": {
            "id": 324,
            "campaign_id": 78,
            "user_id": 1,
            "certification": null,
            "Items": {
                "Users": {
                    "name": "Wally"
                },
                "title": "Item 1"
            }
        }
    },
    {
        "UserCertifications": {
            "id": 325,
            "campaign_id": 78,
            "user_id": 1,
            "certification": null,
            "Items": {
                "Users": {
                    "name": "Admin"
                },
                "title": "Item 1"
            }
        }
    },
        ...this is not the entire result, it is too big, but you can get the idea of what im alking about through this example.

The problem is, the result is correct, with "Items" and "Users" nested as intended, but i would like the result to be nested in a way that all the UserCertifications with "id: 324" are a single object with a single "Item" ("Item 1") and the users as an Array of users. Is there a way to nest the results like this using the dotted notation with Sequelize?
Aditional context: I'm passing the alias of the table names "Items" and "Users" like this to get that result, "UserCertifications.Items.title" and "UserCertifications.Items.Users.name", respectively
Disclaimer: It needs to be a raw query because I am building a query string from scratch using a "query builder", so i can't use the "Include" notation from sequelize.


问:
merge dataframes and duplicate values according to names in first column - pandas
This is dataframe 1:
data = [['aguaagua', 30,20], ['expansion1', 40,50 ]] df1 = pd.DataFrame(data, columns=['Unit', '22-01' , '22-02'])

And dataframe 2:
data2 = [['agua-tg1'], ['agua-tg2' ], ['nepi12030'], ['nepi22030']] df2 = pd.DataFrame(data2, columns=['Unit'])

The result should be:
data3 = [['agua-tg1', 30, 20], ['agua-tg2', 30,20 ], ['nepi12030', 40, 50], ['nepi22030', 40, 50]] df3 = pd.DataFrame(data3, columns=['Unit', '22-01' , '22-02'])

I was using two lists as index, then merging
bb = df2.merge(df1, on='Unit', how='outer')

But I get:
pyresult

How can I solve it?