问:
How to write and read binary (not ASCII encoded) to Arduino with pyserial
I need to make a Raspberry Pi communicate with an Arduino. I will only need values between 0 and 180, and this is within the range of a single byte so I want to only send the value in binary, and not send it with ASCII encoding for faster transfer (i.e.: if I want to write a "123" to the Arduino, I want 0x7B to be sent, and not the ASCII codes for 1, then 2, then 3 (0x31,0x32, 0x33).

In my testing I have been trying to write a Python program that will take an integer within that range and then send it over serial in binary.

Here is a test I was trying. I want to write a binary number to the Arduino, and then have the Arduino print out the value it received.

Python code:

```<code>USB_PORT = "/dev/ttyUSB0"  # Arduino Uno WiFi Rev2

import serial

try:
   usb = serial.Serial(USB_PORT, 9600, timeout=2)
except:
   print("ERROR - Could not open USB serial port.  Please check your port name and permissions.")
   print("Exiting program.")
   exit()

while True:
    command = int(input("Enter command: "))
    usb.write(command)

    value = (usb.readline())
    print("You sent:", value)
</code>```

And here is the Arduino code

byte command;
void setup()
{
    Serial.begin(9600);

}

void loop()
{
    if (Serial.available() > 0)
    {
        command = Serial.read();
        Serial.print(command);
    }

}
```
<p>All this gives me is this:</p>
```Enter command: 1
You sent: b'0'
Enter command: 4
You sent: b'0000'
```

答:
usb.write(command) expects command to be of type bytes not int.

It seems the write method internally calls bytes(), since it sends the number of zero bytes that you called it with.

Calling bytes() with an integer, creates a zero filled byte array, with the specified number of zeroes.

If you want to send a single byte, you need to call it with a list with a single element.

You should do:

command = bytes([int(input("Enter command: "))])
```

Body:
I need to make a Raspberry Pi communicate with an Arduino. I will only need values between 0 and 180, and this is within the range of a single byte so I want to only send the value in binary, and not send it with ASCII encoding for faster transfer (i.e.: if I want to write a "123" to the Arduino, I want 0x7B to be sent, and not the ASCII codes for 1, then 2, then 3 (0x31,0x32, 0x33).

In my testing I have been trying to write a Python program that will take an integer within that range and then send it over serial in binary.

Here is a test I was trying. I want to write a binary number to the Arduino, and then have the Arduino print out the value it received.

Python code:

USB_PORT = "/dev/ttyUSB0"  # Arduino Uno WiFi Rev2

import serial

try:
   usb = serial.Serial(USB_PORT, 9600, timeout=2)
except:
   print("ERROR - Could not open USB serial port.  Please check your port name and permissions.")
   print("Exiting program.")
   exit()

while True:
    command = int(input("Enter command: "))
    usb.write(command)

    value = (usb.readline())
    print("You sent:", value)

And here is the Arduino code

byte command;
void setup()
{
    Serial.begin(9600);

}

void loop()
{
    if (Serial.available() > 0)
    {
        command = Serial.read();
        Serial.print(command);
    }

}

All this gives me is this:

Enter command: 1
You sent: b'0'
Enter command: 4
You sent: b'0000'


来源: stackexchange

问:
Divider color change React Material Ui
I'm working with the divider component of the material ui framework and am stuck with the color changing aspect. With most other components from this framework I've been able to change the color by applying the useStyles() method as such:

```<code>const useStyles = makeStyles(theme => ({
    textPadding: {
      paddingTop: 10,
      paddingBottom: 10,
      color:'white',
    },
}));
</code>```

But I'm not able to change the color of the dividers using the same approach:

```<code>const useStyles = makeStyles(theme => ({
dividerColor: {
  backgroundColor: 'white',
},
}));
</code>```

I of-course then apply it to the component:

```<code><Divider classname={classes.dividerColor}></Divider>
</code>```

I looked up the docs for it but can't figure out what I've done wrong. Could someone give me a helping hand?

答:
To change Divider line color in MUI v5 you need to adjust your approach depending on whether the element has children or not.

For an empty Divider:

<Divider sx={{ bgcolor: "secondary.light" }} />
```
<p>For a <code>Divider</code> with content:</p>
<pre class="lang-js prettyprint-override"><code><Divider
  sx={{
    "&::before, &::after": {
      borderColor: "secondary.light",
    },
  }}
>
  <Typography>Some Text</Typography>
</Divider>
</code>```

Comparing to the other answers for v5, note that you do not need to nest the SX props under &.MuiDivider-root and you can use the theme properties with the SX shorthand strings (e.g., secondary.light instead of (theme) => theme.palette.secondary.light.

ContentLicense: CC BY-SA 4.0

Title: Divider color change React Material Ui

Body:
I'm working with the divider component of the material ui framework and am stuck with the color changing aspect. With most other components from this framework I've been able to change the color by applying the useStyles() method as such:

const useStyles = makeStyles(theme => ({
    textPadding: {
      paddingTop: 10,
      paddingBottom: 10,
      color:'white',
    },
}));

But I'm not able to change the color of the dividers using the same approach:

const useStyles = makeStyles(theme => ({
dividerColor: {
  backgroundColor: 'white',
},
}));

I of-course then apply it to the component:

<Divider classname={classes.dividerColor}></Divider>

I looked up the docs for it but can't figure out what I've done wrong. Could someone give me a helping hand?

Body:
To change Divider line color in MUI v5 you need to adjust your approach depending on whether the element has children or not.

For an empty Divider:

<Divider sx={{ bgcolor: "secondary.light" }} />

For a Divider with content:

<Divider
  sx={{
    "&::before, &::after": {
      borderColor: "secondary.light",
    },
  }}
>
  <Typography>Some Text</Typography>
</Divider>

Comparing to the other answers for v5, note that you do not need to nest the SX props under &.MuiDivider-root and you can use the theme properties with the SX shorthand strings (e.g., secondary.light instead of (theme) => theme.palette.secondary.light.


问:
Few issues Inserting and filtering XML data to SQL
I'm having a few issues when inserting data from a XML file to a SQL Server database.

How can I get the file name dynamically? This file is put on that path with a different number every time but always starts with Idoc and it's a .xml file. I'm having troubles setting a variable inside OPENROWSET.

At the moment I'm getting data from just one path (ZPROD) but I want to be able to insert data from EDI_DC40/DOCNUM instead of ZPROD/PLN_ORDER, I would need another CROSS APPLY but I can't get it to work.

I want to ignore the .000 from QTY, I tried setting it to int but it didn't work, 'can't convert to int'.

Is there a way to ignore most of LINENO and just get the last two characters? It's set to varchar(2) but obviously is taking the first two characters, not the last ones.

Below is my code, I hope is all within the rules and understandable:

Thanks.

```<code><?xml version="1.0" encoding="UTF-8" ?>
<ZMPROD01>
   <IDOC BEGIN="1">
      <EDI_DC40 SEGMENT="1">
         <TABNAM>EDI_DC40</TABNAM>
         <DOCNUM>0000003899888135</DOCNUM>
         <CREDAT>20220201</CREDAT>
         <CRETIM>152041</CRETIM>
      </EDI_DC40>
      <ZPROD SEGMENT="1">
         <WERKS>8285</WERKS>
         <LGNUM>0</LGNUM>
         <AUFNR>000915229446</AUFNR>
         <LINENO>RM01PL01</LINENO>
         <CHARG>0006186588</CHARG>
         <START1>20220202</START1>
         <START2>211609</START2>
         <QTY>4166.000</QTY>
         <END1>20220202</END1>
         <END2>240000</END2>
         <MAKTX>579 FUS5 75ML ULTRA SENST GEL</MAKTX>
         <PLN_ORDER>6963701111</PLN_ORDER>
         <Z1PRODI SEGMENT="1">
            <POSNR>000010</POSNR>
            <MATNR>000000000098920665</MATNR>
         </Z1PRODI>
         <Z1PRODI SEGMENT="1">
            <POSNR>000040</POSNR>
            <HRKFT>V010</HRKFT>
         </Z1PRODI>
         <Z1PRODI SEGMENT="1">
            <POSNR>000050</POSNR>
            <MATNR>000000000099396964</MATNR>
         </Z1PRODI>
      </ZPROD>
   </IDOC>
</ZMPROD01>
</code>```
```<code>CREATE TABLE XMLTESTTABLE
(
    PONo int,
    ASP int,
    LOTNo varchar(11),
    EntryDate date,
    StartDate date,
    EndDate date,
    GAS int,
    PlannedQty varchar(15),
    LineNum varchar(2),
    SAPDesc varchar(200),
    StartTime date,
    EndTime date
);
</code>```
```<code>INSERT INTO XMLTESTTABLE(PONo, ASP, LOTNo, EntryDate, StartDate, EndDate, GAS, PlannedQty, LineNum, SAPDesc, StartTime, EndTime)
SELECT
   MY_XML.ZPROD.query('AUFNR').value('.', 'VARCHAR(9)'),
   MY_XML.ZPROD.query('CHARG').value('.', 'VARCHAR(8)'),
   MY_XML.ZPROD.query('PLN_ORDER').value('.', 'VARCHAR(10)'),
   MY_XML.ZPROD.query('START1').value('.', 'date'),
   MY_XML.ZPROD.query('START1').value('.', 'date'),
   MY_XML.ZPROD.query('END1').value('.', 'date'),
   MY_XML.ZPROD.query('CHARG').value('.', 'VARCHAR(8)'),
   MY_XML.ZPROD.query('QTY').value('.', 'VARCHAR(9)'),
   MY_XML.ZPROD.query('LINENO').value('.', 'VARCHAR(2)'),
   MY_XML.ZPROD.query('MAKTX').value('.', 'VARCHAR(200)'),
   MY_XML.ZPROD.query('START2').value('.', 'time'),
   MY_XML.ZPROD.query('END2').value('.', 'time')
FROM (SELECT CAST(MY_XML AS xml)
      FROM OPENROWSET(BULK 'C:\Users\PC_user\Documents\Idoc3899888135.xml', SINGLE_BLOB) AS T(MY_XML)) AS T(MY_XML)
      CROSS APPLY MY_XML.nodes('ZMPROD01/ZPROD') AS MY_XML (ZPROD);
</code>```

答:
Please try the following solution.

No need to use the .query() method.

##3,4 got resolved.

I took a liberty to handle invalid 240000 time as 23:59:59.

Last two columns data type is time(0).

SQL

```<code>USE tempdb;
GO

DROP TABLE IF EXISTS XMLTESTTABLE;

CREATE TABLE XMLTESTTABLE (
    PONo INT,
    ASP int,
    LOTNo varchar(11),
    EntryDate date,
    StartDate date,
    EndDate date,
    GAS int,
    PlannedQty varchar(15),
    LineNum varchar(2),
    SAPDesc varchar(200),
    StartTime time(0),
    EndTime time(0)
);

INSERT INTO XMLTESTTABLE(PONo, ASP, LOTNo, EntryDate, StartDate, EndDate, GAS, PlannedQty, LineNum, SAPDesc, StartTime, EndTime)
SELECT ZPROD.value('(AUFNR/text())[1]', 'VARCHAR(9)')
    , ZPROD.value('(CHARG/text())[1]', 'VARCHAR(8)')
    , ZPROD.value('(PLN_ORDER/text())[1]', 'VARCHAR(10)')
    , ZPROD.value('(START1/text())[1]', 'date')
    , ZPROD.value('(START1/text())[1]', 'date')
    , ZPROD.value('(END1/text())[1]', 'date')
    , ZPROD.value('(CHARG/text())[1]', 'VARCHAR(8)')
    , ZPROD.value('(QTY/text())[1]', 'DECIMAL(10,0)') AS [qty]
    , RIGHT(ZPROD.value('(LINENO/text())[1]', 'VARCHAR(10)'), 2) AS [lineno]
    , ZPROD.value('(MAKTX/text())[1]', 'VARCHAR(200)') AS MAKTX
    , TRY_CAST(STUFF(STUFF(ZPROD.value('(START2/text())[1]', 'CHAR(6)'),3,0,':'),6,0,':') AS TIME)
    , TRY_CAST(IIF(STUFF(STUFF(ZPROD.value('(END2/text())[1]', 'CHAR(6)'),3,0,':'),6,0,':')='24:00:00'
        ,'23:59:59',null) AS TIME)
FROM (SELECT TRY_CAST(MY_XML AS xml)
      FROM OPENROWSET(BULK 'e:\Temp\Idoc3899888135.xml', SINGLE_BLOB) AS T(MY_XML)) AS T(MY_XML)
      CROSS APPLY MY_XML.nodes('/ZMPROD01/IDOC/ZPROD') AS MY_XML(ZPROD);

-- test
SELECT * FROM dbo.XMLTESTTABLE;
</code>```

Body:
I'm having a few issues when inserting data from a XML file to a SQL Server database.

How can I get the file name dynamically? This file is put on that path with a different number every time but always starts with Idoc and it's a .xml file. I'm having troubles setting a variable inside OPENROWSET.

At the moment I'm getting data from just one path (ZPROD) but I want to be able to insert data from EDI_DC40/DOCNUM instead of ZPROD/PLN_ORDER, I would need another CROSS APPLY but I can't get it to work.

I want to ignore the .000 from QTY, I tried setting it to int but it didn't work, 'can't convert to int'.

Is there a way to ignore most of LINENO and just get the last two characters? It's set to varchar(2) but obviously is taking the first two characters, not the last ones.

Below is my code, I hope is all within the rules and understandable:

Thanks.

<?xml version="1.0" encoding="UTF-8" ?>
<ZMPROD01>
   <IDOC BEGIN="1">
      <EDI_DC40 SEGMENT="1">
         <TABNAM>EDI_DC40</TABNAM>
         <DOCNUM>0000003899888135</DOCNUM>
         <CREDAT>20220201</CREDAT>
         <CRETIM>152041</CRETIM>
      </EDI_DC40>
      <ZPROD SEGMENT="1">
         <WERKS>8285</WERKS>
         <LGNUM>0</LGNUM>
         <AUFNR>000915229446</AUFNR>
         <LINENO>RM01PL01</LINENO>
         <CHARG>0006186588</CHARG>
         <START1>20220202</START1>
         <START2>211609</START2>
         <QTY>4166.000</QTY>
         <END1>20220202</END1>
         <END2>240000</END2>
         <MAKTX>579 FUS5 75ML ULTRA SENST GEL</MAKTX>
         <PLN_ORDER>6963701111</PLN_ORDER>
         <Z1PRODI SEGMENT="1">
            <POSNR>000010</POSNR>
            <MATNR>000000000098920665</MATNR>
         </Z1PRODI>
         <Z1PRODI SEGMENT="1">
            <POSNR>000040</POSNR>
            <HRKFT>V010</HRKFT>
         </Z1PRODI>
         <Z1PRODI SEGMENT="1">
            <POSNR>000050</POSNR>
            <MATNR>000000000099396964</MATNR>
         </Z1PRODI>
      </ZPROD>
   </IDOC>
</ZMPROD01>

CREATE TABLE XMLTESTTABLE
(
    PONo int,
    ASP int,
    LOTNo varchar(11),
    EntryDate date,
    StartDate date,
    EndDate date,
    GAS int,
    PlannedQty varchar(15),
    LineNum varchar(2),
    SAPDesc varchar(200),
    StartTime date,
    EndTime date
);

INSERT INTO XMLTESTTABLE(PONo, ASP, LOTNo, EntryDate, StartDate, EndDate, GAS, PlannedQty, LineNum, SAPDesc, StartTime, EndTime)
SELECT
   MY_XML.ZPROD.query('AUFNR').value('.', 'VARCHAR(9)'),
   MY_XML.ZPROD.query('CHARG').value('.', 'VARCHAR(8)'),
   MY_XML.ZPROD.query('PLN_ORDER').value('.', 'VARCHAR(10)'),
   MY_XML.ZPROD.query('START1').value('.', 'date'),
   MY_XML.ZPROD.query('START1').value('.', 'date'),
   MY_XML.ZPROD.query('END1').value('.', 'date'),
   MY_XML.ZPROD.query('CHARG').value('.', 'VARCHAR(8)'),
   MY_XML.ZPROD.query('QTY').value('.', 'VARCHAR(9)'),
   MY_XML.ZPROD.query('LINENO').value('.', 'VARCHAR(2)'),
   MY_XML.ZPROD.query('MAKTX').value('.', 'VARCHAR(200)'),
   MY_XML.ZPROD.query('START2').value('.', 'time'),
   MY_XML.ZPROD.query('END2').value('.', 'time')
FROM (SELECT CAST(MY_XML AS xml)
      FROM OPENROWSET(BULK 'C:\Users\PC_user\Documents\Idoc3899888135.xml', SINGLE_BLOB) AS T(MY_XML)) AS T(MY_XML)
      CROSS APPLY MY_XML.nodes('ZMPROD01/ZPROD') AS MY_XML (ZPROD);

Body:
Please try the following solution.

No need to use the .query() method.

##3,4 got resolved.

I took a liberty to handle invalid 240000 time as 23:59:59.

Last two columns data type is time(0).

SQL

USE tempdb;
GO

DROP TABLE IF EXISTS XMLTESTTABLE;

CREATE TABLE XMLTESTTABLE (
    PONo INT,
    ASP int,
    LOTNo varchar(11),
    EntryDate date,
    StartDate date,
    EndDate date,
    GAS int,
    PlannedQty varchar(15),
    LineNum varchar(2),
    SAPDesc varchar(200),
    StartTime time(0),
    EndTime time(0)
);

INSERT INTO XMLTESTTABLE(PONo, ASP, LOTNo, EntryDate, StartDate, EndDate, GAS, PlannedQty, LineNum, SAPDesc, StartTime, EndTime)
SELECT ZPROD.value('(AUFNR/text())[1]', 'VARCHAR(9)')
    , ZPROD.value('(CHARG/text())[1]', 'VARCHAR(8)')
    , ZPROD.value('(PLN_ORDER/text())[1]', 'VARCHAR(10)')
    , ZPROD.value('(START1/text())[1]', 'date')
    , ZPROD.value('(START1/text())[1]', 'date')
    , ZPROD.value('(END1/text())[1]', 'date')
    , ZPROD.value('(CHARG/text())[1]', 'VARCHAR(8)')
    , ZPROD.value('(QTY/text())[1]', 'DECIMAL(10,0)') AS [qty]
    , RIGHT(ZPROD.value('(LINENO/text())[1]', 'VARCHAR(10)'), 2) AS [lineno]
    , ZPROD.value('(MAKTX/text())[1]', 'VARCHAR(200)') AS MAKTX
    , TRY_CAST(STUFF(STUFF(ZPROD.value('(START2/text())[1]', 'CHAR(6)'),3,0,':'),6,0,':') AS TIME)
    , TRY_CAST(IIF(STUFF(STUFF(ZPROD.value('(END2/text())[1]', 'CHAR(6)'),3,0,':'),6,0,':')='24:00:00'
        ,'23:59:59',null) AS TIME)
FROM (SELECT TRY_CAST(MY_XML AS xml)
      FROM OPENROWSET(BULK 'e:\Temp\Idoc3899888135.xml', SINGLE_BLOB) AS T(MY_XML)) AS T(MY_XML)
      CROSS APPLY MY_XML.nodes('/ZMPROD01/IDOC/ZPROD') AS MY_XML(ZPROD);

-- test
SELECT * FROM dbo.XMLTESTTABLE;


问:
mongoose.connect(), first argument should be String, received undefined
I am trying to set the test database for the testing purpose, but its not working.

I am trying to connect to MongoDB using mongoose, but finding problem in the connection error shows:

```<code>throw new MongooseError('The `uri` parameter to `openUri()` must be a ' +
    ^
MongooseError: The `uri` parameter to `openUri()` must be a string, got "undefined". Make sure the first parameter to `mongoose.connect()` or `mongoose.createConnection()`is a string.
    at new MongooseError (/media/abhigyan/ABHI/programming/node js/Practice/On my Own/Todo/node_modules/mongoose/lib/error/mongooseError.js:11:11)
    at NativeConnection.Connection.openUri (/media/abhigyan/ABHI/programming/node js/Practice/On my Own/Todo/node_modules/mongoose/lib/connection.js:424:11)
    at Mongoose.connect (/media/abhigyan/ABHI/programming/node js/Practice/On my Own/Todo/node_modules/mongoose/lib/index.js:230:15)
    at Object.<anonymous> (/media/abhigyan/ABHI/programming/node js/Practice/On my Own/Todo/server/db/mongoose.js:5:10)
    at Module._compile (module.js:635:30)
    at Object.Module._extensions..js (module.js:646:10)
    at Module.load (module.js:554:32)
    at tryModuleLoad (module.js:497:12)
    at Function.Module._load (module.js:489:3)
    at Module.require (module.js:579:17)
    at require (internal/module.js:11:18)
    at Object.<anonymous> (/media/abhigyan/ABHI/programming/node js/Practice/On my Own/Todo/server/models/Todo.js:1:82)
    at Module._compile (module.js:635:30)
    at Object.Module._extensions..js (module.js:646:10)
    at Module.load (module.js:554:32)
    at tryModuleLoad (module.js:497:12)
    at Function.Module._load (module.js:489:3)
    at Module.require (module.js:579:17)
    at require (internal/module.js:11:18)
    at Object.<anonymous> (/media/abhigyan/ABHI/programming/node js/Practice/On my Own/Todo/server/tests/server.test.js:4:16)
    at Module._compile (module.js:635:30)
    at Object.Module._extensions..js (module.js:646:10)
    at Module.load (module.js:554:32)
    at tryModuleLoad (module.js:497:12)
    at Function.Module._load (module.js:489:3)
    at Module.require (module.js:579:17)
    at require (internal/module.js:11:18)
    at /media/abhigyan/ABHI/programming/node js/Practice/On my Own/Todo/node_modules/mocha/lib/mocha.js:250:27
    at Array.forEach (<anonymous>)
    at Mocha.loadFiles (/media/abhigyan/ABHI/programming/node js/Practice/On my Own/Todo/node_modules/mocha/lib/mocha.js:247:14)
    at Mocha.run (/media/abhigyan/ABHI/programming/node js/Practice/On my Own/Todo/node_modules/mocha/lib/mocha.js:576:10)
    at Object.<anonymous> (/media/abhigyan/ABHI/programming/node js/Practice/On my Own/Todo/node_modules/mocha/bin/_mocha:637:18)
    at Module._compile (module.js:635:30)
    at Object.Module._extensions..js (module.js:646:10)
    at Module.load (module.js:554:32)
    at tryModuleLoad (module.js:497:12)
    at Function.Module._load (module.js:489:3)
    at Function.Module.runMain (module.js:676:10)
    at startup (bootstrap_node.js:187:16)
    at bootstrap_node.js:608:3
error Command failed with exit code 1.
</code>```

I am passing a valid String, but Its not working!

```<code>const mongoose = require('mongoose');

mongoose.Promise = global.Promise;
mongoose.connect(process.env.MONGODB_URI, err => {
    if(err)
        console.log(err);
    }
);

module.exports = {
    mongoose
};
</code>```

Here is the Script to run mocha:

```<code>export NODE_ENV='test' && mocha server/**/*.test.js
</code>```

Here is the configuration code:

```<code>const config = require('./config.json');

const env = process.env.NODE_ENV.toString() || 'development';

if(env === 'test' || env === 'development') {
    const envConfig = config[env];
    Object.keys(envConfig).forEach(key => {
        process.env[key] = envConfig[key];
    });
};

console.log(env);
</code>```

Here is the config.json file:

```<code>{
    "test": {
        "PORT": 3000,
        "MONGODB_URI": "mongodb://localhost:27017/TodoTest"
    },
    "development": {
        "PORT": 3000,
        "MONGODB_URI": "mongodb://localhost:27017/Todo"
    }
}
</code>```

Thanks for the help!

答:
```<code>Also, make sure to call dotenv at the top of the code.
The following code gives you the above error as you mentioned above
// Connecting to DataBase
        connectDB();
        const dotenv = require('dotenv').config();
The Correct Way
        const dotenv = require('dotenv').config();
        // Connecting to DataBase
        connectDB();
</code>```

Body:
I am trying to set the test database for the testing purpose, but its not working.

I am trying to connect to MongoDB using mongoose, but finding problem in the connection error shows:

throw new MongooseError('The `uri` parameter to `openUri()` must be a ' +
    ^
MongooseError: The `uri` parameter to `openUri()` must be a string, got "undefined". Make sure the first parameter to `mongoose.connect()` or `mongoose.createConnection()`is a string.
    at new MongooseError (/media/abhigyan/ABHI/programming/node js/Practice/On my Own/Todo/node_modules/mongoose/lib/error/mongooseError.js:11:11)
    at NativeConnection.Connection.openUri (/media/abhigyan/ABHI/programming/node js/Practice/On my Own/Todo/node_modules/mongoose/lib/connection.js:424:11)
    at Mongoose.connect (/media/abhigyan/ABHI/programming/node js/Practice/On my Own/Todo/node_modules/mongoose/lib/index.js:230:15)
    at Object.<anonymous> (/media/abhigyan/ABHI/programming/node js/Practice/On my Own/Todo/server/db/mongoose.js:5:10)
    at Module._compile (module.js:635:30)
    at Object.Module._extensions..js (module.js:646:10)
    at Module.load (module.js:554:32)
    at tryModuleLoad (module.js:497:12)
    at Function.Module._load (module.js:489:3)
    at Module.require (module.js:579:17)
    at require (internal/module.js:11:18)
    at Object.<anonymous> (/media/abhigyan/ABHI/programming/node js/Practice/On my Own/Todo/server/models/Todo.js:1:82)
    at Module._compile (module.js:635:30)
    at Object.Module._extensions..js (module.js:646:10)
    at Module.load (module.js:554:32)
    at tryModuleLoad (module.js:497:12)
    at Function.Module._load (module.js:489:3)
    at Module.require (module.js:579:17)
    at require (internal/module.js:11:18)
    at Object.<anonymous> (/media/abhigyan/ABHI/programming/node js/Practice/On my Own/Todo/server/tests/server.test.js:4:16)
    at Module._compile (module.js:635:30)
    at Object.Module._extensions..js (module.js:646:10)
    at Module.load (module.js:554:32)
    at tryModuleLoad (module.js:497:12)
    at Function.Module._load (module.js:489:3)
    at Module.require (module.js:579:17)
    at require (internal/module.js:11:18)
    at /media/abhigyan/ABHI/programming/node js/Practice/On my Own/Todo/node_modules/mocha/lib/mocha.js:250:27
    at Array.forEach (<anonymous>)
    at Mocha.loadFiles (/media/abhigyan/ABHI/programming/node js/Practice/On my Own/Todo/node_modules/mocha/lib/mocha.js:247:14)
    at Mocha.run (/media/abhigyan/ABHI/programming/node js/Practice/On my Own/Todo/node_modules/mocha/lib/mocha.js:576:10)
    at Object.<anonymous> (/media/abhigyan/ABHI/programming/node js/Practice/On my Own/Todo/node_modules/mocha/bin/_mocha:637:18)
    at Module._compile (module.js:635:30)
    at Object.Module._extensions..js (module.js:646:10)
    at Module.load (module.js:554:32)
    at tryModuleLoad (module.js:497:12)
    at Function.Module._load (module.js:489:3)
    at Function.Module.runMain (module.js:676:10)
    at startup (bootstrap_node.js:187:16)
    at bootstrap_node.js:608:3
error Command failed with exit code 1.

I am passing a valid String, but Its not working!

const mongoose = require('mongoose');

mongoose.Promise = global.Promise;
mongoose.connect(process.env.MONGODB_URI, err => {
    if(err)
        console.log(err);
    }
);

module.exports = {
    mongoose
};

Here is the Script to run mocha:

export NODE_ENV='test' && mocha server/**/*.test.js

Here is the configuration code:

const config = require('./config.json');

const env = process.env.NODE_ENV.toString() || 'development';

if(env === 'test' || env === 'development') {
    const envConfig = config[env];
    Object.keys(envConfig).forEach(key => {
        process.env[key] = envConfig[key];
    });
};

console.log(env);

Here is the config.json file:

{
    "test": {
        "PORT": 3000,
        "MONGODB_URI": "mongodb://localhost:27017/TodoTest"
    },
    "development": {
        "PORT": 3000,
        "MONGODB_URI": "mongodb://localhost:27017/Todo"
    }
}

Thanks for the help!


问:
Tree Visualisation with Java
I am looking for a libary to generate Graphs or Trees like Organizations Charts.
This library should be able to generate plain Images from this graphs.

Does anyone knows a good, hopefully open source, library for Graph Visualization?

答:
Check these libraries :

JUNG

jgraph

jgrapht

ContentLicense: CC BY-SA 3.0

Title: Tree Visualisation with Java


问:
How to get the (relative) place of values in a dataframe when sorted using Python?
How can I create a Pandas DataFrame that shows the relative position of each value, when those values are sorted from low to high for each column?
So in this case, how can you transform 'df' into 'dfOut'?</p>
```<code>import pandas as pd
import numpy as np

#create DataFrame
df = pd.DataFrame({'A': [12, 18, 9, 21, 24, 15],
                   'B': [18, 22, 19, 14, 14, 11],
                   'C': [5, 7, 7, 9, 12, 9]})

# How to assign a value to the order in the column, when sorted from low to high?

dfOut = pd.DataFrame({'A': [2, 4, 1, 5, 6, 3],
                      'B': [3, 5, 4, 2, 2, 1],
                      'C': [1, 2, 2, 3, 4, 3]})
</code>```

答:
If you need to map the same values to the same output, try using the rank method of a DataFrame. Like this:

```<code>>> dfOut = df.rank(method="dense").astype(int)  # Type transformation added to match your output
>> dfOut
   A  B  C
0  2  3  1
1  4  5  2
2  1  4  2
3  5  2  3
4  6  2  4
5  3  1  3
</code>```

The rank method computes the rank for each column following a specific criteria. According to the Pandas documentation, the "dense" method ensures that "rank always increases by 1 between groups", and that might match your use case.

Original answer: In case that repeated numbers are not required to map to the same out value, np.argsort could be applied on each column to retrieve the position of each value that would sort the column. Combine this with the apply method of a DataFrame to apply the function on each column and you have this:

```<code>>> dfOut = df.apply(lambda column: np.argsort(column.values)))
>> dfOut
   A  B  C
0  2  5  0
1  0  3  1
2  5  4  2
3  1  0  3
4  3  2  5
5  4  1  4
</code>```

Body:
How can I create a Pandas DataFrame that shows the relative position of each value, when those values are sorted from low to high for each column?
So in this case, how can you transform 'df' into 'dfOut'?

import pandas as pd
import numpy as np

#create DataFrame
df = pd.DataFrame({'A': [12, 18, 9, 21, 24, 15],
                   'B': [18, 22, 19, 14, 14, 11],
                   'C': [5, 7, 7, 9, 12, 9]})

# How to assign a value to the order in the column, when sorted from low to high?

dfOut = pd.DataFrame({'A': [2, 4, 1, 5, 6, 3],
                      'B': [3, 5, 4, 2, 2, 1],
                      'C': [1, 2, 2, 3, 4, 3]})

Body:
If you need to map the same values to the same output, try using the rank method of a DataFrame. Like this:

>> dfOut = df.rank(method="dense").astype(int)  # Type transformation added to match your output
>> dfOut
   A  B  C
0  2  3  1
1  4  5  2
2  1  4  2
3  5  2  3
4  6  2  4
5  3  1  3

The rank method computes the rank for each column following a specific criteria. According to the Pandas documentation, the "dense" method ensures that "rank always increases by 1 between groups", and that might match your use case.

Original answer: In case that repeated numbers are not required to map to the same out value, np.argsort could be applied on each column to retrieve the position of each value that would sort the column. Combine this with the apply method of a DataFrame to apply the function on each column and you have this:

>> dfOut = df.apply(lambda column: np.argsort(column.values)))
>> dfOut
   A  B  C
0  2  5  0
1  0  3  1
2  5  4  2
3  1  0  3
4  3  2  5
5  4  1  4


问:
Using css pre-wrap vs br?
I'm wondering if there's any benefits to using
 instead of css white-space:pre-wrap.

Most sites I've come across use
 for line breaks on comment sections and other user written sections, instead of pre-wrap. Left wondering what benefits this has, as pre-wrap seems to be easier to implement, allows users to put their own line spacing, and so forth; which to me, would seem like a nicer user experience.

Only downside to this seems to be that very old browsers (IE 7-), which horrors seem to have been long forgotten as usage and support is now so low, don't fully support white-space.

答:
I have a case where I list a few name/value pairs, separated with line breaks.
Seems I have these options:

a restyled pre (to show each name/value pair on a new line)

make sure the name/value pairs are not html and use

use white-space:pre-wrap

The least work in this situation is the css-option.

The reason why there are so many solutions with
 in the wild, I guess, is because there was a time when that did not exist, and it's easy to stick to a once invented wheel.

Title: Using css pre-wrap vs br?


问:
Applying a custom function on a particular column of multiple dataframes in R
I have multiple dataframes (df1, df2, df3, df4, df5), and each dataframe has the same two columns: date, which has dates (e.g. 2020-11-12) as characters and price column as numeric. For example, df1 looks like this:
df1

Date
Price

2020-11-12
29.75

2020-11-13
29.95

2020-11-14
30.72

2020-11-15
32.83

2020-11-16
33.14

I am trying to use lapply with a custom function that converts the character "date" column to the date class. However, lapply function doesn't give me a reformatted date column. My simple codes are as follows:

```<code>df.list <- list(df1, df2, df3, df4, df5)  # create a list of dataframes

# create a custom function to change the class of date column
date_con <- function(x) {
             x$date <- as.Date(x$date, format="%Y-%m-%d")}
</code>```

When I use

```<code>lapply(df.list, date_con)
</code>```

the date columns still remain as character. For instance, when I check class(df$date), it still shows as "character" but not date. On the other hand, if I go manually for each dataframe to make this conversion, it works but I don't want to go over 100s of dataframes manually. That is,

```<code>df1$date <- as.Date(df1$date, format="%Y-%m-%d")
</code>```

works but it is  obviously not efficient and I am sure there is a way to achieve this. So, how can I use lapply or some other approach to efficiently convert character date column into a date class column for large number of dataframes?

答:
The issue here is that R passes argyments by value and not by reference/pointers. Thus the original objects are not modified. To modify the original objects, you need to invoke the use of names and listing to the environment.

Name the elements in your list:

```<code>df.list <- list(df1 = df1, df2 = df2, df3 = df3, df4 = df4, df5 = df5)

# function
date_con <- function(x) {
    x$date <- as.Date(x$date, format="%Y-%m-%d")
    x
}
</code>```

Now run

```<code>list2env(lapply(df.list, date_con),.GlobalEnv)
</code>```

You can now check the class of the date column in your original dataframe:

```<code>class(df1$date)
[1] "Date"
</code>```

Body:
I have multiple dataframes (df1, df2, df3, df4, df5), and each dataframe has the same two columns: date, which has dates (e.g. 2020-11-12) as characters and price column as numeric. For example, df1 looks like this:
df1

Date
Price

2020-11-12
29.75

2020-11-13
29.95

2020-11-14
30.72

2020-11-15
32.83

2020-11-16
33.14

I am trying to use lapply with a custom function that converts the character "date" column to the date class. However, lapply function doesn't give me a reformatted date column. My simple codes are as follows:

df.list <- list(df1, df2, df3, df4, df5)  # create a list of dataframes

# create a custom function to change the class of date column
date_con <- function(x) {
             x$date <- as.Date(x$date, format="%Y-%m-%d")}

When I use

lapply(df.list, date_con)

the date columns still remain as character. For instance, when I check class(df$date), it still shows as "character" but not date. On the other hand, if I go manually for each dataframe to make this conversion, it works but I don't want to go over 100s of dataframes manually. That is,

df1$date <- as.Date(df1$date, format="%Y-%m-%d")

works but it is  obviously not efficient and I am sure there is a way to achieve this. So, how can I use lapply or some other approach to efficiently convert character date column into a date class column for large number of dataframes?

Body:
The issue here is that R passes argyments by value and not by reference/pointers. Thus the original objects are not modified. To modify the original objects, you need to invoke the use of names and listing to the environment.

Name the elements in your list:

df.list <- list(df1 = df1, df2 = df2, df3 = df3, df4 = df4, df5 = df5)

# function
date_con <- function(x) {
    x$date <- as.Date(x$date, format="%Y-%m-%d")
    x
}

Now run

list2env(lapply(df.list, date_con),.GlobalEnv)

You can now check the class of the date column in your original dataframe:

class(df1$date)
[1] "Date"


问:
How to change maven's Remote Repository URL in the NetBeans IDE (from http to https)?
When trying to run a NetBeans project, I get the following error message:

Failed to execute goal
  org.apache.maven.plugins:maven-surefire-plugin:2.10:test
  (default-test) on project MyNetBeansProject: Execution default-test of goal
  org.apache.maven.plugins:maven-surefire-plugin:2.10:test failed:
  Plugin org.apache.maven.plugins:maven-surefire-plugin:2.10 or one of
  its dependencies could not be resolved: Failed to collect dependencies
  for org.apache.maven.plugins:maven-surefire-plugin:jar:2.10 (): Failed
  to read artifact descriptor for
  org.apache.maven.surefire:surefire-booter:jar:2.10: Could not transfer
  artifact org.apache.maven.surefire:surefire-booter:pom:2.10 from/to
  central (http://repo.maven.apache.org/maven2): Failed to transfer
  file:
  http://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-booter/2.10/surefire-booter-2.10.pom.
  Return code is: 501 , ReasonPhrase:HTTPS Required. -> [Help 1]

The following part of the error message is the most important one:

Failed to transfer file:
  http://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-booter/2.10/surefire-booter-2.10.pom.
  Return code is: 501 , ReasonPhrase:HTTPS Required.

Services -> Maven Repositories -> Central Repository -> right mouse click on "Central Repository" gives the following information:

As one can see, the Remote Repository URL is "http://repo.maven.apache.org/maven2/". I think it should instead be "https://repo.maven.apache.org/maven2/".
However, the problem is that I can't seem to change the Remote Repository URL.

Does anybody know how to change maven's Remote Repository URL in the NetBeans IDE?

UPDATE:

Under NetBeans -> Preferences one can see that the maven version used by my NetBeans IDE is Version 3.0.5:

答:
Go maven>conf and update settings.xml</p>
```<code><profile>
    <id>maven-https</id>
    <activation>
        <activeByDefault>true</activeByDefault>
    </activation>

    <repositories>
        <repository>
            <id>central</id>
            <url>https://repo1.maven.org/maven2</url>
            <snapshots>
                <enabled>false</enabled>
            </snapshots>
        </repository>

    </repositories>
    <pluginRepositories>
        <pluginRepository>
            <id>central</id>
            <url>https://repo1.maven.org/maven2</url>
            <snapshots>
                <enabled>false</enabled>
            </snapshots>
        </pluginRepository>
    </pluginRepositories>
</profile>
</code>```

Body:
Go maven>conf and update settings.xml

<profile>
    <id>maven-https</id>
    <activation>
        <activeByDefault>true</activeByDefault>
    </activation>

    <repositories>
        <repository>
            <id>central</id>
            <url>https://repo1.maven.org/maven2</url>
            <snapshots>
                <enabled>false</enabled>
            </snapshots>
        </repository>

    </repositories>
    <pluginRepositories>
        <pluginRepository>
            <id>central</id>
            <url>https://repo1.maven.org/maven2</url>
            <snapshots>
                <enabled>false</enabled>
            </snapshots>
        </pluginRepository>
    </pluginRepositories>
</profile>


问:
How to fix "listen EADDRINUSE: address already in use :::3000" error in Integration test?
I am trying to test my simple endpoint using jest and superTest in my TypeScript project but I am getting listen EADDRINUSE: address already in use :::3000 error.

I have a app.ts for express work. This is the one I am importing in my test file.

I have a server.ts file which starts the server. I am NOT using this in my test.

I am reading some similar questions on stackoverflow where people are suggesting killing the process. My issue is only with my Integration Test. It works fine if I test with Postman.

Question(s):

Should I stop the server in afterEach? I am not 100% sure if this a valid approach.

Here is the code:

app.ts

```<code>import 'reflect-metadata';
import * as express from 'express';
import cors from 'cors';
import helmet from 'helmet';
import app from './server';
import routes from './routes/routes';
import errorHandler from './utils/errorHandler';
import LoggerService from './services/logger.service';

async function bootstrap() {
  app.disable('x-powered-by');
  app.use(cors());
  app.use(helmet());
  app.use(express.json());
  app.use(express.urlencoded({ extended: true }));
  routes(app);
  /**
 * Configure error handler
 */
  errorHandler(app);
}

// Need for testing purpose
export default app;

bootstrap()
  .then(() => {
    new LoggerService().logger.info('Server is up');
  })
  .catch((error) => {
    new LoggerService().logger.error(`Unknown error. ${error.message}`);
  });
</code>```

server.ts

```<code>import * as dotenv from 'dotenv';
import express from 'express';
import LoggerService from './services/logger.service';

dotenv.config();

const server = express();

/**
 * App Variables
 */
if (!process.env.PORT) {
  process.exit(1);
}

/**
 * Setup listener port
 */
const PORT: string | number = process.env.PORT || 3000;
server.listen(PORT, () => {
  new LoggerService().logger.info(`Listening on port ${PORT}`);
});

export default server;
</code>```

route.ts

```<code>import { Application } from 'express';
import container from '../ioc.container';
import HealthController from '../controllers/health.controller';

export default function RouteManager(application: Application) {
  const healthController = container.get<HealthController>(HealthController);

  application.get('/healthStatus', healthController.healthCheck.bind(healthController));
}
</code>```

some.spec.ts

```<code>import { mockHealthCheck } from '../mockData/health.mockData';
import app from '../app';
import request from 'supertest';

describe('GET /healthStatus API Tests', () => {

  it('Receives 200 on successful call', async () => {
    const result = await request(app).get('/healthStatus');
    expect(result.statusCode).toEqual(200);
    expect(result.body.description).toEqual(mockHealthCheck.description);
    expect(result.body.status).toEqual(mockHealthCheck.status);
  });

  it('Receives 404 - Not Found on unsuccessful call', async () => {
    const result = await request(app).get('/invalidEndPoint');
    expect(result.statusCode).toEqual(401);
  });

});

</code>```

答:
I can fix this issue if I make following changes in my app.ts and server.ts files. Remaininig files stays same (route.ts and some.spec.ts).

app.ts

```<code>/**
 * Required External Modules
 */
import 'reflect-metadata';
import * as dotenv from 'dotenv';
import express from 'express';
import cors from 'cors';
import helmet from 'helmet';
import routes from './routes/routes';

dotenv.config();

const app = express();

/**
 *  App Configuration
 */
app.use(helmet());
app.use(cors());
app.use(express.json());
app.disable('x-powered-by');
routes(app);

export default app;
</code>```

server.ts

```<code>import app from './app';
import LoggerService from './services/logger.service';

/**
 * App Variables
 */
if (!process.env.PORT) {
  process.exit(1);
}

const PORT: number = parseInt(process.env.PORT as string, 10);
app.listen(PORT, () => {
  new LoggerService().logger.info(`Listening on port ${PORT}`);
});
</code>```

Body:
I am trying to test my simple endpoint using jest and superTest in my TypeScript project but I am getting listen EADDRINUSE: address already in use :::3000 error.

I have a app.ts for express work. This is the one I am importing in my test file.

I have a server.ts file which starts the server. I am NOT using this in my test.

I am reading some similar questions on stackoverflow where people are suggesting killing the process. My issue is only with my Integration Test. It works fine if I test with Postman.

Question(s):

Should I stop the server in afterEach? I am not 100% sure if this a valid approach.

Here is the code:

app.ts

import 'reflect-metadata';
import * as express from 'express';
import cors from 'cors';
import helmet from 'helmet';
import app from './server';
import routes from './routes/routes';
import errorHandler from './utils/errorHandler';
import LoggerService from './services/logger.service';

async function bootstrap() {
  app.disable('x-powered-by');
  app.use(cors());
  app.use(helmet());
  app.use(express.json());
  app.use(express.urlencoded({ extended: true }));
  routes(app);
  /**
 * Configure error handler
 */
  errorHandler(app);
}

// Need for testing purpose
export default app;

bootstrap()
  .then(() => {
    new LoggerService().logger.info('Server is up');
  })
  .catch((error) => {
    new LoggerService().logger.error(`Unknown error. ${error.message}`);
  });

server.ts

import * as dotenv from 'dotenv';
import express from 'express';
import LoggerService from './services/logger.service';

dotenv.config();

const server = express();

/**
 * App Variables
 */
if (!process.env.PORT) {
  process.exit(1);
}

/**
 * Setup listener port
 */
const PORT: string | number = process.env.PORT || 3000;
server.listen(PORT, () => {
  new LoggerService().logger.info(`Listening on port ${PORT}`);
});

export default server;

route.ts

import { Application } from 'express';
import container from '../ioc.container';
import HealthController from '../controllers/health.controller';

export default function RouteManager(application: Application) {
  const healthController = container.get<HealthController>(HealthController);

  application.get('/healthStatus', healthController.healthCheck.bind(healthController));
}

some.spec.ts

import { mockHealthCheck } from '../mockData/health.mockData';
import app from '../app';
import request from 'supertest';

describe('GET /healthStatus API Tests', () => {

  it('Receives 200 on successful call', async () => {
    const result = await request(app).get('/healthStatus');
    expect(result.statusCode).toEqual(200);
    expect(result.body.description).toEqual(mockHealthCheck.description);
    expect(result.body.status).toEqual(mockHealthCheck.status);
  });

  it('Receives 404 - Not Found on unsuccessful call', async () => {
    const result = await request(app).get('/invalidEndPoint');
    expect(result.statusCode).toEqual(401);
  });

});

Body:
I can fix this issue if I make following changes in my app.ts and server.ts files. Remaininig files stays same (route.ts and some.spec.ts).

app.ts

/**
 * Required External Modules
 */
import 'reflect-metadata';
import * as dotenv from 'dotenv';
import express from 'express';
import cors from 'cors';
import helmet from 'helmet';
import routes from './routes/routes';

dotenv.config();

const app = express();

/**
 *  App Configuration
 */
app.use(helmet());
app.use(cors());
app.use(express.json());
app.disable('x-powered-by');
routes(app);

export default app;

server.ts

import app from './app';
import LoggerService from './services/logger.service';

/**
 * App Variables
 */
if (!process.env.PORT) {
  process.exit(1);
}

const PORT: number = parseInt(process.env.PORT as string, 10);
app.listen(PORT, () => {
  new LoggerService().logger.info(`Listening on port ${PORT}`);
});


问:
Copy Data From On-Premise SQL Server To Azure SQL - Azure Private Network
Requirement: I wanted to copy data from a specific table/view residing on a on-premise SQL Server to Azure SQL DB.

Infrastructure: As depicted in below picture. Essentially, the Azure network is directly connected with corporate network over Express Route. Thus it's a pure private network connection; as good as the corporate network itself.

Issue/Question: I know there are multiple approaches present to get this operation done and I am not restricted to use ADF copy Data tool only. BUT, for all of these I see some cavets or extra steps needed to be done as below:

ADF Copy Data Tool: Needs a SH-IR and a small MSI package needs to be installed on on-premise machine which hosts the SQL server for registration purpose.

Logic Apps: Needs a Virtual Gateway (OR) ASE

App Service: If the operation is wrapped in a C# application and I choose to deploy to a Azure Web Apps. Then in-order to connect to on-premise SQL Server we need to setup hybrid connection manager and as in #1 we need to install something in on-premise machine.

For my case, none of these extra steps can be done. essentially, the on-premise SQL Server comes under a different BU and thus I don't have any permission there; except they have given grant to a table/view. Thus, none of these extra shitty steps can be done.

Moreover, as mentioned above; since it's connected over express route as direct connection, As can be seen in above picture, both the on-premise and azure SQL are essentially inside the same corporate network. THUS, I should be able to access them directly without configuring any of these extra steps as mentioned above.

Please confirm on these and provide a suggestion.

Thank You.

答:
There are 2 solutions which could work for your scenario but even for them to work ,you would need access to on prem SQL server machine access to some extent atleast for one time config and Azure SQL db should be accessible via SSMS installed on on-prem machine.

Using linked server

You can create a linked server ( process explained  here https://www.sqlshack.com/create-linked-server-azure-sql-database/ ) on on-prem server and create a agent server job to insert data to azure SQL db table.

Via Python Script

This would need Python installation on on-prem machine. Once installed you can write script to transfer data between on-prem SQL server and Azure SQL db. You can schedule this script again by using an agent server job.


问:
Cannot convert python script to exe using pyinstaller due to package not found error
I am trying to convert my script that uses transformers to an exe file. Its a small single file script that performs token classification:

```<code>from transformers import AutoTokenizer, AutoModelForTokenClassification
from transformers import pipeline

# download once to save locally
# tokenizer = AutoTokenizer.from_pretrained("dslim/bert-base-NER-uncased")
# model = AutoModelForTokenClassification.from_pretrained("dslim/bert-base-NER-uncased")

# save model locally
# tokenizer.save_pretrained("./model")
# model.save_pretrained("./model")

# now just load from local file
tokenizer = AutoTokenizer.from_pretrained('./model')
model = AutoModelForTokenClassification.from_pretrained('./model')

nlp = pipeline("token-classification", model=model, tokenizer=tokenizer, aggregation_strategy="simple")

example = """00:00:02 Speaker 1: hi john, it's nice to see you again. how was your weekend? do anything special? 00:00:06 Speaker 2: yep, all good thanks. i was with my sister in derby. We saw, you know, that james bond film. what's it called? then got a couple of drinks at the pitcher and piano, back in nottingham. """
ner_results = nlp(example)
print(ner_results)

for i in range(0, len(ner_results)):
  start = ner_results[i]['start']
  end = ner_results[i]['end']
  example = example.replace(ner_results[i]['word'], ner_results[i]['entity_group'])
print(example)
</code>```

The online models are downloaded only once and then saved locally so that they can be packaged with pyinstaller. I am using the below line to build the exe file (which from reading other similar questions on SO, adds all the required missing libraries that pyinstaller misses)

```<code>pyinstaller --windowed --add-data ./model/config.json;./model/ --add-data ./model/pytorch_model.bin;./model/ --add-data ./model/special_tokens_map.json;./model/ --add-data ./model/tokenizer.json;./model/ --add-data ./model/tokenizer_config.json;./model/ --add-data ./model/vocab.txt;./model/ --collect-data tensorflow --collect-data torch --copy-metadata torch --copy-metadata tqdm --copy-metadata regex --copy-metadata sacremoses --copy-metadata requests --copy-metadata packaging --copy-metadata filelock --copy-metadata numpy --copy-metadata tokenizers --copy-metadata importlib_metadata --hidden-import=“sklearn.utils._cython_blas” --hidden-import=“sklearn.neighbors.typedefs” --hidden-import=“sklearn.neighbors.quad_tree” --hidden-import=“sklearn.tree” --hidden-import=“sklearn.tree._utils” deidentify.py
</code>```

This generates the following .spec file

```<code># -*- mode: python ; coding: utf-8 -*-
from PyInstaller.utils.hooks import collect_data_files
from PyInstaller.utils.hooks import copy_metadata

datas = [('./model/config.json', './model/'), ('./model/pytorch_model.bin', './model/'), ('./model/special_tokens_map.json', './model/'), ('./model/tokenizer.json', './model/'), ('./model/tokenizer_config.json', './model/'), ('./model/vocab.txt', './model/')]
datas += collect_data_files('tensorflow')
datas += collect_data_files('torch')
datas += copy_metadata('torch')
datas += copy_metadata('tqdm')
datas += copy_metadata('regex')
datas += copy_metadata('sacremoses')
datas += copy_metadata('requests')
datas += copy_metadata('packaging')
datas += copy_metadata('filelock')
datas += copy_metadata('numpy')
datas += copy_metadata('tokenizers')
datas += copy_metadata('importlib_metadata')

block_cipher = None

a = Analysis(['deidentify.py'],
             pathex=[],
             binaries=[],
             datas=datas,
             hiddenimports=['“sklearn.utils._cython_blas”', '“sklearn.neighbors.typedefs”', '“sklearn.neighbors.quad_tree”', '“sklearn.tree”', '“sklearn.tree._utils”'],
             hookspath=[],
             hooksconfig={},
             runtime_hooks=[],
             excludes=[],
             win_no_prefer_redirects=False,
             win_private_assemblies=False,
             cipher=block_cipher,
             noarchive=False)
pyz = PYZ(a.pure, a.zipped_data,
             cipher=block_cipher)

exe = EXE(pyz,
          a.scripts,
          [],
          exclude_binaries=True,
          name='deidentify',
          debug=False,
          bootloader_ignore_signals=False,
          strip=False,
          upx=True,
          console=False,
          disable_windowed_traceback=False,
          target_arch=None,
          codesign_identity=None,
          entitlements_file=None )
coll = COLLECT(exe,
               a.binaries,
               a.zipfiles,
               a.datas,
               strip=False,
               upx=True,
               upx_exclude=[],
               name='deidentify')
</code>```

As can be seen all model files and libraries are included.

The following is the console output when generating the exe file

```<code>console output removed due to maximum character limit reached
</code>```

I dont know why so many modules are missing above as I have them installed in my system and in my local environment. They should be picked up. I even asked them to be included in the .spec file.

After the process completes, the error I receive when I run the exe file is:

```<code>Traceback (most recent call last):
  File "transformers\utils\versions.py", line 105, in require_version
  File "importlib_metadata\__init__.py", line 631, in version
  File "importlib_metadata\__init__.py", line 604, in distribution
  File "importlib_metadata\__init__.py", line 229, in from_name
importlib_metadata.PackageNotFoundError: No package metadata was found for dataclasses

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "deidentify.py", line 1, in <module>
  File "PyInstaller\loader\pyimod03_importers.py", line 476, in exec_module
  File "transformers\__init__.py", line 43, in <module>
  File "PyInstaller\loader\pyimod03_importers.py", line 476, in exec_module
  File "transformers\dependency_versions_check.py", line 41, in <module>
  File "transformers\utils\versions.py", line 120, in require_version_core
  File "transformers\utils\versions.py", line 108, in require_version
importlib_metadata.PackageNotFoundError: No package metadata was found for The 'dataclasses' distribution was not found and is required by this application.
Try: pip install transformers -U or pip install -e '.[dev]' if you're working with git master
</code>```

importlib_metadata is installed in pip it should not be missing.

Update

After the comment by @0x26res and updating to python 3.8 I am presented with a new error:

```<code>Traceback (most recent call last):
  File "torch\_sources.py", line 21, in get_source_lines_and_file
    sourcelines, file_lineno = inspect.getsourcelines(obj)
  File "inspect.py", line 979, in getsourcelines
  File "inspect.py", line 798, in findsource
OSError: could not get source code

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "transformers\file_utils.py", line 2704, in _get_module
  File "importlib\__init__.py", line 127, in import_module
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "PyInstaller\loader\pyimod03_importers.py", line 495, in exec_module
  File "transformers\models\deberta\modeling_deberta.py", line 505, in <module>
  File "torch\jit\_script.py", line 1307, in script
    ast = get_jit_def(obj, obj.__name__)
  File "torch\jit\frontend.py", line 233, in get_jit_def
    parsed_def = parse_def(fn)
  File "torch\_sources.py", line 95, in parse_def
    sourcelines, file_lineno, filename = get_source_lines_and_file(fn, ErrorReport.call_stack())
  File "torch\_sources.py", line 28, in get_source_lines_and_file
    raise OSError(msg) from e
OSError: Can't get source for <function c2p_dynamic_expand at 0x000002608019EDC0>. TorchScript requires source access in order to carry out compilation, make sure original .py files are available.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "deidentify.py", line 16, in <module>
  File "transformers\pipelines\__init__.py", line 651, in pipeline
  File "transformers\pipelines\token_classification.py", line 103, in __init__
  File "transformers\pipelines\base.py", line 853, in check_model_type
  File "transformers\models\auto\auto_factory.py", line 601, in items
  File "transformers\models\auto\auto_factory.py", line 604, in <listcomp>
  File "transformers\models\auto\auto_factory.py", line 573, in _load_attr_from_module
  File "transformers\models\auto\auto_factory.py", line 535, in getattribute_from_module
  File "transformers\file_utils.py", line 2694, in __getattr__
  File "transformers\file_utils.py", line 2706, in _get_module
RuntimeError: Failed to import transformers.models.deberta.modeling_deberta because of the following error (look up to see its traceback):
Can't get source for <function c2p_dynamic_expand at 0x000002608019EDC0>. TorchScript requires source access in order to carry out compilation, make sure original .py files are available.
</code>```

I gave the following command after updating to python 3.8

```<code>pyinstaller --windowed --add-data ./model/;./model/ --collect-data torch --copy-metadata torch --copy-metadata tqd
m --copy-metadata regex --copy-metadata sacremoses --copy-metadata requests --copy-metadata packaging --copy-metadata filelock --copy-metadata numpy --copy-metadata tokenizers deidentify.p
y
</code>```

答:
First of all, you don't need to individually include all the files in ./model/, just include the entire model directory and everything else will be included too:

datas=[('model/','model'),...

I don't know why dataclasses isn't being included, but just try including it manually

datas=[('[path-to-your-dataclasses.py]', '.'),...

This will put dataclasses.py in the root directory and it should be found by the exe.

Body:
I am trying to convert my script that uses transformers to an exe file. Its a small single file script that performs token classification:

from transformers import AutoTokenizer, AutoModelForTokenClassification
from transformers import pipeline

# download once to save locally
# tokenizer = AutoTokenizer.from_pretrained("dslim/bert-base-NER-uncased")
# model = AutoModelForTokenClassification.from_pretrained("dslim/bert-base-NER-uncased")

# save model locally
# tokenizer.save_pretrained("./model")
# model.save_pretrained("./model")

# now just load from local file
tokenizer = AutoTokenizer.from_pretrained('./model')
model = AutoModelForTokenClassification.from_pretrained('./model')

nlp = pipeline("token-classification", model=model, tokenizer=tokenizer, aggregation_strategy="simple")

example = """00:00:02 Speaker 1: hi john, it's nice to see you again. how was your weekend? do anything special? 00:00:06 Speaker 2: yep, all good thanks. i was with my sister in derby. We saw, you know, that james bond film. what's it called? then got a couple of drinks at the pitcher and piano, back in nottingham. """
ner_results = nlp(example)
print(ner_results)

for i in range(0, len(ner_results)):
  start = ner_results[i]['start']
  end = ner_results[i]['end']
  example = example.replace(ner_results[i]['word'], ner_results[i]['entity_group'])
print(example)

The online models are downloaded only once and then saved locally so that they can be packaged with pyinstaller. I am using the below line to build the exe file (which from reading other similar questions on SO, adds all the required missing libraries that pyinstaller misses)

pyinstaller --windowed --add-data ./model/config.json;./model/ --add-data ./model/pytorch_model.bin;./model/ --add-data ./model/special_tokens_map.json;./model/ --add-data ./model/tokenizer.json;./model/ --add-data ./model/tokenizer_config.json;./model/ --add-data ./model/vocab.txt;./model/ --collect-data tensorflow --collect-data torch --copy-metadata torch --copy-metadata tqdm --copy-metadata regex --copy-metadata sacremoses --copy-metadata requests --copy-metadata packaging --copy-metadata filelock --copy-metadata numpy --copy-metadata tokenizers --copy-metadata importlib_metadata --hidden-import=“sklearn.utils._cython_blas” --hidden-import=“sklearn.neighbors.typedefs” --hidden-import=“sklearn.neighbors.quad_tree” --hidden-import=“sklearn.tree” --hidden-import=“sklearn.tree._utils” deidentify.py

This generates the following .spec file

# -*- mode: python ; coding: utf-8 -*-
from PyInstaller.utils.hooks import collect_data_files
from PyInstaller.utils.hooks import copy_metadata

datas = [('./model/config.json', './model/'), ('./model/pytorch_model.bin', './model/'), ('./model/special_tokens_map.json', './model/'), ('./model/tokenizer.json', './model/'), ('./model/tokenizer_config.json', './model/'), ('./model/vocab.txt', './model/')]
datas += collect_data_files('tensorflow')
datas += collect_data_files('torch')
datas += copy_metadata('torch')
datas += copy_metadata('tqdm')
datas += copy_metadata('regex')
datas += copy_metadata('sacremoses')
datas += copy_metadata('requests')
datas += copy_metadata('packaging')
datas += copy_metadata('filelock')
datas += copy_metadata('numpy')
datas += copy_metadata('tokenizers')
datas += copy_metadata('importlib_metadata')

block_cipher = None

a = Analysis(['deidentify.py'],
             pathex=[],
             binaries=[],
             datas=datas,
             hiddenimports=['“sklearn.utils._cython_blas”', '“sklearn.neighbors.typedefs”', '“sklearn.neighbors.quad_tree”', '“sklearn.tree”', '“sklearn.tree._utils”'],
             hookspath=[],
             hooksconfig={},
             runtime_hooks=[],
             excludes=[],
             win_no_prefer_redirects=False,
             win_private_assemblies=False,
             cipher=block_cipher,
             noarchive=False)
pyz = PYZ(a.pure, a.zipped_data,
             cipher=block_cipher)

exe = EXE(pyz,
          a.scripts,
          [],
          exclude_binaries=True,
          name='deidentify',
          debug=False,
          bootloader_ignore_signals=False,
          strip=False,
          upx=True,
          console=False,
          disable_windowed_traceback=False,
          target_arch=None,
          codesign_identity=None,
          entitlements_file=None )
coll = COLLECT(exe,
               a.binaries,
               a.zipfiles,
               a.datas,
               strip=False,
               upx=True,
               upx_exclude=[],
               name='deidentify')

As can be seen all model files and libraries are included.

The following is the console output when generating the exe file

console output removed due to maximum character limit reached

I dont know why so many modules are missing above as I have them installed in my system and in my local environment. They should be picked up. I even asked them to be included in the .spec file.

After the process completes, the error I receive when I run the exe file is:

Traceback (most recent call last):
  File "transformers\utils\versions.py", line 105, in require_version
  File "importlib_metadata\__init__.py", line 631, in version
  File "importlib_metadata\__init__.py", line 604, in distribution
  File "importlib_metadata\__init__.py", line 229, in from_name
importlib_metadata.PackageNotFoundError: No package metadata was found for dataclasses

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "deidentify.py", line 1, in <module>
  File "PyInstaller\loader\pyimod03_importers.py", line 476, in exec_module
  File "transformers\__init__.py", line 43, in <module>
  File "PyInstaller\loader\pyimod03_importers.py", line 476, in exec_module
  File "transformers\dependency_versions_check.py", line 41, in <module>
  File "transformers\utils\versions.py", line 120, in require_version_core
  File "transformers\utils\versions.py", line 108, in require_version
importlib_metadata.PackageNotFoundError: No package metadata was found for The 'dataclasses' distribution was not found and is required by this application.
Try: pip install transformers -U or pip install -e '.[dev]' if you're working with git master

importlib_metadata is installed in pip it should not be missing.

Update

After the comment by @0x26res and updating to python 3.8 I am presented with a new error:

Traceback (most recent call last):
  File "torch\_sources.py", line 21, in get_source_lines_and_file
    sourcelines, file_lineno = inspect.getsourcelines(obj)
  File "inspect.py", line 979, in getsourcelines
  File "inspect.py", line 798, in findsource
OSError: could not get source code

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "transformers\file_utils.py", line 2704, in _get_module
  File "importlib\__init__.py", line 127, in import_module
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "PyInstaller\loader\pyimod03_importers.py", line 495, in exec_module
  File "transformers\models\deberta\modeling_deberta.py", line 505, in <module>
  File "torch\jit\_script.py", line 1307, in script
    ast = get_jit_def(obj, obj.__name__)
  File "torch\jit\frontend.py", line 233, in get_jit_def
    parsed_def = parse_def(fn)
  File "torch\_sources.py", line 95, in parse_def
    sourcelines, file_lineno, filename = get_source_lines_and_file(fn, ErrorReport.call_stack())
  File "torch\_sources.py", line 28, in get_source_lines_and_file
    raise OSError(msg) from e
OSError: Can't get source for <function c2p_dynamic_expand at 0x000002608019EDC0>. TorchScript requires source access in order to carry out compilation, make sure original .py files are available.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "deidentify.py", line 16, in <module>
  File "transformers\pipelines\__init__.py", line 651, in pipeline
  File "transformers\pipelines\token_classification.py", line 103, in __init__
  File "transformers\pipelines\base.py", line 853, in check_model_type
  File "transformers\models\auto\auto_factory.py", line 601, in items
  File "transformers\models\auto\auto_factory.py", line 604, in <listcomp>
  File "transformers\models\auto\auto_factory.py", line 573, in _load_attr_from_module
  File "transformers\models\auto\auto_factory.py", line 535, in getattribute_from_module
  File "transformers\file_utils.py", line 2694, in __getattr__
  File "transformers\file_utils.py", line 2706, in _get_module
RuntimeError: Failed to import transformers.models.deberta.modeling_deberta because of the following error (look up to see its traceback):
Can't get source for <function c2p_dynamic_expand at 0x000002608019EDC0>. TorchScript requires source access in order to carry out compilation, make sure original .py files are available.

I gave the following command after updating to python 3.8

pyinstaller --windowed --add-data ./model/;./model/ --collect-data torch --copy-metadata torch --copy-metadata tqd
m --copy-metadata regex --copy-metadata sacremoses --copy-metadata requests --copy-metadata packaging --copy-metadata filelock --copy-metadata numpy --copy-metadata tokenizers deidentify.p
y


问:
Automaticaly execute script inside Kuebrentes pod A when i update pod B with ansible
I have 2 pods , deploy with ansible :
- POD A
- POD B.

Actually, i execute ansible to update pod B.
When update is finished, i need to execute a command inside pod A.

Do you know the best way to automatise the execution of this script ? via ansible ? kubernetes conf ?

thanks

答:
The first example is waiting for a pod that I have updated to be ready.

- name: Deploy all cryoserver-ssh manifests
  kubernetes.core.k8s:
    state: present
    src: "{{ item }}"
    kubeconfig: /deployment/ansible-files/admin.conf
    namespace: "default"
  with_fileglob:
    - "/deployment/ansible-helm-deploy/cryoserver-ssh/templates/*"

- name: Remove saml config files into helm chart
  ansible.builtin.file:
    path: "/project/helm_charts/cryoserver-ssh/{{ item['metadata']['MetadataFile'] }}"
    state: absent
  loop: "{{ metadata_files }}"

- name: Wait till the pods are running
  kubernetes.core.k8s_info:
    kubeconfig: /deployment/ansible-files/admin.conf
    kind: Pod
    label_selectors:
      - "app = cryoserver-ssh"
    wait: yes
    namespace: default
    wait_sleep: 10
    wait_timeout: 360```
</div>
</div>
</p>
<p>In the second example, I am uploading a script that needs to be run in the pod, making it executable and then executing it.</p>
<p><div class="snippet" data-lang="js" data-hide="false" data-console="true" data-babel="false">
<div class="snippet-code">
<pre class="snippet-code-js lang-js prettyprint-override"><code>- name: Copy cryoserver-update-zookeeper.sh local file to remote pod
  kubernetes.core.k8s_cp:
    kubeconfig: /deployment/ansible-files/admin.conf
    namespace: default
    pod: "{{ helm_values['cryoserver']['serviceName'] }}-ssh-0"
    remote_path: "/project/scripts/cryoserver-update-zookeeper.sh"
    local_path: "/project/plays/templates/cryoserver-update-zookeeper.sh"
- name: Run cryoserver-update-zookeeper.sh script
  kubernetes.core.k8s_exec:
    kubeconfig: /deployment/ansible-files/admin.conf
    namespace: default
    pod: "{{ helm_values['cryoserver']['serviceName'] }}-ssh-0"
    command: "chmod +x /project/scripts/cryoserver-update-zookeeper.sh"
- name: Run cryoserver-update-zookeeper.sh script
  kubernetes.core.k8s_exec:
    kubeconfig: /deployment/ansible-files/admin.conf
    namespace: default
    pod: "{{ helm_values['cryoserver']['serviceName'] }}-ssh-0"
    command: "/bin/bash -c '/project/scripts/cryoserver-update-zookeeper.sh'"</code>```

Body:
The first example is waiting for a pod that I have updated to be ready.

- name: Deploy all cryoserver-ssh manifests
  kubernetes.core.k8s:
    state: present
    src: "{{ item }}"
    kubeconfig: /deployment/ansible-files/admin.conf
    namespace: "default"
  with_fileglob:
    - "/deployment/ansible-helm-deploy/cryoserver-ssh/templates/*"

- name: Remove saml config files into helm chart
  ansible.builtin.file:
    path: "/project/helm_charts/cryoserver-ssh/{{ item['metadata']['MetadataFile'] }}"
    state: absent
  loop: "{{ metadata_files }}"

- name: Wait till the pods are running
  kubernetes.core.k8s_info:
    kubeconfig: /deployment/ansible-files/admin.conf
    kind: Pod
    label_selectors:
      - "app = cryoserver-ssh"
    wait: yes
    namespace: default
    wait_sleep: 10
    wait_timeout: 360

In the second example, I am uploading a script that needs to be run in the pod, making it executable and then executing it.

- name: Copy cryoserver-update-zookeeper.sh local file to remote pod
  kubernetes.core.k8s_cp:
    kubeconfig: /deployment/ansible-files/admin.conf
    namespace: default
    pod: "{{ helm_values['cryoserver']['serviceName'] }}-ssh-0"
    remote_path: "/project/scripts/cryoserver-update-zookeeper.sh"
    local_path: "/project/plays/templates/cryoserver-update-zookeeper.sh"
- name: Run cryoserver-update-zookeeper.sh script
  kubernetes.core.k8s_exec:
    kubeconfig: /deployment/ansible-files/admin.conf
    namespace: default
    pod: "{{ helm_values['cryoserver']['serviceName'] }}-ssh-0"
    command: "chmod +x /project/scripts/cryoserver-update-zookeeper.sh"
- name: Run cryoserver-update-zookeeper.sh script
  kubernetes.core.k8s_exec:
    kubeconfig: /deployment/ansible-files/admin.conf
    namespace: default
    pod: "{{ helm_values['cryoserver']['serviceName'] }}-ssh-0"
    command: "/bin/bash -c '/project/scripts/cryoserver-update-zookeeper.sh'"


问:
How do I annotate Id column if value is generated by a trigger?
I have a setup with Oracle XE 10g, Hibernate 3.5, JPA 2.0. There is a simple table with a primary key, which is generated by a database trigger at insertion. The trigger gets the value from a sequence. The trigger/sequence construction was made by Oracle XE.

The actual question is: How do I get the current value of the Id in my entity after a EntityManager.persist?

I tried:

```<code>@Id
private long id;
</code>```

-> id is 0;

```<code>@Id
@Generated(GenerationTime.ALWAYS)
@Column(insertable = false, updatable = false)
private long id;
</code>```

-> id is 0;

```<code>@Id
@GeneratedValue(strategy=GenerationType.AUTO)
private long id;
</code>```

-> Fails because Hibernate is trying to query the a sequence directly (which does not exist).

IDs are generated in the database in the first two approaches, but I don't have the value in my object.

答:
Unless you are bound to the trigger, that seems like a level of obfuscation over the sequence, and it seems to go outside the normal Hibernate lifecycle.  Why not directly call the sequence:

```<code>@SequenceGenerator(name="alias_for_my_sequence", sequenceName="seq_name_in_oracle")
@GeneratedValue(strategy=GenerationType.SEQUENCE, generator="alias_for_my_sequence")
@Id
private Long id;
</code>```

Then you will get the value back, as Hibernate is directly involved in the generation, and there isn't something happening after the fact.

Body:
I have a setup with Oracle XE 10g, Hibernate 3.5, JPA 2.0. There is a simple table with a primary key, which is generated by a database trigger at insertion. The trigger gets the value from a sequence. The trigger/sequence construction was made by Oracle XE.

The actual question is: How do I get the current value of the Id in my entity after a EntityManager.persist?

I tried:

@Id
private long id;

-> id is 0;

@Id
@Generated(GenerationTime.ALWAYS)
@Column(insertable = false, updatable = false)
private long id;

-> id is 0;

@Id
@GeneratedValue(strategy=GenerationType.AUTO)
private long id;

-> Fails because Hibernate is trying to query the a sequence directly (which does not exist).

IDs are generated in the database in the first two approaches, but I don't have the value in my object.

Body:
Unless you are bound to the trigger, that seems like a level of obfuscation over the sequence, and it seems to go outside the normal Hibernate lifecycle.  Why not directly call the sequence:

@SequenceGenerator(name="alias_for_my_sequence", sequenceName="seq_name_in_oracle")
@GeneratedValue(strategy=GenerationType.SEQUENCE, generator="alias_for_my_sequence")
@Id
private Long id;

Then you will get the value back, as Hibernate is directly involved in the generation, and there isn't something happening after the fact.


问:
Getting the OAuth Token from FedEx to use for Track API
I'm trying to get the OAuth Token to get auth access to some of the FedEx APIs ( like Track API for tracking shipments ), but I get a
401 (NOT.AUTHORIZED.ERROR -> "The given client credentials were not valid. Please modify your request and try again") error.
(At the moment, I'm using Postman to try and test the APIs.)

Here is the url I'm using, found provided by FedEx:
https://apis-sandbox.fedex.com/oauth/token

I've followed ( to my understanding ) how the body and headers should be set:

```<code>Headers: Content-Type: application/x-www-form-urlencoded
Body: x-www-form-urlencoded (Postman option):
   grant_type: client_crendentials
   client_id: ***PROJECT_API_KEY
   client_secret: ***PROJECT_SECRET KEY
</code>```

After sending, I only get the error message above. I checked / doubled-checked my API keys, but I can't seem to get it to go through.
Any ideas?

```<code>postman settings:
POST - URL https://apis-sandbox.fedex.com/oauth/token
Headers - Content-Type: application/x-www-form-urlencoded
Body - x-www-form-urlencoded:
   grant_type: client_credentials
   client_id: *******
   client_secret: *******
</code>```

(information gotten from https://developer.fedex.com/api/en-us/catalog/authorization/v1/docs.html)

答: The sandbox API credentials are not immediately usable when creating a new account. It took >1 hour for me to get a registration complete email from FedEx after which the sandbox credentials became valid.

Body:
I'm trying to get the OAuth Token to get auth access to some of the FedEx APIs ( like Track API for tracking shipments ), but I get a
401 (NOT.AUTHORIZED.ERROR -> "The given client credentials were not valid. Please modify your request and try again") error.
(At the moment, I'm using Postman to try and test the APIs.)

Here is the url I'm using, found provided by FedEx:
https://apis-sandbox.fedex.com/oauth/token

I've followed ( to my understanding ) how the body and headers should be set:

Headers: Content-Type: application/x-www-form-urlencoded
Body: x-www-form-urlencoded (Postman option):
   grant_type: client_crendentials
   client_id: ***PROJECT_API_KEY
   client_secret: ***PROJECT_SECRET KEY

After sending, I only get the error message above. I checked / doubled-checked my API keys, but I can't seem to get it to go through.
Any ideas?

postman settings:
POST - URL https://apis-sandbox.fedex.com/oauth/token
Headers - Content-Type: application/x-www-form-urlencoded
Body - x-www-form-urlencoded:
   grant_type: client_credentials
   client_id: *******
   client_secret: *******

(information gotten from https://developer.fedex.com/api/en-us/catalog/authorization/v1/docs.html)


问:
Calculate difference in variable within same day in two nc file
I have two NC files with ERA5land precipitation data from the months AMJJASO of 2001-2020. One NC file of precipitation data from UTC 00.00 and one NC file with precipitation data from UTC 12.00. I would like to subtract the precipitation from UTC 00.00 from the precipitation at timeslot 12.00, hence;

UTC12.00 - UTC00.00 = UTCprecip

I have previously used cdo as

```<code>cdo sub UTC12.nc UTC00.nc UTCprecip.nc
</code>```

but was looking for something like daysub, not ydaysub though(!).

But does this simple subtraction using cdo sub occur within each day only?

Or do I have to use a different call?

答:
Based on what you have said the sub method will correctly handle your problem.

The sub method in CDO will do the subtraction for matching time steps. It only matches time steps, not dates etc. So time step 1 in UTC00.nc is subtracted step 1 from UTC12.nc and so on until the last remaining time step that is in both files.

Body:
I have two NC files with ERA5land precipitation data from the months AMJJASO of 2001-2020. One NC file of precipitation data from UTC 00.00 and one NC file with precipitation data from UTC 12.00. I would like to subtract the precipitation from UTC 00.00 from the precipitation at timeslot 12.00, hence;

UTC12.00 - UTC00.00 = UTCprecip

I have previously used cdo as

cdo sub UTC12.nc UTC00.nc UTCprecip.nc

but was looking for something like daysub, not ydaysub though(!).

But does this simple subtraction using cdo sub occur within each day only?

Or do I have to use a different call?


问:
How to apply Lemmatization to a column in a pandas dataframe
If i had the following dataframe:

```<code>import pandas as pd

d = {'col1': ['challenging', 'swimming'], 'col2': [3, 4]}
df = pd.DataFrame(data=d)

Output
          col1  col2
0  challenging     3
1     swimming     4
</code>```

I am using the WordNetLemmatizer:

```<code>print(wordnet_lemmatizer.lemmatize('challenging',pos='v'))
print(wordnet_lemmatizer.lemmatize('swimming',pos='v'))

Output
challenge
swim
</code>```

How can I apply this lemmatization function to all elements of col1 from the original dataframe?

I have tried the following but no luck since it requires an input of pos so no change to dataframe

```<code>df['col1'] =df['col1'].apply(wordnet_lemmatizer.lemmatize)
</code>```

If i try:

```<code>df['col1'] =df['col1'].apply(wordnet_lemmatizer.lemmatize(pos='v'))
</code>```

I get

TypeError: lemmatize() missing 1 required positional argument: 'word'

The desired output is:

```<code>        col1  col2
0       challenge     3
1       swim     4
</code>```

答:
For a best output, you can use spacy

```<code>import spacy
nlp = spacy.load("en_core_web_sm")  # load an existing English template
df['col1'] = [j.lemma_ for i in df['col1'] for j in nlp(i)]
</code>```

You must install spacy, then install english langage

```<code>python -m spacy download en_core_web_sm
</code>```

Body:
If i had the following dataframe:

import pandas as pd

d = {'col1': ['challenging', 'swimming'], 'col2': [3, 4]}
df = pd.DataFrame(data=d)

Output
          col1  col2
0  challenging     3
1     swimming     4

I am using the WordNetLemmatizer:

print(wordnet_lemmatizer.lemmatize('challenging',pos='v'))
print(wordnet_lemmatizer.lemmatize('swimming',pos='v'))

Output
challenge
swim

How can I apply this lemmatization function to all elements of col1 from the original dataframe?

I have tried the following but no luck since it requires an input of pos so no change to dataframe

df['col1'] =df['col1'].apply(wordnet_lemmatizer.lemmatize)

If i try:

df['col1'] =df['col1'].apply(wordnet_lemmatizer.lemmatize(pos='v'))

I get

TypeError: lemmatize() missing 1 required positional argument: 'word'

The desired output is:

        col1  col2
0       challenge     3
1       swim     4

Body:
For a best output, you can use spacy

import spacy
nlp = spacy.load("en_core_web_sm")  # load an existing English template
df['col1'] = [j.lemma_ for i in df['col1'] for j in nlp(i)]

You must install spacy, then install english langage

python -m spacy download en_core_web_sm


问:
Create Array of same Size
I have an array

```<code>a = zeros(2,3)

2×3 Matrix{Float64}:
 0.0  0.0  0.0
 0.0  0.0  0.0
</code>```

and I want to create a Bool array of same size with all values set to false.

答:
One solution would be:</p>
```<code>size(a) |> falses

2×3 BitMatrix:
 0  0  0
 0  0  0
</code>```

Title: Create Array of same Size

Body:
I have an array

a = zeros(2,3)

2×3 Matrix{Float64}:
 0.0  0.0  0.0
 0.0  0.0  0.0

and I want to create a Bool array of same size with all values set to false.

Body:
One solution would be:

size(a) |> falses

2×3 BitMatrix:
 0  0  0
 0  0  0


问:
Excel formula to extract string between 2 specific character
I have a data table that looks like this:

And am using the following forumula:
=MID(LEFT(F2,FIND("D",F2)-1),FIND(" ",F2)+1,LEN(F2))

With the goal of extracting the numeric value after the "D" character in my Visit column. Some visits have 1 digit after the "D", some are 2, max will be 3 digits. However, the formula I've tried to use above is returning blanks and #VALUE! only. I figured the number character between the D and the space would work, but then I realize not all will have the space (see C2D16). Can someone explain what I am doing incorrectly?

TIA

答:
Here is a formula that I think is a little cleaner

```<code>=MID(F1,FIND("D",F1)+1,IFERROR(FIND(" ",F1),LEN(F1)+1)-(FIND("D",F1)+1))
</code>```

I think this actually represents the thought process of what you are trying to do.

Breaking it down:

```<code>=MID(F1,X,Y)
</code>```

You want the middle part of F1, starting at X with a Length of Y

```<code>X=FIND("D",F1)+1
</code>```

Starting point (X) is one space after the first D

```<code>Y=IFERROR(FIND(" ",F1),LEN(F1)+1)-(FIND("D",F1)+1)
</code>```

Length (Y) is X subtracted from either the position of the space character or 1 more than the length of the entire string

Body:
Here is a formula that I think is a little cleaner

=MID(F1,FIND("D",F1)+1,IFERROR(FIND(" ",F1),LEN(F1)+1)-(FIND("D",F1)+1))

I think this actually represents the thought process of what you are trying to do.

Breaking it down:

=MID(F1,X,Y)

You want the middle part of F1, starting at X with a Length of Y

X=FIND("D",F1)+1

Starting point (X) is one space after the first D

Y=IFERROR(FIND(" ",F1),LEN(F1)+1)-(FIND("D",F1)+1)

Length (Y) is X subtracted from either the position of the space character or 1 more than the length of the entire string


问:
how to use a custom functional component in react-native?
how to use a functional custom component in react native??? i actualy build a login functional component like a custom react but not exsist a  "div" i made a "View"
why should this not work??</p>
```<code>    import React from 'react';
    import { Text, View } from 'react-native';

    const Login = () => {
    return (
        <View>
          <Text>Hi</Text>
        </View>
      );
    }
    export default Login;

---------------------------------
    import React from 'react';
    import {Login} from './components/Login/Login';
    import { Text, View } from 'react-native';

    const App = () => {
        return (
        <View>
          <Login/>
       </View>
        )
     }
     export default App;
</code>```

答:
You've got your named exports and default exports turned around.

This would work instead:

```<code> import React from 'react';
    import { Text, View } from 'react-native';

    const Login = () => {
    return (
        <View>
          <Text>Hi</Text>
        </View>
      );
    }
    export default Login;

---------------------------------
    import React from 'react';
    import Login from './components/Login/Login';
    import { Text, View } from 'react-native';

    const App = () => {
        return (
        <View>
          <Login/>
       </View>
        )
     }
     export default App;
</code>```

When you export default Login, you must import it as a default import with import Login from './components/Login/Login'

Here is an Expo Snack with the working code as well

Body:
how to use a functional custom component in react native??? i actualy build a login functional component like a custom react but not exsist a  "div" i made a "View"
why should this not work??

    import React from 'react';
    import { Text, View } from 'react-native';

    const Login = () => {
    return (
        <View>
          <Text>Hi</Text>
        </View>
      );
    }
    export default Login;

---------------------------------
    import React from 'react';
    import {Login} from './components/Login/Login';
    import { Text, View } from 'react-native';

    const App = () => {
        return (
        <View>
          <Login/>
       </View>
        )
     }
     export default App;

Body:
You've got your named exports and default exports turned around.

This would work instead:

 import React from 'react';
    import { Text, View } from 'react-native';

    const Login = () => {
    return (
        <View>
          <Text>Hi</Text>
        </View>
      );
    }
    export default Login;

---------------------------------
    import React from 'react';
    import Login from './components/Login/Login';
    import { Text, View } from 'react-native';

    const App = () => {
        return (
        <View>
          <Login/>
       </View>
        )
     }
     export default App;

When you export default Login, you must import it as a default import with import Login from './components/Login/Login'

Here is an Expo Snack with the working code as well


答:
Here is my attempt using some functions:

```<code>def sorted_idx(l, num):
    x = sorted(list(set(l)))
    for i in range(len(x)):
        if x[i]==num:
            return i+1

def output_list(l):
    ret = [sorted_idx(l, elem) for elem in l]
    return ret

dfOut = df.apply(lambda column: output_list(column))

print(dfOut)
</code>```

I make reduce the original list to unique values and then sort. Finally, I return the index+1 where the element in the original list matches this unique, sorted list to get the values you have in your expected output.

Output:

```<code>   A  B  C
0  2  3  1
1  4  5  2
2  1  4  2
3  5  2  3
4  6  2  4
5  3  1  3
</code>```

Title: How to get the (relative) place of values in a dataframe when sorted using Python?

Body:
How can I create a Pandas DataFrame that shows the relative position of each value, when those values are sorted from low to high for each column?
So in this case, how can you transform 'df' into 'dfOut'?

import pandas as pd
import numpy as np

#create DataFrame
df = pd.DataFrame({'A': [12, 18, 9, 21, 24, 15],
                   'B': [18, 22, 19, 14, 14, 11],
                   'C': [5, 7, 7, 9, 12, 9]})

# How to assign a value to the order in the column, when sorted from low to high?

dfOut = pd.DataFrame({'A': [2, 4, 1, 5, 6, 3],
                      'B': [3, 5, 4, 2, 2, 1],
                      'C': [1, 2, 2, 3, 4, 3]})

Body:
Here is my attempt using some functions:

def sorted_idx(l, num):
    x = sorted(list(set(l)))
    for i in range(len(x)):
        if x[i]==num:
            return i+1

def output_list(l):
    ret = [sorted_idx(l, elem) for elem in l]
    return ret

dfOut = df.apply(lambda column: output_list(column))

print(dfOut)

I make reduce the original list to unique values and then sort. Finally, I return the index+1 where the element in the original list matches this unique, sorted list to get the values you have in your expected output.

Output:

   A  B  C
0  2  3  1
1  4  5  2
2  1  4  2
3  5  2  3
4  6  2  4
5  3  1  3


问:
dropdown selector in R Markdown html output without Shniy?
Is it possible to add a dropdown selector in R Markdown html output? I do not want to use Shiny because I want it to be functional without a server.

Any ideas?

答:
See here https://github.com/rstudio/rmarkdown/issues/1116
Save this as tabset-dropdown.html in your working directory:

```<code><style>
/* Notwendiges Zeug für Dropdowns */
.tabset-dropdown > .nav-tabs {
    display: inline-block;
    max-height: 500px;
    min-height: 44px;
    overflow-y: auto;
    background: white;
    border: 1px solid #ddd;
    border-radius: 4px;
}

.tabset-dropdown > .nav-tabs > li.active:before {
    content: "";
    font-family: 'Glyphicons Halflings';
    display: inline-block;
    padding: 10px;
    border-right: 1px solid #ddd;
}

.tabset-dropdown > .nav-tabs.nav-tabs-open > li.active:before {
    content: "";
    border: none;
}

.tabset-dropdown > .nav-tabs.nav-tabs-open:before {
    content: "";
    font-family: 'Glyphicons Halflings';
    display: inline-block;
    padding: 10px;
    border-right: 1px solid #ddd;
}

.tabset-dropdown > .nav-tabs > li.active {
    display: block;
}

.tabset-dropdown > .nav-tabs > li > a,
.tabset-dropdown > .nav-tabs > li > a:focus,
.tabset-dropdown > .nav-tabs > li > a:hover {
    border: none;
    display: inline-block;
    border-radius: 4px;
}

.tabset-dropdown > .nav-tabs.nav-tabs-open > li {
    display: block;
    float: none;
}

.tabset-dropdown > .nav-tabs > li {
    display: none;
}
</style>

<script>
$(document).ready(function () {
    $('.tabset-dropdown > .nav-tabs > li').click(function () {
        $(this).parent().toggleClass('nav-tabs-open')
    });
});
</script>
</code>```

The corresponding rmarkdown code is:

```<code>---
  title: "Untitled"
output:
  html_document:
  toc: true
toc_float: true
code_folding: hide
number_sections: true
includes:
  after_body: tabset-dropdown.html
---

  ```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
```

# R Markdown {.tabset .tabset-dropdown}

## Tab A

This is an R Markdown document. Markdown is a simple formatting syntax for authoring HTML, PDF, and MS Word documents. For more details on using R Markdown see <http://rmarkdown.rstudio.com>.

## Tab B
When you click the **Knit** button a document will be generated that includes both content as well as the output of any embedded R code chunks within the document. You can embed an R code chunk like this:

  ## Tab C
  ```{r cars}
summary(cars)
```

## Including Plots

You can also embed plots, for example:

  ```{r pressure, echo=FALSE}
plot(pressure)
```

Note that the `echo = FALSE` parameter was added to the code chunk to prevent printing of the R code that generated the plot.
</code>```

Body:
See here https://github.com/rstudio/rmarkdown/issues/1116
Save this as tabset-dropdown.html in your working directory:

<style>
/* Notwendiges Zeug für Dropdowns */
.tabset-dropdown > .nav-tabs {
    display: inline-block;
    max-height: 500px;
    min-height: 44px;
    overflow-y: auto;
    background: white;
    border: 1px solid #ddd;
    border-radius: 4px;
}

.tabset-dropdown > .nav-tabs > li.active:before {
    content: "";
    font-family: 'Glyphicons Halflings';
    display: inline-block;
    padding: 10px;
    border-right: 1px solid #ddd;
}

.tabset-dropdown > .nav-tabs.nav-tabs-open > li.active:before {
    content: "&#xe258;";
    border: none;
}

.tabset-dropdown > .nav-tabs.nav-tabs-open:before {
    content: "";
    font-family: 'Glyphicons Halflings';
    display: inline-block;
    padding: 10px;
    border-right: 1px solid #ddd;
}

.tabset-dropdown > .nav-tabs > li.active {
    display: block;
}

.tabset-dropdown > .nav-tabs > li > a,
.tabset-dropdown > .nav-tabs > li > a:focus,
.tabset-dropdown > .nav-tabs > li > a:hover {
    border: none;
    display: inline-block;
    border-radius: 4px;
}

.tabset-dropdown > .nav-tabs.nav-tabs-open > li {
    display: block;
    float: none;
}

.tabset-dropdown > .nav-tabs > li {
    display: none;
}
</style>

<script>
$(document).ready(function () {
    $('.tabset-dropdown > .nav-tabs > li').click(function () {
        $(this).parent().toggleClass('nav-tabs-open')
    });
});
</script>

The corresponding rmarkdown code is:

---
  title: "Untitled"
output:
  html_document:
  toc: true
toc_float: true
code_folding: hide
number_sections: true
includes:
  after_body: tabset-dropdown.html
---

  ```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
```

# R Markdown {.tabset .tabset-dropdown}

## Tab A

This is an R Markdown document. Markdown is a simple formatting syntax for authoring HTML, PDF, and MS Word documents. For more details on using R Markdown see <http://rmarkdown.rstudio.com>.

## Tab B
When you click the **Knit** button a document will be generated that includes both content as well as the output of any embedded R code chunks within the document. You can embed an R code chunk like this:

  ## Tab C
  ```{r cars}
summary(cars)
```

## Including Plots

You can also embed plots, for example:

  ```{r pressure, echo=FALSE}
plot(pressure)
```

Note that the `echo = FALSE` parameter was added to the code chunk to prevent printing of the R code that generated the plot.