问:
Deep level xml parsing to csv using NodeJS
I have a medium sized xml ~ 5mb that needs to be converted to csv.

Obviously wont go for reinventing the wheel,
so a two layer approach -
1> xml to json
2> json to csv

My current code is :

```<code>const xml_obj = {}
const htt = require('http-status-code-node');
var fs = require('fs');
var xml2js = require('xml2js');
var converter = require('json-2-csv');

xml_obj["convert"] = (req, res, next) => {
  var parser = new xml2js.Parser();
  fs.readFile(__dirname + '/directoryexport.xml', function (err, data) {
    parser.parseString(data, function (err, result) {
      console.log('Done');
      var callback = function (err, ycsv) {
        if (err) return console.log(err);
        ///
        res.setHeader('Content-Disposition', 'attachment; filename=testing.csv');
        res.set('Content-Type', 'text/csv');
        res.status(200).send(result);
        ///
      }
      var documents = [];
      documents.push(result)
      converter.json2csv(documents, callback);
    })
  });
}
module.exports = xml_obj.convert
</code>```

However the xml being nested gives a multi layered json which the yields a single string instead of a proper delimited csv..

The current output CSV

The Original xml

The XML structure

The Json I get on converting xml

Also as per the documentation of the json to csv converter
if the input json is in a proper structure like :

```<code>[
    {
        Make: 'Nissan',
        Model: 'Murano',
        Year: '2013',
        Specifications: {
            Mileage: '7106',
            Trim: 'S AWD'
        }
    },
    {
        Make: 'BMW',
        Model: 'X5',
        Year: '2014',
        Specifications: {
            Mileage: '3287',
            Trim: 'M'
        }
    }
];
</code>```

This yields a very nicely formatted csv like this : Example Perfect CSV From JSON

Edit 1 :
The format I'm looking for is somewhat like,
It's important to capture all parent organization and organizationalUnit details for each person node.
For example,

organizationalUnit UUID "b3b05b77-a8a7-43ed-ab74-b7d898c60296" should
  produce a CSV lines like:

```<code>"Mr Shayne Howard","Howard","Shayne","Mr","","Branch Manager","(02) 6121 5492","","Level 1, 12 Mort Street, Canberra, ACT, 2601","Shayne.Howard@employment.gov.au","","b43e0864-1b9a-40f0-8049-c90af5f9141c","","GPO Box 9880 CANBERRA ACT 2601 Australia",1392,"","Department of Employment","","1300 488 064","","","","http://www.employment.gov.au","GPO Box 9880, Canberra ACT 2601","EMPLOYMENT"
"Mr Luke de Jong","De Jong","Luke","Mr","","Branch Manager, General Counsel","(02) 6240 0909",""(02) 6123 5100"","","Luke.deJong@employment.gov.au","","58a503a8-ce8b-41c0-b690-b9f9efd98a89","","GPO Box 9880 CANBERRA ACT 2601",1393,"","Department of Employment","","1300 488 064","","","","http://www.employment.gov.au","GPO Box 9880, Canberra ACT 2601","EMPLOYMENT"
</code>```

Edit 2 :
Flattening out the json is a good idea, but its not capturing the entire data.
Using the camaro nodejs module  with the following template :

```<code>persons: ['//person', {
      root_organization_name: '../../../../name',
      main_organization_name: '../../../name',
      main_organization_website: '../../../website',
      fullName: 'fullName',
      familyName: 'familyName',
      firstName: 'firstName',
      personalTitle: 'personalTitle',
      title: 'title',
      person_phone: 'phone',
      person_location: 'location',
      person_fax: 'fax',
      otherRolesDN: 'otherRolesDN',
      person_mail: 'mail',
      informationPublicationScheme: '../informationPublicationScheme',
      publications: '../../publications',
      annualReport: '../../annualReport',
      mediaReleases: '../../mediaReleases',
      organizationUnit_1_name: '../../name',
      organizationUnit_1_description: '../../description',
      organizationUnit_1_location: '../../location',
      organizationUnit_1_phone: '../../phone',
      organizationUnit_1_fax: '../../fax',
      organizationUnit_1_website: '../../website',
      organizationUnit_2_name: '../name',
      organizationUnit_2_location: '../location',
      organizationUnit_2_phone: '../phone',
      organizationUnit_2_fax: '../fax',
      organizationUnit_2_website: '../website',
      occupantName: './role/occupantName',
      roleName: './role/roleName',
      occupantUUID: './role/occupantUUID',
      role_phone: './role/phone',
      role_fax: './role/fax',
      role_location: './role/location',
      role_mail: './role/ mail'
    }]
</code>```

How could I also get the roles array.
Also current csv gets some rows of data in wrong columns :

Wrong csv after camaro
Any tips on how to make this work with my input.

答:
Because the output of json structure is deep, if you want it to convert to csv properly, you would have to flatten it.

Seems like you only interest in deepest level. here's an example, if you want to add more data, feel free to add to the template

```<code>const transform = require('camaro')
const tocsv = require('json2csv')
const fs = require('fs')

const xml = fs.readFileSync('so.xml', 'utf-8')
const template = {
    persons: ['//person', {
        root_organization_name: '../../../../name',
        main_organization_name: '../../../name',
        main_organization_website: '../../../website',
        fullName: 'fullName',
        familyName: 'familyName',
        firstName: 'firstName',
        personalTitle: 'personalTitle',
        title: 'title',
        person_phone: 'phone',
        person_location: 'location',
        person_fax: 'fax',
        otherRolesDN: 'otherRolesDN',
        person_mail: 'mail',
        informationPublicationScheme: '../informationPublicationScheme',
        publications: '../../publications',
        annualReport: '../../annualReport',
        mediaReleases: '../../mediaReleases',
        organizationUnit_1_name: '../../name',
        organizationUnit_1_description: '../../description',
        organizationUnit_1_location: '../../location',
        organizationUnit_1_phone: '../../phone',
        organizationUnit_1_fax: '../../fax',
        organizationUnit_1_website: '../../website',
        organizationUnit_2_name: '../name',
        organizationUnit_2_location: '../location',
        organizationUnit_2_phone: '../phone',
        organizationUnit_2_fax: '../fax',
        organizationUnit_2_website: '../website',
        roles: ['../role', {
            occupantName: 'occupantName',
            roleName: 'roleName',
            occupantUUID: 'occupantUUID',
            role_phone: 'phone',
            role_fax: 'fax',
            role_location: 'location',
            role_mail: ' mail'
        }]
    }]
}

const result = transform(xml, template)
console.log(JSON.stringify(result.roles, null, 4))
</code>```

Example of output json (use json2csv to convert to csv if you want)

Body:
I have a medium sized xml ~ 5mb that needs to be converted to csv.

Obviously wont go for reinventing the wheel,
so a two layer approach -
1> xml to json
2> json to csv

My current code is :

const xml_obj = {}
const htt = require('http-status-code-node');
var fs = require('fs');
var xml2js = require('xml2js');
var converter = require('json-2-csv');

xml_obj["convert"] = (req, res, next) => {
  var parser = new xml2js.Parser();
  fs.readFile(__dirname + '/directoryexport.xml', function (err, data) {
    parser.parseString(data, function (err, result) {
      console.log('Done');
      var callback = function (err, ycsv) {
        if (err) return console.log(err);
        ///
        res.setHeader('Content-Disposition', 'attachment; filename=testing.csv');
        res.set('Content-Type', 'text/csv');
        res.status(200).send(result);
        ///
      }
      var documents = [];
      documents.push(result)
      converter.json2csv(documents, callback);
    })
  });
}
module.exports = xml_obj.convert

However the xml being nested gives a multi layered json which the yields a single string instead of a proper delimited csv..

The current output CSV

The Original xml

The XML structure

The Json I get on converting xml

Also as per the documentation of the json to csv converter
if the input json is in a proper structure like :

[
    {
        Make: 'Nissan',
        Model: 'Murano',
        Year: '2013',
        Specifications: {
            Mileage: '7106',
            Trim: 'S AWD'
        }
    },
    {
        Make: 'BMW',
        Model: 'X5',
        Year: '2014',
        Specifications: {
            Mileage: '3287',
            Trim: 'M'
        }
    }
];

This yields a very nicely formatted csv like this : Example Perfect CSV From JSON

Edit 1 :
The format I'm looking for is somewhat like,
It's important to capture all parent organization and organizationalUnit details for each person node.
For example,

organizationalUnit UUID "b3b05b77-a8a7-43ed-ab74-b7d898c60296" should
  produce a CSV lines like:

"Mr Shayne Howard","Howard","Shayne","Mr","","Branch Manager","(02) 6121 5492","","Level 1, 12 Mort Street, Canberra, ACT, 2601","Shayne.Howard@employment.gov.au","","b43e0864-1b9a-40f0-8049-c90af5f9141c","","GPO Box 9880 CANBERRA ACT 2601 Australia",1392,"","Department of Employment","","1300 488 064","","","","http://www.employment.gov.au","GPO Box 9880, Canberra ACT 2601","EMPLOYMENT"
"Mr Luke de Jong","De Jong","Luke","Mr","","Branch Manager, General Counsel","(02) 6240 0909",""(02) 6123 5100"","","Luke.deJong@employment.gov.au","","58a503a8-ce8b-41c0-b690-b9f9efd98a89","","GPO Box 9880 CANBERRA ACT 2601",1393,"","Department of Employment","","1300 488 064","","","","http://www.employment.gov.au","GPO Box 9880, Canberra ACT 2601","EMPLOYMENT"

Edit 2 :
Flattening out the json is a good idea, but its not capturing the entire data.
Using the camaro nodejs module  with the following template :

persons: ['//person', {
      root_organization_name: '../../../../name',
      main_organization_name: '../../../name',
      main_organization_website: '../../../website',
      fullName: 'fullName',
      familyName: 'familyName',
      firstName: 'firstName',
      personalTitle: 'personalTitle',
      title: 'title',
      person_phone: 'phone',
      person_location: 'location',
      person_fax: 'fax',
      otherRolesDN: 'otherRolesDN',
      person_mail: 'mail',
      informationPublicationScheme: '../informationPublicationScheme',
      publications: '../../publications',
      annualReport: '../../annualReport',
      mediaReleases: '../../mediaReleases',
      organizationUnit_1_name: '../../name',
      organizationUnit_1_description: '../../description',
      organizationUnit_1_location: '../../location',
      organizationUnit_1_phone: '../../phone',
      organizationUnit_1_fax: '../../fax',
      organizationUnit_1_website: '../../website',
      organizationUnit_2_name: '../name',
      organizationUnit_2_location: '../location',
      organizationUnit_2_phone: '../phone',
      organizationUnit_2_fax: '../fax',
      organizationUnit_2_website: '../website',
      occupantName: './role/occupantName',
      roleName: './role/roleName',
      occupantUUID: './role/occupantUUID',
      role_phone: './role/phone',
      role_fax: './role/fax',
      role_location: './role/location',
      role_mail: './role/ mail'
    }]

How could I also get the roles array.
Also current csv gets some rows of data in wrong columns :

Wrong csv after camaro
Any tips on how to make this work with my input.

Body:
Because the output of json structure is deep, if you want it to convert to csv properly, you would have to flatten it.

Seems like you only interest in deepest level. here's an example, if you want to add more data, feel free to add to the template

const transform = require('camaro')
const tocsv = require('json2csv')
const fs = require('fs')

const xml = fs.readFileSync('so.xml', 'utf-8')
const template = {
    persons: ['//person', {
        root_organization_name: '../../../../name',
        main_organization_name: '../../../name',
        main_organization_website: '../../../website',
        fullName: 'fullName',
        familyName: 'familyName',
        firstName: 'firstName',
        personalTitle: 'personalTitle',
        title: 'title',
        person_phone: 'phone',
        person_location: 'location',
        person_fax: 'fax',
        otherRolesDN: 'otherRolesDN',
        person_mail: 'mail',
        informationPublicationScheme: '../informationPublicationScheme',
        publications: '../../publications',
        annualReport: '../../annualReport',
        mediaReleases: '../../mediaReleases',
        organizationUnit_1_name: '../../name',
        organizationUnit_1_description: '../../description',
        organizationUnit_1_location: '../../location',
        organizationUnit_1_phone: '../../phone',
        organizationUnit_1_fax: '../../fax',
        organizationUnit_1_website: '../../website',
        organizationUnit_2_name: '../name',
        organizationUnit_2_location: '../location',
        organizationUnit_2_phone: '../phone',
        organizationUnit_2_fax: '../fax',
        organizationUnit_2_website: '../website',
        roles: ['../role', {
            occupantName: 'occupantName',
            roleName: 'roleName',
            occupantUUID: 'occupantUUID',
            role_phone: 'phone',
            role_fax: 'fax',
            role_location: 'location',
            role_mail: ' mail'
        }]
    }]
}

const result = transform(xml, template)
console.log(JSON.stringify(result.roles, null, 4))

Example of output json (use json2csv to convert to csv if you want)


来源: stackexchange

问:
Ruta not able to find white space
I want to rule to trigger on white space.
for sentence:"My name is Prabhav."

Here is my ruta code:

```<code> DECLARE Alert(STRING comment);

    (WS{->LOG("WS")} )
    {-> CREATE(Alert,1,2,3,"comment"="Space not allowed")};
</code>```

note:
I think code is syntactically ok because it works fine if I use "ANY"
Instead of "WS".

Thanks in advance !

答:
Whitespaces are not visible by default. You need to remove them from the filtered types, e.g., with RETAINTYPE:

```<code>DECLARE Alert(STRING comment);
RETAINTYPE(WS);
WS{->LOG("WS"), CREATE(Alert,"comment"="Space not allowed")};
</code>```

DISCLAIMER: I am a developer of UIMA Ruta

ContentLicense: CC BY-SA 3.0

Title: Ruta not able to find white space

Body:
I want to rule to trigger on white space.
for sentence:"My name is Prabhav."

Here is my ruta code:

 DECLARE Alert(STRING comment);

    (WS{->LOG("WS")} )
    {-> CREATE(Alert,1,2,3,"comment"="Space not allowed")};

note:
I think code is syntactically ok because it works fine if I use "ANY"
Instead of "WS".

Thanks in advance !

Body:
Whitespaces are not visible by default. You need to remove them from the filtered types, e.g., with RETAINTYPE:

DECLARE Alert(STRING comment);
RETAINTYPE(WS);
WS{->LOG("WS"), CREATE(Alert,"comment"="Space not allowed")};

DISCLAIMER: I am a developer of UIMA Ruta


问:
NotImplementedError: Use module Crypto.Cipher.PKCS1_OAEP instead error
I am trying to RSA encryption in Python. So I have a public/private key being generated, encrypting the message using the public key and writing the ciphertext to a text file. The code I am using is as follows:

```<code>from Crypto.PublicKey import RSA
from Crypto import Random
import ast

random_generator = Random.new().read
key = RSA.generate(1024, random_generator)

publickey = key.publickey()

encrypted = publickey.encrypt('encrypt this message', 32)

print('encrypted message:', encrypted)
f = open('encryption.txt', 'w')
f.write(str(encrypted))
f.close()

f = open('encryption.txt', 'r')
message = f.read()

decrypted = key.decrypt(ast.literal_eval(str(encrypted)))

print('decrypted', decrypted)

f = open('encryption.txt', 'w')
f.write(str(message))
f.write(str(decrypted))
f.close()
</code>```

But now when I run the application, I get the following error:

```<code>Traceback (most recent call last):
  File "C:/Users/RedCode/PycharmProjects/AdvancedApps/Encryption/RSA Example.py", line 10, in <module>
    encrypted = publickey.encrypt('encrypt this message', 32)
  File "C:\Users\RedCode\AppData\Local\Programs\Python\Python36-32\lib\site-packages\Crypto\PublicKey\RSA.py", line 390, in encrypt
    raise NotImplementedError("Use module Crypto.Cipher.PKCS1_OAEP instead")
NotImplementedError: Use module Crypto.Cipher.PKCS1_OAEP instead
</code>```

No matter how I try implementing Crypto.Cipher.PKCS1_OAEP, the error persists. I have tried import Crypto.Cipher.PKCS1_OAEP, from Crypto.Cipher.PKCS1_OAEP import RSA, from Crypto.Cipher.PKCS1_OAEP import Random, from Crypto.Cipher.PKCS1_OAEP import ast, and import Crypto.Cipher and none of those helped.

I tried from Crypto.Cipher.PKCS1_OAEP import RSA but then the error was:

```<code>Traceback (most recent call last):
  File "C:/Users/RedCode/PycharmProjects/AdvancedApps/Encryption/RSA Example.py", line 3, in <module>
    from Crypto.Cipher.PKCS1_OAEP import RSA
ImportError: cannot import name 'RSA'
</code>```

I checked my files and I do have the RSA package.

How can I correct this issue?

答:
You need to create an instance of PKCS1_OAEP using new, and use that to encrypt/decrypt your message.

```<code>from Crypto.Cipher import PKCS1_OAEP

encryptor = PKCS1_OAEP.new(publickey)
encrypted = encryptor.encrypt(b'encrypt this message')
</code>```

and the same for decryption

```<code>decryptor = PKCS1_OAEP.new(key)
decrypted = decryptor.decrypt(ast.literal_eval(str(encrypted)))
</code>```

Body:
I am trying to RSA encryption in Python. So I have a public/private key being generated, encrypting the message using the public key and writing the ciphertext to a text file. The code I am using is as follows:

from Crypto.PublicKey import RSA
from Crypto import Random
import ast

random_generator = Random.new().read
key = RSA.generate(1024, random_generator)

publickey = key.publickey()

encrypted = publickey.encrypt('encrypt this message', 32)

print('encrypted message:', encrypted)
f = open('encryption.txt', 'w')
f.write(str(encrypted))
f.close()

f = open('encryption.txt', 'r')
message = f.read()

decrypted = key.decrypt(ast.literal_eval(str(encrypted)))

print('decrypted', decrypted)

f = open('encryption.txt', 'w')
f.write(str(message))
f.write(str(decrypted))
f.close()

But now when I run the application, I get the following error:

Traceback (most recent call last):
  File "C:/Users/RedCode/PycharmProjects/AdvancedApps/Encryption/RSA Example.py", line 10, in <module>
    encrypted = publickey.encrypt('encrypt this message', 32)
  File "C:\Users\RedCode\AppData\Local\Programs\Python\Python36-32\lib\site-packages\Crypto\PublicKey\RSA.py", line 390, in encrypt
    raise NotImplementedError("Use module Crypto.Cipher.PKCS1_OAEP instead")
NotImplementedError: Use module Crypto.Cipher.PKCS1_OAEP instead

No matter how I try implementing Crypto.Cipher.PKCS1_OAEP, the error persists. I have tried import Crypto.Cipher.PKCS1_OAEP, from Crypto.Cipher.PKCS1_OAEP import RSA, from Crypto.Cipher.PKCS1_OAEP import Random, from Crypto.Cipher.PKCS1_OAEP import ast, and import Crypto.Cipher and none of those helped.

I tried from Crypto.Cipher.PKCS1_OAEP import RSA but then the error was:

Traceback (most recent call last):
  File "C:/Users/RedCode/PycharmProjects/AdvancedApps/Encryption/RSA Example.py", line 3, in <module>
    from Crypto.Cipher.PKCS1_OAEP import RSA
ImportError: cannot import name 'RSA'

I checked my files and I do have the RSA package.

How can I correct this issue?

Body:
You need to create an instance of PKCS1_OAEP using new, and use that to encrypt/decrypt your message.

from Crypto.Cipher import PKCS1_OAEP

encryptor = PKCS1_OAEP.new(publickey)
encrypted = encryptor.encrypt(b'encrypt this message')

and the same for decryption

decryptor = PKCS1_OAEP.new(key)
decrypted = decryptor.decrypt(ast.literal_eval(str(encrypted)))


问:
Indexing a CSV running into inconsistent number of samples for logistic regression
I'm currently indexing a CSV with values below and running into the error:

ValueError: Found input variables with inconsistent numbers of
  samples: [1, 514]

It's examining it as 1 row with 514 columns which emphasize that I have called a specific parameter wrong or is it due to me removing NaN's (which most of the data would default as?)

```<code>"Classification","DGMLEN","IPLEN","TTL","IP"
"1","0.000000","192.168.1.5","185.60.216.35","TLSv1.2"
"2","0.000160","192.168.1.5","185.60.216.35","TCP"
"3","0.000161","192.168.1.5","185.60.216.35","TLSv1.2"

import pandas
df = pandas.read_csv('wcdemo.csv', header=0,
                  names = ["Classification", "DGMLEN", "IPLEN", "TTL", "IP"],
                  na_values='.')

df = df.apply(pandas.to_numeric, errors='coerce')
#Data=pd.read_csv ('wcdemo.csv').reset_index()#index_col='false')
feature_cols=['Classification','DGMLEN','IPLEN','IP']

X=df[feature_cols]

    #datanewframe = pandas.Series(['Classification', 'DGMLEN', 'IPLEN', 'TTL', 'IP'], dtype='object')

#df = pandas.read_csv('wcdemo.csv')
#indexed_df = df.set_index(['Classification', 'DGMLEN','IPLEN','TTL','IP']

df['IPLEN'] = pandas.to_numeric(df['IPLEN'], errors='coerce').fillna(0)
df['TTL'] = pandas.to_numeric(df['TTL'], errors='coerce').fillna(0)

#DEFINE X TRAIN
X_train = df['IPLEN']
y_train = df['TTL']

#s = pandas.Series(['Classification', 'DGMLEN', 'IPLEN', 'TTL', 'IP'])

Y=df['TTL']

from sklearn.linear_model import LogisticRegression

logreg=LogisticRegression()
logreg.fit(X_train,y_train,).fillna(0.0)

#with the error being triggered here
logreg.fit(X_train,y_train,).fillna(0.0)
</code>```

答:
As there is only 1 feature in your X_train, its current shape is (n_samples,). But scikit estimators require X to be of shape (n_samples, n_features). So you need to reshape your data.

Use this:

logreg.fit(X_train.reshape(-1,1), y_train).fillna(0.0)

Body:
I'm currently indexing a CSV with values below and running into the error:

ValueError: Found input variables with inconsistent numbers of
  samples: [1, 514]

It's examining it as 1 row with 514 columns which emphasize that I have called a specific parameter wrong or is it due to me removing NaN's (which most of the data would default as?)

"Classification","DGMLEN","IPLEN","TTL","IP"
"1","0.000000","192.168.1.5","185.60.216.35","TLSv1.2"
"2","0.000160","192.168.1.5","185.60.216.35","TCP"
"3","0.000161","192.168.1.5","185.60.216.35","TLSv1.2"

import pandas
df = pandas.read_csv('wcdemo.csv', header=0,
                  names = ["Classification", "DGMLEN", "IPLEN", "TTL", "IP"],
                  na_values='.')

df = df.apply(pandas.to_numeric, errors='coerce')
#Data=pd.read_csv ('wcdemo.csv').reset_index()#index_col='false')
feature_cols=['Classification','DGMLEN','IPLEN','IP']

X=df[feature_cols]

    #datanewframe = pandas.Series(['Classification', 'DGMLEN', 'IPLEN', 'TTL', 'IP'], dtype='object')

#df = pandas.read_csv('wcdemo.csv')
#indexed_df = df.set_index(['Classification', 'DGMLEN','IPLEN','TTL','IP']

df['IPLEN'] = pandas.to_numeric(df['IPLEN'], errors='coerce').fillna(0)
df['TTL'] = pandas.to_numeric(df['TTL'], errors='coerce').fillna(0)

#DEFINE X TRAIN
X_train = df['IPLEN']
y_train = df['TTL']

#s = pandas.Series(['Classification', 'DGMLEN', 'IPLEN', 'TTL', 'IP'])

Y=df['TTL']

from sklearn.linear_model import LogisticRegression

logreg=LogisticRegression()
logreg.fit(X_train,y_train,).fillna(0.0)

#with the error being triggered here
logreg.fit(X_train,y_train,).fillna(0.0)


问:
How to convert List of Maps to CSV
I need to convert List of Maps to CSV object which should like below

```<code>List<Map<String,Object>> maps = new ArrayList<Map<String,Object>>();
</code>```

maps object contains the value in below format

map 1 :

```<code>(header 1, value1)

(header 2, value2)

(header 3, value3)

(header 4, value4)
</code>```

map 2 :

```<code>(header 1, value5)

(header 2, value6)

(header 3, value7)

(header 4, value8)
</code>```

I am looking for CSV which should like below

```<code>header1, header2,header3, header4

value1, value2,value3,value4

value5,value6,value7,value8
</code>```

I have tried to read the map (key, value) and writing to CSV files, but its writing to the format

header 1, value1

header2, value2

header 3,value3

header4,value4

header1,value5

like this below is the code snippet which I have tried for

```<code>(Map<String, Object> map : maps) {

 for (Map.Entry<String, Object> entry : map.entrySet()) {

w.append(entry.getKey()).append(",").append(entry.getValue().toString()).append("\n"); } }
</code>```

答:
The following code will help you. It's not ready for your purpose. You'll need to change it, so it prints to a file instead of returning a String.

```<code>private static String toCSV(List<Map<String, Object>> list) {
    List<String> headers = list.stream().flatMap(map -> map.keySet().stream()).distinct().collect(toList());
    final StringBuffer sb = new StringBuffer();
    for (int i = 0; i < headers.size(); i++) {
        sb.append(headers.get(i));
        sb.append(i == headers.size()-1 ? "\n" : ",");
    }
    for (Map<String, Object> map : list) {
        for (int i = 0; i < headers.size(); i++) {
            sb.append(map.get(headers.get(i)));
            sb.append(i == headers.size()-1 ? "\n" : ",");
        }
    }
    return sb.toString();
}
</code>```

For the following input, it gave the following output:

```<code>public static void main(String[] args) {
    List<Map<String, Object>> list = new ArrayList<>();
    Map<String, Object> map1 = new HashMap<>();
    map1.put("header1", "value1");
    map1.put("header2", "value2");
    map1.put("header3", "value3");
    map1.put("header4", "value4");
    Map<String, Object> map2 = new HashMap<>();
    map2.put("header1", "value5");
    map2.put("header2", "value6");
    map2.put("header3", "value7");
    map2.put("header4", "value8");
    list.add(map1);
    list.add(map2);
    System.out.println(toCSV(list));
}
</code>```

Such form is quite reversible, so you can write your own method to read from such CSV to a List<Map<String, String>

Another example, when key is not present in one of the maps:

Title: How to convert List of Maps to CSV

Body:
I need to convert List of Maps to CSV object which should like below

List<Map<String,Object>> maps = new ArrayList<Map<String,Object>>();

maps object contains the value in below format

map 1 :

(header 1, value1)

(header 2, value2)

(header 3, value3)

(header 4, value4)

map 2 :

(header 1, value5)

(header 2, value6)

(header 3, value7)

(header 4, value8)

I am looking for CSV which should like below

header1, header2,header3, header4

value1, value2,value3,value4

value5,value6,value7,value8

I have tried to read the map (key, value) and writing to CSV files, but its writing to the format

header 1, value1

header2, value2

header 3,value3

header4,value4

header1,value5

like this below is the code snippet which I have tried for

(Map<String, Object> map : maps) {

 for (Map.Entry<String, Object> entry : map.entrySet()) {

w.append(entry.getKey()).append(",").append(entry.getValue().toString()).append("\n"); } }

Body:
The following code will help you. It's not ready for your purpose. You'll need to change it, so it prints to a file instead of returning a String.

private static String toCSV(List<Map<String, Object>> list) {
    List<String> headers = list.stream().flatMap(map -> map.keySet().stream()).distinct().collect(toList());
    final StringBuffer sb = new StringBuffer();
    for (int i = 0; i < headers.size(); i++) {
        sb.append(headers.get(i));
        sb.append(i == headers.size()-1 ? "\n" : ",");
    }
    for (Map<String, Object> map : list) {
        for (int i = 0; i < headers.size(); i++) {
            sb.append(map.get(headers.get(i)));
            sb.append(i == headers.size()-1 ? "\n" : ",");
        }
    }
    return sb.toString();
}

For the following input, it gave the following output:

public static void main(String[] args) {
    List<Map<String, Object>> list = new ArrayList<>();
    Map<String, Object> map1 = new HashMap<>();
    map1.put("header1", "value1");
    map1.put("header2", "value2");
    map1.put("header3", "value3");
    map1.put("header4", "value4");
    Map<String, Object> map2 = new HashMap<>();
    map2.put("header1", "value5");
    map2.put("header2", "value6");
    map2.put("header3", "value7");
    map2.put("header4", "value8");
    list.add(map1);
    list.add(map2);
    System.out.println(toCSV(list));
}

Such form is quite reversible, so you can write your own method to read from such CSV to a List<Map<String, String>

Another example, when key is not present in one of the maps:


问:
VLOOKUP with a MINIF statement
I am trying return the value in column B based on the MIN IF function below.

```<code>=MIN(IF($A:$A=$A2,E:E))
</code>```

The formula for the MIN IF is fine. I have tried various functions but VLOOKUP is the closest I have come in order to return the corresponding column to the minimum value calculated. This is the closest I have come:

```<code>=VLOOKUP(MIN(IF($A:$A=$A2,E:E)),A:N,2,0)
</code>```

To help, the Min function returns the value in E4 (72.8) and and I need to display the value in B4(1-03).

Any help will be greatly appreciated.

答:
Your VLOOKUP won't work because it is trying to find 72.8 in column A, and failing. When the value you are searching for is not in the first column of your table, use INDEX and MATCH instead:

```<code>=INDEX($B:$B,MATCH(MIN(IF($A:$A=$A2,E:E)),$E:$E,0))
</code>```

As long as you don't have any duplicate minimum values in column E you'll be fine.

Title: VLOOKUP with a MINIF statement

Body:
I am trying return the value in column B based on the MIN IF function below.

=MIN(IF($A:$A=$A2,E:E))

The formula for the MIN IF is fine. I have tried various functions but VLOOKUP is the closest I have come in order to return the corresponding column to the minimum value calculated. This is the closest I have come:

=VLOOKUP(MIN(IF($A:$A=$A2,E:E)),A:N,2,0)

To help, the Min function returns the value in E4 (72.8) and and I need to display the value in B4(1-03).

Any help will be greatly appreciated.

Body:
Your VLOOKUP won't work because it is trying to find 72.8 in column A, and failing. When the value you are searching for is not in the first column of your table, use INDEX and MATCH instead:

=INDEX($B:$B,MATCH(MIN(IF($A:$A=$A2,E:E)),$E:$E,0))

As long as you don't have any duplicate minimum values in column E you'll be fine.


问:
Use of cut command in unix
Suppose I have string as follwing:

Rajat;Harshit Srivastava;Mayank 123;5

Now i want result as following using cut command
Rajat
Harshit Srivastava
Mayank 123
5
I have tried but cut is not working on string containing spaces.

答:
man cut would tell you:

```<code>   -d, --delimiter=DELIM
          use DELIM instead of TAB for field delimiter

   --output-delimiter=STRING
          use  STRING  as  the  output delimiter the default is to use the
          input delimiter
</code>```

If you insist on using cut for changing the delimiters:

```<code>$ echo "Rajat;Harshit Srivastava;Mayank 123;5" | cut -d \; --output-delimiter=\  -f 1-
Rajat Harshit Srivastava Mayank 123 5
</code>```

but instead you should use sed or tr or awk for it. Try man tr, for example.

Title: Use of cut command in unix

Body:
man cut would tell you:

   -d, --delimiter=DELIM
          use DELIM instead of TAB for field delimiter

   --output-delimiter=STRING
          use  STRING  as  the  output delimiter the default is to use the
          input delimiter

If you insist on using cut for changing the delimiters:

$ echo "Rajat;Harshit Srivastava;Mayank 123;5" | cut -d \; --output-delimiter=\  -f 1-
Rajat Harshit Srivastava Mayank 123 5

but instead you should use sed or tr or awk for it. Try man tr, for example.


问:
Combining multiple json and displaying it in Angular UI Grid
i am trying to get json from multiple source and want to display it in the single Angular UI grid. in all these source first field is common .

Data format :

Source :1

```<code>var one=[
      { col1: "Heading 1", col2: "First 12", col3: "First 13"},
      { col1: "Heading 2", col2: "First 22", col3: "First 23"},
      { col1: "Heading 3", col2: "First 32", col3: "First 33"},
      { col1: "Heading 4", col2: "First 42", col3: "First 43"},
      { col1: "Heading 5", col2: "First 52", col3: "First 53"}
    ];
</code>```

Source: 2

```<code>var two=[
      { col1: "Heading 1", col2: "Second 12", col3: "Second 13"},
      { col1: "Heading 2", col2: "Second 22", col3: "Second 23"},
      { col1: "Heading 3", col2: "Second 32", col3: "Second 33"},
      { col1: "Heading 4", col2: "Second 42", col3: "Second 43"},
      { col1: "Heading 5", col2: "Second 52", col3: "Second 53"}
    ];
</code>```

I want it to be displayed in gird like this

Source:

```<code>var app = angular.module('plunker', ['ui.grid', 'ui.grid.selection']);

app.controller('MainCtrl', ['$scope', 'uiGridConstants', function($scope, uiGridConstants) {

var one=[
      { col1: "Heading 1", col2: "First 12", col3: "First 13"},
      { col1: "Heading 2", col2: "First 22", col3: "First 23"},
      { col1: "Heading 3", col2: "First 32", col3: "First 33"},
      { col1: "Heading 4", col2: "First 42", col3: "First 43"},
      { col1: "Heading 5", col2: "First 52", col3: "First 53"}
    ]; 1
     1
var two=[
      { col1: "Heading 1", col2: "Second 12", col3: "Second 13"},
      { col1: "Heading 2", col2: "Second 22", col3: "Second 23"},
      { col1: "Heading 3", col2: "Second 32", col3: "Second 33"},
      { col1: "Heading 4", col2: "Second 42", col3: "Second 43"},
      { col1: "Heading 5", col2: "Second 52", col3: "Second 53"}
    ];

  var finalObj = {

    "firstData":one,
    "secondData":two
  }

  console.log("finalObj"+JSON.stringify(finalObj));

  $scope.gridOptions = {
    columnDefs: [
        {field: 'firstData.col1', displayName: 'Column 1', width: 175},
      {field: 'firstData.col2', displayName: 'Column 2', width: '*'},
      {field: 'firstData.col3', displayName: 'Column 3', width: 120},
      {field: 'secondData.col2', displayName: 'Column 4', width: '*'},
      {field: 'secondData.col3', displayName: 'Column 5', width: 120}
    ],
    enableRowSelection: true,
    onRegisterApi: function(gridApi) {
      $scope.gridApi = gridApi;
    }
    };
  $scope.gridOptions=finalObj;

}]);
</code>```

Code : see the attaced Plunkr http://plnkr.co/edit/ZUyJQInnygirvfohIhQo

答:
1) $scope.gridOptions=finalObj; needs to be changed to $scope.gridOptions.data=finalObj;

2) $scope.gridOptions.data needs an array.

Try this:

```<code>function merge(obj1, obj2){
    return obj1.map(function(o1){
      return Object.assign({}, o1, {
        col4: obj2.find(function(o2){ return o1.col1 == o2.col1 }).col2,
        col5: obj2.find(function(o2){ return o1.col1 == o2.col1 }).col3
      })
    })
}

var finalObj = merge(one, two);

$scope.gridOptions.data=finalObj;
</code>```

Plunker

Body:
i am trying to get json from multiple source and want to display it in the single Angular UI grid. in all these source first field is common .

Data format :

Source :1

var one=[
      { col1: "Heading 1", col2: "First 12", col3: "First 13"},
      { col1: "Heading 2", col2: "First 22", col3: "First 23"},
      { col1: "Heading 3", col2: "First 32", col3: "First 33"},
      { col1: "Heading 4", col2: "First 42", col3: "First 43"},
      { col1: "Heading 5", col2: "First 52", col3: "First 53"}
    ];

Source: 2

var two=[
      { col1: "Heading 1", col2: "Second 12", col3: "Second 13"},
      { col1: "Heading 2", col2: "Second 22", col3: "Second 23"},
      { col1: "Heading 3", col2: "Second 32", col3: "Second 33"},
      { col1: "Heading 4", col2: "Second 42", col3: "Second 43"},
      { col1: "Heading 5", col2: "Second 52", col3: "Second 53"}
    ];

I want it to be displayed in gird like this

Source:

var app = angular.module('plunker', ['ui.grid', 'ui.grid.selection']);

app.controller('MainCtrl', ['$scope', 'uiGridConstants', function($scope, uiGridConstants) {

var one=[
      { col1: "Heading 1", col2: "First 12", col3: "First 13"},
      { col1: "Heading 2", col2: "First 22", col3: "First 23"},
      { col1: "Heading 3", col2: "First 32", col3: "First 33"},
      { col1: "Heading 4", col2: "First 42", col3: "First 43"},
      { col1: "Heading 5", col2: "First 52", col3: "First 53"}
    ]; 1
     1
var two=[
      { col1: "Heading 1", col2: "Second 12", col3: "Second 13"},
      { col1: "Heading 2", col2: "Second 22", col3: "Second 23"},
      { col1: "Heading 3", col2: "Second 32", col3: "Second 33"},
      { col1: "Heading 4", col2: "Second 42", col3: "Second 43"},
      { col1: "Heading 5", col2: "Second 52", col3: "Second 53"}
    ];

  var finalObj = {

    "firstData":one,
    "secondData":two
  }

  console.log("finalObj"+JSON.stringify(finalObj));

  $scope.gridOptions = {
    columnDefs: [
        {field: 'firstData.col1', displayName: 'Column 1', width: 175},
      {field: 'firstData.col2', displayName: 'Column 2', width: '*'},
      {field: 'firstData.col3', displayName: 'Column 3', width: 120},
      {field: 'secondData.col2', displayName: 'Column 4', width: '*'},
      {field: 'secondData.col3', displayName: 'Column 5', width: 120}
    ],
    enableRowSelection: true,
    onRegisterApi: function(gridApi) {
      $scope.gridApi = gridApi;
    }
    };
  $scope.gridOptions=finalObj;

}]);

Code : see the attaced Plunkr http://plnkr.co/edit/ZUyJQInnygirvfohIhQo

Body:
1) $scope.gridOptions=finalObj; needs to be changed to $scope.gridOptions.data=finalObj;

2) $scope.gridOptions.data needs an array.

Try this:

function merge(obj1, obj2){
    return obj1.map(function(o1){
      return Object.assign({}, o1, {
        col4: obj2.find(function(o2){ return o1.col1 == o2.col1 }).col2,
        col5: obj2.find(function(o2){ return o1.col1 == o2.col1 }).col3
      })
    })
}

var finalObj = merge(one, two);

$scope.gridOptions.data=finalObj;

Plunker


问:
nginx rule to append additional segment after the url
I have a dynamic URL's as follows

```<code>http://www.xxxx.com/black/xxxx/xxxx
http://www.xxxx.com/blue/xxxx/xxxx
http://www.xxxx.com/red/xxxx/xxxx
</code>```

I want to do 301 redirect for the above url to

```<code>http://www.xxxx.com/color/black/xxxx/xxxx
http://www.xxxx.com/color/blue/xxxx/xxxx
http://www.xxxx.com/color/red/xxxx/xxxx
</code>```

I'm trying the matching like this

```<code>location ~  /(black|blue|red)(/(.+)?)?$ {

}
</code>```

Thanks in advcance!

答:
The location regular expression should include an anchor for the beginning of the URI.

If you are trying to match only URIs containing the suffix /xxxx/xxxx, you can use the character class [^/] to match non-slash characters.

The redirect is straightforward using a return statement:

```<code>location ~ ^/(black|blue|red)/[^/]+/[^/]+$ {
    return 301 /color$request_uri;
}
</code>```

The evaluation order of regular expression location blocks is significant. Place this block above other blocks with potentially conflicting rules.

See this document for details.

Body:
I have a dynamic URL's as follows

http://www.xxxx.com/black/xxxx/xxxx
http://www.xxxx.com/blue/xxxx/xxxx
http://www.xxxx.com/red/xxxx/xxxx

I want to do 301 redirect for the above url to

http://www.xxxx.com/color/black/xxxx/xxxx
http://www.xxxx.com/color/blue/xxxx/xxxx
http://www.xxxx.com/color/red/xxxx/xxxx

I'm trying the matching like this

location ~  /(black|blue|red)(/(.+)?)?$ {

}

Thanks in advcance!

Body:
The location regular expression should include an anchor for the beginning of the URI.

If you are trying to match only URIs containing the suffix /xxxx/xxxx, you can use the character class [^/] to match non-slash characters.

The redirect is straightforward using a return statement:

location ~ ^/(black|blue|red)/[^/]+/[^/]+$ {
    return 301 /color$request_uri;
}

The evaluation order of regular expression location blocks is significant. Place this block above other blocks with potentially conflicting rules.

See this document for details.


问:
Good way to sanitize input in classic asp
I have to update old projects at work. I do not have any experience with classic asp, although i'm familiar with php scripting.

Are there any functions I should use?

Can you provide me with a good function for some basic protection?

Is there something like a parameterized query in asp?

Thanks!

答: Watch out for SQL injection. Do not concatenate user input to a SQL string and then execute it. Instead, always used parameterized queries.</p>

OwnerDisplayName: Sander Versluys

ContentLicense: CC BY-SA 2.5

LastEditorDisplayName: Sander Versluys

OwnerDisplayName: OrbMan


问:
combine tables related to each other by a linking table and not to worry if the linking table is missing some data
I have a simple relational database set up similar to the example below

```<code>[Customer] --< [CGLink] >-- [Order]

Person:
Alias Name(PK)
A     Bill
B     Ben
C     Bob
D     Jim
E     John

CGLink:
ID(PK) Alias Type
1      A     W
2      A     X
3      B     W
4      B     X
5      B     Y
6      B     Z
7      C     Y
8      E     Z

Group:
Type(PK) Group
W        Double-U
X        Eks
Y        Whai
Z        Zed
</code>```

And I want to return a set of results like

```<code>Alias   Name   Group
A       Bill   Double-U
A       Bill   Eks
B       Ben    Double-U
B       Ben    Eks
B       Ben    Whai
B       Ben    Zed
C       Bob    Whai
D       Jim
E       John   Zed
</code>```

As you can see Jim had no associated data in the linking table and I want to avoid having to place a row of dummy data that I can link all people to who don't have a group in order to let me return this data.

The query I am using is

```<code>SELECT p.Alias, p.Name, g.Group
FROM Person AS p, Group AS s, CGLink AS l
WHERE (p.Alias=l.Alias
AND l.Type=s.Type)
ORDER BY p.Alias, p.Name;
</code>```

This returns

```<code>Alias   Name   Group
A       Bill   Double-U
A       Bill   Eks
B       Ben    Double-U
B       Ben    Eks
B       Ben    Whai
B       Ben    Zed
C       Bob    Whai
E       John   Zed
</code>```

Which as you can see, is missing Jim because he had no associated group.
How can I obtain the results I really want?

Cheers

答: left join</p>

Body:
I have a simple relational database set up similar to the example below

[Customer] --< [CGLink] >-- [Order]

Person:
Alias Name(PK)
A     Bill
B     Ben
C     Bob
D     Jim
E     John

CGLink:
ID(PK) Alias Type
1      A     W
2      A     X
3      B     W
4      B     X
5      B     Y
6      B     Z
7      C     Y
8      E     Z

Group:
Type(PK) Group
W        Double-U
X        Eks
Y        Whai
Z        Zed

And I want to return a set of results like

Alias   Name   Group
A       Bill   Double-U
A       Bill   Eks
B       Ben    Double-U
B       Ben    Eks
B       Ben    Whai
B       Ben    Zed
C       Bob    Whai
D       Jim
E       John   Zed

As you can see Jim had no associated data in the linking table and I want to avoid having to place a row of dummy data that I can link all people to who don't have a group in order to let me return this data.

The query I am using is

SELECT p.Alias, p.Name, g.Group
FROM Person AS p, Group AS s, CGLink AS l
WHERE (p.Alias=l.Alias
AND l.Type=s.Type)
ORDER BY p.Alias, p.Name;

This returns

Alias   Name   Group
A       Bill   Double-U
A       Bill   Eks
B       Ben    Double-U
B       Ben    Eks
B       Ben    Whai
B       Ben    Zed
C       Bob    Whai
E       John   Zed

Which as you can see, is missing Jim because he had no associated group.
How can I obtain the results I really want?

Cheers

Body: left join


问:
Can't obtain token from Reckon API - Returning SSL Error
I haven't been coding for many years so please forgive me.  I'm trying to obtain a token from the Reckon API using the following code, but getting this error returned:

"SSL certificate problem: unable to get local issuer certificate"

```<code><?php
$code= $_GET['code'];
$url = 'https://identity.reckon.com/connect/token';
$auth_creds = "CLIENTID:CLIENTSECRET";
$headers = array(
    "Content-Type: application/x-www-form-urlencoded",
    "Authorization: Basic" . base64_encode($auth_creds));
$body = 'grant_type=authorization_code&code=' . $code .
'&redirect_uri=http://localhost';
$ch = curl_init();
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_POSTFIELDS, $body);
$response = curl_exec($ch);
if($response === false)$response= curl_error($ch);
curl_close($ch);
echo $response;
?>
</code>```

I'm following the Reckon API oauth help docs: https://reckon.helpdocsonline.com/reckon-api-authorisation-services

答:
I managed to work it out.  I had to add this line of code:
    curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
However I was then getting another error "{"error":"invalid_client"}"...
This was due to encoding my client ID and client Secret incorrectly.  But now that is fixed and I have my token and refresh token.</p>

Body:
I haven't been coding for many years so please forgive me.  I'm trying to obtain a token from the Reckon API using the following code, but getting this error returned:

"SSL certificate problem: unable to get local issuer certificate"

<?php
$code= $_GET['code'];
$url = 'https://identity.reckon.com/connect/token';
$auth_creds = "CLIENTID:CLIENTSECRET";
$headers = array(
    "Content-Type: application/x-www-form-urlencoded",
    "Authorization: Basic" . base64_encode($auth_creds));
$body = 'grant_type=authorization_code&code=' . $code .
'&redirect_uri=http://localhost';
$ch = curl_init();
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_POSTFIELDS, $body);
$response = curl_exec($ch);
if($response === false)$response= curl_error($ch);
curl_close($ch);
echo $response;
?>

I'm following the Reckon API oauth help docs: https://reckon.helpdocsonline.com/reckon-api-authorisation-services


问:
Sequelize-Postgres error: values.map is not a function
I'm working on Node + Postgres app and I have trouble in setting the DataType.ARRAY using Sequelize model schema.

Here is my model schema,

```<code>var ArtistRoles = sequelize.define('ArtistRoles', {
    id:{
      primaryKey: true,
      type: DataTypes.UUID,
      defaultValue: DataTypes.UUIDV1
    },
    name:{
      type: DataTypes.STRING,
      allowNull: false
    },
    tags: {
      type: DataTypes.ARRAY(DataTypes.STRING),
      defaultValue: []
    }
  },{
    hooks:{
      beforeCreate: function (role, options) {
        //role.tags = JSON.stringify(role.tags);
        console.log(role.tags);
      }
    },
    classMethods: {
      associate: function (models) {
        // add relations model relations here
        //ArtistRoles.belongsTo(models.Artists);
      }
    }
  });
</code>```

I used Postman to test the API and here is how I POST the data to server,

Here is the Error trace on Node console.

```<code>TypeError: values.map is not a function
    at ARRAY.BaseTypes.ARRAY.$stringify (~/PROJECTS/my-api/node_modules/sequelize/lib/dialects/postgres/data-types.js:385:33)
    at ARRAY.ABSTRACT.stringify (~/PROJECTS/my-api/node_modules/sequelize/lib/data-types.js:77:17)
    at Object.escape (~/PROJECTS/my-api/node_modules/sequelize/lib/dialects/abstract/query-generator.js:967:32)
    at Object.insertQuery (~/PROJECTS/my-api/node_modules/sequelize/lib/dialects/abstract/query-generator.js:299:28)
    at QueryInterface.insert (~/PROJECTS/my-api/node_modules/sequelize/lib/query-interface.js:497:33)
    at .<anonymous> (~/PROJECTS/my-api/node_modules/sequelize/lib/instance.js:672:56)
    at tryCatcher (~/PROJECTS/my-api/node_modules/bluebird/js/release/util.js:16:23)
    at Promise._settlePromiseFromHandler (~/PROJECTS/my-api/node_modules/bluebird/js/release/promise.js:510:31)
    at Promise._settlePromise (~/PROJECTS/my-api/node_modules/bluebird/js/release/promise.js:567:18)
    at Promise._settlePromise0 (~/PROJECTS/my-api/node_modules/bluebird/js/release/promise.js:612:10)
    at Promise._settlePromises (~/PROJECTS/my-api/node_modules/bluebird/js/release/promise.js:691:18)
    at Async._drainQueue (~/PROJECTS/my-api/node_modules/bluebird/js/release/async.js:138:16)
    at Async._drainQueues (~/PROJECTS/my-api/node_modules/bluebird/js/release/async.js:148:10)
    at Immediate.Async.drainQueues (~/PROJECTS/my-api/node_modules/bluebird/js/release/async.js:17:14)
    at runCallback (timers.js:637:20)
    at tryOnImmediate (timers.js:610:5)
POST /api/1.0/role 500 46.114 ms - 45
</code>```

What could be the reason? Please help.

答:
For application/x-www-form-urlencoded, the body of the HTTP message sent to the server is essentially one giant query string.

So, the value of tags will be in string format when received in backend.
Convert it back into Array type in backend and you should be good to go

Body:
I'm working on Node + Postgres app and I have trouble in setting the DataType.ARRAY using Sequelize model schema.

Here is my model schema,

var ArtistRoles = sequelize.define('ArtistRoles', {
    id:{
      primaryKey: true,
      type: DataTypes.UUID,
      defaultValue: DataTypes.UUIDV1
    },
    name:{
      type: DataTypes.STRING,
      allowNull: false
    },
    tags: {
      type: DataTypes.ARRAY(DataTypes.STRING),
      defaultValue: []
    }
  },{
    hooks:{
      beforeCreate: function (role, options) {
        //role.tags = JSON.stringify(role.tags);
        console.log(role.tags);
      }
    },
    classMethods: {
      associate: function (models) {
        // add relations model relations here
        //ArtistRoles.belongsTo(models.Artists);
      }
    }
  });

I used Postman to test the API and here is how I POST the data to server,

Here is the Error trace on Node console.

TypeError: values.map is not a function
    at ARRAY.BaseTypes.ARRAY.$stringify (~/PROJECTS/my-api/node_modules/sequelize/lib/dialects/postgres/data-types.js:385:33)
    at ARRAY.ABSTRACT.stringify (~/PROJECTS/my-api/node_modules/sequelize/lib/data-types.js:77:17)
    at Object.escape (~/PROJECTS/my-api/node_modules/sequelize/lib/dialects/abstract/query-generator.js:967:32)
    at Object.insertQuery (~/PROJECTS/my-api/node_modules/sequelize/lib/dialects/abstract/query-generator.js:299:28)
    at QueryInterface.insert (~/PROJECTS/my-api/node_modules/sequelize/lib/query-interface.js:497:33)
    at .<anonymous> (~/PROJECTS/my-api/node_modules/sequelize/lib/instance.js:672:56)
    at tryCatcher (~/PROJECTS/my-api/node_modules/bluebird/js/release/util.js:16:23)
    at Promise._settlePromiseFromHandler (~/PROJECTS/my-api/node_modules/bluebird/js/release/promise.js:510:31)
    at Promise._settlePromise (~/PROJECTS/my-api/node_modules/bluebird/js/release/promise.js:567:18)
    at Promise._settlePromise0 (~/PROJECTS/my-api/node_modules/bluebird/js/release/promise.js:612:10)
    at Promise._settlePromises (~/PROJECTS/my-api/node_modules/bluebird/js/release/promise.js:691:18)
    at Async._drainQueue (~/PROJECTS/my-api/node_modules/bluebird/js/release/async.js:138:16)
    at Async._drainQueues (~/PROJECTS/my-api/node_modules/bluebird/js/release/async.js:148:10)
    at Immediate.Async.drainQueues (~/PROJECTS/my-api/node_modules/bluebird/js/release/async.js:17:14)
    at runCallback (timers.js:637:20)
    at tryOnImmediate (timers.js:610:5)
POST /api/1.0/role 500 46.114 ms - 45

What could be the reason? Please help.


答:
what about the following perl one-liner?

```<code>perl -ne '/(.*?)\s+(.*)/; print "$1\t$2\n"' your_input_file
</code>```

Title: Squeezing spaces between columns in Unix shell

Body:
I want the spaces to be removed between two columns.
After running a sql query from shell, I'm getting the output as below:

23554402243                       0584940772;2TZ0584940772001U;
23554402272                       0423721840;7TT0423721840001B;
23554402303                       0110770863;BBTU500248822001Q;
23554402305                       02311301;BTB02311301001J;
23554402563                       0550503408;PPTU004984208001O;
23554402605                       0457553223;Q0T0457553223001I;
23554367602                       0454542427;TB8U501674990001V;
23554378584                       0383071261;HTHU500374797001Y;
23554404965                       059792244;ST3059792244005C;
23554405503                       0571632586;QTO0571632586001D;

But the desired output should be like below:

23554400043     0117601738;22TU003719388001V;
23554402883     0823973229;TTT0823973229001C;
23554402950     024071080;MNT024071080001D;
23554405827     0415260614;TL20415260614001R;
23554405828     08119270800;TL2U003010407001G;
23554406553     011306895;VBT011306895001E;
23554406557     054121509;TL2054121509001M;
23554406563     065069209;TL2065069209005M;
23554409085     0803434328;QTO0803434328001B;
23553396219     062004063;G6T062004063001C;

Remember, there should be only one tabspace between two columns in the desired output.

Body:
what about the following perl one-liner?

perl -ne '/(.*?)\s+(.*)/; print "$1\t$2\n"' your_input_file


问:
How to get all the single number from [number -(to) number] with comma-seperated in a column in sql server
column of data in sqlserver like
    numbers
    1000-1050, 1054, 1090-1230, 1245

```<code>numbers
-------
1000-1050, 1054, 1090-1230, 1245
</code>```

how to get the style like below:

```<code>numbers
-------
1000
1001
1002
1003
1004
...
1050
1054
1090
1091
1092
...
1245
</code>```

答:
Create a split function:

```<code>CREATE FUNCTION [dbo].[SplitIDsTest]
(
    @idList varchar(4000)
)
RETURNS @parsedList TABLE
(
    Id varchar(50),
    Nr int
)
AS
BEGIN

    DECLARE @id varchar(10), @pos int
    DECLARE @nr int = 0;

    SET @idList = LTRIM(RTRIM(@idList)) + ','
    SET @Pos = CHARINDEX(',', @idList)

    IF REPLACE(@idList, ',', '') <> ''
        BEGIN
            WHILE @Pos > 0
                BEGIN
                    SET @id = LTRIM(RTRIM(LEFT(@idList, @pos - 1)))

                    IF @id <> ''
                    BEGIN
                        set @nr += 1;
                        INSERT INTO @ParsedList (Id, Nr)
                        VALUES (@id, @nr);
                    END

                    SET @idList = RIGHT(@idList, LEN(@idList) - @pos) -- 'WMPC,' (inklusive Komma) links vom Eingabeparameter abschneiden, weil jetzt der nächste Wert gesucht wird
                    SET @pos = CHARINDEX(',', @idList, 1)             -- Nächste Position eines Kommas suchen und in der WHILE-Schleife weitermachen
                END
        END

    RETURN
END
</code>```

Then a tally table:

```<code>select top 1000000 N=identity(int, 1, 1)
  into dbo.Tally
  from master.dbo.syscolumns a cross join master.dbo.syscolumns b;
</code>```

Then use:

```<code>select distinct Tally.N
  from SplitIDsTest('10-12, 34, 9') splitted
  join Tally on 1 = case
                     when CHARINDEX('-', splitted.Id) > 0 then
                      case
                       when Tally.N between cast(left(splitted.Id, CHARINDEX('-', splitted.Id) - 1) as int)
                                        and cast(right(splitted.Id, len(splitted.Id) - CHARINDEX('-', splitted.Id)) as int) then 1
                       else 0
                      end
                     when Tally.N = cast(splitted.Id as int) then 1
                     else 0
                    end
 order by Tally.N
</code>```

Body:
column of data in sqlserver like
    numbers
    1000-1050, 1054, 1090-1230, 1245

numbers
-------
1000-1050, 1054, 1090-1230, 1245

how to get the style like below:

numbers
-------
1000
1001
1002
1003
1004
...
1050
1054
1090
1091
1092
...
1245

Body:
Create a split function:

CREATE FUNCTION [dbo].[SplitIDsTest]
(
    @idList varchar(4000)
)
RETURNS @parsedList TABLE
(
    Id varchar(50),
    Nr int
)
AS
BEGIN

    DECLARE @id varchar(10), @pos int
    DECLARE @nr int = 0;

    SET @idList = LTRIM(RTRIM(@idList)) + ','
    SET @Pos = CHARINDEX(',', @idList)

    IF REPLACE(@idList, ',', '') <> ''
        BEGIN
            WHILE @Pos > 0
                BEGIN
                    SET @id = LTRIM(RTRIM(LEFT(@idList, @pos - 1)))

                    IF @id <> ''
                    BEGIN
                        set @nr += 1;
                        INSERT INTO @ParsedList (Id, Nr)
                        VALUES (@id, @nr);
                    END

                    SET @idList = RIGHT(@idList, LEN(@idList) - @pos) -- 'WMPC,' (inklusive Komma) links vom Eingabeparameter abschneiden, weil jetzt der nächste Wert gesucht wird
                    SET @pos = CHARINDEX(',', @idList, 1)             -- Nächste Position eines Kommas suchen und in der WHILE-Schleife weitermachen
                END
        END

    RETURN
END

Then a tally table:

select top 1000000 N=identity(int, 1, 1)
  into dbo.Tally
  from master.dbo.syscolumns a cross join master.dbo.syscolumns b;

Then use:

select distinct Tally.N
  from SplitIDsTest('10-12, 34, 9') splitted
  join Tally on 1 = case
                     when CHARINDEX('-', splitted.Id) > 0 then
                      case
                       when Tally.N between cast(left(splitted.Id, CHARINDEX('-', splitted.Id) - 1) as int)
                                        and cast(right(splitted.Id, len(splitted.Id) - CHARINDEX('-', splitted.Id)) as int) then 1
                       else 0
                      end
                     when Tally.N = cast(splitted.Id as int) then 1
                     else 0
                    end
 order by Tally.N


问:
Slow execution of customized is_element_present
I have in method body of another method:

```<code>for i in range(60):
    try:
        if sel.is_element_present("//div[@id='result']/form[3]/strong/div/button"): break
    except: pass
    time.sleep(1)
</code>```

and it executes in 5 seconds.

Nothing changes on site and I execute this line:

```<code>self.WaitForElement(u"//div[@id='result']/form[3]/strong/div/button")

def WaitForElement(self,name):
    for i in range(60):
        try:
            if sel.is_element_present(name): break
        except: pass
        time.sleep(1)
</code>```

and it executes in almost 30 seconds, so it's very weird.

Do you have any idea?

答: Looking quickly over your code, the only thing I can see is you defined a standard string in the first example but a unicode string in the second.</p>

Body:
I have in method body of another method:

for i in range(60):
    try:
        if sel.is_element_present("//div[@id='result']/form[3]/strong/div/button"): break
    except: pass
    time.sleep(1)

and it executes in 5 seconds.

Nothing changes on site and I execute this line:

self.WaitForElement(u"//div[@id='result']/form[3]/strong/div/button")

def WaitForElement(self,name):
    for i in range(60):
        try:
            if sel.is_element_present(name): break
        except: pass
        time.sleep(1)

and it executes in almost 30 seconds, so it's very weird.

Do you have any idea?


问:
Java.lang.nullpointer exception in junit testing
My code for junit is

```<code>@Autowired
private ABC abc;

@Test
public void checkZonePresent() {
    assertThat(abc.getZoneList().size()).isEqualTo(1);
}
</code>```

SO, here I am getting error like java.lang.nullpointer exception

答:
At first glance abc.getZoneList() is null. But also NPE  might if you didn't configured tests with spring (didn't ). It's difficult to say without code , show pls how you configured test class

zero step (in your case it helps , but in general you don't need to test it if you configured spring tests) :

assertThat(abc ,is(notNullValue());

First you should check that  :

assertThat(abc.getZoneList(),is(notNullValue());

and looks like ,abc.getZoneList() ,it's a problem
the next check should be

assertThat(abc.getZoneList().size()).isEqualTo(1)

your test fail and there is no explanation why, but test should explains what's going wrong

Body:
My code for junit is

@Autowired
private ABC abc;

@Test
public void checkZonePresent() {
    assertThat(abc.getZoneList().size()).isEqualTo(1);
}

SO, here I am getting error like java.lang.nullpointer exception


问:
cant open image in android studio (file not found)
I'm using android studio and I have some pics located here: ..\app\src\main\res\pics

I have this method to open images in the adapter java class

```<code>private Drawable drawImage(String iconString) {
    Log.i(TAG , "icon string = " + iconString) ;
    AssetManager assetManager=ctx.getAssets();
    InputStream inputStream;

    try {
        inputStream=assetManager.open("pics/"+iconString+".png");
        Log.i(TAG , "icon string try = " + iconString) ;
        Drawable drawable=Drawable.createFromStream(inputStream, null);
        return  drawable ;

    } catch (Exception e) {
        e.printStackTrace();
    }
    return  null;

}
</code>```

I did invoke the method
but when I build and run it I get the following error:

```<code>java.io.FileNotFoundException: pics/ic_payments.png
06-08 10:10:44.577 8002-8002/ ... W/System.err:
at android.content.res.AssetManager.openAsset(Native Method)
</code>```

What did I miss here?

答:
Android code "ic_launcher" replace with your image name

1.Create asset folder and

create pics folder(inside of your asset folder)

paste the image in pics folder

```<code>ImageView imageview;
</code>```

2.Inside of Oncreate

```<code> imageview=(ImageView)findViewById(R.id.imageview);
 imageview.setImageDrawable(drawImage("ic_launcher"));
</code>```

3,Outside of Oncreate

```<code>private Drawable drawImage(String iconString) {
    Log.i(TAG , "icon string = " + iconString) ;
    AssetManager assetManager=getAssets();
    InputStream inputStream;

    try {
        inputStream=assetManager.open("pics/"+iconString+".png");
        Log.i(TAG , "icon string try = " + iconString) ;
        Drawable drawable=Drawable.createFromStream(inputStream, null);
        return  drawable ;

    } catch (Exception e) {
        e.printStackTrace();
    }
    return  null;

}
</code>```

Body:
I'm using android studio and I have some pics located here: ..\app\src\main\res\pics

I have this method to open images in the adapter java class

private Drawable drawImage(String iconString) {
    Log.i(TAG , "icon string = " + iconString) ;
    AssetManager assetManager=ctx.getAssets();
    InputStream inputStream;

    try {
        inputStream=assetManager.open("pics/"+iconString+".png");
        Log.i(TAG , "icon string try = " + iconString) ;
        Drawable drawable=Drawable.createFromStream(inputStream, null);
        return  drawable ;

    } catch (Exception e) {
        e.printStackTrace();
    }
    return  null;

}

I did invoke the method
but when I build and run it I get the following error:

java.io.FileNotFoundException: pics/ic_payments.png
06-08 10:10:44.577 8002-8002/ ... W/System.err:
at android.content.res.AssetManager.openAsset(Native Method)

What did I miss here?

Body:
Android code "ic_launcher" replace with your image name

1.Create asset folder and

create pics folder(inside of your asset folder)

paste the image in pics folder

ImageView imageview;

2.Inside of Oncreate

 imageview=(ImageView)findViewById(R.id.imageview);
 imageview.setImageDrawable(drawImage("ic_launcher"));

3,Outside of Oncreate

private Drawable drawImage(String iconString) {
    Log.i(TAG , "icon string = " + iconString) ;
    AssetManager assetManager=getAssets();
    InputStream inputStream;

    try {
        inputStream=assetManager.open("pics/"+iconString+".png");
        Log.i(TAG , "icon string try = " + iconString) ;
        Drawable drawable=Drawable.createFromStream(inputStream, null);
        return  drawable ;

    } catch (Exception e) {
        e.printStackTrace();
    }
    return  null;

}


问:
Querying using DynamoDB vs AWS Lambda
I am creating an iOS app that will query my database without doing any object manipulation or calculation, I can do this in several ways:

Query directly using DynamoDB iOS SDK

Query using AWS Lambda iOS SDK, and then executing a lambda function that queries DynamoDB directly.

My question is, which is more execution-time and cost efficient?

My thoughts are that option 1 is the best as it does not require a function or any execution time.

I know that I am charged for the DynamoDB query and I know that I am charged for the execution time of the lambda function. However, I am not sure if I will be charged for using the DynamoDB SDK method or for something else or if I might forgetting another possible cost or thing that I am not taking into account.

Thank you.

答:
Your thought is correct, executing the query using DynamoDB IOS SDK is better for execution time and cost.

Additionally you will be charged for data transfer (for both approaches).

However its not advisable to store the Access Keys & Secrets inside IOS app, where you need to consider using temporary security credentials possibly with AWS Cognito.

Title: Querying using DynamoDB vs AWS Lambda


问:
List all Kafka 0.10 topics using - -zookeeper flag without access to Zookeeper
I'm using kafka 0.10 without zookeeper. I want to get kafka topics list. This command is not working since we're not using zookeeper:
bin/kafka-topics.sh --list --zookeeper localhost:2181. How can I get the same output without zookeeper?

答:
Kafka is a distributed system and needs Zookeeper.
you have to start zookeeper too.
Follow "Quick Start" here : https://kafka.apache.org/0100/documentation.html#quickstart</p>

ContentLicense: CC BY-SA 4.0


答:
Assuming you need to remove space between all the columns:

If you need tab spaced result between first two columns. Put g to apply changes between all the columns.

```<code>sed -r 's/\s+/\t/' inputfile
</code>```

if -r is not available:

```<code>sed 's/\s\+/\t/'
</code>```

or  If you need single space between every multi-space

tr -s ' '

Body:
Assuming you need to remove space between all the columns:

If you need tab spaced result between first two columns. Put g to apply changes between all the columns.

sed -r 's/\s+/\t/' inputfile

if -r is not available:

sed 's/\s\+/\t/'

or  If you need single space between every multi-space

tr -s ' '


问:
Having multiple versions of Rails installed on OS X
OS X 10.6 has Rails 2.2.2 installed by default.

How to upgrade to Rails 2.3.8?

Since Rails 3 is out, is it possible to have multiple versions of Rails (like 2.3.8 and 3) installed on OS X?

(For example, I could be working on two Rails projects, one is Rails 2.x and the other is Rails 3.x).

答:
To upgrade your OS version of the rails gem:

```<code>sudo gem install rails --version 2.3.8
</code>```

While you're at it, you can also upgrade your OS rubygems system itself, overwriting the binary that OS X comes with. This might seem kind of hacky, but it's a stable thing that everyone does.

```<code>sudo gem install rubygems-update
sudo update_rubygems
</code>```

Regarding managing rails versions in your projects, if your only needs are:

Specifying the version of rails that you want on a rails project

Specifying different versions of rails and/or other gems in each of multiple rails projects

Then rvm is overkill. The only thing you need, and what is also a complete and utter pleasure to work with, is Bundler. Bundler is like Software Update for your rails project. In fact, Rails 3 comes with Bundler by default, you just have to list the gems you need in Gemfile and it takes care of the rest. You don't need to worry about which versions of your gems are installed in the OS.

(You do however need the appropriate version of the rails gem installed in the OS in order to generate the rails project in the first place)

Body:
To upgrade your OS version of the rails gem:

sudo gem install rails --version 2.3.8

While you're at it, you can also upgrade your OS rubygems system itself, overwriting the binary that OS X comes with. This might seem kind of hacky, but it's a stable thing that everyone does.

sudo gem install rubygems-update
sudo update_rubygems

Regarding managing rails versions in your projects, if your only needs are:

Specifying the version of rails that you want on a rails project

Specifying different versions of rails and/or other gems in each of multiple rails projects

Then rvm is overkill. The only thing you need, and what is also a complete and utter pleasure to work with, is Bundler. Bundler is like Software Update for your rails project. In fact, Rails 3 comes with Bundler by default, you just have to list the gems you need in Gemfile and it takes care of the rest. You don't need to worry about which versions of your gems are installed in the OS.

(You do however need the appropriate version of the rails gem installed in the OS in order to generate the rails project in the first place)


问:
Loading large number of items in recycler view
I have a recycler view within a fragment and basically I m trying to load song list in the recycler view .Each row of recycler view contains an imageview (for album art) and textview ( for song name). I am having trouble when the size of the dataset is huge, that is when there are too many songs, the recycler view lags and the app ends up giving an ANR.I am using Glide to load album arts in each row's imageview.
How is google music player able to show such large number of songs without any lag?

Edit:
This is my SongsFragment

```<code>public class SongsFragment extends Fragment {
static {
    AppCompatDelegate.setCompatVectorFromResourcesEnabled(true);
}
ProgressBar progressBar;    // progress bar to show after every 30 items
NestedScrollView nestedScrollView;  //for smooth scrolling of recyclerview as well as to detect the end of recyclerview
RecyclerView recyclerView;
ArrayList<Song> songMainList = new ArrayList<>();  //partial list in which items are added
ArrayList<Song> songAllList = new ArrayList<>(); //Complete List of songs
SongAdapter songsAdapter;
private LinearLayoutManager layoutManager;

@Override
public View onCreateView(LayoutInflater inflater, ViewGroup container, Bundle savedInstanceState) {
    View rootView = inflater.inflate(R.layout.fragment_songs, container, false);

    nestedScrollView = (NestedScrollView) rootView.findViewById(R.id.nestedScrollView);
    progressBar = (ProgressBar) rootView.findViewById(R.id.progressBar);

    String songJson = getActivity().getIntent().getStringExtra("songList");
    songAllList = new Gson().fromJson(songJson, new TypeToken<ArrayList<Song>>() {
    }.getType());
            //Getting list of all songs in songAllList

    if (songAllList.size() > 30) {
        songMainList = new ArrayList<>(songAllList.subList(0,30));
    } else {
        songMainList = songAllList;
    }

    //if size of fetched songAllList>30 then add only 30 rows to songMainList

    recyclerView = (RecyclerView) rootView.findViewById(R.id.songs);
    int spanCount = 1; // 2 columns
    int spacing = 4; // 50px
    recyclerView.addItemDecoration(new GridItemDecoration(spanCount, spacing, true));
    recyclerView.setHasFixedSize(true);
    recyclerView.setNestedScrollingEnabled(false);
    recyclerView.setLayoutManager(new LinearLayoutManager(getActivity()));
    songsAdapter = new SongAdapter(getActivity(), songMainList, recyclerView);

    nestedScrollView.getViewTreeObserver().addOnScrollChangedListener(new ViewTreeObserver.OnScrollChangedListener() {
        @Override
        public void onScrollChanged() {
            View view = (View) nestedScrollView.getChildAt(nestedScrollView.getChildCount() - 1);
            int diff = (view.getBottom() - (nestedScrollView.getHeight() + nestedScrollView
                    .getScrollY()));

            if (diff == 0) {    //NestedScrollView scrolled to bottom
                progressBar.setVisibility(View.VISIBLE);    //show progressbar
                new Handler().postDelayed(new Runnable() {
                    @Override
                    public void run() {
                        if (songMainList.size() < songAllList.size()) {
                            int x = 0, y = 0;
                            if ((songAllList.size() - songMainList.size()) >= 30) {
                                x = songMainList.size();
                                y = x + 30;
                            } else {
                                x = songMainList.size();
                                y = x + songAllList.size() - songMainList.size();
                            }
                            for (int i = x; i < y; i++) {
                                songMainList.add(songAllList.get(i));   //Adding new items from songAllList to songMainList one by one
                                songsAdapter.notifyDataSetChanged();
                            }
                        }
                        progressBar.setVisibility(View.GONE);
                    }
                }, 1500);

            }
        }
    });
    recyclerView.setAdapter(songsAdapter);

    return rootView;
    }
}
</code>```

And this is my RecyclerViewAdapter along with viewholder

```<code>public class SongAdapter extends RecyclerView.Adapter {

private List<Song> songsList;
private Context c;

private RecyclerView.ViewHolder holder;

public SongAdapter(Context context) {
    mainActivityContext = context;
}

public SongAdapter(Context context, List<Song> songs, RecyclerView recyclerView) {
    songsList = songs;
    LinearLayoutManager linearLayoutManager = (LinearLayoutManager) recyclerView.getLayoutManager();
    c = context;
}

public SongAdapter getInstance() {
    return SongAdapter.this;
}

@Override
public RecyclerView.ViewHolder onCreateViewHolder(ViewGroup parent, int viewType) {
    View view = LayoutInflater.from(parent.getContext()).inflate(R.layout.song_list_row, parent, false);
    return new SongViewHolder(view,c);
}

@Override
public void onBindViewHolder(final RecyclerView.ViewHolder holder, int position) {

    if (holder instanceof SongViewHolder) {
        Song song = songsList.get(position);
        this.holder = holder;

        String name = song.getName();
        String artist = song.getArtist();
        String imagepath = song.getImagepath();

        ((SongViewHolder) holder).name.setText(name);

        ((SongViewHolder) holder).artist.setText(artist);

        if (!imagepath.equalsIgnoreCase("no_image")) //if the album art has valid  imagepath for this song

            Glide.with(c).load(imagepath)
                    .centerCrop()
                    .into(((SongViewHolder) holder).iv);
        else
            ((SongViewHolder) holder).iv.setImageResource(R.drawable.empty);

        ((SongViewHolder) holder).song = song;
    }
}

@Override
public int getItemCount() {
    return songsList.size();
}

static class SongViewHolder extends RecyclerView.ViewHolder{

    ImageView iv;
    TextView name, artist;
    CardView songListCard;
    private Context ctx;

    private OnLongPressListener mListener;

    SongViewHolder(View v, Context context) {
        super(v);
        this.ctx = context;
        iv= (ImageView) v.findViewById(R.id.album_art);
        name= (TextView) v.findViewById(R.id.name);
        artist= (TextView) v.findViewById(R.id.artist_mini);
        songListCard = (CardView) v.findViewById(R.id.song_list_card);
    }
}
</code>```

The recyclerview works fine when there are only 150-200 items but when reaching to 600-700 items , the whole app slows down. Could this be because of the way I have used glide in onBindViewHolder?

答:
Solved the problem by removing the NestedScrollView over the recyclerview.
The nestedscrollview was not allowing the recyclerview.addOnScrollListener() to be called because of which I was getting a lag on loading more items.
Here is how i implemented the loadOnScroll for RecyclerView-</p>

```<code> recyclerView.addOnScrollListener(new RecyclerView.OnScrollListener() {
        @Override
        public void onScrolled(RecyclerView recyclerView, int dx, int dy) {
            super.onScrolled(recyclerView, dx, dy);
            if (!recyclerView.canScrollVertically(1))
                onScrolledToBottom();
        }
    });

private void onScrolledToBottom() {
    if (songMainList.size() < songAllList.size()) {
        int x, y;
        if ((songAllList.size() - songMainList.size()) >= 50) {
            x = songMainList.size();
            y = x + 50;
        } else {
            x = songMainList.size();
            y = x + songAllList.size() - songMainList.size();
        }
        for (int i = x; i < y; i++) {
            songMainList.add(songAllList.get(i));
        }
        songsAdapter.notifyDataSetChanged();
    }

}
</code>```

Body:
I have a recycler view within a fragment and basically I m trying to load song list in the recycler view .Each row of recycler view contains an imageview (for album art) and textview ( for song name). I am having trouble when the size of the dataset is huge, that is when there are too many songs, the recycler view lags and the app ends up giving an ANR.I am using Glide to load album arts in each row's imageview.
How is google music player able to show such large number of songs without any lag?

Edit:
This is my SongsFragment

public class SongsFragment extends Fragment {
static {
    AppCompatDelegate.setCompatVectorFromResourcesEnabled(true);
}
ProgressBar progressBar;    // progress bar to show after every 30 items
NestedScrollView nestedScrollView;  //for smooth scrolling of recyclerview as well as to detect the end of recyclerview
RecyclerView recyclerView;
ArrayList<Song> songMainList = new ArrayList<>();  //partial list in which items are added
ArrayList<Song> songAllList = new ArrayList<>(); //Complete List of songs
SongAdapter songsAdapter;
private LinearLayoutManager layoutManager;

@Override
public View onCreateView(LayoutInflater inflater, ViewGroup container, Bundle savedInstanceState) {
    View rootView = inflater.inflate(R.layout.fragment_songs, container, false);

    nestedScrollView = (NestedScrollView) rootView.findViewById(R.id.nestedScrollView);
    progressBar = (ProgressBar) rootView.findViewById(R.id.progressBar);

    String songJson = getActivity().getIntent().getStringExtra("songList");
    songAllList = new Gson().fromJson(songJson, new TypeToken<ArrayList<Song>>() {
    }.getType());
            //Getting list of all songs in songAllList

    if (songAllList.size() > 30) {
        songMainList = new ArrayList<>(songAllList.subList(0,30));
    } else {
        songMainList = songAllList;
    }

    //if size of fetched songAllList>30 then add only 30 rows to songMainList

    recyclerView = (RecyclerView) rootView.findViewById(R.id.songs);
    int spanCount = 1; // 2 columns
    int spacing = 4; // 50px
    recyclerView.addItemDecoration(new GridItemDecoration(spanCount, spacing, true));
    recyclerView.setHasFixedSize(true);
    recyclerView.setNestedScrollingEnabled(false);
    recyclerView.setLayoutManager(new LinearLayoutManager(getActivity()));
    songsAdapter = new SongAdapter(getActivity(), songMainList, recyclerView);

    nestedScrollView.getViewTreeObserver().addOnScrollChangedListener(new ViewTreeObserver.OnScrollChangedListener() {
        @Override
        public void onScrollChanged() {
            View view = (View) nestedScrollView.getChildAt(nestedScrollView.getChildCount() - 1);
            int diff = (view.getBottom() - (nestedScrollView.getHeight() + nestedScrollView
                    .getScrollY()));

            if (diff == 0) {    //NestedScrollView scrolled to bottom
                progressBar.setVisibility(View.VISIBLE);    //show progressbar
                new Handler().postDelayed(new Runnable() {
                    @Override
                    public void run() {
                        if (songMainList.size() < songAllList.size()) {
                            int x = 0, y = 0;
                            if ((songAllList.size() - songMainList.size()) >= 30) {
                                x = songMainList.size();
                                y = x + 30;
                            } else {
                                x = songMainList.size();
                                y = x + songAllList.size() - songMainList.size();
                            }
                            for (int i = x; i < y; i++) {
                                songMainList.add(songAllList.get(i));   //Adding new items from songAllList to songMainList one by one
                                songsAdapter.notifyDataSetChanged();
                            }
                        }
                        progressBar.setVisibility(View.GONE);
                    }
                }, 1500);

            }
        }
    });
    recyclerView.setAdapter(songsAdapter);

    return rootView;
    }
}

And this is my RecyclerViewAdapter along with viewholder

public class SongAdapter extends RecyclerView.Adapter {

private List<Song> songsList;
private Context c;

private RecyclerView.ViewHolder holder;

public SongAdapter(Context context) {
    mainActivityContext = context;
}

public SongAdapter(Context context, List<Song> songs, RecyclerView recyclerView) {
    songsList = songs;
    LinearLayoutManager linearLayoutManager = (LinearLayoutManager) recyclerView.getLayoutManager();
    c = context;
}

public SongAdapter getInstance() {
    return SongAdapter.this;
}

@Override
public RecyclerView.ViewHolder onCreateViewHolder(ViewGroup parent, int viewType) {
    View view = LayoutInflater.from(parent.getContext()).inflate(R.layout.song_list_row, parent, false);
    return new SongViewHolder(view,c);
}

@Override
public void onBindViewHolder(final RecyclerView.ViewHolder holder, int position) {

    if (holder instanceof SongViewHolder) {
        Song song = songsList.get(position);
        this.holder = holder;

        String name = song.getName();
        String artist = song.getArtist();
        String imagepath = song.getImagepath();

        ((SongViewHolder) holder).name.setText(name);

        ((SongViewHolder) holder).artist.setText(artist);

        if (!imagepath.equalsIgnoreCase("no_image")) //if the album art has valid  imagepath for this song

            Glide.with(c).load(imagepath)
                    .centerCrop()
                    .into(((SongViewHolder) holder).iv);
        else
            ((SongViewHolder) holder).iv.setImageResource(R.drawable.empty);

        ((SongViewHolder) holder).song = song;
    }
}

@Override
public int getItemCount() {
    return songsList.size();
}

static class SongViewHolder extends RecyclerView.ViewHolder{

    ImageView iv;
    TextView name, artist;
    CardView songListCard;
    private Context ctx;

    private OnLongPressListener mListener;

    SongViewHolder(View v, Context context) {
        super(v);
        this.ctx = context;
        iv= (ImageView) v.findViewById(R.id.album_art);
        name= (TextView) v.findViewById(R.id.name);
        artist= (TextView) v.findViewById(R.id.artist_mini);
        songListCard = (CardView) v.findViewById(R.id.song_list_card);
    }
}

The recyclerview works fine when there are only 150-200 items but when reaching to 600-700 items , the whole app slows down. Could this be because of the way I have used glide in onBindViewHolder?

Body:
Solved the problem by removing the NestedScrollView over the recyclerview.
The nestedscrollview was not allowing the recyclerview.addOnScrollListener() to be called because of which I was getting a lag on loading more items.
Here is how i implemented the loadOnScroll for RecyclerView-

 recyclerView.addOnScrollListener(new RecyclerView.OnScrollListener() {
        @Override
        public void onScrolled(RecyclerView recyclerView, int dx, int dy) {
            super.onScrolled(recyclerView, dx, dy);
            if (!recyclerView.canScrollVertically(1))
                onScrolledToBottom();
        }
    });

private void onScrolledToBottom() {
    if (songMainList.size() < songAllList.size()) {
        int x, y;
        if ((songAllList.size() - songMainList.size()) >= 50) {
            x = songMainList.size();
            y = x + 50;
        } else {
            x = songMainList.size();
            y = x + songAllList.size() - songMainList.size();
        }
        for (int i = x; i < y; i++) {
            songMainList.add(songAllList.get(i));
        }
        songsAdapter.notifyDataSetChanged();
    }

}


问:
Storing birthday in database, what type to use if there's missing values
I am collecting sports data for a database I will be creating soon, and one of the columns I have for one of my tables is player birthdays. I am using R to scrape and collect the data, and here is what it currently looks like:

```<code>head(my_df)

        Name        BirthDate     Age                 Place  Height  Weight
1       Mike                       NA                            NA      NA
2     Austin  August 17, 1994  22.295 Roseville, California      73     210
3       Koby January 19, 1997  20.140      Wolfforth, Texas      70     255
4         Ty             1991  26.000       Amarillo, Texas      70     165
5       Cole                       NA                            NA      NA
6       Jeff    July 24, 1995  21.320     Boulder, Colorado      72     200

dput(head(my_df))
structure(list(Name = c("Mike", "Austin", "Koby", "Ty", "Cole",
"Jeff"), BirthDate = c("", "August 17, 1994", "January 19, 1997",
"1991", "", "July 24, 1995"), Age = c(NA, 22.295, 20.14, 26,
NA, 21.32), Place = c("", "Roseville, California", "Wolfforth, Texas",
"Amarillo, Texas", "", "Boulder, Colorado"), Height = c(NA, 73,
70, 70, NA, 72), Weight = c(NA, 210L, 255L, 165L, NA, 200L)), .Names = c("Name",
"BirthDate", "Age", "Place", "Height", "Weight"), row.names = 25:30, class = "data.frame")
</code>```

The BirthDate column has two inconsistencies that make appropriate formatting difficult:

there are entirely missing values

for some columns, I have the birthyear but am missing Day and Month.

I'm relatively new with databases and am trying to plan ahead with this. Does anybody have any thoughts on the best way to handle this? To be more specific, how can I write the R code to format the column the right way?

答:
If we intend to do calculations on date as date, or on year and month columns, for example: the age of the player, how many of them were born on January, etc. then we can can separate it into 3 numeric columns.</p>

```<code>do.call(rbind,
        lapply(my_df$BirthDate, function(i){
          x <- rev(unlist(strsplit(i, " ")))
          res <- data.frame(
            BirthDate = i,
            YYYY = as.numeric(x[1]),
            DD = as.numeric(sub(",", "", x[2], fixed = TRUE)),
            MM = match(x[3], month.name))
          res$DOB <- as.Date(paste(res$DD, res$MM, res$YYYY, sep = "/"),
                             format = "%d/%m/%Y")
          res
        }))

#          BirthDate YYYY DD MM        DOB
# 1                    NA NA NA       <NA>
# 2  August 17, 1994 1994 17  8 1994-08-17
# 3 January 19, 1997 1997 19  1 1997-01-19
# 4             1991 1991 NA NA       <NA>
# 5                    NA NA NA       <NA>
# 6    July 24, 1995 1995 24  7 1995-07-24
</code>```

Body:
I am collecting sports data for a database I will be creating soon, and one of the columns I have for one of my tables is player birthdays. I am using R to scrape and collect the data, and here is what it currently looks like:

head(my_df)

        Name        BirthDate     Age                 Place  Height  Weight
1       Mike                       NA                            NA      NA
2     Austin  August 17, 1994  22.295 Roseville, California      73     210
3       Koby January 19, 1997  20.140      Wolfforth, Texas      70     255
4         Ty             1991  26.000       Amarillo, Texas      70     165
5       Cole                       NA                            NA      NA
6       Jeff    July 24, 1995  21.320     Boulder, Colorado      72     200

dput(head(my_df))
structure(list(Name = c("Mike", "Austin", "Koby", "Ty", "Cole",
"Jeff"), BirthDate = c("", "August 17, 1994", "January 19, 1997",
"1991", "", "July 24, 1995"), Age = c(NA, 22.295, 20.14, 26,
NA, 21.32), Place = c("", "Roseville, California", "Wolfforth, Texas",
"Amarillo, Texas", "", "Boulder, Colorado"), Height = c(NA, 73,
70, 70, NA, 72), Weight = c(NA, 210L, 255L, 165L, NA, 200L)), .Names = c("Name",
"BirthDate", "Age", "Place", "Height", "Weight"), row.names = 25:30, class = "data.frame")

The BirthDate column has two inconsistencies that make appropriate formatting difficult:

there are entirely missing values

for some columns, I have the birthyear but am missing Day and Month.

I'm relatively new with databases and am trying to plan ahead with this. Does anybody have any thoughts on the best way to handle this? To be more specific, how can I write the R code to format the column the right way?

Body:
If we intend to do calculations on date as date, or on year and month columns, for example: the age of the player, how many of them were born on January, etc. then we can can separate it into 3 numeric columns.

do.call(rbind,
        lapply(my_df$BirthDate, function(i){
          x <- rev(unlist(strsplit(i, " ")))
          res <- data.frame(
            BirthDate = i,
            YYYY = as.numeric(x[1]),
            DD = as.numeric(sub(",", "", x[2], fixed = TRUE)),
            MM = match(x[3], month.name))
          res$DOB <- as.Date(paste(res$DD, res$MM, res$YYYY, sep = "/"),
                             format = "%d/%m/%Y")
          res
        }))

#          BirthDate YYYY DD MM        DOB
# 1                    NA NA NA       <NA>
# 2  August 17, 1994 1994 17  8 1994-08-17
# 3 January 19, 1997 1997 19  1 1997-01-19
# 4             1991 1991 NA NA       <NA>
# 5                    NA NA NA       <NA>
# 6    July 24, 1995 1995 24  7 1995-07-24


问:
Thin grey lines at top and bottom of ios app
I'm trying to make a cordova app with a completely black background. I've got pretty much all the way, but I just can't figure out what is causing 2 thin grey bars top and bottom of the screen. After the LaunchImage is displayed, I see a brief plain grey background, and then the webview is overlaid. But it is 4 pixels 'shorter' (2 at the top, and 2 at the bottom). I can't figure out where this is coming from.

I've added the cordova-plugin-webviewcolor plugin, and set that to black. I can see that's working because on rotation, and on overscroll, the 'revealed' backround is indeed black (or any other colour I set with that plugin). But I still see the 2 grey lines between my app and the 'background'.

Here's a screenshot - I've overlaid the actual screenshot onto a slightly larger red canvas, so you can see the grey more easily. On the phone, there's no red border, just the thin lines top and bottom of the screen. Any pointers as to where this comes from, and if it's possible to remove?

答: Thanks to @filipvkovic for the pointers on debugging. This turned out to be an empty 'header' and 'footer' div that the web framework (VueJS and Quasar) I'm using inserted. They are invisible on desktop browsers, but show up for some reason when baked into an app. So, recoded the layout to avoid them, and all is good.</p>


问:
How to delete Image from NSMutableArray and UI in objective C
I am new in iOS and I am facing problem regarding to delete image from mutable array and also from UIImage.

To add an image in array my code is like this:

In connectionDidFinishLoading

```<code>   imageArray =[[NSMutableArray alloc]init];
    for (int i=0; i<idarray.count; i++) {
        [imageArray addObject:[UIImage new]];
    }

- (void)imagePickerController:(UIImagePickerController *)picker didFinishPickingImage:(UIImage *)image editingInfo:(NSDictionary *)editingInfo
{
    CGSize newSizeClient=CGSizeMake(200,200); // I am giving resolution 50*50 , you can change your need
    UIGraphicsBeginImageContext(newSizeClient);
    [image drawInRect:CGRectMake(0, 0, newSizeClient.width, newSizeClient.height)];
    UIImage* newImageClient = UIGraphicsGetImageFromCurrentImageContext();
    UIGraphicsEndImageContext();

    captureimg.image = newImageClient;

    [imageArray replaceObjectAtIndex:index withObject:captureimg.image];

    dispatch_async(dispatch_get_main_queue(), ^{
        [Audittable reloadData];
    });

    NSData *imgData = [[NSData alloc] initWithData:UIImageJPEGRepresentation((newImageClient), 0.5)];
    OCSSignString = [imgData base64EncodedStringWithOptions:NSDataBase64Encoding64CharacterLineLength];

    [picker dismissModalViewControllerAnimated:YES];

    imageView = [[UIImageView alloc] initWithFrame:CGRectMake(20, 30,40,40)];
    UIImage *wonImage = captureimg.image;
    imageView.contentMode=UIViewContentModeCenter;
    [imageView setImage:wonImage];
}

</code>```

In cellForRowAtIndexPath

```<code>cell.CaptureImage.image = [imageArray objectAtIndex:indexPath.row];
</code>```

When deleting I am using code like this

```<code>[imageArray removeObjectAtIndex:index];
</code>```

But it crashes. I am using a custom TableView.

How can I remove image from mutable array?

More code:

CellForrowAtIndexPath

```<code>cell.btnRemove.tag=indexPath.row;
 [cell.btnRemove addTarget:self action:@selector(btnRemoveClick:) forControlEvents:UIControlEventTouchUpInside];
</code>```

On Button Click

```<code>-(void)btnRemoveClick:(UIButton *)sender
{
    AuditNextTableViewCell *cell = sender.superview.superview;
    // NSLog(@"%@",cell.actuallbl.text);

    NSIndexPath *indexPath = [Audittable indexPathForCell:cell];
    CurrentIndexPath=indexPath.row;

    UIAlertView *Alertview =[[UIAlertView alloc] initWithTitle:@"Warning!" message:@"Are You Sure you want to delete Image" delegate:self cancelButtonTitle:@"NO" otherButtonTitles:@"YES", nil];
    CheckStringShow=@"7";
    [Alertview show];
}

- (void)alertView:(UIAlertView *)alertView didDismissWithButtonIndex:(NSInteger)buttonIndex {
            [imageArray removeObjectAtIndex:CurrentIndexPath];
            [Audittable reloadData];
}
</code>```

It is removing current object only means last added object in array. I want to remove image at current index.

答: After removing the image from the mutable array you have to reload the data of the table view. ;)</p>

Body:
I am new in iOS and I am facing problem regarding to delete image from mutable array and also from UIImage.

To add an image in array my code is like this:

In connectionDidFinishLoading

   imageArray =[[NSMutableArray alloc]init];
    for (int i=0; i<idarray.count; i++) {
        [imageArray addObject:[UIImage new]];
    }

- (void)imagePickerController:(UIImagePickerController *)picker didFinishPickingImage:(UIImage *)image editingInfo:(NSDictionary *)editingInfo
{
    CGSize newSizeClient=CGSizeMake(200,200); // I am giving resolution 50*50 , you can change your need
    UIGraphicsBeginImageContext(newSizeClient);
    [image drawInRect:CGRectMake(0, 0, newSizeClient.width, newSizeClient.height)];
    UIImage* newImageClient = UIGraphicsGetImageFromCurrentImageContext();
    UIGraphicsEndImageContext();

    captureimg.image = newImageClient;

    [imageArray replaceObjectAtIndex:index withObject:captureimg.image];

    dispatch_async(dispatch_get_main_queue(), ^{
        [Audittable reloadData];
    });

    NSData *imgData = [[NSData alloc] initWithData:UIImageJPEGRepresentation((newImageClient), 0.5)];
    OCSSignString = [imgData base64EncodedStringWithOptions:NSDataBase64Encoding64CharacterLineLength];

    [picker dismissModalViewControllerAnimated:YES];

    imageView = [[UIImageView alloc] initWithFrame:CGRectMake(20, 30,40,40)];
    UIImage *wonImage = captureimg.image;
    imageView.contentMode=UIViewContentModeCenter;
    [imageView setImage:wonImage];
}

In cellForRowAtIndexPath

cell.CaptureImage.image = [imageArray objectAtIndex:indexPath.row];

When deleting I am using code like this

[imageArray removeObjectAtIndex:index];

But it crashes. I am using a custom TableView.

How can I remove image from mutable array?

More code:

CellForrowAtIndexPath

cell.btnRemove.tag=indexPath.row;
 [cell.btnRemove addTarget:self action:@selector(btnRemoveClick:) forControlEvents:UIControlEventTouchUpInside];

On Button Click

-(void)btnRemoveClick:(UIButton *)sender
{
    AuditNextTableViewCell *cell = sender.superview.superview;
    // NSLog(@"%@",cell.actuallbl.text);

    NSIndexPath *indexPath = [Audittable indexPathForCell:cell];
    CurrentIndexPath=indexPath.row;

    UIAlertView *Alertview =[[UIAlertView alloc] initWithTitle:@"Warning!" message:@"Are You Sure you want to delete Image" delegate:self cancelButtonTitle:@"NO" otherButtonTitles:@"YES", nil];
    CheckStringShow=@"7";
    [Alertview show];
}

- (void)alertView:(UIAlertView *)alertView didDismissWithButtonIndex:(NSInteger)buttonIndex {
            [imageArray removeObjectAtIndex:CurrentIndexPath];
            [Audittable reloadData];
}

It is removing current object only means last added object in array. I want to remove image at current index.