问:
How to detect android contact updated and sync to firestore
I am trying to get all the android contacts which been updated.

I am saving on firebase the last contact id i added and the last updated timestamp

I am using the next function to get back a cursor of all the updated contacts to compare with firebase server

```<code>private fun getUpdatedContacts(): Cursor? {

    val projection = arrayOf(
            ContactsContract.Contacts._ID,
            ContactsContract.Contacts.DISPLAY_NAME,
            ContactsContract.Contacts.HAS_PHONE_NUMBER,
            ContactsContract.Contacts.CONTACT_LAST_UPDATED_TIMESTAMP)

    val selection = ContactsContract.Contacts.CONTACT_LAST_UPDATED_TIMESTAMP + " > ? AND " +
            ContactsContract.Contacts._ID + "<= ?"

    val selectionArgs = arrayOf(mFireContactDetails!!.lcu_ms.toString(), mFireContactDetails!!.lcid.toString())

    val sortOrder = ContactsContract.Contacts.CONTACT_LAST_UPDATED_TIMESTAMP + " ASC"

    return mContentResolver.query(
            ContactsContract.Contacts.CONTENT_URI,
            projection,
            selection,
            selectionArgs,
            sortOrder)
}
</code>```

But when i change one contact in my phone this cursor is returned MANY un-related contacts that i never used and mark them as changed. Last time when i just added a phone number to an existing contact, i got back from this cursor more than 50 contacts as been updated.

What is going on Android?? I am trying to sync contacts for the past 3 months now. Why is it so hard???

答:
I been testing this solution for couple of days and it seems OK but i think i need to test it much more. If you using this method, do your own testing and above all, PLEASE LET ME KNOW IF I MISSED ANYTHING and don't be hurry to downgrade. Thx!

I built an App class that extents Application and implements the
ActivityLifecycleCallbacks. In which i create a ContactSync class for the
first time and activate it everytime the app goes to foregound

In ContactSync class, i am using Kotlin withContext(Dispatchers.IO) to suspend any code for easier flow

I use .get() to get all the contacts from firestore related to current user

at the .get() addOnSuccessListener, i add all the contacts to a HashMap with the normalized phone number as key and name + firestore id as values (using internal class)

While making the HashMap i also make sure there are no duplicates on firestore with smae phone number and if so delete them (using batch)

i then retrieve all the contacts from android phone. I sort them by NORMALIZED_NUMBER first and DISPLAY_NAME (will explain later)

I am now creating a batchArray with index and count to avoid exceeding the 500 limit

I start scanning through the contacts cursor,

I first get the normalized number, if not available (null) i create it my self using a function i made (it might be that a null value is only returned for phone numbers not in correct format, not sure)

I then compare the normalized number with previous cursor value. If the same i ignore it to avoid duplicates in firestore (remember the cursor is sorted by NORMALIZED_NUMBER)

I then check if the normalized number already in HashMap.

If in HashMap: i compare the name in HashMap to the cursor name. if different, i conclude the name was changed and i update the firestore contact in batch array (remember to increment counter and if exceeds 500 increase index). I then remove the normalized number from the HashMap to avoid its deletion later

If not in HashMap: i conclude the contact is new and i add it to firestore via batch

I iterate through all the cursor until completed.

When cursor complete i close it

Any remaining records found in HashMap are ones that were not found on firestore hence deleted. I iterate and delete them using batch

sync is done on the phone side

Now, since making the actual sync needs access to all users, i user firebase functions in node. I create 2 functions:

function that fires when new user is created (signed via phone)

function that fires when new contact document is created.

Both functions compare the users to the normalized number in document and if matching, writing the uid of that user to the firestore document "friend_uid" field.

Note you might have errors in these functions if you try to use them in free firebase plan. I suggest changing to Blaze plan and limit the charging to couple of dollars. By changing to Blaze, google also gives you free extras and avoid actual payment

By that, the sync is completed. The sync takes only couple of seconds

To display all the contacts which are users to the app, query all user contacts with "friend_uid" that are not null.

Some extra notes:

The .get() will retrieve all the contacts every time a sync is made. That might be a lot of reads if user has couple of hundreds contacts. To minimize, i use .get(Source.DEFAULT) when launching the app and .get(Source.CACHE) for the other times. Since these documents name and number only modified by user, i believe it will not be a problem most of the times (still testing)

To minimize the sync process as much as possible, i initiate it only if any contact changed its timestamp. I save the last timestamp to SharedPreferences and compare it. I found it mostly saves sync when app re-opened fast.

I also save the last user logged in. If any change in user, i re-initialize the current user contacts

Some source code (still testing, please let me know if any error):

```<code>private fun getContacts(): Cursor? {
    val projection = arrayOf(
            ContactsContract.CommonDataKinds.Phone._ID,
            ContactsContract.CommonDataKinds.Phone.NUMBER,
            ContactsContract.CommonDataKinds.Phone.NORMALIZED_NUMBER,
            ContactsContract.CommonDataKinds.Phone.DISPLAY_NAME,
            ContactsContract.CommonDataKinds.Phone.CONTACT_LAST_UPDATED_TIMESTAMP)

    //sort by NORMALIZED_NUMBER to detect duplicates and then by name to keep order and avoiding name change
    val sortOrder = ContactsContract.CommonDataKinds.Phone.NORMALIZED_NUMBER + " ASC, " +
            ContactsContract.CommonDataKinds.Phone.DISPLAY_NAME + " ASC"

    return mContentResolver.query(
            ContactsContract.CommonDataKinds.Phone.CONTENT_URI,
            projection,
            null,
            null,
            sortOrder)
}

    private suspend fun syncContactsAsync() = withContext(Dispatchers.IO)  {

    if (isAnythingChanged() || mFirstRun) {

        if (getValues() == Result.SUCCESS) {
            myPrintln("values retrieved success")
        } else {
            myPrintln("values retrieved failed. Aborting.")
            return@withContext
        }

        val cursor: Cursor? = getContacts()

        if (cursor == null) {
            myPrintln("cursor cannot be null")
            mFireContactHashMap.clear()
            return@withContext
        }

        if (cursor.count == 0) {
            cursor.close()
            mFireContactHashMap.clear()
            myPrintln("cursor empty")
            return@withContext
        }

        var contactName: String?
        var internalContact: InternalContact?
        val batchArray = mutableListOf(FirebaseFirestore.getInstance().batch())
        var batchIndex = 0
        var batchCount = 0
        var normalizedNumber:String?
        var prevNumber = ""
        var firestoreId: String

        while (cursor.moveToNext()) {

            normalizedNumber = cursor.getString(COLUMN_UPDATED_NORMALIZED_NUMBER)

            if (normalizedNumber == null) {
                normalizedNumber = cursor.getString(COLUMN_UPDATED_PHONE_NUMBER)
                normalizedNumber = Phone.getParsedPhoneNumber(mDeviceCountryIso,normalizedNumber,mContext)
            }

            //cursor sorted by normalized numbers so if same as previous, do not check
            if (normalizedNumber != prevNumber) {

                prevNumber = normalizedNumber

                contactName = cursor.getString(COLUMN_UPDATED_DISPLAY_NAME)
                internalContact = mFireContactHashMap[normalizedNumber]

                //if phone number exists on firestore
                if (internalContact != null) {

                    //if name changed, update in firestore
                    if (internalContact.name != contactName) {
                        myPrintln("updating $normalizedNumber from name: ${internalContact.name} to: $contactName")
                        batchArray[batchIndex].update(
                                mFireContactRef.document(internalContact.id),
                                FireContact.COLUMN_NAME,
                                contactName)

                        batchCount++
                    }

                    //remove to avoid deletions
                    mFireContactHashMap.remove(normalizedNumber)
                } else {
                    //New item. Insert
                    if (normalizedNumber != mUserPhoneNumber) {
                        myPrintln("adding $normalizedNumber / $contactName")
                        firestoreId = mFireContactRef.document().id

                        batchArray[batchIndex].set(mFireContactRef.document(firestoreId),
                                FireContact(firestoreId, -1, contactName,
                                        cursor.getString(COLUMN_UPDATED_PHONE_NUMBER),
                                        normalizedNumber))
                        batchCount++
                    }
                }

                if (BATCH_HALF_MAX < batchCount ) {

                    batchArray += FirebaseFirestore.getInstance().batch()
                    batchCount = 0
                    batchIndex++
                }
            }
        }

        cursor.close()

        //Remaining contacts not found on cursor so assumed deleted. Delete from firestore
        mFireContactHashMap.forEach { (key, value) ->
            myPrintln("deleting ${value.name} / $key")
            batchArray[batchIndex].delete(mFireContactRef.document(value.id))
            batchCount++
            if (BATCH_HALF_MAX < batchCount ) {
                batchArray += FirebaseFirestore.getInstance().batch()
                batchCount = 0
                batchIndex++
            }
        }

        //execute all batches

        if ((batchCount > 0) || (batchIndex > 0)) {
            myPrintln("committing changes...")
            batchArray.forEach { batch ->
                batch.commit()
            }
        } else {
            myPrintln("no records to commit")
        }

        myPrintln("end sync")

        mFireContactHashMap.clear()
        mPreferenceManager.edit().putLong(PREF_LAST_TIMESTAMP,mLastContactUpdated).apply()

        mFirstRun = false
    } else {
        myPrintln("no change in contacts")
    }
}

private suspend fun putAllUserContactsToHashMap() : Result {

    var result = Result.FAILED

    val batchArray = mutableListOf(FirebaseFirestore.getInstance().batch())
    var batchIndex = 0
    var batchCount = 0

    mFireContactHashMap.clear()

    var source = Source.CACHE

    if (mFirstRun) {
        source = Source.DEFAULT
        myPrintln("get contacts via Source.DEFAULT")
    } else {
        myPrintln("get contacts via Source.CACHE")
    }

    mFireContactRef.whereEqualTo( FireContact.COLUMN_USER_ID,mUid ).get(source)
    .addOnSuccessListener {documents ->

        var fireContact : FireContact

        for (doc in documents) {

            fireContact = doc.toObject(FireContact::class.java)

            if (!mFireContactHashMap.containsKey(fireContact.paPho)) {
                mFireContactHashMap[fireContact.paPho] = InternalContact(fireContact.na, doc.id)
            } else {
                myPrintln("duplicate will be removed from firestore: ${fireContact.paPho} / ${fireContact.na} / ${doc.id}")

                batchArray[batchIndex].delete(mFireContactRef.document(doc.id))

                batchCount++

                if (BATCH_HALF_MAX < batchCount) {
                    batchArray += FirebaseFirestore.getInstance().batch()
                    batchCount = 0
                    batchIndex++
                }
            }
        }

        result = Result.SUCCESS
    }.addOnFailureListener { exception ->
        myPrintln("Error getting documents: $exception")
    }.await()

    //execute all batches
    if ((batchCount > 0) || (batchIndex > 0)) {
        myPrintln("committing duplicate delete... ")
        batchArray.forEach { batch ->
            batch.commit()
        }
    } else {
        myPrintln("no duplicates to delete")
    }

    return result
}
</code>```

Body:
I am trying to get all the android contacts which been updated.

I am saving on firebase the last contact id i added and the last updated timestamp

I am using the next function to get back a cursor of all the updated contacts to compare with firebase server

private fun getUpdatedContacts(): Cursor? {

    val projection = arrayOf(
            ContactsContract.Contacts._ID,
            ContactsContract.Contacts.DISPLAY_NAME,
            ContactsContract.Contacts.HAS_PHONE_NUMBER,
            ContactsContract.Contacts.CONTACT_LAST_UPDATED_TIMESTAMP)

    val selection = ContactsContract.Contacts.CONTACT_LAST_UPDATED_TIMESTAMP + " > ? AND " +
            ContactsContract.Contacts._ID + "<= ?"

    val selectionArgs = arrayOf(mFireContactDetails!!.lcu_ms.toString(), mFireContactDetails!!.lcid.toString())

    val sortOrder = ContactsContract.Contacts.CONTACT_LAST_UPDATED_TIMESTAMP + " ASC"

    return mContentResolver.query(
            ContactsContract.Contacts.CONTENT_URI,
            projection,
            selection,
            selectionArgs,
            sortOrder)
}

But when i change one contact in my phone this cursor is returned MANY un-related contacts that i never used and mark them as changed. Last time when i just added a phone number to an existing contact, i got back from this cursor more than 50 contacts as been updated.

What is going on Android?? I am trying to sync contacts for the past 3 months now. Why is it so hard???

Body:
I been testing this solution for couple of days and it seems OK but i think i need to test it much more. If you using this method, do your own testing and above all, PLEASE LET ME KNOW IF I MISSED ANYTHING and don't be hurry to downgrade. Thx!

I built an App class that extents Application and implements the
ActivityLifecycleCallbacks. In which i create a ContactSync class for the
first time and activate it everytime the app goes to foregound

In ContactSync class, i am using Kotlin withContext(Dispatchers.IO) to suspend any code for easier flow

I use .get() to get all the contacts from firestore related to current user

at the .get() addOnSuccessListener, i add all the contacts to a HashMap with the normalized phone number as key and name + firestore id as values (using internal class)

While making the HashMap i also make sure there are no duplicates on firestore with smae phone number and if so delete them (using batch)

i then retrieve all the contacts from android phone. I sort them by NORMALIZED_NUMBER first and DISPLAY_NAME (will explain later)

I am now creating a batchArray with index and count to avoid exceeding the 500 limit

I start scanning through the contacts cursor,

I first get the normalized number, if not available (null) i create it my self using a function i made (it might be that a null value is only returned for phone numbers not in correct format, not sure)

I then compare the normalized number with previous cursor value. If the same i ignore it to avoid duplicates in firestore (remember the cursor is sorted by NORMALIZED_NUMBER)

I then check if the normalized number already in HashMap.

If in HashMap: i compare the name in HashMap to the cursor name. if different, i conclude the name was changed and i update the firestore contact in batch array (remember to increment counter and if exceeds 500 increase index). I then remove the normalized number from the HashMap to avoid its deletion later

If not in HashMap: i conclude the contact is new and i add it to firestore via batch

I iterate through all the cursor until completed.

When cursor complete i close it

Any remaining records found in HashMap are ones that were not found on firestore hence deleted. I iterate and delete them using batch

sync is done on the phone side

Now, since making the actual sync needs access to all users, i user firebase functions in node. I create 2 functions:

function that fires when new user is created (signed via phone)

function that fires when new contact document is created.

Both functions compare the users to the normalized number in document and if matching, writing the uid of that user to the firestore document "friend_uid" field.

Note you might have errors in these functions if you try to use them in free firebase plan. I suggest changing to Blaze plan and limit the charging to couple of dollars. By changing to Blaze, google also gives you free extras and avoid actual payment

By that, the sync is completed. The sync takes only couple of seconds

To display all the contacts which are users to the app, query all user contacts with "friend_uid" that are not null.

Some extra notes:

The .get() will retrieve all the contacts every time a sync is made. That might be a lot of reads if user has couple of hundreds contacts. To minimize, i use .get(Source.DEFAULT) when launching the app and .get(Source.CACHE) for the other times. Since these documents name and number only modified by user, i believe it will not be a problem most of the times (still testing)

To minimize the sync process as much as possible, i initiate it only if any contact changed its timestamp. I save the last timestamp to SharedPreferences and compare it. I found it mostly saves sync when app re-opened fast.

I also save the last user logged in. If any change in user, i re-initialize the current user contacts

Some source code (still testing, please let me know if any error):

private fun getContacts(): Cursor? {
    val projection = arrayOf(
            ContactsContract.CommonDataKinds.Phone._ID,
            ContactsContract.CommonDataKinds.Phone.NUMBER,
            ContactsContract.CommonDataKinds.Phone.NORMALIZED_NUMBER,
            ContactsContract.CommonDataKinds.Phone.DISPLAY_NAME,
            ContactsContract.CommonDataKinds.Phone.CONTACT_LAST_UPDATED_TIMESTAMP)

    //sort by NORMALIZED_NUMBER to detect duplicates and then by name to keep order and avoiding name change
    val sortOrder = ContactsContract.CommonDataKinds.Phone.NORMALIZED_NUMBER + " ASC, " +
            ContactsContract.CommonDataKinds.Phone.DISPLAY_NAME + " ASC"

    return mContentResolver.query(
            ContactsContract.CommonDataKinds.Phone.CONTENT_URI,
            projection,
            null,
            null,
            sortOrder)
}

    private suspend fun syncContactsAsync() = withContext(Dispatchers.IO)  {

    if (isAnythingChanged() || mFirstRun) {

        if (getValues() == Result.SUCCESS) {
            myPrintln("values retrieved success")
        } else {
            myPrintln("values retrieved failed. Aborting.")
            return@withContext
        }

        val cursor: Cursor? = getContacts()

        if (cursor == null) {
            myPrintln("cursor cannot be null")
            mFireContactHashMap.clear()
            return@withContext
        }

        if (cursor.count == 0) {
            cursor.close()
            mFireContactHashMap.clear()
            myPrintln("cursor empty")
            return@withContext
        }

        var contactName: String?
        var internalContact: InternalContact?
        val batchArray = mutableListOf(FirebaseFirestore.getInstance().batch())
        var batchIndex = 0
        var batchCount = 0
        var normalizedNumber:String?
        var prevNumber = ""
        var firestoreId: String

        while (cursor.moveToNext()) {

            normalizedNumber = cursor.getString(COLUMN_UPDATED_NORMALIZED_NUMBER)

            if (normalizedNumber == null) {
                normalizedNumber = cursor.getString(COLUMN_UPDATED_PHONE_NUMBER)
                normalizedNumber = Phone.getParsedPhoneNumber(mDeviceCountryIso,normalizedNumber,mContext)
            }

            //cursor sorted by normalized numbers so if same as previous, do not check
            if (normalizedNumber != prevNumber) {

                prevNumber = normalizedNumber

                contactName = cursor.getString(COLUMN_UPDATED_DISPLAY_NAME)
                internalContact = mFireContactHashMap[normalizedNumber]

                //if phone number exists on firestore
                if (internalContact != null) {

                    //if name changed, update in firestore
                    if (internalContact.name != contactName) {
                        myPrintln("updating $normalizedNumber from name: ${internalContact.name} to: $contactName")
                        batchArray[batchIndex].update(
                                mFireContactRef.document(internalContact.id),
                                FireContact.COLUMN_NAME,
                                contactName)

                        batchCount++
                    }

                    //remove to avoid deletions
                    mFireContactHashMap.remove(normalizedNumber)
                } else {
                    //New item. Insert
                    if (normalizedNumber != mUserPhoneNumber) {
                        myPrintln("adding $normalizedNumber / $contactName")
                        firestoreId = mFireContactRef.document().id

                        batchArray[batchIndex].set(mFireContactRef.document(firestoreId),
                                FireContact(firestoreId, -1, contactName,
                                        cursor.getString(COLUMN_UPDATED_PHONE_NUMBER),
                                        normalizedNumber))
                        batchCount++
                    }
                }

                if (BATCH_HALF_MAX < batchCount ) {

                    batchArray += FirebaseFirestore.getInstance().batch()
                    batchCount = 0
                    batchIndex++
                }
            }
        }

        cursor.close()

        //Remaining contacts not found on cursor so assumed deleted. Delete from firestore
        mFireContactHashMap.forEach { (key, value) ->
            myPrintln("deleting ${value.name} / $key")
            batchArray[batchIndex].delete(mFireContactRef.document(value.id))
            batchCount++
            if (BATCH_HALF_MAX < batchCount ) {
                batchArray += FirebaseFirestore.getInstance().batch()
                batchCount = 0
                batchIndex++
            }
        }

        //execute all batches

        if ((batchCount > 0) || (batchIndex > 0)) {
            myPrintln("committing changes...")
            batchArray.forEach { batch ->
                batch.commit()
            }
        } else {
            myPrintln("no records to commit")
        }

        myPrintln("end sync")

        mFireContactHashMap.clear()
        mPreferenceManager.edit().putLong(PREF_LAST_TIMESTAMP,mLastContactUpdated).apply()

        mFirstRun = false
    } else {
        myPrintln("no change in contacts")
    }
}

private suspend fun putAllUserContactsToHashMap() : Result {

    var result = Result.FAILED

    val batchArray = mutableListOf(FirebaseFirestore.getInstance().batch())
    var batchIndex = 0
    var batchCount = 0

    mFireContactHashMap.clear()

    var source = Source.CACHE

    if (mFirstRun) {
        source = Source.DEFAULT
        myPrintln("get contacts via Source.DEFAULT")
    } else {
        myPrintln("get contacts via Source.CACHE")
    }

    mFireContactRef.whereEqualTo( FireContact.COLUMN_USER_ID,mUid ).get(source)
    .addOnSuccessListener {documents ->

        var fireContact : FireContact

        for (doc in documents) {

            fireContact = doc.toObject(FireContact::class.java)

            if (!mFireContactHashMap.containsKey(fireContact.paPho)) {
                mFireContactHashMap[fireContact.paPho] = InternalContact(fireContact.na, doc.id)
            } else {
                myPrintln("duplicate will be removed from firestore: ${fireContact.paPho} / ${fireContact.na} / ${doc.id}")

                batchArray[batchIndex].delete(mFireContactRef.document(doc.id))

                batchCount++

                if (BATCH_HALF_MAX < batchCount) {
                    batchArray += FirebaseFirestore.getInstance().batch()
                    batchCount = 0
                    batchIndex++
                }
            }
        }

        result = Result.SUCCESS
    }.addOnFailureListener { exception ->
        myPrintln("Error getting documents: $exception")
    }.await()

    //execute all batches
    if ((batchCount > 0) || (batchIndex > 0)) {
        myPrintln("committing duplicate delete... ")
        batchArray.forEach { batch ->
            batch.commit()
        }
    } else {
        myPrintln("no duplicates to delete")
    }

    return result
}


来源: stackexchange

问:
Why when I write noexcept before const i get an error
Why when I write definition of a class member function such as

int returnSize() const noexcept
```

<p>the code compiles, but when I write</p>

<pre class="lang-cpp prettyprint-override"><code>int returnSize() noexcept const
</code>```

it doesn't compile

答:
That's just the way the language is defined.

Exception specifiers need to come after const/volatile qualification:
https://en.cppreference.com/w/cpp/language/function

ContentLicense: CC BY-SA 4.0

Body:
Why when I write definition of a class member function such as

int returnSize() const noexcept

the code compiles, but when I write

int returnSize() noexcept const

it doesn't compile


问:
PM2 process manager for Node across multiple cores
I have a simple stateless Node app that I want to instantiate across a multi-core (multi-vCPU AWS instance) server, and I understand how PM2's cluster mode works to obviate the need for using the Cluster module in the app code.

I have a dual core AWS t2.medium EC2 instance, PM2 I believe is configured correctly and at startup it invokes two processes for the app with distinct PM2 IDs and PIDs.

PM2 is starting the app as follows:

```<code>pm2 start [app_name] -i max
</code>```

PM2 lists the two processes with distinct PM2 IDs and distinct PIDs as expected.

However...

```<code>ps -U [username] -au
</code>```

...suggests both processes are running on the same core.

Am I missing something? (Probably!)

Thanks in advance to anyone who can shed some light on this.

答: Processes aren't bound to cores, but rather assigned by the OS's scheduler. When your clustered program comes under load, the OS will use both cores to schedule your processes and, of course, all the other stuff it needs to run.</p>

Body:
I have a simple stateless Node app that I want to instantiate across a multi-core (multi-vCPU AWS instance) server, and I understand how PM2's cluster mode works to obviate the need for using the Cluster module in the app code.

I have a dual core AWS t2.medium EC2 instance, PM2 I believe is configured correctly and at startup it invokes two processes for the app with distinct PM2 IDs and PIDs.

PM2 is starting the app as follows:

pm2 start [app_name] -i max

PM2 lists the two processes with distinct PM2 IDs and distinct PIDs as expected.

However...

ps -U [username] -au

...suggests both processes are running on the same core.

Am I missing something? (Probably!)

Thanks in advance to anyone who can shed some light on this.


问:
Scraping data from span tag with class using BS
Im trying to scrape project's names from Gitlab. When I inspect source code I see that name of project is in:

```<code><span class='project-name'>Project Name</span>
</code>```

Unfortunately, when I try to scrape this date I got empty list, My code looks like:

```<code>url = 'https://gitlab.com/users/USER/projects'
source = urllib.request.urlopen(url).read()
soup = bs.BeautifulSoup(source,'lxml')
repos = [repo.text for repo in soup.find_all('span',{'class':'project-name'})]
</code>```

I was trying other solutions like using attrs, class_ or using other HTML tags, but nothing works. What can be wrong here?

答:
Ok, so it looks like when you inspect the page in the network tabs in chrome developer tools, you can see that projects are not rendered when request is made:

What that means is, project information is requested after. In order to get the projects you need to send a request to https://gitlab.com/users/USER/projects.json endpoint:

After that you can inspect the response from that endpoint. As you can see the response here is json so we can load json data with json module and then in that dictionary there is an entry called html which has html data in it, so we can parse it with beautifulsoup and the rest of the code stays the same:

import bs4 as bs
import urllib, json
url = 'https://gitlab.com/users/USER/projects.json'
source = urllib.request.urlopen(url).read()
soup = bs.BeautifulSoup(json.loads(source)["html"],'html.parser')
repos = [repo.text for repo in soup.find_all('span',{'class':'project-name'})]
print(repos)
```

<p><strong>Output:</strong></p>

```['freebsd', 'freebsd-ports', 'freebsd-test', 'risc-vhdl', 'dotfiles', 'tideyBot']
```

Body:
Im trying to scrape project's names from Gitlab. When I inspect source code I see that name of project is in:

Project Name

Unfortunately, when I try to scrape this date I got empty list, My code looks like:

url = 'https://gitlab.com/users/USER/projects'
source = urllib.request.urlopen(url).read()
soup = bs.BeautifulSoup(source,'lxml')
repos = [repo.text for repo in soup.find_all('span',{'class':'project-name'})]

I was trying other solutions like using attrs, class_ or using other HTML tags, but nothing works. What can be wrong here?

Body:
Ok, so it looks like when you inspect the page in the network tabs in chrome developer tools, you can see that projects are not rendered when request is made:

What that means is, project information is requested after. In order to get the projects you need to send a request to https://gitlab.com/users/USER/projects.json endpoint:

After that you can inspect the response from that endpoint. As you can see the response here is json so we can load json data with json module and then in that dictionary there is an entry called html which has html data in it, so we can parse it with beautifulsoup and the rest of the code stays the same:

import bs4 as bs
import urllib, json
url = 'https://gitlab.com/users/USER/projects.json'
source = urllib.request.urlopen(url).read()
soup = bs.BeautifulSoup(json.loads(source)["html"],'html.parser')
repos = [repo.text for repo in soup.find_all('span',{'class':'project-name'})]
print(repos)

Output:

['freebsd', 'freebsd-ports', 'freebsd-test', 'risc-vhdl', 'dotfiles', 'tideyBot']


问:
I want the CSS of a page to change when a button is clicked
I found how you could do this with individual elements but I couldn't find anything that would change the the body CSS of a page. I am trying to make a thing where it will switch between light and dark mode when you click the button.

```<code><!DOCTYPE html>
<button style="background-color: #252525;
border: 2px;
border-style: solid;
border-color: white;
color: white;
padding: 10px 32px;
text-align: center;
text-decoration: none;
display: inline-block;
font-size: 16px;
font-weight: bold;
margin: 4px 2px;
cursor: pointer;" id="changethemebutton" onclick="changeTheme()">Change Theme</button>
</code>```

```<code>function changeTheme() {
document.getElementById(".background").style.background = "black";
}
</script>
<style>
body{
    background-color:white
}
</style>
</code>```

答:
Best way to set a dark-mode on your website is to toggle a class .dark-mode on your body using javascript and then stylize it using css:

body {
  /* light-mode styles */
  background-color: white;
}

body.dark-mode {
  /* dark-mode styles */
  background-color: black;
}```
<pre class="snippet-code-html lang-html prettyprint-override"><code><button style="background-color: #252525;
border: 2px;
border-style: solid;
border-color: white;
color: white;
padding: 10px 32px;
text-align: center;
text-decoration: none;
display: inline-block;
font-size: 16px;
font-weight: bold;
margin: 4px 2px;
cursor: pointer;" id="changethemebutton" onclick="changeTheme()">Change Theme
</button>

<script>
  function changeTheme() {
    document.body.classList.toggle('dark-mode');  // <-- Toogle dark-mode class to <body>
  }
</script></code>```

Body:
I found how you could do this with individual elements but I couldn't find anything that would change the the body CSS of a page. I am trying to make a thing where it will switch between light and dark mode when you click the button.

<!DOCTYPE html>
<button style="background-color: #252525;
border: 2px;
border-style: solid;
border-color: white;
color: white;
padding: 10px 32px;
text-align: center;
text-decoration: none;
display: inline-block;
font-size: 16px;
font-weight: bold;
margin: 4px 2px;
cursor: pointer;" id="changethemebutton" onclick="changeTheme()">Change Theme</button>

function changeTheme() {
document.getElementById(".background").style.background = "black";
}
</script>

Body:
Best way to set a dark-mode on your website is to toggle a class .dark-mode on your body using javascript and then stylize it using css:

body {
  /* light-mode styles */
  background-color: white;
}

body.dark-mode {
  /* dark-mode styles */
  background-color: black;
}

<button style="background-color: #252525;
border: 2px;
border-style: solid;
border-color: white;
color: white;
padding: 10px 32px;
text-align: center;
text-decoration: none;
display: inline-block;
font-size: 16px;
font-weight: bold;
margin: 4px 2px;
cursor: pointer;" id="changethemebutton" onclick="changeTheme()">Change Theme
</button>

<script>
  function changeTheme() {
    document.body.classList.toggle('dark-mode');  // <-- Toogle dark-mode class to <body>
  }
</script>


问:
React Native Expo How to upload an image to Firebase using expo Image picker
I'm trying to upload an Image to my firebase but I can't figure it out.

This is my code

```<code>_pickImage = async () => {
    let result = await ImagePicker.launchImageLibraryAsync({
        mediaTypes: ImagePicker.MediaTypeOptions.All,
        allowsEditing: true,
        aspect: [4, 3],
        quality: 1
    });

    if (!result.cancelled) {
        this.uploadImage(result.uri, 'test-image')
            .then(() => {
                console.log('it work')
            })
            .catch(error => {
                console.log('it does not work')
                console.error(error)
            })
    }
};

uploadImage = async (uri, imageName) => {
    const response = await fetch(uri);
    const blob = response.blob();

    const ref = firebase.storage().ref().child(`images/${imageName}`);
    return ref.put(blob);
}
</code>```

I'm getting this error.

```<code>FirebaseStorageError {
 "code_": "storage/invalid-argument",
 "message_": "Firebase Storage: Invalid argument in `put` at index 0: Expected Blob or File.",
 "name_": "FirebaseError",
 "serverResponse_": null,
}
</code>```

答: try var ref= firebase.storage().ref().child('images/'+imageName) instead of yours

Body:
I'm trying to upload an Image to my firebase but I can't figure it out.

This is my code

_pickImage = async () => {
    let result = await ImagePicker.launchImageLibraryAsync({
        mediaTypes: ImagePicker.MediaTypeOptions.All,
        allowsEditing: true,
        aspect: [4, 3],
        quality: 1
    });

    if (!result.cancelled) {
        this.uploadImage(result.uri, 'test-image')
            .then(() => {
                console.log('it work')
            })
            .catch(error => {
                console.log('it does not work')
                console.error(error)
            })
    }
};

uploadImage = async (uri, imageName) => {
    const response = await fetch(uri);
    const blob = response.blob();

    const ref = firebase.storage().ref().child(`images/${imageName}`);
    return ref.put(blob);
}

I'm getting this error.

FirebaseStorageError {
 "code_": "storage/invalid-argument",
 "message_": "Firebase Storage: Invalid argument in `put` at index 0: Expected Blob or File.",
 "name_": "FirebaseError",
 "serverResponse_": null,
}


问:
access var outside function scope
I havethe following code and I'm trying to access selectedBrand var outside so to use it elsewhere in may code

```<code>var selectedBrand;
var brand_sel = d3.select('select')
    .on("change",function(b) {
        selectedBrand = d3.select("#brand").node().value;
    });

  console.log(selectedBrand)
</code>```

when I try to consollog selectedBrand I get undefined results.

I try almost everything I understand without success

many thanks for any help

答:
just invoke your function brand_sel().

 Because right now you have only defined it, but never actually invoked it, hence it never fired and selectedBrand remains undefined.

Also that's some dirty dirty scoping that I would not want to deal with in my code. In general what you're doing is a bad pattern that will lead to bugs down the road. Start using const / let instead to avoid issues with scoping and in general try to keep the scope clean - this can easily be a function return, rather than directly mutating a variable through a function call.

At the very least, "use strict"; on the top of your code might be a good idea.

Overall, without meaning to sound condescending, it looks like you're developing some "not so good"  practices. There's a lot of good tutorials on the web or even book that do teach JS "the proper way". I know it may sound like I'm bashing you, but trust me, it will be worth it in the long run, if you don't feel like spending 3 hours debugging why are not your values updating properly in loops and whatnot.

Title: access var outside function scope

Body:
I havethe following code and I'm trying to access selectedBrand var outside so to use it elsewhere in may code

var selectedBrand;
var brand_sel = d3.select('select')
    .on("change",function(b) {
        selectedBrand = d3.select("#brand").node().value;
    });

  console.log(selectedBrand)

when I try to consollog selectedBrand I get undefined results.

I try almost everything I understand without success

many thanks for any help


问:
Button not clicking/calling function when pressed
I was adding a new window in a bigger project and im having trouble with buttons... When you click on them when the program is running, there is no sign of them being pressed and nothing happens. Im pretty new to tkinter so if anyone could help that would be apprecated!</p>

```<code>from tkinter import *

def window():

    def sign_up_clicked():
        return True
        print ('test')

    def log_in_clicked():
        return False
        print ('test')

    window = Tk()
    window.title('Welcome')
    window.geometry('480x200')

    welcome_lbl = Label(window, text='Welcome', font=('Arial', 26), fg='#71C2FE') #Label with text saying 'welcome' in light blue
    welcome_lbl.place(x=15, y=10) #placing welcome label in top left of sign_up_window

    under_line_lbl = Label(window, text='_______________________', font=('Arial', 35), fg='#FFB56B') #orange line under blue for looks
    under_line_lbl.place(x=0, y=40)

    intro_lbl = Label(window, text='Please press login if you already have an account,\notherwiseyou can create an account by pressing sign-up. ', font=('Arial', 14, 'bold italic'), justify='left')#lable with text underlined and italics
    intro_lbl.place(x=10, y=90)

    sign_up_btn = Button(window, text='Sign up', relief='raised', width=6, font=('candara', 14), command=sign_up_clicked)
    sign_up_btn.place(x=10, y=130)

    log_in_btn = Button(window, text='Login', relief='raised', width=6, font=('candara', 14), command=log_in_clicked)
    log_in_btn.place(x=10, y=160)

    window.mainloop()

window()
</code>```

答:
You are returning before your print statement is reached. Place print('test') first and then return True. When a function reaches a return it ends the execution of that function, hands back the value if your return is assigned one, and returns to the place it left off in your code.

Working example:

```<code>from tkinter import *

def window():

    def sign_up_clicked():
        print ('test')
        # returns control to mainloop
        return True

    def log_in_clicked():
        print ('test')
        # returns control to mainloop
        return False

    window = Tk()
    window.title('Welcome')
    window.geometry('480x200')

    welcome_lbl = Label(window, text='Welcome', font=('Arial', 26), fg='#71C2FE') #Label with text saying 'welcome' in light blue
    welcome_lbl.place(x=15, y=10) #placing welcome label in top left of sign_up_window

    under_line_lbl = Label(window, text='_______________________', font=('Arial', 35), fg='#FFB56B') #orange line under blue for looks
    under_line_lbl.place(x=0, y=40)

    intro_lbl = Label(window, text='Please press login if you already have an account,\notherwiseyou can create an account by pressing sign-up. ', font=('Arial', 14, 'bold italic'), justify='left')#lable with text underlined and italics
    intro_lbl.place(x=10, y=90)

    sign_up_btn = Button(window, text='Sign up', relief='raised', width=6, font=('candara', 14), command=sign_up_clicked)
    sign_up_btn.place(x=10, y=130)

    log_in_btn = Button(window, text='Login', relief='raised', width=6, font=('candara', 14), command=log_in_clicked)
    log_in_btn.place(x=10, y=160)

    window.mainloop()

window()
</code>```

Body:
I was adding a new window in a bigger project and im having trouble with buttons... When you click on them when the program is running, there is no sign of them being pressed and nothing happens. Im pretty new to tkinter so if anyone could help that would be apprecated!

from tkinter import *

def window():

    def sign_up_clicked():
        return True
        print ('test')

    def log_in_clicked():
        return False
        print ('test')

    window = Tk()
    window.title('Welcome')
    window.geometry('480x200')

    welcome_lbl = Label(window, text='Welcome', font=('Arial', 26), fg='#71C2FE') #Label with text saying 'welcome' in light blue
    welcome_lbl.place(x=15, y=10) #placing welcome label in top left of sign_up_window

    under_line_lbl = Label(window, text='_______________________', font=('Arial', 35), fg='#FFB56B') #orange line under blue for looks
    under_line_lbl.place(x=0, y=40)

    intro_lbl = Label(window, text='Please press login if you already have an account,\notherwiseyou can create an account by pressing sign-up. ', font=('Arial', 14, 'bold italic'), justify='left')#lable with text underlined and italics
    intro_lbl.place(x=10, y=90)

    sign_up_btn = Button(window, text='Sign up', relief='raised', width=6, font=('candara', 14), command=sign_up_clicked)
    sign_up_btn.place(x=10, y=130)

    log_in_btn = Button(window, text='Login', relief='raised', width=6, font=('candara', 14), command=log_in_clicked)
    log_in_btn.place(x=10, y=160)

    window.mainloop()

window()

Body:
You are returning before your print statement is reached. Place print('test') first and then return True. When a function reaches a return it ends the execution of that function, hands back the value if your return is assigned one, and returns to the place it left off in your code.

Working example:

from tkinter import *

def window():

    def sign_up_clicked():
        print ('test')
        # returns control to mainloop
        return True

    def log_in_clicked():
        print ('test')
        # returns control to mainloop
        return False

    window = Tk()
    window.title('Welcome')
    window.geometry('480x200')

    welcome_lbl = Label(window, text='Welcome', font=('Arial', 26), fg='#71C2FE') #Label with text saying 'welcome' in light blue
    welcome_lbl.place(x=15, y=10) #placing welcome label in top left of sign_up_window

    under_line_lbl = Label(window, text='_______________________', font=('Arial', 35), fg='#FFB56B') #orange line under blue for looks
    under_line_lbl.place(x=0, y=40)

    intro_lbl = Label(window, text='Please press login if you already have an account,\notherwiseyou can create an account by pressing sign-up. ', font=('Arial', 14, 'bold italic'), justify='left')#lable with text underlined and italics
    intro_lbl.place(x=10, y=90)

    sign_up_btn = Button(window, text='Sign up', relief='raised', width=6, font=('candara', 14), command=sign_up_clicked)
    sign_up_btn.place(x=10, y=130)

    log_in_btn = Button(window, text='Login', relief='raised', width=6, font=('candara', 14), command=log_in_clicked)
    log_in_btn.place(x=10, y=160)

    window.mainloop()

window()


问:
How to pass dependencies to a custom .NET Core ILoggerProvider
I am creating a custom .NET Core ILoggerProvider that requires some dependencies to be passed into its constructor.

I believe I am using a fairly common pattern to initialize my logging implementation; it looks something like this:

```<code>var services = new ServiceCollection();

// Register some services here

services.AddLogging(builder =>
{
    builder.AddProvider(new DebugLoggerProvider());
});

var provider = services.BuildServiceProvider();
</code>```

I want to add my new provider within the AddLogging block, in the same way that the DebugLoggerProvider is currently added.

My custom provider requires some other services to be passed into its constructor and since these are already registered with the ServiceCollection, I assume that I should be able to reference them. However, unlike methods such as AddSingleton, which have an overload that exposes the IServiceProvider, AddLogging doesn't seem to offer an equivalent.

Is there a simple way to achieve this, or am I attempting to do something that contradicts the way .NET Core logging was designed to be deployed?

UPDATE:

After experimenting with the suggestions proposed by @Nkosi, I can confirm that it is possible to get this to work by bypassing AddLogging and directly implementing what it does internally, as follows:

```<code>var services = new ServiceCollection();

// Register some services
services.AddSingleton<IMyService, MyService>();

// Initialize logging
services.AddOptions();
services.AddSingleton<ILoggerFactory, LoggerFactory>();
services.AddSingleton(typeof(ILogger<>), typeof(Logger<>));
services.AddSingleton<ILoggerProvider>(p => new DebugLoggerProvider());
services.AddSingleton<ILoggerProvider>(p => new MyLoggerProvider("Constant value", p.GetService<IMyService>()));

var provider = services.BuildServiceProvider();
</code>```

答:
Apologies for being a bit late to the party on this one, but I ran into exactly the same problem after having searched high and low. Inspired by the excellent entries in this page, I ended up with the solution below.

```<code>services.AddTransient<IMyLogRepository, LogRepository>();
var loggerFactory = LoggerFactory.Create(builder =>
        {
            builder.AddConsole()
                   .AddDbLoggerProvider(services.BuildServiceProvider().GetService<IMyLogRepository>());
        });
        services.AddSingleton(loggerFactory.CreateLogger("MyLogging"));
</code>```

The key to this being:

```<code>services.BuildServiceProvider().GetService<IMyLogRepository>())
</code>```

Which allowed me to link my database repository to the dbLogger object I created in a single extra line. In essence, it gives me the ability to pluck my DI database object an send it to the Logging service via standard ILoggerProvider and ILogger interfaces

Body:
I am creating a custom .NET Core ILoggerProvider that requires some dependencies to be passed into its constructor.

I believe I am using a fairly common pattern to initialize my logging implementation; it looks something like this:

var services = new ServiceCollection();

// Register some services here

services.AddLogging(builder =>
{
    builder.AddProvider(new DebugLoggerProvider());
});

var provider = services.BuildServiceProvider();

I want to add my new provider within the AddLogging block, in the same way that the DebugLoggerProvider is currently added.

My custom provider requires some other services to be passed into its constructor and since these are already registered with the ServiceCollection, I assume that I should be able to reference them. However, unlike methods such as AddSingleton, which have an overload that exposes the IServiceProvider, AddLogging doesn't seem to offer an equivalent.

Is there a simple way to achieve this, or am I attempting to do something that contradicts the way .NET Core logging was designed to be deployed?

UPDATE:

After experimenting with the suggestions proposed by @Nkosi, I can confirm that it is possible to get this to work by bypassing AddLogging and directly implementing what it does internally, as follows:

var services = new ServiceCollection();

// Register some services
services.AddSingleton<IMyService, MyService>();

// Initialize logging
services.AddOptions();
services.AddSingleton<ILoggerFactory, LoggerFactory>();
services.AddSingleton(typeof(ILogger<>), typeof(Logger<>));
services.AddSingleton<ILoggerProvider>(p => new DebugLoggerProvider());
services.AddSingleton<ILoggerProvider>(p => new MyLoggerProvider("Constant value", p.GetService<IMyService>()));

var provider = services.BuildServiceProvider();

Body:
Apologies for being a bit late to the party on this one, but I ran into exactly the same problem after having searched high and low. Inspired by the excellent entries in this page, I ended up with the solution below.

services.AddTransient<IMyLogRepository, LogRepository>();
var loggerFactory = LoggerFactory.Create(builder =>
        {
            builder.AddConsole()
                   .AddDbLoggerProvider(services.BuildServiceProvider().GetService<IMyLogRepository>());
        });
        services.AddSingleton(loggerFactory.CreateLogger("MyLogging"));

The key to this being:

services.BuildServiceProvider().GetService<IMyLogRepository>())

Which allowed me to link my database repository to the dbLogger object I created in a single extra line. In essence, it gives me the ability to pluck my DI database object an send it to the Logging service via standard ILoggerProvider and ILogger interfaces


问:
ios 13 DeviceOrientationEvent.requestPermission: how to force device to ask again for user permission
In iOS 13 Apple has introduced the API DeviceOrientationEvent.requestPermission. It must be triggered on user action (click, tap or equivalent).
My problem here is that the result seems to be cached, so in case the user denies permission I can't ask access again (the promise is automatically fulfilled with the cached value).
Is there any way to force the device to forgot the cached value and ask again for the user permission to access orientation data (I mean it should display again the popup window where the user can allow or deny access)?

This is the relevant code:

```<code>if (DeviceOrientationEvent && typeof(DeviceOrientationEvent.requestPermission) === "function") {
    const permissionState = await DeviceOrientationEvent.requestPermission();

    if (permissionState === "granted") {
        // Permission granted
    } else {
        // Permission denied
    }
}
</code>```

答:
Try simply to quit Safari and launch it back.
The prompt will come back.</p>

Body:
In iOS 13 Apple has introduced the API DeviceOrientationEvent.requestPermission. It must be triggered on user action (click, tap or equivalent).
My problem here is that the result seems to be cached, so in case the user denies permission I can't ask access again (the promise is automatically fulfilled with the cached value).
Is there any way to force the device to forgot the cached value and ask again for the user permission to access orientation data (I mean it should display again the popup window where the user can allow or deny access)?

This is the relevant code:

if (DeviceOrientationEvent && typeof(DeviceOrientationEvent.requestPermission) === "function") {
    const permissionState = await DeviceOrientationEvent.requestPermission();

    if (permissionState === "granted") {
        // Permission granted
    } else {
        // Permission denied
    }
}


问:
Validate Regex groups with OR when both groups are present in Java code
Need to validate a format where either of transport and protocol can be present or both can be. I have applied this regex to validate both the group. But when any one validation succeeds, it does not validate other group, like in the code.
Note: transport can accepts either tcp, sctp or udp and protocol can either accept radius, diameter and tacacs+.

Like example, if both transport and protocol are part of URL, and if format of transport is correct and it validates with the value provided in Regex, then it returns true even if the value of protocol is invalid.

Java Code:

```<code>String pattern = "((?=.*;transport=(tcp|sctp|udp)\\b)|(?=.*;protocol=(diameter|radius|tacacs+)\\b))";
String url = "transport=tcp;protocol=aradius";
    Pattern regExPattern = Pattern.compile(pattern);
    if(regExPattern.matcher(url).find()) {
        return true;
    } else {
        return false;
    }
</code>```

This returns true, since it validates transport successfully but unable to validate protocol values.

答:
The complication is that the transport and protocol are both optional and appear in any order. To do this in one regex I have an alternation using | where in the first alternative the transport parameter is scanned first and the protocol parameter is optional and in the second alternative things are reversed. To implement an optional regular expression A that can appear far into the input stream, my approach is to use:

(.*(?<tag>A)|) If regular expression A appears in the input stream, scan for A, else scan no characters. But either way there is a match.

You cannot use:

.*(?<tag>A)? because the regex engine satisfies this trivially by scanning 0 characters and never scanning A.

The regex:

```<code>;transport=(?<transport1>tcp|sctp|udp)\b(.*;protocol=(?<protocol1>diameter|radius|tacacs+)\b|)|;protocol=(?<protocol2>diameter|radius|tacacs+)\b(.*;transport=(?<transport2>tcp|sctp|udp)\b|)
</code>```

See regex demo

The program:

```<code>import java.util.regex.Pattern;
import java.util.regex.Matcher;

public class Test
{
    public static void main(String[] args) {
        String tests[] = {
            "aaa;transport=tcp  ;protocol=radius",
            "aaa;protocol=diameter    ;transport=udp",
            "aaa;transport=sctp",
            "aaa;protocol=tacacs",
            "aaaa"
        };
        for (String test: tests) {
            Pattern pattern = Pattern.compile(";transport=(?<transport1>tcp|sctp|udp)\\b(.*;protocol=(?<protocol1>diameter|radius|tacacs+)\\b|)|;protocol=(?<protocol2>diameter|radius|tacacs+)\\b(.*;transport=(?<transport2>tcp|sctp|udp)\\b|)");
            System.out.println(test);
            Matcher m = pattern.matcher(test);
            if (m.find()) {
                String protocol = m.group("protocol1") == null ? m.group("protocol2") :  m.group("protocol1");
                String transport = m.group("transport1") == null ? m.group("transport2") : m.group("transport1");
                if (protocol != null) {
                    System.out.println("   protocol=" + protocol);
                }
                if (transport != null) {
                    System.out.println("   transport=" + transport);
                }
            }
        }
    }
}
</code>```

Prints:

```<code>aaa;transport=tcp  ;protocol=radius
   protocol=radius
   transport=tcp
aaa;protocol=diameter    ;transport=udp
   protocol=diameter
   transport=udp
aaa;transport=sctp
   transport=sctp
aaa;protocol=tacacs
   protocol=tacacs
aaaa
</code>```

Note

It would be simpler to use two separate regular expressions and two calls to find to scan the transport and protocol values. At least the regular expressions would be far simpler.

Body:
Need to validate a format where either of transport and protocol can be present or both can be. I have applied this regex to validate both the group. But when any one validation succeeds, it does not validate other group, like in the code.
Note: transport can accepts either tcp, sctp or udp and protocol can either accept radius, diameter and tacacs+.

Like example, if both transport and protocol are part of URL, and if format of transport is correct and it validates with the value provided in Regex, then it returns true even if the value of protocol is invalid.

Java Code:

String pattern = "((?=.*;transport=(tcp|sctp|udp)\\b)|(?=.*;protocol=(diameter|radius|tacacs+)\\b))";
String url = "transport=tcp;protocol=aradius";
    Pattern regExPattern = Pattern.compile(pattern);
    if(regExPattern.matcher(url).find()) {
        return true;
    } else {
        return false;
    }

This returns true, since it validates transport successfully but unable to validate protocol values.

Body:
The complication is that the transport and protocol are both optional and appear in any order. To do this in one regex I have an alternation using | where in the first alternative the transport parameter is scanned first and the protocol parameter is optional and in the second alternative things are reversed. To implement an optional regular expression A that can appear far into the input stream, my approach is to use:

(.*(?<tag>A)|) If regular expression A appears in the input stream, scan for A, else scan no characters. But either way there is a match.

You cannot use:

.*(?<tag>A)? because the regex engine satisfies this trivially by scanning 0 characters and never scanning A.

The regex:

;transport=(?<transport1>tcp|sctp|udp)\b(.*;protocol=(?<protocol1>diameter|radius|tacacs+)\b|)|;protocol=(?<protocol2>diameter|radius|tacacs+)\b(.*;transport=(?<transport2>tcp|sctp|udp)\b|)

See regex demo

The program:

import java.util.regex.Pattern;
import java.util.regex.Matcher;

public class Test
{
    public static void main(String[] args) {
        String tests[] = {
            "aaa;transport=tcp  ;protocol=radius",
            "aaa;protocol=diameter    ;transport=udp",
            "aaa;transport=sctp",
            "aaa;protocol=tacacs",
            "aaaa"
        };
        for (String test: tests) {
            Pattern pattern = Pattern.compile(";transport=(?<transport1>tcp|sctp|udp)\\b(.*;protocol=(?<protocol1>diameter|radius|tacacs+)\\b|)|;protocol=(?<protocol2>diameter|radius|tacacs+)\\b(.*;transport=(?<transport2>tcp|sctp|udp)\\b|)");
            System.out.println(test);
            Matcher m = pattern.matcher(test);
            if (m.find()) {
                String protocol = m.group("protocol1") == null ? m.group("protocol2") :  m.group("protocol1");
                String transport = m.group("transport1") == null ? m.group("transport2") : m.group("transport1");
                if (protocol != null) {
                    System.out.println("   protocol=" + protocol);
                }
                if (transport != null) {
                    System.out.println("   transport=" + transport);
                }
            }
        }
    }
}

Prints:

aaa;transport=tcp  ;protocol=radius
   protocol=radius
   transport=tcp
aaa;protocol=diameter    ;transport=udp
   protocol=diameter
   transport=udp
aaa;transport=sctp
   transport=sctp
aaa;protocol=tacacs
   protocol=tacacs
aaaa

Note

It would be simpler to use two separate regular expressions and two calls to find to scan the transport and protocol values. At least the regular expressions would be far simpler.


问:
How to ignore case for POJO if it extends an abstract class in Spring Jackson?
I know we can ignore case for input JSON by adding property in application.yml as:

```<code>spring:
  jackson:
    mapper:
      accept_case_insensitive_properties: true
</code>```

But if my POJO extends an abstract class, it is not working and my JSON is not being parsed.

My abstract class:

```<code>@JsonIgnoreProperties(ignoreUnknown = true)
@JsonTypeInfo(
        use = JsonTypeInfo.Id.NAME,
        include = JsonTypeInfo.As.PROPERTY,
        property = "event")
@JsonSubTypes({
        @JsonSubTypes.Type(value = Orders.class, name = "orders"),
        @JsonSubTypes.Type(value = WorkOrders.class, name = "workOrders")
})
public abstract class ElasticDocument {
// Fields and getter/setter
}
</code>```

My Pojo:

```<code>@JsonIgnoreProperties(ignoreUnknown = true)
@Data
public class Orders extends ElasticDocument {
//other fields

private List<OrderLine> orderLines;

}
</code>```

Input JSON which I am getting from input has different case e.g.

```<code>{
  "event": "orders",
  "OrderNo": 12345,
  "Status": "Created",
  "CustomerZipCode": "23456",
  "CustomerFirstName": "firstname1",
  "orderType": "PHONEORDER",
  "customerLastName": "lastname1",
  "OrderLines": [
    {
      "LineName": "sample"
    }
  ]
}
</code>```

My contoller method where I am using this ElasticDocument object:

```<code>@PostMapping("save")
public Orders save(@RequestBody ElasticDocument elasticDocument) {
    return elasticsearchRepository.save((Orders) elasticDocument);
}
</code>```

I am using Spring-boot version 2.2.4

答:
I think you forgot to add @type to your request JSON.@type is to identify the type of ElasticDocument being serialized.

Here is a example that i tried in my local system with minimum fields in class:

ElasticDocument.java

```<code>@JsonIgnoreProperties(ignoreUnknown = true)
@JsonTypeInfo(use = JsonTypeInfo.Id.NAME, include = JsonTypeInfo.As.PROPERTY)
@JsonSubTypes({
        @JsonSubTypes.Type(value = Orders.class, name = "Orders"),
        @JsonSubTypes.Type(value = WorkOrders.class, name = "workOrders")
})
public abstract class ElasticDocument {
    private Integer docId;
    private String docName;
   // getters and setters
}
</code>```

Orders.java

```<code>public class Orders extends ElasticDocument{
    private Integer orderId;
    private String orderName;
    // getters and setters
}
</code>```

WorkOrders.java

```<code>public class WorkOrders extends ElasticDocument{
    private Integer workOrderId;
    private String workOrderName;
    // getters and setters
}
</code>```

StackOverflowController.java

```<code>@RestController
@RequestMapping("/api/v1")
public class StackOverflowController {

    @PostMapping("/orders")
    ElasticDocument createOrder(@RequestBody ElasticDocument order){
        return order;
    }
}
</code>```

When i send data like this to my endpoint (Please note the attributes name in json are lowercase)

```<code>{
    "@type":"workOrders",
    "docId":123,
    "docName":"XXXX",
    "orderid":45,
    "ordername":"shoe",
    "workorderid":324,
    "workordername":"dsf"
}
</code>```

It is converted to workOrders response:

```<code>{
    "@type": "workOrders",
    "docId": 123,
    "docName": "XXXX",
    "workOrderId": 324,
    "workOrderName": "dsf"
}
</code>```

And when i changed the @type to Orders in request then i will get Order response:

```<code>{
    "@type": "Orders",
    "docId": 123,
    "docName": "XXXX",
    "orderId": 45,
    "orderName": "shoe"
}
</code>```

Body:
I know we can ignore case for input JSON by adding property in application.yml as:

spring:
  jackson:
    mapper:
      accept_case_insensitive_properties: true

But if my POJO extends an abstract class, it is not working and my JSON is not being parsed.

My abstract class:

@JsonIgnoreProperties(ignoreUnknown = true)
@JsonTypeInfo(
        use = JsonTypeInfo.Id.NAME,
        include = JsonTypeInfo.As.PROPERTY,
        property = "event")
@JsonSubTypes({
        @JsonSubTypes.Type(value = Orders.class, name = "orders"),
        @JsonSubTypes.Type(value = WorkOrders.class, name = "workOrders")
})
public abstract class ElasticDocument {
// Fields and getter/setter
}

My Pojo:

@JsonIgnoreProperties(ignoreUnknown = true)
@Data
public class Orders extends ElasticDocument {
//other fields

private List<OrderLine> orderLines;

}

Input JSON which I am getting from input has different case e.g.

{
  "event": "orders",
  "OrderNo": 12345,
  "Status": "Created",
  "CustomerZipCode": "23456",
  "CustomerFirstName": "firstname1",
  "orderType": "PHONEORDER",
  "customerLastName": "lastname1",
  "OrderLines": [
    {
      "LineName": "sample"
    }
  ]
}

My contoller method where I am using this ElasticDocument object:

@PostMapping("save")
public Orders save(@RequestBody ElasticDocument elasticDocument) {
    return elasticsearchRepository.save((Orders) elasticDocument);
}

I am using Spring-boot version 2.2.4

Body:
I think you forgot to add @type to your request JSON.@type is to identify the type of ElasticDocument being serialized.

Here is a example that i tried in my local system with minimum fields in class:

ElasticDocument.java

@JsonIgnoreProperties(ignoreUnknown = true)
@JsonTypeInfo(use = JsonTypeInfo.Id.NAME, include = JsonTypeInfo.As.PROPERTY)
@JsonSubTypes({
        @JsonSubTypes.Type(value = Orders.class, name = "Orders"),
        @JsonSubTypes.Type(value = WorkOrders.class, name = "workOrders")
})
public abstract class ElasticDocument {
    private Integer docId;
    private String docName;
   // getters and setters
}

Orders.java

public class Orders extends ElasticDocument{
    private Integer orderId;
    private String orderName;
    // getters and setters
}

WorkOrders.java

public class WorkOrders extends ElasticDocument{
    private Integer workOrderId;
    private String workOrderName;
    // getters and setters
}

StackOverflowController.java

@RestController
@RequestMapping("/api/v1")
public class StackOverflowController {

    @PostMapping("/orders")
    ElasticDocument createOrder(@RequestBody ElasticDocument order){
        return order;
    }
}

When i send data like this to my endpoint (Please note the attributes name in json are lowercase)

{
    "@type":"workOrders",
    "docId":123,
    "docName":"XXXX",
    "orderid":45,
    "ordername":"shoe",
    "workorderid":324,
    "workordername":"dsf"
}

It is converted to workOrders response:

{
    "@type": "workOrders",
    "docId": 123,
    "docName": "XXXX",
    "workOrderId": 324,
    "workOrderName": "dsf"
}

And when i changed the @type to Orders in request then i will get Order response:

{
    "@type": "Orders",
    "docId": 123,
    "docName": "XXXX",
    "orderId": 45,
    "orderName": "shoe"
}


问:
How to extract a json object only using basic commands like sed/awk in shell?
Like this:
I have a json file with below format:

```<code>{
    "Item": {
        "summary": {
            "B": "ABCDE"
        },
        "name": {
            "S": "sider"
        },
        "age": {
            "N": "1"
        },
        "data": {
            "B": "abcde"
        }
    }
}
</code>```

How can I get the object "Item" only using existed commands like sed/awk without installing any external tools in shell?

expected output:

```<code>{
    "summary": {
        "B": "ABCDE"
    },
    "name": {
        "S": "sider"
    },
    "age": {
        "N": "1"
    },
    "data": {
        "B": "abcde"
    }
}
</code>```

答:
As chepner suggests, you need to question your constraints.  In some cases, you are dealt a bad hand and have to deal with it.  So here is a sed approach:

Here's a revised solution with awk.  Sed is thrown in also to un-indent:

```<code>awk '
  /^    }/  { p = 0 ; print  }
  p == 1 { print }
  /"Item": {/ { print "   {" ; p = 1 }
'  | sed 's/^   //'
</code>```

Originally posted this (but noticed that the output didn't match your expectations):

```<code>sed -n '/^    "Item": {/,/^    }/p'
</code>```

The above sed approach assumes that Item is indented as you have it in the sample input above.

Here's an approach using my favorite, jq:

```<code>jq '.Item'
</code>```

You may do well to check if python with the json package is installed on your machine.  Here's a python3 script that would suit your needs:

```<code>#!/usr/bin/env python3

import json
import sys
j = json.load( sys.stdin )
print(json.dumps(j["Item"]))
</code>```

Body:
Like this:
I have a json file with below format:

{
    "Item": {
        "summary": {
            "B": "ABCDE"
        },
        "name": {
            "S": "sider"
        },
        "age": {
            "N": "1"
        },
        "data": {
            "B": "abcde"
        }
    }
}

How can I get the object "Item" only using existed commands like sed/awk without installing any external tools in shell?

expected output:

{
    "summary": {
        "B": "ABCDE"
    },
    "name": {
        "S": "sider"
    },
    "age": {
        "N": "1"
    },
    "data": {
        "B": "abcde"
    }
}

Body:
As chepner suggests, you need to question your constraints.  In some cases, you are dealt a bad hand and have to deal with it.  So here is a sed approach:

Here's a revised solution with awk.  Sed is thrown in also to un-indent:

awk '
  /^    }/  { p = 0 ; print  }
  p == 1 { print }
  /"Item": {/ { print "   {" ; p = 1 }
'  | sed 's/^   //'

Originally posted this (but noticed that the output didn't match your expectations):

sed -n '/^    "Item": {/,/^    }/p'

The above sed approach assumes that Item is indented as you have it in the sample input above.

Here's an approach using my favorite, jq:

jq '.Item'

You may do well to check if python with the json package is installed on your machine.  Here's a python3 script that would suit your needs:

#!/usr/bin/env python3

import json
import sys
j = json.load( sys.stdin )
print(json.dumps(j["Item"]))


问:
How to create an url from an absolute path in elm
I want to create an url (not a link) for a text field for a share functionality.
Like for example the one stackoverflow uses

I already hava function that produces the path part for the url, like

```<code>toUrl : Route -> String
toUrl route = ...

toUrl (Home (Just "hallo")) --> "/?b=hallo"
</code>```

and using this string for a link as a href attribute works, but I'm wondering how I could create a complete url from this string.

PS: I'm using a single page application so I get an Url at the beginning.

答:
What you say is "the path part for the url" isn't actually just the path part, but the path and query parts of the URL. Ideally you'd separate them so you can create a well-formed URL representation:

```<code>{ initialUrl
    | path = "/"
    , query = Just "b=hallo"
}
</code>```

But since it's just a record, with no validation, it'll work if you just use it as the path as well. At least if you later just use Url.toString on it. Other operations might cause unexpected results.

```<code>{ initialUrl | path = "/?b=hallo" }
</code>```

Body:
I want to create an url (not a link) for a text field for a share functionality.
Like for example the one stackoverflow uses

I already hava function that produces the path part for the url, like

toUrl : Route -> String
toUrl route = ...

toUrl (Home (Just "hallo")) --> "/?b=hallo"

and using this string for a link as a href attribute works, but I'm wondering how I could create a complete url from this string.

PS: I'm using a single page application so I get an Url at the beginning.

Body:
What you say is "the path part for the url" isn't actually just the path part, but the path and query parts of the URL. Ideally you'd separate them so you can create a well-formed URL representation:

{ initialUrl
    | path = "/"
    , query = Just "b=hallo"
}

But since it's just a record, with no validation, it'll work if you just use it as the path as well. At least if you later just use Url.toString on it. Other operations might cause unexpected results.

{ initialUrl | path = "/?b=hallo" }


问:
Most efficient way to check strings between two HTML <select>?
I'm kinda a newbie with Javascript and was proposed a problem of this sort. I have to fetch all files in a directory in PHP, then I have to list these files in two HTML <select> lists. The first one contains the extensions of the files I find in the directory while the second one contains the full filenames. When i select an extension (with an onchange in the first select) ALL the files that don't have that extension must be removed from the second <select>. The thing should work like this: I select "jpg" and all the files that aren't .jpg files must be removed from the second select. I made a function to manage this but it doesn't work as it should. Here I am posting the code I made:

```<code>echo("function visible_selection(){
    var ext = document.getElementById(\"ext\");
    var slen = document.getElementById(\"files\").length;
    var selected = ext.options[ext.selectedIndex].text;
    var filez = document.getElementById(\"files\");
    for(var i=0; i < slen; i++){
        var vdump = filez.options[i].text;
        var xcase = vdump.split(\".\");
        if (!vdump.includes(selected) && xcase[1] != selected){
            filez.remove(i);
        }
    }
    filez.style.display = \"inline\";
}");
</code>```

ext is obviously the first <select> that contains the extensions, while files is the second one with the full filenames. My question is, how can I correctly check if the extension is contained in the filename and therefore delete the files that don't have that kind of extension?

答:
-Edit. Apologies to @David. I miss-read the name. Thank you for your help.

Thank you @CBCrusher for breaking down the steps to debug the issue.

As OP has narrowed the issue down to the for loop I will help out with a working example.

You are currently loop through the array from 0 -> length. However, you are removing elements inside the loop, this will change the length of the array, (giving you out of bounds errors). The trick to this is looping through the array backwards.

function visible_selection() {
  var ext = document.getElementById("ext");
  var slen = document.getElementById("files").length;
  var selected = ext.options[ext.selectedIndex].text;
  var filez = document.getElementById("files");

  for (var i = slen - 1; i >= 0; i--) {
    var vdump = filez.options[i].text;
    var xcase = vdump.split(".");
    if (!vdump.includes(selected) && xcase[1] != selected) {
      filez.remove(i);
    }
  }
  filez.style.display = "inline";
}

document.getElementById("ext").onchange = () => {
  visible_selection()
}```
<pre class="snippet-code-html lang-html prettyprint-override"><code><label for="ext">Extentions:</label>

<select id="ext">
  <option disabled selected value> -- select an extention -- </option>
  <option value=".jpeg">.jpeg</option>
  <option value=".png">.png</option>
  <option value=".pdf">.pdf</option>
</select>

<label for="files">Files:</label>

<select id="files">
  <option disabled selected value> -- all files -- </option>
  <option value="File1.jpeg">File1.jpeg</option>
  <option value="Fiel2.jpeg">Fiel2.jpeg</option>
  <option value="Fiel3.png">Fiel3.png</option>
  <option value="Fiel4.pdf">Fiel4.pdf</option>
  <option value="Fiel5.pdf">Fiel5.pdf</option>
</select></code>```

Body:
I'm kinda a newbie with Javascript and was proposed a problem of this sort. I have to fetch all files in a directory in PHP, then I have to list these files in two HTML <select> lists. The first one contains the extensions of the files I find in the directory while the second one contains the full filenames. When i select an extension (with an onchange in the first select) ALL the files that don't have that extension must be removed from the second <select>. The thing should work like this: I select "jpg" and all the files that aren't .jpg files must be removed from the second select. I made a function to manage this but it doesn't work as it should. Here I am posting the code I made:

echo("function visible_selection(){
    var ext = document.getElementById(\"ext\");
    var slen = document.getElementById(\"files\").length;
    var selected = ext.options[ext.selectedIndex].text;
    var filez = document.getElementById(\"files\");
    for(var i=0; i < slen; i++){
        var vdump = filez.options[i].text;
        var xcase = vdump.split(\".\");
        if (!vdump.includes(selected) && xcase[1] != selected){
            filez.remove(i);
        }
    }
    filez.style.display = \"inline\";
}");

ext is obviously the first <select> that contains the extensions, while files is the second one with the full filenames. My question is, how can I correctly check if the extension is contained in the filename and therefore delete the files that don't have that kind of extension?

Body:
-Edit. Apologies to @David. I miss-read the name. Thank you for your help.

Thank you @CBCrusher for breaking down the steps to debug the issue.

As OP has narrowed the issue down to the for loop I will help out with a working example.

You are currently loop through the array from 0 -> length. However, you are removing elements inside the loop, this will change the length of the array, (giving you out of bounds errors). The trick to this is looping through the array backwards.

function visible_selection() {
  var ext = document.getElementById("ext");
  var slen = document.getElementById("files").length;
  var selected = ext.options[ext.selectedIndex].text;
  var filez = document.getElementById("files");

  for (var i = slen - 1; i >= 0; i--) {
    var vdump = filez.options[i].text;
    var xcase = vdump.split(".");
    if (!vdump.includes(selected) && xcase[1] != selected) {
      filez.remove(i);
    }
  }
  filez.style.display = "inline";
}

document.getElementById("ext").onchange = () => {
  visible_selection()
}

<label for="ext">Extentions:</label>

<select id="ext">
  <option disabled selected value> -- select an extention -- </option>
  <option value=".jpeg">.jpeg</option>
  <option value=".png">.png</option>
  <option value=".pdf">.pdf</option>
</select>

<label for="files">Files:</label>

<select id="files">
  <option disabled selected value> -- all files -- </option>
  <option value="File1.jpeg">File1.jpeg</option>
  <option value="Fiel2.jpeg">Fiel2.jpeg</option>
  <option value="Fiel3.png">Fiel3.png</option>
  <option value="Fiel4.pdf">Fiel4.pdf</option>
  <option value="Fiel5.pdf">Fiel5.pdf</option>
</select>


问:
How to ignore ssl certificate in codeception chrome headless browser?
I added in acceptance.suite.yml

```<code>chromeOptions:
    args: ["--headless", "--disable-gpu","--test-type","--ignore-certificate-errors"],
</code>```

but no luck? what can i do now

description  edit:

When i start tests in headless mode they get stuck, in the _output file (fail.png) i get blank page. So i think that they get to "Insecure Connection" page and can't get through it, and my question is how to avoid that page

答:
I suspect the argument you want is allow-insecure-localhost. This line worked for me to configure my acceptance.suite.yml file in CodeCeption.

```<code>- WebDriver:
    url: xxx.com
    window_size: false # disabled in ChromeDriver
    port: 9515
    browser: chrome
    capabilities:
        goog:chromeOptions:
            args: ["allow-insecure-localhost","headless","start-maximized"]
</code>```

This page lists all the options that chrome supports https://peter.sh/experiments/chromium-command-line-switches/#allow-insecure-localhost. Google themselves link to that (3rd party) page from their own page describing ChromeDriver config https://sites.google.com/a/chromium.org/chromedriver/capabilities.

ContentLicense: CC BY-SA 3.0

Body:
I added in acceptance.suite.yml

chromeOptions:
    args: ["--headless", "--disable-gpu","--test-type","--ignore-certificate-errors"],

but no luck? what can i do now

description  edit:

When i start tests in headless mode they get stuck, in the _output file (fail.png) i get blank page. So i think that they get to "Insecure Connection" page and can't get through it, and my question is how to avoid that page

Body:
I suspect the argument you want is allow-insecure-localhost. This line worked for me to configure my acceptance.suite.yml file in CodeCeption.

- WebDriver:
    url: xxx.com
    window_size: false # disabled in ChromeDriver
    port: 9515
    browser: chrome
    capabilities:
        goog:chromeOptions:
            args: ["allow-insecure-localhost","headless","start-maximized"]

This page lists all the options that chrome supports https://peter.sh/experiments/chromium-command-line-switches/#allow-insecure-localhost. Google themselves link to that (3rd party) page from their own page describing ChromeDriver config https://sites.google.com/a/chromium.org/chromedriver/capabilities.


问:
Create a Multi-Index pandas df by multiplying a df by N number of constants
I have a df:

```<code>dates         p1     p2    p3    p4
2019-12-01   0.4    0.4   0.4   0.3
2019-12-02   0.41   0.3   0.3   0.3
</code>```

And a constants df like this:

```<code>   v1      v2      v3
0   43  0.680068  680.068
1  210  0.319932  319.932
2  211  0.319932  319.932
</code>```

I would like to multiply each element in the first df by each value of v3 and create a multi-index which takes the v1 as the top-level index - something that looks like this:

```<code>                                 43                                 210  ...
dates            p1             p2         p3         p4       ...   p1 ...
2019-12-01   0.4 * 680.068    0.4* v3    0.4 * v3   0.3 * v3   ...  0.3 * 319.932
2019-12-02   0.41 * v3   0.3 * v3   0.3 * v3  0.3 * v3   ...   ...  ...
</code>```

So this would result in 1 df with 3 top-level indexes (43, 210, 211) and then to lower level will be 3 dfs that have had element-by-element multiplication

Body:
I have a df:

dates         p1     p2    p3    p4
2019-12-01   0.4    0.4   0.4   0.3
2019-12-02   0.41   0.3   0.3   0.3

And a constants df like this:

   v1      v2      v3
0   43  0.680068  680.068
1  210  0.319932  319.932
2  211  0.319932  319.932

I would like to multiply each element in the first df by each value of v3 and create a multi-index which takes the v1 as the top-level index - something that looks like this:

                                 43                                 210  ...
dates            p1             p2         p3         p4       ...   p1 ...
2019-12-01   0.4 * 680.068    0.4* v3    0.4 * v3   0.3 * v3   ...  0.3 * 319.932
2019-12-02   0.41 * v3   0.3 * v3   0.3 * v3  0.3 * v3   ...   ...  ...

So this would result in 1 df with 3 top-level indexes (43, 210, 211) and then to lower level will be 3 dfs that have had element-by-element multiplication

Body:
This is a simple extension of your previous question. I will still assume that dates is used as the index for the initial dataframe, say df:

              p1   p2   p3   p4
dates
2019-12-01  0.40  0.4  0.4  0.3
2019-12-02  0.41  0.3  0.3  0.3

Let us call v the second dataframe, and dg the result.

We can simply compute the values with concat:

dg = pd.concat([df * val for val in v['v3']], axis = 1)

Then, we compute the columns label with MultiIndex.from_products:

dg.columns = pd.MultiIndex.from_product([v['v1'], df.columns])

to obtain:

                  43                                       210                                     211
                   p1        p2        p3        p4         p1        p2        p3       p4         p1        p2        p3       p4
dates
2019-12-01  272.02720  272.0272  272.0272  204.0204  127.97280  127.9728  127.9728  95.9796  127.97280  127.9728  127.9728  95.9796
2019-12-02  278.82788  204.0204  204.0204  204.0204  131.17212   95.9796   95.9796  95.9796  131.17212   95.9796   95.9796  95.9796

It can even be done in one single pass (thank to anky_91 for the trick):

pd.concat([df * val for val in v['v3']], axis = 1,keys=v['v1'])

A multiindex can have names, so you can even do:

dg.columns.names=('v1', '')

to get:

v1                43                                       210                                     211
                   p1        p2        p3        p4         p1        p2        p3       p4         p1        p2        p3       p4
dates
2019-12-01  272.02720  272.0272  272.0272  204.0204  127.97280  127.9728  127.9728  95.9796  127.97280  127.9728  127.9728  95.9796
2019-12-02  278.82788  204.0204  204.0204  204.0204  131.17212   95.9796   95.9796  95.9796  131.17212   95.9796   95.9796  95.9796


问:
How big payload we can send in single transaction in Hyperledger Fabric. How many assets we can put in a single transaction?
1)  How big payload in terms of size, we can add in a single transaction in Hyperledger Fabric. I have millions of assets, ingesting into Fabric network. Maximum assets(Key-Value) can be added into the single transaction are 1000(Correct if I am wrong). Can we customize this number, so that a high number of an asset can be added in a single transaction in a short time?

2) What is an efficient way to trigger the huge number of an asset into the network?

I am trying to improve performance in terms of TPS and latency. I have already added necessary indexes and composite key (Couch DB)

答:
1) At the transport level, Fabric currently has a 100MB limit.  It cannot be changed.  There is some additional overhead added in some of the transaction wrappers, so to be safe, the total size of the key/values you write plus the size of the key/values you read should be no greater than ~90MB.

2) You can try implementing an operation on your chaincode which allows for multiple assets to be created.


问:
Firebase is not defined?
I'm working on an Angular project and use Firebase, and it's erroring with ReferenceError: Firebase is not defined, but I can not figure out why.

this is my index.html

```<code><!DOCTYPE html>
<head>
  <meta charset="utf-8">
  <meta http-equiv="X-UA-Compatible" content="IE=edge">
  <title>My Contacts App</title>
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <link rel="stylesheet" href="bower_components/foundation/css/foundation.css">
  <link rel="stylesheet" href="app.css">
</head>
<body>
  <div class="container">
    <div class="row">
      <div class="large-12 columns">
        <h1>myContacts</h1>
        <hr>
      </div>
    </div>
    <div ng-view></div>
  </div>
  <script src="bower_components/jquery/dist/jquery.js"></script>
  <script src="bower_components/angular/angular.js"></script>
  <script src="bower_components/angular-route/angular-route.js"></script>
  <script src="bower_components/firebase/firebase.js"></script>
  <script src="bower_components/angularfire/dist/angularfire.js"></script>
  <script src="bower_components/foundation/js/foundation.js"></script>
  <script src="app.js"></script>
  <script src="contacts/contacts.js"></script>
</body>
</html>
</code>```

my contact.js

```<code>'use strict';

angular.module('myContacts.contacts', ['ngRoute', 'firebase'])

.config(['$routeProvider', function($routeProvider) {
  $routeProvider.when('/contacts', {
    templateUrl: 'contacts/contacts.html',
    controller: 'ContactsCtrl'
  });
}])

.controller('ContactsCtrl', ['$scope', '$firebaseArray', function($scope, $firebaseArray) {
  var ref = new Firebase('https://<my_project_name>.firebaseio.com/contacts');

  $scope.contacts = $firebaseArray(ref);
  console.log($scope.contacts);
}]);
</code>```

my app.js

```<code>'use strict';

angular.module('myContacts', [
  'ngRoute',
  'firebase',
  'myContacts.contacts'
]).
config(['$routeProvider', function($routeProvider) {

  $routeProvider.otherwise({redirectTo: '/contacts'});
}]);
</code>```

and my package.json

```<code>{
  "name": "angular-seed",
  "private": true,
  "version": "0.0.0",
  "description": "A starter project for AngularJS",
  "repository": "https://github.com/angular/angular-seed",
  "license": "MIT",
  "devDependencies": {
    "bower": "^1.7.7",
    "http-server": "^0.9.0",
    "jasmine-core": "^2.4.1",
    "karma": "^0.13.22",
    "karma-chrome-launcher": "^0.2.3",
    "karma-firefox-launcher": "^0.1.7",
    "karma-jasmine": "^0.3.8",
    "karma-junit-reporter": "^0.4.1",
    "protractor": "^3.2.2",
    "shelljs": "^0.6.0",
    "firebase": "*"
  },
  "scripts": {
    "postinstall": "bower install",

    "prestart": "npm install",
    "start": "http-server -a localhost -p 8000 -c-1 ./app",

    "pretest": "npm install",
    "test": "karma start karma.conf.js",
    "test-single-run": "karma start karma.conf.js --single-run",

    "preupdate-webdriver": "npm install",
    "update-webdriver": "webdriver-manager update",

    "preprotractor": "npm run update-webdriver",
    "protractor": "protractor e2e-tests/protractor.conf.js",

    "update-index-async": "node -e \"require('shelljs/global'); sed('-i', /\\/\\/@@NG_LOADER_START@@[\\s\\S]*\\/\\/@@NG_LOADER_END@@/, '//@@NG_LOADER_START@@\\n' + sed(/sourceMappingURL=angular-loader.min.js.map/,'sourceMappingURL=bower_components/angular-loader/angular-loader.min.js.map','app/bower_components/angular-loader/angular-loader.min.js') + '\\n//@@NG_LOADER_END@@', 'app/index-async.html');\""
  }
}
</code>```

In package.json file I firebase assigned "*", I do not know if it can do it? Everything else looks good...

firebase from console

this is bower.json file

```<code>{
  "name": "angular-seed",
  "description": "A starter project for AngularJS",
  "version": "0.0.0",
  "homepage": "https://github.com/angular/angular-seed",
  "license": "MIT",
  "private": true,
  "dependencies": {
    "angular": "~1.5.0",
    "angular-route": "~1.5.0",
    "angular-loader": "~1.5.0",
    "angular-mocks": "~1.5.0",
    "html5-boilerplate": "^5.3.0"
  }
}
</code>```

whether there might have to give Firebase?

答:
In case of a hosting project the catch is that local script files are loading before Firebase. To prevent that, simply load them 'deferred', like this:

```<code><script defer src="app.js"></script>
</code>```

Thanks to this post by Paul De Jong!

Title: Firebase is not defined?

Body:
I'm working on an Angular project and use Firebase, and it's erroring with ReferenceError: Firebase is not defined, but I can not figure out why.

this is my index.html

<!DOCTYPE html>

  <meta charset="utf-8">
  <meta http-equiv="X-UA-Compatible" content="IE=edge">
  <title>My Contacts App</title>
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <link rel="stylesheet" href="bower_components/foundation/css/foundation.css">
  <link rel="stylesheet" href="app.css">

myContacts

my contact.js

'use strict';

angular.module('myContacts.contacts', ['ngRoute', 'firebase'])

.config(['$routeProvider', function($routeProvider) {
  $routeProvider.when('/contacts', {
    templateUrl: 'contacts/contacts.html',
    controller: 'ContactsCtrl'
  });
}])

.controller('ContactsCtrl', ['$scope', '$firebaseArray', function($scope, $firebaseArray) {
  var ref = new Firebase('https://<my_project_name>.firebaseio.com/contacts');

  $scope.contacts = $firebaseArray(ref);
  console.log($scope.contacts);
}]);

my app.js

'use strict';

angular.module('myContacts', [
  'ngRoute',
  'firebase',
  'myContacts.contacts'
]).
config(['$routeProvider', function($routeProvider) {

  $routeProvider.otherwise({redirectTo: '/contacts'});
}]);

and my package.json

{
  "name": "angular-seed",
  "private": true,
  "version": "0.0.0",
  "description": "A starter project for AngularJS",
  "repository": "https://github.com/angular/angular-seed",
  "license": "MIT",
  "devDependencies": {
    "bower": "^1.7.7",
    "http-server": "^0.9.0",
    "jasmine-core": "^2.4.1",
    "karma": "^0.13.22",
    "karma-chrome-launcher": "^0.2.3",
    "karma-firefox-launcher": "^0.1.7",
    "karma-jasmine": "^0.3.8",
    "karma-junit-reporter": "^0.4.1",
    "protractor": "^3.2.2",
    "shelljs": "^0.6.0",
    "firebase": "*"
  },
  "scripts": {
    "postinstall": "bower install",

    "prestart": "npm install",
    "start": "http-server -a localhost -p 8000 -c-1 ./app",

    "pretest": "npm install",
    "test": "karma start karma.conf.js",
    "test-single-run": "karma start karma.conf.js --single-run",

    "preupdate-webdriver": "npm install",
    "update-webdriver": "webdriver-manager update",

    "preprotractor": "npm run update-webdriver",
    "protractor": "protractor e2e-tests/protractor.conf.js",

    "update-index-async": "node -e \"require('shelljs/global'); sed('-i', /\\/\\/@@NG_LOADER_START@@[\\s\\S]*\\/\\/@@NG_LOADER_END@@/, '//@@NG_LOADER_START@@\\n' + sed(/sourceMappingURL=angular-loader.min.js.map/,'sourceMappingURL=bower_components/angular-loader/angular-loader.min.js.map','app/bower_components/angular-loader/angular-loader.min.js') + '\\n//@@NG_LOADER_END@@', 'app/index-async.html');\""
  }
}

In package.json file I firebase assigned "*", I do not know if it can do it? Everything else looks good...

firebase from console

this is bower.json file

{
  "name": "angular-seed",
  "description": "A starter project for AngularJS",
  "version": "0.0.0",
  "homepage": "https://github.com/angular/angular-seed",
  "license": "MIT",
  "private": true,
  "dependencies": {
    "angular": "~1.5.0",
    "angular-route": "~1.5.0",
    "angular-loader": "~1.5.0",
    "angular-mocks": "~1.5.0",
    "html5-boilerplate": "^5.3.0"
  }
}

whether there might have to give Firebase?

Body:
In case of a hosting project the catch is that local script files are loading before Firebase. To prevent that, simply load them 'deferred', like this:

<script defer src="app.js"></script>

Thanks to this post by Paul De Jong!