问:
Web scraping multiple google scholar pages in python
I want to scrape multiple Google scholar user profiles - publications, journals, citations etc. I have already written the python code for scraping a user profile given the url. Now, suppose I have 100 names and the corresponding urls in an excel file like this.

```<code>name       link

Autor      https://scholar.google.com/citations?user=cp-8uaAAAAAJ&hl=en
Dorn       https://scholar.google.com/citations?user=w3Dri00AAAAJ&hl=en
Hanson     https://scholar.google.com/citations?user=nMtHiQsAAAAJ&hl=en
Borjas     https://scholar.google.com/citations?user=Patm-BEAAAAJ&hl=en
....

</code>```

My question is can I read the 'link' column of this file and write a for loop for the urls so that I can scrape each of these profiles and append the results in the same file. I seems a bit far fetched but I hope there is a way to do so. Thanks in advance!

答:
To read data from an Excel file you can use pandas read_excel() method like so:

# https://pandas.pydata.org/docs/reference/api/pandas.read_excel.html
authors_df = pd.read_excel("google_scholar_scrape_multiple_authors.xlsx", sheet_name="authors") # sheet_name is optional in this case

print(authors_df["author_link"])

'''
0    https://scholar.google.com/citations?hl=en&use...
1    https://scholar.google.com/citations?hl=en&use...
2    https://scholar.google.com/citations?hl=en&use...
3    https://scholar.google.com/citations?hl=en&use...
4    https://scholar.google.com/citations?hl=en&use...
Name: author_link, dtype: object
'''

print(authors_df)

'''
         author_name                                        author_link
0      Masatoshi Nei  https://scholar.google.com/citations?hl=en&use...
1  Ulrich K. Laemmli  https://scholar.google.com/citations?hl=en&use...
2         Gene Myers  https://scholar.google.com/citations?hl=en&use...
3       Sudhir Kumar  https://scholar.google.com/citations?hl=en&use...
4    Irving Weissman  https://scholar.google.com/citations?hl=en&use...
'''
```
<hr />
<p>To scrape from multiple authors you can use a <code>for</code> loop to iterate over <code>["author_link"]</code> and extract desired data using <code>beautifulsoup</code>, <code>lxml</code>, and <code>requests</code> libraries.</p>
<hr />
<p>Code and full example in the online IDE:</p>
<pre class="lang-py prettyprint-override"><code>from bs4 import BeautifulSoup
import requests, lxml
import pandas as pd

# https://docs.python-requests.org/en/master/user/quickstart/#custom-headers
headers = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3538.102 Safari/537.36 Edge/18.19582",
}

# https://pandas.pydata.org/docs/reference/api/pandas.read_excel.html
authors_df = pd.read_excel("google_scholar_scrape_multiple_authors.xlsx", sheet_name="authors") # sheet_name is optional in this case

# to_list() returns a list of author links so we can iterate over them
for author_link in authors_df["author_link"].to_list():
    html = requests.get(author_link, headers=headers, timeout=30)
    soup = BeautifulSoup(html.text, "lxml")

    print(f"Currently extracting: {soup.select_one('#gsc_prf_in').text}")

    author_email = soup.select_one("#gsc_prf_ivh").text
    author_image = f'https://scholar.google.com{soup.select_one("#gsc_prf_pup-img")["src"]}'

    print(author_image, f"Author email: {author_email}", sep="\n")

    # iterating over container with all needed data by accessing the right CSS selector
    # have a look at SelectorGadget Chrome extension to easily grab CSS selectors
    for article in soup.select("#gsc_a_b .gsc_a_t"):
        article_title = article.select_one(".gsc_a_at").text
        article_link = f'https://scholar.google.com{article.select_one(".gsc_a_at")["href"]}'
        article_authors = article.select_one(".gsc_a_at+ .gs_gray").text
        article_publication = article.select_one(".gs_gray+ .gs_gray").text

        print(article_title, article_link, article_authors, article_publication, sep="\n")

    print("-" * 15)

# part of the output:
'''
Currently extracting: Masatoshi Nei
https://scholar.google.comhttps://scholar.googleusercontent.com/citations?view_op=view_photo&user=VxOmZDgAAAAJ&citpid=3
Author email: Verified email at temple.edu
The neighbor-joining method: a new method for reconstructing phylogenetic trees.
https://scholar.google.com/citations?view_op=view_citation&hl=en&user=VxOmZDgAAAAJ&citation_for_view=VxOmZDgAAAAJ:u5HHmVD_uO8C
N Saitou, M Nei
Molecular biology and evolution 4 (4), 406-425, 1987

... other results

---------------
Currently extracting: Irving Weissman
https://scholar.google.com/citations/images/avatar_scholar_128.png
Author email: Verified email at stanford.edu
Stem cells, cancer, and cancer stem cells
https://scholar.google.com/citations?view_op=view_citation&hl=en&user=Y66bJgUAAAAJ&citation_for_view=Y66bJgUAAAAJ:u5HHmVD_uO8C
T Reya, SJ Morrison, MF Clarke, IL Weissman
nature 414 (6859), 105-111, 2001
'''
</code>```

Alternatively, you can achieve the same thing using Google Scholar Author API from SeprApi. It's a paid API with a free plan.

Essentially, you only need to grab the data you want from the received dictionary without the need to figure out what selectors to use in order to scrape the proper data, how to bypass blocks from Google, how to increase the number of requests.

Example code to integrate:

import re
import pandas as pd
from serpapi import GoogleSearch

authors_df = pd.read_excel("google_scholar_scrape_multiple_authors.xlsx", sheet_name="authors")  # sheet_name is optional in this case

for author in authors_df["author_link"].to_list():
    params = {
        "api_key": "YOUR_API_KEY",
        "engine": "google_scholar_author",
        "hl": "en",
        # using basic regular expression to grab user ID from the passed URL
        "author_id": re.search(r"user=(.*)", author).group(1)  # -> VxOmZDgAAAAJ, unique author ID from the URL
    }

    search = GoogleSearch(params)
    results = search.get_dict()

    print(f"Extracting data from: {results['author']['name']}\n"
          f"Author info: {results['author']}\n\n"
          f"Author articles:\n{results['articles']}\n")

# part of the output:
'''
Extracting data from: Masatoshi Nei
Author info: {'name': 'Masatoshi Nei', 'affiliations': 'Laura Carnell Professor of Biology, Temple University', 'email': 'Verified email at temple.edu', 'interests': [{'title': 'Evolution', 'link': 'https://scholar.google.com/citations?view_op=search_authors&hl=en&mauthors=label:evolution', 'serpapi_link': 'https://serpapi.com/search.json?engine=google_scholar_profiles&hl=en&mauthors=label%3Aevolution'}, {'title': 'Evolutionary biology', 'link': 'https://scholar.google.com/citations?view_op=search_authors&hl=en&mauthors=label:evolutionary_biology', 'serpapi_link': 'https://serpapi.com/search.json?engine=google_scholar_profiles&hl=en&mauthors=label%3Aevolutionary_biology'}, {'title': 'Molecular evolution', 'link': 'https://scholar.google.com/citations?view_op=search_authors&hl=en&mauthors=label:molecular_evolution', 'serpapi_link': 'https://serpapi.com/search.json?engine=google_scholar_profiles&hl=en&mauthors=label%3Amolecular_evolution'}, {'title': 'Population genetics', 'link': 'https://scholar.google.com/citations?view_op=search_authors&hl=en&mauthors=label:population_genetics', 'serpapi_link': 'https://serpapi.com/search.json?engine=google_scholar_profiles&hl=en&mauthors=label%3Apopulation_genetics'}, {'title': 'Phylogenetics', 'link': 'https://scholar.google.com/citations?view_op=search_authors&hl=en&mauthors=label:phylogenetics', 'serpapi_link': 'https://serpapi.com/search.json?engine=google_scholar_profiles&hl=en&mauthors=label%3Aphylogenetics'}], 'thumbnail': 'https://scholar.googleusercontent.com/citations?view_op=view_photo&user=VxOmZDgAAAAJ&citpid=3'}

Author articles:
[{'title': 'The neighbor-joining method: a new method for reconstructing phylogenetic trees.', 'link': 'https://scholar.google.com/citations?view_op=view_citation&hl=en&user=VxOmZDgAAAAJ&citation_for_view=VxOmZDgAAAAJ:u5HHmVD_uO8C', 'citation_id': 'VxOmZDgAAAAJ:u5HHmVD_uO8C', 'authors': 'N Saitou, M Nei', 'publication': 'Molecular biology and evolution 4 (4), 406-425, 1987', 'cited_by': {'value': 64841, 'link': 'https://scholar.google.com/scholar?oi=bibs&hl=en&cites=7672721046330422437,346314157833338191', 'serpapi_link': 'https://serpapi.com/search.json?cites=7672721046330422437%2C346314157833338191&engine=google_scholar&hl=en', 'cites_id': '7672721046330422437,346314157833338191'}, 'year': '1987'}, {'title': 'MEGA5: molecular evolutionary genetics analysis using maximum likelihood, evolutionary distance, and maximum parsimony methods', 'link': 'https://scholar.google.com/citations?view_op=view_citation&hl=en&user=VxOmZDgAAAAJ&citation_for_view=VxOmZDgAAAAJ:Tyk-4Ss8FVUC', 'citation_id': 'VxOmZDgAAAAJ:Tyk-4Ss8FVUC', 'authors': 'K Tamura, D Peterson, N Peterson, G Stecher, M Nei, S Kumar', 'publication': 'Molecular biology and evolution 28 (10), 2731-2739, 2011', 'cited_by': {'value': 44316, 'link': 'https://scholar.google.com/scholar?oi=bibs&hl=en&cites=5624029996178252455,5910675136328950108,13537318717249213469', 'serpapi_link': 'https://serpapi.com/search.json?cites=5624029996178252455%2C5910675136328950108%2C13537318717249213469&engine=google_scholar&hl=en', 'cites_id': '5624029996178252455,5910675136328950108,13537318717249213469'}, 'year': '2011'}, {'title': 'MEGA6: molecular evolutionary genetics analysis version 6.0', 'link': 'https://scholar.google.com/citations?view_op=view_citation&hl=en&user=VxOmZDgAAAAJ&citation_for_view=VxOmZDgAAAAJ:qmtmRrLr0tkC', 'citation_id': 'VxOmZDgAAAAJ:qmtmRrLr0tkC', 'authors': 'K Tamura, G Stecher, D Peterson, A Filipski, S Kumar', 'publication': 'Molecular biology and evolution 30 (12), 2725-2729, 2013', 'cited_by': {'value': 40558, 'link': 'https://scholar.google.com/scholar?oi=bibs&hl=en&cites=5258359186493639031', 'serpapi_link': 'https://serpapi.com/search.json?cites=5258359186493639031&engine=google_scholar&hl=en', 'cites_id': '5258359186493639031'}, 'year': '2013'}, {'title': 'MEGA4: molecular evolutionary genetics analysis (MEGA) software version 4.0', 'link': 'https://scholar.google.com/citations?view_op=view_citation&hl=en&user=VxOmZDgAAAAJ&citation_for_view=VxOmZDgAAAAJ:u-x6o8ySG0sC', 'citation_id': 'VxOmZDgAAAAJ:u-x6o8ySG0sC', 'authors': 'K Tamura, J Dudley, M Nei, S Kumar', 'publication': 'Molecular biology and evolution 24 (8), 1596-1599, 2007', 'cited_by': {'value': 34245, 'link': 'https://scholar.google.com/scholar?oi=bibs&hl=en&cites=8480751610153565117', 'serpapi_link': 'https://serpapi.com/search.json?cites=8480751610153565117&engine=google_scholar&hl=en', 'cites_id': '8480751610153565117'}, 'year': '2007'}, {'title': 'Molecular evolutionary genetics', 'link': 'https://scholar.google.com/citations?view_op=view_citation&hl=en&user=VxOmZDgAAAAJ&citation_for_view=VxOmZDgAAAAJ:d1gkVwhDpl0C', 'citation_id': 'VxOmZDgAAAAJ:d1gkVwhDpl0C', 'authors': 'M Nei', 'publication': 'Columbia university press, 1987', 'cited_by': {'value': 20704, 'link': 'https://scholar.google.com/scholar?oi=bibs&hl=en&cites=7660515423132980153', 'serpapi_link': 'https://serpapi.com/search.json?cites=7660515423132980153&engine=google_scholar&hl=en', 'cites_id': '7660515423132980153'}, 'year': '1987'}, {'title': 'MEGA2: molecular evolutionary genetics analysis software', 'link': 'https://scholar.google.com/citations?view_op=view_citation&hl=en&user=VxOmZDgAAAAJ&citation_for_view=VxOmZDgAAAAJ:IjCSPb-OGe4C', 'citation_id': 'VxOmZDgAAAAJ:IjCSPb-OGe4C', 'authors': 'S Kumar, K Tamura, IB Jakobsen, M Nei', 'publication': 'Bioinformatics 17 (12), 1244-1245, 2001', 'cited_by': {'value': 16078, 'link': 'https://scholar.google.com/scholar?oi=bibs&hl=en&cites=14171206204658643394,531426008085525562,5869149036159079676,8067244568899724142,12929609819447339488,15783386726452728786', 'serpapi_link': 'https://serpapi.com/search.json?cites=14171206204658643394%2C531426008085525562%2C5869149036159079676%2C8067244568899724142%2C12929609819447339488%2C15783386726452728786&engine=google_scholar&hl=en', 'cites_id': '14171206204658643394,531426008085525562,5869149036159079676,8067244568899724142,12929609819447339488,15783386726452728786'}, 'year': '2001'}, {'title': 'Estimation of average heterozygosity and genetic distance from a small number of individuals', 'link': 'https://scholar.google.com/citations?view_op=view_citation&hl=en&user=VxOmZDgAAAAJ&citation_for_view=VxOmZDgAAAAJ:2osOgNQ5qMEC', 'citation_id': 'VxOmZDgAAAAJ:2osOgNQ5qMEC', 'authors': 'M Nei', 'publication': 'Genetics 89 (3), 583-590, 1978', 'cited_by': {'value': 14504, 'link': 'https://scholar.google.com/scholar?oi=bibs&hl=en&cites=11038674224870321151', 'serpapi_link': 'https://serpapi.com/search.json?cites=11038674224870321151&engine=google_scholar&hl=en', 'cites_id': '11038674224870321151'}, 'year': '1978'}, {'title': 'MEGA3: integrated software for molecular evolutionary genetics analysis and sequence alignment', 'link': 'https://scholar.google.com/citations?view_op=view_citation&hl=en&user=VxOmZDgAAAAJ&citation_for_view=VxOmZDgAAAAJ:9yKSN-GCB0IC', 'citation_id': 'VxOmZDgAAAAJ:9yKSN-GCB0IC', 'authors': 'S Kumar, K Tamura, M Nei', 'publication': 'Briefings in bioinformatics 5 (2), 150-163, 2004', 'cited_by': {'value': 14403, 'link': 'https://scholar.google.com/scholar?oi=bibs&hl=en&cites=10013295782066828040,15148316572039251274', 'serpapi_link': 'https://serpapi.com/search.json?cites=10013295782066828040%2C15148316572039251274&engine=google_scholar&hl=en', 'cites_id': '10013295782066828040,15148316572039251274'}, 'year': '2004'}, {'title': 'Mathematical model for studying genetic variation in terms of restriction endonucleases', 'link': 'https://scholar.google.com/citations?view_op=view_citation&hl=en&user=VxOmZDgAAAAJ&citation_for_view=VxOmZDgAAAAJ:qjMakFHDy7sC', 'citation_id': 'VxOmZDgAAAAJ:qjMakFHDy7sC', 'authors': 'M Nei, WH Li', 'publication': 'Proceedings of the National Academy of Sciences 76 (10), 5269-5273, 1979', 'cited_by': {'value': 13619, 'link': 'https://scholar.google.com/scholar?oi=bibs&hl=en&cites=5179626164554275201,1942230974501958280', 'serpapi_link': 'https://serpapi.com/search.json?cites=5179626164554275201%2C1942230974501958280&engine=google_scholar&hl=en', 'cites_id': '5179626164554275201,1942230974501958280'}, 'year': '1979'}, {'title': 'Genetic distance between populations', 'link': 'https://scholar.google.com/citations?view_op=view_citation&hl=en&user=VxOmZDgAAAAJ&citation_for_view=VxOmZDgAAAAJ:UeHWp8X0CEIC', 'citation_id': 'VxOmZDgAAAAJ:UeHWp8X0CEIC', 'authors': 'M Nei', 'publication': 'The American Naturalist 106 (949), 283-292, 1972', 'cited_by': {'value': 12980, 'link': 'https://scholar.google.com/scholar?oi=bibs&hl=en&cites=4154924214026252226,7115074001272219295', 'serpapi_link': 'https://serpapi.com/search.json?cites=4154924214026252226%2C7115074001272219295&engine=google_scholar&hl=en', 'cites_id': '4154924214026252226,7115074001272219295'}, 'year': '1972'}, {'title': 'Estimation of the number of nucleotide substitutions in the control region of mitochondrial DNA in humans and chimpanzees.', 'link': 'https://scholar.google.com/citations?view_op=view_citation&hl=en&user=VxOmZDgAAAAJ&citation_for_view=VxOmZDgAAAAJ:Y0pCki6q_DkC', 'citation_id': 'VxOmZDgAAAAJ:Y0pCki6q_DkC', 'authors': 'K Tamura, M Nei', 'publication': 'Molecular biology and evolution 10 (3), 512-526, 1993', 'cited_by': {'value': 11093, 'link': 'https://scholar.google.com/scholar?oi=bibs&hl=en&cites=13509507708085673250', 'serpapi_link': 'https://serpapi.com/search.json?cites=13509507708085673250&engine=google_scholar&hl=en', 'cites_id': '13509507708085673250'}, 'year': '1993'}, {'title': 'Analysis of gene diversity in subdivided populations', 'link': 'https://scholar.google.com/citations?view_op=view_citation&hl=en&user=VxOmZDgAAAAJ&citation_for_view=VxOmZDgAAAAJ:zYLM7Y9cAGgC', 'citation_id': 'VxOmZDgAAAAJ:zYLM7Y9cAGgC', 'authors': 'M Nei', 'publication': 'Proceedings of the national academy of sciences 70 (12), 3321-3323, 1973', 'cited_by': {'value': 10714, 'link': 'https://scholar.google.com/scholar?oi=bibs&hl=en&cites=11712109356391350421', 'serpapi_link': 'https://serpapi.com/search.json?cites=11712109356391350421&engine=google_scholar&hl=en', 'cites_id': '11712109356391350421'}, 'year': '1973'}, {'title': 'Molecular evolution and phylogenetics', 'link': 'https://scholar.google.com/citations?view_op=view_citation&hl=en&user=VxOmZDgAAAAJ&citation_for_view=VxOmZDgAAAAJ:YsMSGLbcyi4C', 'citation_id': 'VxOmZDgAAAAJ:YsMSGLbcyi4C', 'authors': 'M Nei, S Kumar', 'publication': 'Oxford University Press, USA, 2000', 'cited_by': {'value': 8795, 'link': 'https://scholar.google.com/scholar?oi=bibs&hl=en&cites=703217195301701212,1351927951694611906', 'serpapi_link': 'https://serpapi.com/search.json?cites=703217195301701212%2C1351927951694611906&engine=google_scholar&hl=en', 'cites_id': '703217195301701212,1351927951694611906'}, 'year': '2000'}, {'title': 'Simple methods for estimating the numbers of synonymous and nonsynonymous nucleotide substitutions.', 'link': 'https://scholar.google.com/citations?view_op=view_citation&hl=en&user=VxOmZDgAAAAJ&citation_for_view=VxOmZDgAAAAJ:W7OEmFMy1HYC', 'citation_id': 'VxOmZDgAAAAJ:W7OEmFMy1HYC', 'authors': 'M Nei, T Gojobori', 'publication': 'Molecular biology and evolution 3 (5), 418-426, 1986', 'cited_by': {'value': 5279, 'link': 'https://scholar.google.com/scholar?oi=bibs&hl=en&cites=12106160511321461626', 'serpapi_link': 'https://serpapi.com/search.json?cites=12106160511321461626&engine=google_scholar&hl=en', 'cites_id': '12106160511321461626'}, 'year': '1986'}, {'title': 'Prospects for inferring very large phylogenies by using the neighbor-joining method', 'link': 'https://scholar.google.com/citations?view_op=view_citation&hl=en&user=VxOmZDgAAAAJ&citation_for_view=VxOmZDgAAAAJ:0EnyYjriUFMC', 'citation_id': 'VxOmZDgAAAAJ:0EnyYjriUFMC', 'authors': 'K Tamura, M Nei, S Kumar', 'publication': 'Proceedings of the National Academy of Sciences 101 (30), 11030-11035, 2004', 'cited_by': {'value': 4882, 'link': 'https://scholar.google.com/scholar?oi=bibs&hl=en&cites=9650987578903829104', 'serpapi_link': 'https://serpapi.com/search.json?cites=9650987578903829104&engine=google_scholar&hl=en', 'cites_id': '9650987578903829104'}, 'year': '2004'}, {'title': 'The bottleneck effect and genetic variability in populations', 'link': 'https://scholar.google.com/citations?view_op=view_citation&hl=en&user=VxOmZDgAAAAJ&citation_for_view=VxOmZDgAAAAJ:eQOLeE2rZwMC', 'citation_id': 'VxOmZDgAAAAJ:eQOLeE2rZwMC', 'authors': 'M Nei, T Maruyama, R Chakraborty', 'publication': 'Evolution, 1-10, 1975', 'cited_by': {'value': 3906, 'link': 'https://scholar.google.com/scholar?oi=bibs&hl=en&cites=13149273985544466189', 'serpapi_link': 'https://serpapi.com/search.json?cites=13149273985544466189&engine=google_scholar&hl=en', 'cites_id': '13149273985544466189'}, 'year': '1975'}, {'title': 'Accuracy of estimated phylogenetic trees from molecular data', 'link': 'https://scholar.google.com/citations?view_op=view_citation&hl=en&user=VxOmZDgAAAAJ&citation_for_view=VxOmZDgAAAAJ:Se3iqnhoufwC', 'citation_id': 'VxOmZDgAAAAJ:Se3iqnhoufwC', 'authors': 'M Nei, F Tajima, Y Tateno', 'publication': 'Journal of molecular evolution 19 (2), 153-170, 1983', 'cited_by': {'value': 2877, 'link': 'https://scholar.google.com/scholar?oi=bibs&hl=en&cites=10638180566709737898', 'serpapi_link': 'https://serpapi.com/search.json?cites=10638180566709737898&engine=google_scholar&hl=en', 'cites_id': '10638180566709737898'}, 'year': '1983'}, {'title': 'Molecular population genetics and evolution.', 'link': 'https://scholar.google.com/citations?view_op=view_citation&hl=en&user=VxOmZDgAAAAJ&citation_for_view=VxOmZDgAAAAJ:WF5omc3nYNoC', 'citation_id': 'VxOmZDgAAAAJ:WF5omc3nYNoC', 'authors': 'M Nei', 'publication': 'Molecular population genetics and evolution., 1975', 'cited_by': {'value': 2795, 'link': 'https://scholar.google.com/scholar?oi=bibs&hl=en&cites=7886550802885580479', 'serpapi_link': 'https://serpapi.com/search.json?cites=7886550802885580479&engine=google_scholar&hl=en', 'cites_id': '7886550802885580479'}, 'year': '1975'}, {'title': 'Pattern of nucleotide substitution at major histocompatibility complex class I loci reveals overdominant selection', 'link': 'https://scholar.google.com/citations?view_op=view_citation&hl=en&user=VxOmZDgAAAAJ&citation_for_view=VxOmZDgAAAAJ:roLk4NBRz8UC', 'citation_id': 'VxOmZDgAAAAJ:roLk4NBRz8UC', 'authors': 'AL Hughes, M Nei', 'publication': 'Nature 335 (6186), 167-170, 1988', 'cited_by': {'value': 2169, 'link': 'https://scholar.google.com/scholar?oi=bibs&hl=en&cites=2966744676732650646', 'serpapi_link': 'https://serpapi.com/search.json?cites=2966744676732650646&engine=google_scholar&hl=en', 'cites_id': '2966744676732650646'}, 'year': '1988'}, {'title': 'Sampling variances of heterozygosity and genetic distance', 'link': 'https://scholar.google.com/citations?view_op=view_citation&hl=en&user=VxOmZDgAAAAJ&citation_for_view=VxOmZDgAAAAJ:UebtZRa9Y70C', 'citation_id': 'VxOmZDgAAAAJ:UebtZRa9Y70C', 'authors': 'M Nei, AK Roychoudhury', 'publication': 'Genetics 76 (2), 379-390, 1974', 'cited_by': {'value': 1918, 'link': 'https://scholar.google.com/scholar?oi=bibs&hl=en&cites=5978996318059495400', 'serpapi_link': 'https://serpapi.com/search.json?cites=5978996318059495400&engine=google_scholar&hl=en', 'cites_id': '5978996318059495400'}, 'year': '1974'}]
'''
```

Disclaimer, I work for SerpApi.

Body:
I want to scrape multiple Google scholar user profiles - publications, journals, citations etc. I have already written the python code for scraping a user profile given the url. Now, suppose I have 100 names and the corresponding urls in an excel file like this.

name       link

Autor      https://scholar.google.com/citations?user=cp-8uaAAAAAJ&hl=en
Dorn       https://scholar.google.com/citations?user=w3Dri00AAAAJ&hl=en
Hanson     https://scholar.google.com/citations?user=nMtHiQsAAAAJ&hl=en
Borjas     https://scholar.google.com/citations?user=Patm-BEAAAAJ&hl=en
....

My question is can I read the 'link' column of this file and write a for loop for the urls so that I can scrape each of these profiles and append the results in the same file. I seems a bit far fetched but I hope there is a way to do so. Thanks in advance!

Body:
To read data from an Excel file you can use pandas read_excel() method like so:

# https://pandas.pydata.org/docs/reference/api/pandas.read_excel.html
authors_df = pd.read_excel("google_scholar_scrape_multiple_authors.xlsx", sheet_name="authors") # sheet_name is optional in this case

print(authors_df["author_link"])

'''
0    https://scholar.google.com/citations?hl=en&use...
1    https://scholar.google.com/citations?hl=en&use...
2    https://scholar.google.com/citations?hl=en&use...
3    https://scholar.google.com/citations?hl=en&use...
4    https://scholar.google.com/citations?hl=en&use...
Name: author_link, dtype: object
'''

print(authors_df)

'''
         author_name                                        author_link
0      Masatoshi Nei  https://scholar.google.com/citations?hl=en&use...
1  Ulrich K. Laemmli  https://scholar.google.com/citations?hl=en&use...
2         Gene Myers  https://scholar.google.com/citations?hl=en&use...
3       Sudhir Kumar  https://scholar.google.com/citations?hl=en&use...
4    Irving Weissman  https://scholar.google.com/citations?hl=en&use...
'''

To scrape from multiple authors you can use a for loop to iterate over ["author_link"] and extract desired data using beautifulsoup, lxml, and requests libraries.

Code and full example in the online IDE:

from bs4 import BeautifulSoup
import requests, lxml
import pandas as pd

# https://docs.python-requests.org/en/master/user/quickstart/#custom-headers
headers = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3538.102 Safari/537.36 Edge/18.19582",
}

# https://pandas.pydata.org/docs/reference/api/pandas.read_excel.html
authors_df = pd.read_excel("google_scholar_scrape_multiple_authors.xlsx", sheet_name="authors") # sheet_name is optional in this case

# to_list() returns a list of author links so we can iterate over them
for author_link in authors_df["author_link"].to_list():
    html = requests.get(author_link, headers=headers, timeout=30)
    soup = BeautifulSoup(html.text, "lxml")

    print(f"Currently extracting: {soup.select_one('#gsc_prf_in').text}")

    author_email = soup.select_one("#gsc_prf_ivh").text
    author_image = f'https://scholar.google.com{soup.select_one("#gsc_prf_pup-img")["src"]}'

    print(author_image, f"Author email: {author_email}", sep="\n")

    # iterating over container with all needed data by accessing the right CSS selector
    # have a look at SelectorGadget Chrome extension to easily grab CSS selectors
    for article in soup.select("#gsc_a_b .gsc_a_t"):
        article_title = article.select_one(".gsc_a_at").text
        article_link = f'https://scholar.google.com{article.select_one(".gsc_a_at")["href"]}'
        article_authors = article.select_one(".gsc_a_at+ .gs_gray").text
        article_publication = article.select_one(".gs_gray+ .gs_gray").text

        print(article_title, article_link, article_authors, article_publication, sep="\n")

    print("-" * 15)

# part of the output:
'''
Currently extracting: Masatoshi Nei
https://scholar.google.comhttps://scholar.googleusercontent.com/citations?view_op=view_photo&user=VxOmZDgAAAAJ&citpid=3
Author email: Verified email at temple.edu
The neighbor-joining method: a new method for reconstructing phylogenetic trees.
https://scholar.google.com/citations?view_op=view_citation&hl=en&user=VxOmZDgAAAAJ&citation_for_view=VxOmZDgAAAAJ:u5HHmVD_uO8C
N Saitou, M Nei
Molecular biology and evolution 4 (4), 406-425, 1987

... other results

---------------
Currently extracting: Irving Weissman
https://scholar.google.com/citations/images/avatar_scholar_128.png
Author email: Verified email at stanford.edu
Stem cells, cancer, and cancer stem cells
https://scholar.google.com/citations?view_op=view_citation&hl=en&user=Y66bJgUAAAAJ&citation_for_view=Y66bJgUAAAAJ:u5HHmVD_uO8C
T Reya, SJ Morrison, MF Clarke, IL Weissman
nature 414 (6859), 105-111, 2001
'''

Alternatively, you can achieve the same thing using Google Scholar Author API from SeprApi. It's a paid API with a free plan.

Essentially, you only need to grab the data you want from the received dictionary without the need to figure out what selectors to use in order to scrape the proper data, how to bypass blocks from Google, how to increase the number of requests.

Example code to integrate:

import re
import pandas as pd
from serpapi import GoogleSearch

authors_df = pd.read_excel("google_scholar_scrape_multiple_authors.xlsx", sheet_name="authors")  # sheet_name is optional in this case

for author in authors_df["author_link"].to_list():
    params = {
        "api_key": "YOUR_API_KEY",
        "engine": "google_scholar_author",
        "hl": "en",
        # using basic regular expression to grab user ID from the passed URL
        "author_id": re.search(r"user=(.*)", author).group(1)  # -> VxOmZDgAAAAJ, unique author ID from the URL
    }

    search = GoogleSearch(params)
    results = search.get_dict()

    print(f"Extracting data from: {results['author']['name']}\n"
          f"Author info: {results['author']}\n\n"
          f"Author articles:\n{results['articles']}\n")

# part of the output:
'''
Extracting data from: Masatoshi Nei
Author info: {'name': 'Masatoshi Nei', 'affiliations': 'Laura Carnell Professor of Biology, Temple University', 'email': 'Verified email at temple.edu', 'interests': [{'title': 'Evolution', 'link': 'https://scholar.google.com/citations?view_op=search_authors&hl=en&mauthors=label:evolution', 'serpapi_link': 'https://serpapi.com/search.json?engine=google_scholar_profiles&hl=en&mauthors=label%3Aevolution'}, {'title': 'Evolutionary biology', 'link': 'https://scholar.google.com/citations?view_op=search_authors&hl=en&mauthors=label:evolutionary_biology', 'serpapi_link': 'https://serpapi.com/search.json?engine=google_scholar_profiles&hl=en&mauthors=label%3Aevolutionary_biology'}, {'title': 'Molecular evolution', 'link': 'https://scholar.google.com/citations?view_op=search_authors&hl=en&mauthors=label:molecular_evolution', 'serpapi_link': 'https://serpapi.com/search.json?engine=google_scholar_profiles&hl=en&mauthors=label%3Amolecular_evolution'}, {'title': 'Population genetics', 'link': 'https://scholar.google.com/citations?view_op=search_authors&hl=en&mauthors=label:population_genetics', 'serpapi_link': 'https://serpapi.com/search.json?engine=google_scholar_profiles&hl=en&mauthors=label%3Apopulation_genetics'}, {'title': 'Phylogenetics', 'link': 'https://scholar.google.com/citations?view_op=search_authors&hl=en&mauthors=label:phylogenetics', 'serpapi_link': 'https://serpapi.com/search.json?engine=google_scholar_profiles&hl=en&mauthors=label%3Aphylogenetics'}], 'thumbnail': 'https://scholar.googleusercontent.com/citations?view_op=view_photo&user=VxOmZDgAAAAJ&citpid=3'}

Author articles:
[{'title': 'The neighbor-joining method: a new method for reconstructing phylogenetic trees.', 'link': 'https://scholar.google.com/citations?view_op=view_citation&hl=en&user=VxOmZDgAAAAJ&citation_for_view=VxOmZDgAAAAJ:u5HHmVD_uO8C', 'citation_id': 'VxOmZDgAAAAJ:u5HHmVD_uO8C', 'authors': 'N Saitou, M Nei', 'publication': 'Molecular biology and evolution 4 (4), 406-425, 1987', 'cited_by': {'value': 64841, 'link': 'https://scholar.google.com/scholar?oi=bibs&hl=en&cites=7672721046330422437,346314157833338191', 'serpapi_link': 'https://serpapi.com/search.json?cites=7672721046330422437%2C346314157833338191&engine=google_scholar&hl=en', 'cites_id': '7672721046330422437,346314157833338191'}, 'year': '1987'}, {'title': 'MEGA5: molecular evolutionary genetics analysis using maximum likelihood, evolutionary distance, and maximum parsimony methods', 'link': 'https://scholar.google.com/citations?view_op=view_citation&hl=en&user=VxOmZDgAAAAJ&citation_for_view=VxOmZDgAAAAJ:Tyk-4Ss8FVUC', 'citation_id': 'VxOmZDgAAAAJ:Tyk-4Ss8FVUC', 'authors': 'K Tamura, D Peterson, N Peterson, G Stecher, M Nei, S Kumar', 'publication': 'Molecular biology and evolution 28 (10), 2731-2739, 2011', 'cited_by': {'value': 44316, 'link': 'https://scholar.google.com/scholar?oi=bibs&hl=en&cites=5624029996178252455,5910675136328950108,13537318717249213469', 'serpapi_link': 'https://serpapi.com/search.json?cites=5624029996178252455%2C5910675136328950108%2C13537318717249213469&engine=google_scholar&hl=en', 'cites_id': '5624029996178252455,5910675136328950108,13537318717249213469'}, 'year': '2011'}, {'title': 'MEGA6: molecular evolutionary genetics analysis version 6.0', 'link': 'https://scholar.google.com/citations?view_op=view_citation&hl=en&user=VxOmZDgAAAAJ&citation_for_view=VxOmZDgAAAAJ:qmtmRrLr0tkC', 'citation_id': 'VxOmZDgAAAAJ:qmtmRrLr0tkC', 'authors': 'K Tamura, G Stecher, D Peterson, A Filipski, S Kumar', 'publication': 'Molecular biology and evolution 30 (12), 2725-2729, 2013', 'cited_by': {'value': 40558, 'link': 'https://scholar.google.com/scholar?oi=bibs&hl=en&cites=5258359186493639031', 'serpapi_link': 'https://serpapi.com/search.json?cites=5258359186493639031&engine=google_scholar&hl=en', 'cites_id': '5258359186493639031'}, 'year': '2013'}, {'title': 'MEGA4: molecular evolutionary genetics analysis (MEGA) software version 4.0', 'link': 'https://scholar.google.com/citations?view_op=view_citation&hl=en&user=VxOmZDgAAAAJ&citation_for_view=VxOmZDgAAAAJ:u-x6o8ySG0sC', 'citation_id': 'VxOmZDgAAAAJ:u-x6o8ySG0sC', 'authors': 'K Tamura, J Dudley, M Nei, S Kumar', 'publication': 'Molecular biology and evolution 24 (8), 1596-1599, 2007', 'cited_by': {'value': 34245, 'link': 'https://scholar.google.com/scholar?oi=bibs&hl=en&cites=8480751610153565117', 'serpapi_link': 'https://serpapi.com/search.json?cites=8480751610153565117&engine=google_scholar&hl=en', 'cites_id': '8480751610153565117'}, 'year': '2007'}, {'title': 'Molecular evolutionary genetics', 'link': 'https://scholar.google.com/citations?view_op=view_citation&hl=en&user=VxOmZDgAAAAJ&citation_for_view=VxOmZDgAAAAJ:d1gkVwhDpl0C', 'citation_id': 'VxOmZDgAAAAJ:d1gkVwhDpl0C', 'authors': 'M Nei', 'publication': 'Columbia university press, 1987', 'cited_by': {'value': 20704, 'link': 'https://scholar.google.com/scholar?oi=bibs&hl=en&cites=7660515423132980153', 'serpapi_link': 'https://serpapi.com/search.json?cites=7660515423132980153&engine=google_scholar&hl=en', 'cites_id': '7660515423132980153'}, 'year': '1987'}, {'title': 'MEGA2: molecular evolutionary genetics analysis software', 'link': 'https://scholar.google.com/citations?view_op=view_citation&hl=en&user=VxOmZDgAAAAJ&citation_for_view=VxOmZDgAAAAJ:IjCSPb-OGe4C', 'citation_id': 'VxOmZDgAAAAJ:IjCSPb-OGe4C', 'authors': 'S Kumar, K Tamura, IB Jakobsen, M Nei', 'publication': 'Bioinformatics 17 (12), 1244-1245, 2001', 'cited_by': {'value': 16078, 'link': 'https://scholar.google.com/scholar?oi=bibs&hl=en&cites=14171206204658643394,531426008085525562,5869149036159079676,8067244568899724142,12929609819447339488,15783386726452728786', 'serpapi_link': 'https://serpapi.com/search.json?cites=14171206204658643394%2C531426008085525562%2C5869149036159079676%2C8067244568899724142%2C12929609819447339488%2C15783386726452728786&engine=google_scholar&hl=en', 'cites_id': '14171206204658643394,531426008085525562,5869149036159079676,8067244568899724142,12929609819447339488,15783386726452728786'}, 'year': '2001'}, {'title': 'Estimation of average heterozygosity and genetic distance from a small number of individuals', 'link': 'https://scholar.google.com/citations?view_op=view_citation&hl=en&user=VxOmZDgAAAAJ&citation_for_view=VxOmZDgAAAAJ:2osOgNQ5qMEC', 'citation_id': 'VxOmZDgAAAAJ:2osOgNQ5qMEC', 'authors': 'M Nei', 'publication': 'Genetics 89 (3), 583-590, 1978', 'cited_by': {'value': 14504, 'link': 'https://scholar.google.com/scholar?oi=bibs&hl=en&cites=11038674224870321151', 'serpapi_link': 'https://serpapi.com/search.json?cites=11038674224870321151&engine=google_scholar&hl=en', 'cites_id': '11038674224870321151'}, 'year': '1978'}, {'title': 'MEGA3: integrated software for molecular evolutionary genetics analysis and sequence alignment', 'link': 'https://scholar.google.com/citations?view_op=view_citation&hl=en&user=VxOmZDgAAAAJ&citation_for_view=VxOmZDgAAAAJ:9yKSN-GCB0IC', 'citation_id': 'VxOmZDgAAAAJ:9yKSN-GCB0IC', 'authors': 'S Kumar, K Tamura, M Nei', 'publication': 'Briefings in bioinformatics 5 (2), 150-163, 2004', 'cited_by': {'value': 14403, 'link': 'https://scholar.google.com/scholar?oi=bibs&hl=en&cites=10013295782066828040,15148316572039251274', 'serpapi_link': 'https://serpapi.com/search.json?cites=10013295782066828040%2C15148316572039251274&engine=google_scholar&hl=en', 'cites_id': '10013295782066828040,15148316572039251274'}, 'year': '2004'}, {'title': 'Mathematical model for studying genetic variation in terms of restriction endonucleases', 'link': 'https://scholar.google.com/citations?view_op=view_citation&hl=en&user=VxOmZDgAAAAJ&citation_for_view=VxOmZDgAAAAJ:qjMakFHDy7sC', 'citation_id': 'VxOmZDgAAAAJ:qjMakFHDy7sC', 'authors': 'M Nei, WH Li', 'publication': 'Proceedings of the National Academy of Sciences 76 (10), 5269-5273, 1979', 'cited_by': {'value': 13619, 'link': 'https://scholar.google.com/scholar?oi=bibs&hl=en&cites=5179626164554275201,1942230974501958280', 'serpapi_link': 'https://serpapi.com/search.json?cites=5179626164554275201%2C1942230974501958280&engine=google_scholar&hl=en', 'cites_id': '5179626164554275201,1942230974501958280'}, 'year': '1979'}, {'title': 'Genetic distance between populations', 'link': 'https://scholar.google.com/citations?view_op=view_citation&hl=en&user=VxOmZDgAAAAJ&citation_for_view=VxOmZDgAAAAJ:UeHWp8X0CEIC', 'citation_id': 'VxOmZDgAAAAJ:UeHWp8X0CEIC', 'authors': 'M Nei', 'publication': 'The American Naturalist 106 (949), 283-292, 1972', 'cited_by': {'value': 12980, 'link': 'https://scholar.google.com/scholar?oi=bibs&hl=en&cites=4154924214026252226,7115074001272219295', 'serpapi_link': 'https://serpapi.com/search.json?cites=4154924214026252226%2C7115074001272219295&engine=google_scholar&hl=en', 'cites_id': '4154924214026252226,7115074001272219295'}, 'year': '1972'}, {'title': 'Estimation of the number of nucleotide substitutions in the control region of mitochondrial DNA in humans and chimpanzees.', 'link': 'https://scholar.google.com/citations?view_op=view_citation&hl=en&user=VxOmZDgAAAAJ&citation_for_view=VxOmZDgAAAAJ:Y0pCki6q_DkC', 'citation_id': 'VxOmZDgAAAAJ:Y0pCki6q_DkC', 'authors': 'K Tamura, M Nei', 'publication': 'Molecular biology and evolution 10 (3), 512-526, 1993', 'cited_by': {'value': 11093, 'link': 'https://scholar.google.com/scholar?oi=bibs&hl=en&cites=13509507708085673250', 'serpapi_link': 'https://serpapi.com/search.json?cites=13509507708085673250&engine=google_scholar&hl=en', 'cites_id': '13509507708085673250'}, 'year': '1993'}, {'title': 'Analysis of gene diversity in subdivided populations', 'link': 'https://scholar.google.com/citations?view_op=view_citation&hl=en&user=VxOmZDgAAAAJ&citation_for_view=VxOmZDgAAAAJ:zYLM7Y9cAGgC', 'citation_id': 'VxOmZDgAAAAJ:zYLM7Y9cAGgC', 'authors': 'M Nei', 'publication': 'Proceedings of the national academy of sciences 70 (12), 3321-3323, 1973', 'cited_by': {'value': 10714, 'link': 'https://scholar.google.com/scholar?oi=bibs&hl=en&cites=11712109356391350421', 'serpapi_link': 'https://serpapi.com/search.json?cites=11712109356391350421&engine=google_scholar&hl=en', 'cites_id': '11712109356391350421'}, 'year': '1973'}, {'title': 'Molecular evolution and phylogenetics', 'link': 'https://scholar.google.com/citations?view_op=view_citation&hl=en&user=VxOmZDgAAAAJ&citation_for_view=VxOmZDgAAAAJ:YsMSGLbcyi4C', 'citation_id': 'VxOmZDgAAAAJ:YsMSGLbcyi4C', 'authors': 'M Nei, S Kumar', 'publication': 'Oxford University Press, USA, 2000', 'cited_by': {'value': 8795, 'link': 'https://scholar.google.com/scholar?oi=bibs&hl=en&cites=703217195301701212,1351927951694611906', 'serpapi_link': 'https://serpapi.com/search.json?cites=703217195301701212%2C1351927951694611906&engine=google_scholar&hl=en', 'cites_id': '703217195301701212,1351927951694611906'}, 'year': '2000'}, {'title': 'Simple methods for estimating the numbers of synonymous and nonsynonymous nucleotide substitutions.', 'link': 'https://scholar.google.com/citations?view_op=view_citation&hl=en&user=VxOmZDgAAAAJ&citation_for_view=VxOmZDgAAAAJ:W7OEmFMy1HYC', 'citation_id': 'VxOmZDgAAAAJ:W7OEmFMy1HYC', 'authors': 'M Nei, T Gojobori', 'publication': 'Molecular biology and evolution 3 (5), 418-426, 1986', 'cited_by': {'value': 5279, 'link': 'https://scholar.google.com/scholar?oi=bibs&hl=en&cites=12106160511321461626', 'serpapi_link': 'https://serpapi.com/search.json?cites=12106160511321461626&engine=google_scholar&hl=en', 'cites_id': '12106160511321461626'}, 'year': '1986'}, {'title': 'Prospects for inferring very large phylogenies by using the neighbor-joining method', 'link': 'https://scholar.google.com/citations?view_op=view_citation&hl=en&user=VxOmZDgAAAAJ&citation_for_view=VxOmZDgAAAAJ:0EnyYjriUFMC', 'citation_id': 'VxOmZDgAAAAJ:0EnyYjriUFMC', 'authors': 'K Tamura, M Nei, S Kumar', 'publication': 'Proceedings of the National Academy of Sciences 101 (30), 11030-11035, 2004', 'cited_by': {'value': 4882, 'link': 'https://scholar.google.com/scholar?oi=bibs&hl=en&cites=9650987578903829104', 'serpapi_link': 'https://serpapi.com/search.json?cites=9650987578903829104&engine=google_scholar&hl=en', 'cites_id': '9650987578903829104'}, 'year': '2004'}, {'title': 'The bottleneck effect and genetic variability in populations', 'link': 'https://scholar.google.com/citations?view_op=view_citation&hl=en&user=VxOmZDgAAAAJ&citation_for_view=VxOmZDgAAAAJ:eQOLeE2rZwMC', 'citation_id': 'VxOmZDgAAAAJ:eQOLeE2rZwMC', 'authors': 'M Nei, T Maruyama, R Chakraborty', 'publication': 'Evolution, 1-10, 1975', 'cited_by': {'value': 3906, 'link': 'https://scholar.google.com/scholar?oi=bibs&hl=en&cites=13149273985544466189', 'serpapi_link': 'https://serpapi.com/search.json?cites=13149273985544466189&engine=google_scholar&hl=en', 'cites_id': '13149273985544466189'}, 'year': '1975'}, {'title': 'Accuracy of estimated phylogenetic trees from molecular data', 'link': 'https://scholar.google.com/citations?view_op=view_citation&hl=en&user=VxOmZDgAAAAJ&citation_for_view=VxOmZDgAAAAJ:Se3iqnhoufwC', 'citation_id': 'VxOmZDgAAAAJ:Se3iqnhoufwC', 'authors': 'M Nei, F Tajima, Y Tateno', 'publication': 'Journal of molecular evolution 19 (2), 153-170, 1983', 'cited_by': {'value': 2877, 'link': 'https://scholar.google.com/scholar?oi=bibs&hl=en&cites=10638180566709737898', 'serpapi_link': 'https://serpapi.com/search.json?cites=10638180566709737898&engine=google_scholar&hl=en', 'cites_id': '10638180566709737898'}, 'year': '1983'}, {'title': 'Molecular population genetics and evolution.', 'link': 'https://scholar.google.com/citations?view_op=view_citation&hl=en&user=VxOmZDgAAAAJ&citation_for_view=VxOmZDgAAAAJ:WF5omc3nYNoC', 'citation_id': 'VxOmZDgAAAAJ:WF5omc3nYNoC', 'authors': 'M Nei', 'publication': 'Molecular population genetics and evolution., 1975', 'cited_by': {'value': 2795, 'link': 'https://scholar.google.com/scholar?oi=bibs&hl=en&cites=7886550802885580479', 'serpapi_link': 'https://serpapi.com/search.json?cites=7886550802885580479&engine=google_scholar&hl=en', 'cites_id': '7886550802885580479'}, 'year': '1975'}, {'title': 'Pattern of nucleotide substitution at major histocompatibility complex class I loci reveals overdominant selection', 'link': 'https://scholar.google.com/citations?view_op=view_citation&hl=en&user=VxOmZDgAAAAJ&citation_for_view=VxOmZDgAAAAJ:roLk4NBRz8UC', 'citation_id': 'VxOmZDgAAAAJ:roLk4NBRz8UC', 'authors': 'AL Hughes, M Nei', 'publication': 'Nature 335 (6186), 167-170, 1988', 'cited_by': {'value': 2169, 'link': 'https://scholar.google.com/scholar?oi=bibs&hl=en&cites=2966744676732650646', 'serpapi_link': 'https://serpapi.com/search.json?cites=2966744676732650646&engine=google_scholar&hl=en', 'cites_id': '2966744676732650646'}, 'year': '1988'}, {'title': 'Sampling variances of heterozygosity and genetic distance', 'link': 'https://scholar.google.com/citations?view_op=view_citation&hl=en&user=VxOmZDgAAAAJ&citation_for_view=VxOmZDgAAAAJ:UebtZRa9Y70C', 'citation_id': 'VxOmZDgAAAAJ:UebtZRa9Y70C', 'authors': 'M Nei, AK Roychoudhury', 'publication': 'Genetics 76 (2), 379-390, 1974', 'cited_by': {'value': 1918, 'link': 'https://scholar.google.com/scholar?oi=bibs&hl=en&cites=5978996318059495400', 'serpapi_link': 'https://serpapi.com/search.json?cites=5978996318059495400&engine=google_scholar&hl=en', 'cites_id': '5978996318059495400'}, 'year': '1974'}]
'''

Disclaimer, I work for SerpApi.


来源: stackexchange

问:
Little bit confused about using Condition Variable for concurrent programming
I am currently learning about Game Programming with the book 'Game Engine Architecture' authored by Jason Gregory.

In this book, he showed an example with the reason for using 'Condition Variable'

[Without Condition Variable]

```<code>Queue g_queue;
pthread_mutex_t g_mutex; bool
g_ready = false;

void* ProducerThread(void*)
{
    // keep on producing forever...
    while (true)
    {
        pthread_mutex_lock(&g_mutex);
        // fill the queue with data
        ProduceDataInto(&g_queue);
        g_ready = true;

        pthread_mutex_unlock(&g_mutex);
        // yield the remainder of my timeslice
        // to give the consumer a chance to run pthread_yield();
    }
    return nullptr;
}

void* ConsumerThread(void*)
{
    // keep on consuming forever...
    while (true)
    {
        // wait for the data to be ready
        while (true)
        {
             // read the value into a local,
             // making sure to lock the mutex

             pthread_mutex_lock(&g_mutex);
             const bool ready = g_ready;
             pthread_mutex_unlock(&g_mutex);

             if (ready) break;
        }
        // consume the data
        pthread_mutex_lock(&g_mutex);
        ConsumeDataFrom(&g_queue);
        g_ready = false;
        pthread_mutex_unlock(&g_mutex);
        // yield the remainder of my timeslice
        // to give the producer a chance to run pthread_yield();
    }
    return nullptr;
}
</code>```

In this example, he said 'Besides the fact that this example is somewhat contrived, there’s one big problem with it: The consumer thread spins in a tight loop, polling the value of g_ready'

I found that the function 'pthread_mutex_lock(&g_mutex)' is a blocking function that if the calling thread can't acquire the mutex, it falls to asleep.

Then, isn't that the consumer thread is not on the state of 'busy-wait'?
I mean, doesn't it spin at all if it does not acquire the mutex?

答:
Though pthread_mutex_lock is a blocking function, the producer and consumer loop will spin tightly. Because  either ProduceDataInto  or ConsumeDataFrom executed and returns immediately, the mutex repeats lock/unlock after each calling ProduceDataInto/ProduceDataInto.

So there must be a queue-full Condition Variable to make the producer wait and a queue-empty Condition Variable to make the consumer wait.

ContentLicense: CC BY-SA 4.0

Body:
I am currently learning about Game Programming with the book 'Game Engine Architecture' authored by Jason Gregory.

In this book, he showed an example with the reason for using 'Condition Variable'

[Without Condition Variable]

Queue g_queue;
pthread_mutex_t g_mutex; bool
g_ready = false;

void* ProducerThread(void*)
{
    // keep on producing forever...
    while (true)
    {
        pthread_mutex_lock(&g_mutex);
        // fill the queue with data
        ProduceDataInto(&g_queue);
        g_ready = true;

        pthread_mutex_unlock(&g_mutex);
        // yield the remainder of my timeslice
        // to give the consumer a chance to run pthread_yield();
    }
    return nullptr;
}

void* ConsumerThread(void*)
{
    // keep on consuming forever...
    while (true)
    {
        // wait for the data to be ready
        while (true)
        {
             // read the value into a local,
             // making sure to lock the mutex

             pthread_mutex_lock(&g_mutex);
             const bool ready = g_ready;
             pthread_mutex_unlock(&g_mutex);

             if (ready) break;
        }
        // consume the data
        pthread_mutex_lock(&g_mutex);
        ConsumeDataFrom(&g_queue);
        g_ready = false;
        pthread_mutex_unlock(&g_mutex);
        // yield the remainder of my timeslice
        // to give the producer a chance to run pthread_yield();
    }
    return nullptr;
}

In this example, he said 'Besides the fact that this example is somewhat contrived, there’s one big problem with it: The consumer thread spins in a tight loop, polling the value of g_ready'

I found that the function 'pthread_mutex_lock(&g_mutex)' is a blocking function that if the calling thread can't acquire the mutex, it falls to asleep.

Then, isn't that the consumer thread is not on the state of 'busy-wait'?
I mean, doesn't it spin at all if it does not acquire the mutex?


问:
PHP/Javascript - Add payment method with Stripe API
I am using Wordpress, but I am using an "out of the box" solution because my client requested it specifically. I have this Javascript which creates a form and when the form is submitted it creates a cookie with the token in it, so I can retrieve it from PHP, it looks like this:

```<code><form action="/wp-content/themes/betheme/includes/add_payment_method.php" method="post" id="add-card-form">
      <div class="form-row">
        <label for="card-element">
          Credit or debit card
        </label>

        <div id="card-element"></div>

        <!-- Used to display Element errors. -->
        <div id="card-errors" role="alert"></div>
      </div>

      <button>Add payment method</button>
    </form>

    <script src="https://js.stripe.com/v3/"></script>

    <script>
        var style = {
          base: {
            color: '#ffffff',
            fontSize: '14px',
            fontSmoothing: 'antialiased',
            '::placeholder': {
              color: '#ccc',
            },
            iconColor: "#fff"
          },
          invalid: {
            color: '#e5424d',
            ':focus': {
              color: '#303238',
            },
          },
        };
        var stripe = Stripe('xxxxxxxxxxxxxxxxxxxxxxx');
        var elements = stripe.elements();
        var cardElement = elements.create('card', {style: style, hidePostalCode: true});

        window.mobilecheck = function() {
          var check = false;
          (function(a){if(/(android|bb\d+|meego).+mobile|avantgo|bada\/|blackberry|blazer|compal|elaine|fennec|hiptop|iemobile|ip(hone|od)|iris|kindle|lge |maemo|midp|mmp|mobile.+firefox|netfront|opera m(ob|in)i|palm( os)?|phone|p(ixi|re)\/|plucker|pocket|psp|series(4|6)0|symbian|treo|up\.(browser|link)|vodafone|wap|windows ce|xda|xiino/i.test(a)||/1207|6310|6590|3gso|4thp|50[1-6]i|770s|802s|a wa|abac|ac(er|oo|s\-)|ai(ko|rn)|al(av|ca|co)|amoi|an(ex|ny|yw)|aptu|ar(ch|go)|as(te|us)|attw|au(di|\-m|r |s )|avan|be(ck|ll|nq)|bi(lb|rd)|bl(ac|az)|br(e|v)w|bumb|bw\-(n|u)|c55\/|capi|ccwa|cdm\-|cell|chtm|cldc|cmd\-|co(mp|nd)|craw|da(it|ll|ng)|dbte|dc\-s|devi|dica|dmob|do(c|p)o|ds(12|\-d)|el(49|ai)|em(l2|ul)|er(ic|k0)|esl8|ez([4-7]0|os|wa|ze)|fetc|fly(\-|_)|g1 u|g560|gene|gf\-5|g\-mo|go(\.w|od)|gr(ad|un)|haie|hcit|hd\-(m|p|t)|hei\-|hi(pt|ta)|hp( i|ip)|hs\-c|ht(c(\-| |_|a|g|p|s|t)|tp)|hu(aw|tc)|i\-(20|go|ma)|i230|iac( |\-|\/)|ibro|idea|ig01|ikom|im1k|inno|ipaq|iris|ja(t|v)a|jbro|jemu|jigs|kddi|keji|kgt( |\/)|klon|kpt |kwc\-|kyo(c|k)|le(no|xi)|lg( g|\/(k|l|u)|50|54|\-[a-w])|libw|lynx|m1\-w|m3ga|m50\/|ma(te|ui|xo)|mc(01|21|ca)|m\-cr|me(rc|ri)|mi(o8|oa|ts)|mmef|mo(01|02|bi|de|do|t(\-| |o|v)|zz)|mt(50|p1|v )|mwbp|mywa|n10[0-2]|n20[2-3]|n30(0|2)|n50(0|2|5)|n7(0(0|1)|10)|ne((c|m)\-|on|tf|wf|wg|wt)|nok(6|i)|nzph|o2im|op(ti|wv)|oran|owg1|p800|pan(a|d|t)|pdxg|pg(13|\-([1-8]|c))|phil|pire|pl(ay|uc)|pn\-2|po(ck|rt|se)|prox|psio|pt\-g|qa\-a|qc(07|12|21|32|60|\-[2-7]|i\-)|qtek|r380|r600|raks|rim9|ro(ve|zo)|s55\/|sa(ge|ma|mm|ms|ny|va)|sc(01|h\-|oo|p\-)|sdk\/|se(c(\-|0|1)|47|mc|nd|ri)|sgh\-|shar|sie(\-|m)|sk\-0|sl(45|id)|sm(al|ar|b3|it|t5)|so(ft|ny)|sp(01|h\-|v\-|v )|sy(01|mb)|t2(18|50)|t6(00|10|18)|ta(gt|lk)|tcl\-|tdg\-|tel(i|m)|tim\-|t\-mo|to(pl|sh)|ts(70|m\-|m3|m5)|tx\-9|up(\.b|g1|si)|utst|v400|v750|veri|vi(rg|te)|vk(40|5[0-3]|\-v)|vm40|voda|vulc|vx(52|53|60|61|70|80|81|83|85|98)|w3c(\-| )|webc|whit|wi(g |nc|nw)|wmlb|wonu|x700|yas\-|your|zeto|zte\-/i.test(a.substr(0,4))) check = true;})(navigator.userAgent||navigator.vendor||window.opera);
          return check;
        };

        if(mobilecheck()) {
            cardElement.mount('#Content > div > div > div > div.section.mcb-section.hide-desktop > div > div > div > div > div > div > div > form > div.form-row >div#card-element');
        }
        else {
            cardElement.mount('#Content > div > div > div > div.section.mcb-section.hide-tablet.hide-mobile > div > div > div > div > div > div > div > form > div.form-row > div#card-element');
        }

        cardElement.addEventListener('change', function(event) {
          var displayError = document.getElementById('card-errors');
          if (event.error) {
            displayError.textContent = event.error.message;
          } else {
            displayError.textContent = '';
          }
        });

        // Create a token or display an error when the form is submitted.
        var form = document.getElementById('add-card-form');
        form.addEventListener('submit', function(event) {
          event.preventDefault();

          stripe.createToken(cardElement).then(function(result) {
            if (result.error) {
              // Inform the customer that there was an error.
              var errorElement = document.getElementById('card-errors');
              errorElement.textContent = result.error.message;
            } else {
              // Send the token to your server.
              stripeTokenHandler(result.token);
            }
          });
        });

        function createCookie(name, value, days) {
          var expires;
          if (days) {
            var date = new Date();
            date.setTime(date.getTime() + (days * 24 * 60 * 60 * 1000));
            expires = "; expires=" + date.toGMTString();
          } else {
           expires = "";
          }
          document.cookie = escape(name) + "=" + escape(value) + expires + "; path=/";
        }

        function stripeTokenHandler(token) {
            createCookie("stripe_token", token, 1);

            var form = document.getElementById('add-card-form');
            form.submit();
        }
</script>
</code>```

When the form is submitted, it executes this PHP in add_payment_method.php:

```<code><?php
    require_once("../../../../wp-load.php");
    submit_stripe_payment_method($_COOKIE['stripe_token']);
?>
</code>```

In my functions.php file for my theme, I have:

```<code>require_once('stripe-php/init.php');

....

function submit_stripe_payment_method($token) {
    print_r($token);

    $current_user_id = get_current_user_id();
    echo 'Your User ID is: ' .$current_user_id;

    $customer_id = get_user_meta( $current_user_id, '_stripe_customer_id', true );
    echo "CUSTOMER_ID: ".(string)$customer_id;

    \Stripe\Stripe::setApiKey("xxxxxxxxxxxxxxxxxxxxxxx");

    echo "MADE IT HERE";

    $customer = \Stripe\Customer::retrieve((string)$customer_id);

    echo "CUSTOMER: ".$customer;

    $customer_card = $customer->sources->create(["source" => $token]);
    echo "```";
    var_dump($customer_card);
    echo "```";
}
</code>```

The output of the above is:

[object Object]Your User ID is: 1CUSTOMER_ID: cus_DsoF18uw1dR0ujMADE IT HERE

As you can see it only gets as far as "MADE IT HERE" echo.

I have verified in the Stripe web console that the $customer_id I am using is correct, and I don't have any errors, PHP or otherwise. The only thing I can think of is I am using the token wrong but it seems pretty straight forward. As you can see, the $token prints as [Object object], and it does this no matter how I try to log it, so I can't see exactly what it is, but this makes sense because it contains sensitive information used to validate the credit card...also maybe it never gets to that point because I never see the echo "CUSTOMER: ".$customer; statement, so maybe it gets hung up on:

$customer = \Stripe\Customer::retrieve((string)$customer_id);

Also, I tried the above without (string) first but got the same result.

So after all of this, I check the Stripe console, and no card has been added.

UPDATE

Apparently the API version I am using is not the latest version, I can upgrade, I just want to know if anyone thinks this is a possible solution before I do it. As it might change how stripe works throughout the site (with the plugin). I am using 2018-02-28 and I can upgrade to 2018-09-24
. I am also testing on localhost environment so I am not https, but I don't think this matters for testing.

UPDATE

I realized I hadn't added the endpoint for my localhost setup to the stripe console admin area. To do this I had to use ngrok (as suggested by Stripe if you are using localhost). I am using localhost on port 8888, so I did ./ngrok http 8888 and then tried again and it didn't work. It does say in Wordpress that the endpoint will enable you to receive notifications on the charge statuses. So maybe it isn't relevant for adding a payment method. I used the http url for ngrok not the https url.

UPDATE

I manually installed stripe-php so I tried with composer but it didn't help, same result.

UPDATE

I changed my key to the secret key and that allowed me to retrieve the $customer but I don't see any output for $customer_card and when I look in the dashboard no card has been created.

UPDATE

I checked the Stripe dashboard and I am receiving an error when I try to create the source:

```<code>{
  "error": {
    "code": "resource_missing",
    "doc_url": "https://stripe.com/docs/error-codes/resource-missing",
    "message": "No such token: [object Object]",
    "param": "source",
    "type": "invalid_request_error"
  }
}
</code>```

What am I doing wrong with the token?

UPDATE

Solution to token problem: in test mode, the token is just a string that corresponds with the test credit card number you are using, I needed "tok_visa" for 4242424242424242. https://stripe.com/docs/testing

答:
You can't call Stripe\PaymentMethod::attach() statically. You have to fetch the PaymentMethod first and then attach it to the Customer. The code should be:

$pm = \Stripe\PaymentMethod::retrieve('pm_123');
$pm->attach(['customer' => 'cus_123']);

I'll work on updating the docs to avoid that confusion!

Body:
I am using Wordpress, but I am using an "out of the box" solution because my client requested it specifically. I have this Javascript which creates a form and when the form is submitted it creates a cookie with the token in it, so I can retrieve it from PHP, it looks like this:

<form action="/wp-content/themes/betheme/includes/add_payment_method.php" method="post" id="add-card-form">

          Credit or debit card

      <button>Add payment method</button>
    </form>

When the form is submitted, it executes this PHP in add_payment_method.php:

<?php
    require_once("../../../../wp-load.php");
    submit_stripe_payment_method($_COOKIE['stripe_token']);
?>

In my functions.php file for my theme, I have:

require_once('stripe-php/init.php');

....

function submit_stripe_payment_method($token) {
    print_r($token);

    $current_user_id = get_current_user_id();
    echo 'Your User ID is: ' .$current_user_id;

    $customer_id = get_user_meta( $current_user_id, '_stripe_customer_id', true );
    echo "CUSTOMER_ID: ".(string)$customer_id;

    \Stripe\Stripe::setApiKey("xxxxxxxxxxxxxxxxxxxxxxx");

    echo "MADE IT HERE";

    $customer = \Stripe\Customer::retrieve((string)$customer_id);

    echo "CUSTOMER: ".$customer;

    $customer_card = $customer->sources->create(["source" => $token]);
    echo "
";
    var_dump($customer_card);
    echo "
";
}

The output of the above is:

[object Object]Your User ID is: 1CUSTOMER_ID: cus_DsoF18uw1dR0ujMADE IT HERE

As you can see it only gets as far as "MADE IT HERE" echo.

I have verified in the Stripe web console that the $customer_id I am using is correct, and I don't have any errors, PHP or otherwise. The only thing I can think of is I am using the token wrong but it seems pretty straight forward. As you can see, the $token prints as [Object object], and it does this no matter how I try to log it, so I can't see exactly what it is, but this makes sense because it contains sensitive information used to validate the credit card...also maybe it never gets to that point because I never see the echo "CUSTOMER: ".$customer; statement, so maybe it gets hung up on:

$customer = \Stripe\Customer::retrieve((string)$customer_id);

Also, I tried the above without (string) first but got the same result.

So after all of this, I check the Stripe console, and no card has been added.

UPDATE

Apparently the API version I am using is not the latest version, I can upgrade, I just want to know if anyone thinks this is a possible solution before I do it. As it might change how stripe works throughout the site (with the plugin). I am using 2018-02-28 and I can upgrade to 2018-09-24
. I am also testing on localhost environment so I am not https, but I don't think this matters for testing.

UPDATE

I realized I hadn't added the endpoint for my localhost setup to the stripe console admin area. To do this I had to use ngrok (as suggested by Stripe if you are using localhost). I am using localhost on port 8888, so I did ./ngrok http 8888 and then tried again and it didn't work. It does say in Wordpress that the endpoint will enable you to receive notifications on the charge statuses. So maybe it isn't relevant for adding a payment method. I used the http url for ngrok not the https url.

UPDATE

I manually installed stripe-php so I tried with composer but it didn't help, same result.

UPDATE

I changed my key to the secret key and that allowed me to retrieve the $customer but I don't see any output for $customer_card and when I look in the dashboard no card has been created.

UPDATE

I checked the Stripe dashboard and I am receiving an error when I try to create the source:

{
  "error": {
    "code": "resource_missing",
    "doc_url": "https://stripe.com/docs/error-codes/resource-missing",
    "message": "No such token: [object Object]",
    "param": "source",
    "type": "invalid_request_error"
  }
}

What am I doing wrong with the token?

UPDATE

Solution to token problem: in test mode, the token is just a string that corresponds with the test credit card number you are using, I needed "tok_visa" for 4242424242424242. https://stripe.com/docs/testing


问:
Extracting a file name from path
I'm trying to get the files name for a path. But every time I run this code below I only get the name of the parent folder instead

```<code>    for file in files:
        if file.endswith("png") or file.endswith("jpg"):
            path = os.path.join(root,file)
            print(path)
            label = os.path.basename(os.path.dirname(path))
            print(label)
</code>```

I get these results :

```<code>D:\AI\Deep learning\face generator\images\chris evans 1.jpg
images
D:\AI\Deep learning\face generator\images\chris evans 2.jpg
images
</code>```

and so on

My expected results are

```<code>D:\AI\Deep learning\face generator\images\chris evans 1.jpg
chris evans 1.jpg
D:\AI\Deep learning\face generator\images\chris evans 2.jpg
chris evans 2.jpg
</code>```

答:
You should use pathlib (Python >= 3.4) for this.

```<code>from pathlib import Path

p = Path('D:\AI\Deep learning\face generator\images\chris evans 1.jpg')
filename = p.name
print(filename)
</code>```

Title: Extracting a file name from path

Body:
I'm trying to get the files name for a path. But every time I run this code below I only get the name of the parent folder instead

    for file in files:
        if file.endswith("png") or file.endswith("jpg"):
            path = os.path.join(root,file)
            print(path)
            label = os.path.basename(os.path.dirname(path))
            print(label)

I get these results :

D:\AI\Deep learning\face generator\images\chris evans 1.jpg
images
D:\AI\Deep learning\face generator\images\chris evans 2.jpg
images

and so on

My expected results are

D:\AI\Deep learning\face generator\images\chris evans 1.jpg
chris evans 1.jpg
D:\AI\Deep learning\face generator\images\chris evans 2.jpg
chris evans 2.jpg

Body:
You should use pathlib (Python >= 3.4) for this.

from pathlib import Path

p = Path('D:\AI\Deep learning\face generator\images\chris evans 1.jpg')
filename = p.name
print(filename)


问:
Is it possible to do relative C# namespace reference?
I have namespaces:

```<code>MyProject.Core.Db
MyProject.Core.Model
</code>```

And I have classes:

```<code>MyProject.Core.Db.User
MyProject.Core.Model.User
</code>```

Is it possible something like:

```<code>using MyProject.Core;

namespace MyProject.BLL
{
    public class Logic
    {
        public static void DoSomething()
        {
            var userEntity = new Db.User();
            var userModel = new Model.User();
        }
    }
}
</code>```

I just want to avoid using suffixes in class names (UserModel, UserEntity).

Is it possible to do in somehow in C#?

答:
One thing you can do, and we probably should do a lot more, is to specify usings relative to the current namespace. To do this, just move your usings inside the namespace declaration. It doesn't fix your stated problem, but the shorter relative paths are less brittle and your project will be easier to refactor.</p>
```<code>namespace MyProject.Core{
    using Db;
    using Model;
</code>```

ContentLicense: CC BY-SA 3.0

Body:
I have namespaces:

MyProject.Core.Db
MyProject.Core.Model

And I have classes:

MyProject.Core.Db.User
MyProject.Core.Model.User

Is it possible something like:

using MyProject.Core;

namespace MyProject.BLL
{
    public class Logic
    {
        public static void DoSomething()
        {
            var userEntity = new Db.User();
            var userModel = new Model.User();
        }
    }
}

I just want to avoid using suffixes in class names (UserModel, UserEntity).

Is it possible to do in somehow in C#?

Body:
One thing you can do, and we probably should do a lot more, is to specify usings relative to the current namespace. To do this, just move your usings inside the namespace declaration. It doesn't fix your stated problem, but the shorter relative paths are less brittle and your project will be easier to refactor.

namespace MyProject.Core{
    using Db;
    using Model;


问:
How to call randomly selected number from contact list in flutter?
```<code>import 'package:flutter_phone_direct_caller/flutter_phone_direct_caller.dart';
import 'package:contacts_service/contacts_service.dart';

void checkContactPermission() async {
  var status = await Permission.contacts.status;
  var contacts = ContactsService.getContacts();
  var list = [contacts];
  var randomItem = (list.shuffle());
  if (status.isGranted) {
    FlutterPhoneDirectCaller.callNumber('randomItem');
  }
  if (!status.isGranted) {
    PermissionStatus permissionStatus = await Permission.contacts.request();
    if (status.isDenied) {
      PermissionStatus permissionStatus = await Permission.contacts.request();
    }
  }
}
</code>```

Here is my code to call randomly selected number from contact list yet it is calling a same number that isn't even on the contact list. What should I change?
Thanks in advance

答:
```<code>Future<void> checkContactPermission() async {
  var status = await Permission.contacts.status;
  if (status.isGranted) {
    var contacts = await ContactsService.getContacts();
    var list = contacts;
    list.shuffle();
    var randomItem = (list.first.phones?.first.value);
    await FlutterPhoneDirectCaller.callNumber('$randomItem');
  }
  if (!status.isGranted) {
    PermissionStatus permissionStatus = await Permission.contacts.request();
    if (status.isDenied) {
      PermissionStatus permissionStatus = await Permission.contacts.request();
      }
    }
  }
</code>```

Here is what I found to be working

Body:
import 'package:flutter_phone_direct_caller/flutter_phone_direct_caller.dart';
import 'package:contacts_service/contacts_service.dart';

void checkContactPermission() async {
  var status = await Permission.contacts.status;
  var contacts = ContactsService.getContacts();
  var list = [contacts];
  var randomItem = (list.shuffle());
  if (status.isGranted) {
    FlutterPhoneDirectCaller.callNumber('randomItem');
  }
  if (!status.isGranted) {
    PermissionStatus permissionStatus = await Permission.contacts.request();
    if (status.isDenied) {
      PermissionStatus permissionStatus = await Permission.contacts.request();
    }
  }
}

Here is my code to call randomly selected number from contact list yet it is calling a same number that isn't even on the contact list. What should I change?
Thanks in advance

Body:
Future<void> checkContactPermission() async {
  var status = await Permission.contacts.status;
  if (status.isGranted) {
    var contacts = await ContactsService.getContacts();
    var list = contacts;
    list.shuffle();
    var randomItem = (list.first.phones?.first.value);
    await FlutterPhoneDirectCaller.callNumber('$randomItem');
  }
  if (!status.isGranted) {
    PermissionStatus permissionStatus = await Permission.contacts.request();
    if (status.isDenied) {
      PermissionStatus permissionStatus = await Permission.contacts.request();
      }
    }
  }

Here is what I found to be working


问:
Validating Indentation within YAML file
I am trying to write Python code that checks if indentation within a YAML file is indented correctly and flags an error if any inconsistencies exist.

For example, the second occurrence of the key-value pair mapping "class" has 4 spaces before it when it should instead have 6 spaces (like the first occurrence).

I have dozens of these YAML files with thousands of entries. So, I need an automated way to check if the indentation is inaccurate.

How could I achieve this within Python?

```<code>students:
  incoming:
  - enrolled: TRUE
    semester: final
    destination:
      name:
      - John Walsh
      - Heather Dunbar
      class:
      - 1258
  - enrolled: TRUE
    semester: final
    destination:
      name:
      - Alfred Flynn
      - Joe Diaz
    class: ## incorrectly indented entry.
      - 3662
</code>```

Here's my code:

```<code>class_indentation = "      class:"

with open("yamls/students.yaml", "r") as file:
    for line_number, line in enumerate(file, start=1):
        if class_indentation in line:
          print(f"Indentation for '{class_indentation}' is valid: {line_number}")
          break
        else:
          print(f"Indentation for '{class_indentation}' is NOT valid: {line_number}")
print("Search completed.")
</code>```

答:
TL;DR: use a YAML parser and test valid nesting: if the class node is child of destination.

YAML parsers

There are 2 major YAML parsers for Python:

ruamel.yaml, preserves more of the original (like comments, ordering, etc.)

pyyaml, which can be seen as the predecessor to ruamel.yaml

For simplicity I will use pyyaml below.

Using pyyaml to test valid nesting

I found How to parse deeply nested yaml data structures in python and reused the answered functions here.

Below code looks for invalid indentation, if a class element is not child of destination:

import yaml

yaml_text = '''
students:
  incoming:
  - enrolled: TRUE
    semester: final
    destination:
      name:
      - John Walsh
      - Heather Dunbar
      class:
      - 1258
  - enrolled: TRUE
    semester: final
    destination:
      name:
      - Alfred Flynn
      - Joe Diaz
    class: ## incorrectly indented entry.
      - 3662
'''

def lookup(sk, d, path=[]):
   # lookup the values for key(s) sk return as list the tuple (path to the value, value)
   if isinstance(d, dict):
       for k, v in d.items():
           if k == sk:
               yield (path + [k], v)
           for res in lookup(sk, v, path + [k]):
               yield res
   elif isinstance(d, list):
       for item in d:
           for res in lookup(sk, item, path + [item]):
               yield res

tree_dict = yaml.safe_load(yaml_text)
for (segments, value) in lookup("class", tree_dict):
    if segments[-2] != 'destination':
        print("Invalid indentation!  Not child of 'destination':")
    else:
       print("OK:")
    print(f"\tpath-segments: {segments}\n\tvalue: {value}")
```
<p>Prints:</p>
```OK:
    path-segments: ['students', 'incoming', {'enrolled': True, 'semester': 'final', 'destination': {'name': ['John Walsh', 'Heather Dunbar'], 'class': [1258]}}, 'destination', 'class']
    value: [1258]
Invalid indentation!  Not child of 'destination':
    path-segments: ['students', 'incoming', {'enrolled': True, 'semester': 'final', 'destination': {'name': ['Alfred Flynn', 'Joe Diaz']}, 'class': [3662]}, 'class']
    value: [3662]
```
<h2>Using <code>ruamel.yaml</code> to test valid nesting</h2>
<p>You can also adapt to <code>ruamel.yaml</code> without loss of functionality.
Simply change import and loading:</p>
```# import yaml
from ruamel.yaml import YAML

# tree_dict = yaml.safe_load(yaml_text)
tree_dict = YAML(typ='safe').load(yaml_text)
```

Alternative: validate YAML using a schema

Alternatively you can also validate your YAML files against a schema.
For example using JSON-schame since YAML can be seen as superset to JSON.

See Validating a yaml document in python for more.

Title: Validating Indentation within YAML file

Body:
I am trying to write Python code that checks if indentation within a YAML file is indented correctly and flags an error if any inconsistencies exist.

For example, the second occurrence of the key-value pair mapping "class" has 4 spaces before it when it should instead have 6 spaces (like the first occurrence).

I have dozens of these YAML files with thousands of entries. So, I need an automated way to check if the indentation is inaccurate.

How could I achieve this within Python?

students:
  incoming:
  - enrolled: TRUE
    semester: final
    destination:
      name:
      - John Walsh
      - Heather Dunbar
      class:
      - 1258
  - enrolled: TRUE
    semester: final
    destination:
      name:
      - Alfred Flynn
      - Joe Diaz
    class: ## incorrectly indented entry.
      - 3662

Here's my code:

class_indentation = "      class:"

with open("yamls/students.yaml", "r") as file:
    for line_number, line in enumerate(file, start=1):
        if class_indentation in line:
          print(f"Indentation for '{class_indentation}' is valid: {line_number}")
          break
        else:
          print(f"Indentation for '{class_indentation}' is NOT valid: {line_number}")
print("Search completed.")

Body:
TL;DR: use a YAML parser and test valid nesting: if the class node is child of destination.

YAML parsers

There are 2 major YAML parsers for Python:

ruamel.yaml, preserves more of the original (like comments, ordering, etc.)

pyyaml, which can be seen as the predecessor to ruamel.yaml

For simplicity I will use pyyaml below.

Using pyyaml to test valid nesting

I found How to parse deeply nested yaml data structures in python and reused the answered functions here.

Below code looks for invalid indentation, if a class element is not child of destination:

import yaml

yaml_text = '''
students:
  incoming:
  - enrolled: TRUE
    semester: final
    destination:
      name:
      - John Walsh
      - Heather Dunbar
      class:
      - 1258
  - enrolled: TRUE
    semester: final
    destination:
      name:
      - Alfred Flynn
      - Joe Diaz
    class: ## incorrectly indented entry.
      - 3662
'''

def lookup(sk, d, path=[]):
   # lookup the values for key(s) sk return as list the tuple (path to the value, value)
   if isinstance(d, dict):
       for k, v in d.items():
           if k == sk:
               yield (path + [k], v)
           for res in lookup(sk, v, path + [k]):
               yield res
   elif isinstance(d, list):
       for item in d:
           for res in lookup(sk, item, path + [item]):
               yield res

tree_dict = yaml.safe_load(yaml_text)
for (segments, value) in lookup("class", tree_dict):
    if segments[-2] != 'destination':
        print("Invalid indentation!  Not child of 'destination':")
    else:
       print("OK:")
    print(f"\tpath-segments: {segments}\n\tvalue: {value}")

Prints:

OK:
    path-segments: ['students', 'incoming', {'enrolled': True, 'semester': 'final', 'destination': {'name': ['John Walsh', 'Heather Dunbar'], 'class': [1258]}}, 'destination', 'class']
    value: [1258]
Invalid indentation!  Not child of 'destination':
    path-segments: ['students', 'incoming', {'enrolled': True, 'semester': 'final', 'destination': {'name': ['Alfred Flynn', 'Joe Diaz']}, 'class': [3662]}, 'class']
    value: [3662]

Using ruamel.yaml to test valid nesting

You can also adapt to ruamel.yaml without loss of functionality.
Simply change import and loading:

# import yaml
from ruamel.yaml import YAML

# tree_dict = yaml.safe_load(yaml_text)
tree_dict = YAML(typ='safe').load(yaml_text)

Alternative: validate YAML using a schema

Alternatively you can also validate your YAML files against a schema.
For example using JSON-schame since YAML can be seen as superset to JSON.

See Validating a yaml document in python for more.


问:
Adding multi level X axis to matplotlib/Seaborn (month and year)
I am having difficulty adding a multi level axis with month and then year to my plot and I have been unable to find any answers anywhere. I have a dataframe which contains the upload date as a datetime dtype and then the year and month for each row. See Below:

```<code>    Upload Date Year    Month      DocID
0   2021-03-22  2021    March      DOC146984
1   2021-12-16  2021    December   DOC173111
2   2021-12-07  2021    December   DOC115350
3   2021-10-29  2021    October    DOC150149
4   2021-03-12  2021    March      DOC125480
5   2021-06-25  2021    June       DOC101062
6   2021-05-03  2021    May        DOC155916
7   2021-11-14  2021    November   DOC198519
8   2021-03-20  2021    March      DOC159523
9   2021-07-19  2021    July       DOC169328
10  2021-04-13  2021    April      DOC182660
11  2021-10-08  2021    October    DOC176871
12  2021-09-19  2021    September  DOC185854
13  2021-05-16  2021    May        DOC192329
14  2021-06-29  2021    June       DOC142190
15  2021-11-30  2021    November   DOC140231
16  2021-11-12  2021    November   DOC145392
17  2021-11-10  2021    November   DOC178159
18  2021-11-06  2021    November   DOC160932
19  2021-06-16  2021    June       DOC131448
</code>```

What I am trying to achieve is to build a bar chart which has the count for number of documents in each month and year. The graph would look something like this:

The main thing is that the x axis is split by each month and then further by each year, rather than me labelling each column with month and year (e.g 'March 2021'). However I can't figure out how to achieve this. I've tried using a countplot but it only allows me to choose month or year (See Below). I have also tried groupby but the end product is always the same. Any Ideas?

This is using randomly generated data, see the code to replicate below:

```<code>import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.style as style
import seaborn as sns
from datetime import date, timedelta
from random import choices
np.random.seed(42)

# initializing dates ranges
test_date1, test_date2 = date(2020, 1, 1), date(2021, 6, 30)

# initializing K
K = 2000

res_dates = [test_date1]

# loop to get each date till end date
while test_date1 != test_date2:
    test_date1 += timedelta(days=1)
    res_dates.append(test_date1)

# random K dates from pack
res = choices(res_dates, k=K)

# Generating dataframe
df = pd.DataFrame(res, columns=['Upload Date'])

# Generate other columns
df['Upload Date'] = pd.to_datetime(df['Upload Date'])
df['Year'] = df['Upload Date'].dt.year
df['Month'] = df['Upload Date'].dt.month_name()
df['DocID'] = np.random.randint(100000,200000, df.shape[0]).astype('str')
df['DocID'] = 'DOC' + df['DocID']

# plotting graph
sns.set_color_codes("pastel")
f, ax = plt.subplots(figsize=(20,8))
sns.countplot(x='Month', data=df)
</code>```

答:
A new column with year and month in numeric form can serve to indicate the x-positions, correctly ordered.  The x-tick labels can be renamed to the month names. Vertical lines and manual placing of the year labels lead to the final plot:

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns

test_date1, test_date2 = '20200101', '20210630'

months = pd.date_range('2021-01-01', periods=12, freq='M').strftime('%B')
K = 2000
df = pd.DataFrame(np.random.choice(pd.date_range(test_date1, test_date2), K), columns=['Upload Date'])
df['Year'] = df['Upload Date'].dt.year
# df['Month'] = pd.Categorical(df['Upload Date'].dt.strftime('%B'), categories=months)
df['YearMonth'] = df['Upload Date'].dt.strftime('%Y%m').astype(int)
df['DocID'] = np.random.randint(100000, 200000, df.shape[0]).astype('str')
df['DocID'] = 'DOC' + df['DocID']

sns.set_style("white")
sns.set_color_codes("pastel")
fig, ax = plt.subplots(figsize=(20, 8))
sns.countplot(x='YearMonth', data=df, ax=ax)
sns.despine()
yearmonth_labels = [int(l.get_text()) for l in ax.get_xticklabels()]
ax.set_xticklabels([months[ym % 100 - 1] for ym in yearmonth_labels])
ax.set_xlabel('')

# calculate the positions of the borders between the years
pos = []
years = []
prev = None
for i, ym in enumerate(yearmonth_labels):
    if ym // 100 != prev:
        pos.append(i)
        prev = ym // 100
        years.append(prev)
pos.append(len(yearmonth_labels))
pos = np.array(pos) - 0.5
# vertical lines to separate the years
ax.vlines(pos, 0, -0.12, color='black', lw=0.8, clip_on=False, transform=ax.get_xaxis_transform())
# years at the center of their range
for year, pos0, pos1 in zip(years, pos[:-1], pos[1:]):
    ax.text((pos0 + pos1) / 2, -0.07, year, ha='center', clip_on=False, transform=ax.get_xaxis_transform())

ax.set_xlim(pos[0], pos[-1])
ax.set_ylim(ymin=0)
plt.tight_layout()
plt.show()
```

Body:
I am having difficulty adding a multi level axis with month and then year to my plot and I have been unable to find any answers anywhere. I have a dataframe which contains the upload date as a datetime dtype and then the year and month for each row. See Below:

    Upload Date Year    Month      DocID
0   2021-03-22  2021    March      DOC146984
1   2021-12-16  2021    December   DOC173111
2   2021-12-07  2021    December   DOC115350
3   2021-10-29  2021    October    DOC150149
4   2021-03-12  2021    March      DOC125480
5   2021-06-25  2021    June       DOC101062
6   2021-05-03  2021    May        DOC155916
7   2021-11-14  2021    November   DOC198519
8   2021-03-20  2021    March      DOC159523
9   2021-07-19  2021    July       DOC169328
10  2021-04-13  2021    April      DOC182660
11  2021-10-08  2021    October    DOC176871
12  2021-09-19  2021    September  DOC185854
13  2021-05-16  2021    May        DOC192329
14  2021-06-29  2021    June       DOC142190
15  2021-11-30  2021    November   DOC140231
16  2021-11-12  2021    November   DOC145392
17  2021-11-10  2021    November   DOC178159
18  2021-11-06  2021    November   DOC160932
19  2021-06-16  2021    June       DOC131448

What I am trying to achieve is to build a bar chart which has the count for number of documents in each month and year. The graph would look something like this:

The main thing is that the x axis is split by each month and then further by each year, rather than me labelling each column with month and year (e.g 'March 2021'). However I can't figure out how to achieve this. I've tried using a countplot but it only allows me to choose month or year (See Below). I have also tried groupby but the end product is always the same. Any Ideas?

This is using randomly generated data, see the code to replicate below:

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.style as style
import seaborn as sns
from datetime import date, timedelta
from random import choices
np.random.seed(42)

# initializing dates ranges
test_date1, test_date2 = date(2020, 1, 1), date(2021, 6, 30)

# initializing K
K = 2000

res_dates = [test_date1]

# loop to get each date till end date
while test_date1 != test_date2:
    test_date1 += timedelta(days=1)
    res_dates.append(test_date1)

# random K dates from pack
res = choices(res_dates, k=K)

# Generating dataframe
df = pd.DataFrame(res, columns=['Upload Date'])

# Generate other columns
df['Upload Date'] = pd.to_datetime(df['Upload Date'])
df['Year'] = df['Upload Date'].dt.year
df['Month'] = df['Upload Date'].dt.month_name()
df['DocID'] = np.random.randint(100000,200000, df.shape[0]).astype('str')
df['DocID'] = 'DOC' + df['DocID']

# plotting graph
sns.set_color_codes("pastel")
f, ax = plt.subplots(figsize=(20,8))
sns.countplot(x='Month', data=df)


问:
R remove header line after placing Kable side by side in Rmarkdown generated HTML
I'm using Rmarkdown to generate an HTML output, where two kables are placed side by side using kable::kables. However, there's an extra line above the side-by-side kables that I would like to remove. I've read the documentation of kableExtra, but seems like they didn't cover this issue.

The horizontal line that I would like to remove is indicated by the red arrow below.

Here's my Rmarkdown code for your reference:

```<code>---
title: "Untitled"
author: "benson23"
output:
  html_document:
    theme: default
    toc: yes
    toc_depth: 4
    toc_float: true
    code_folding: hide
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
library(tidyverse)
library(rmarkdown)
library(kableExtra)
library(knitr)
```

## Help with Kable

As you can see, there's an extra line above the aligned tables, which I would like to remove.

```{r kable_extra_line}
df1 <- tibble(Dummy = c(90748, 91006, 90748, 91006),
                       A = c(0, 6, 1, 110),
                       B = c(8, 0, 339, 0),
                       C = c(6, 7, 909, 309),
                       D = c(0, 0, 0, 0))

df2 <- tibble(Dummy = c(90748, 91006, 90748, 91006),
                       A = c(0, 901, 0, 50),
                       B = c(0, 1, 122, 0),
                       C = c(1847, 1, 754, 625),
                       D = c(1, 0, 0, 0))
kables(list(
  kable(df1) %>%
    kable_styling(bootstrap_options = c("striped", "bordered"),
                  full_width = F) %>%
    row_spec(0, bold = T) %>%
    pack_rows("Pack some rows here", 1, 2,
              label_row_css = "background-color: #666; color: #fff;") %>%
    pack_rows("Pack some rows here", 3, 4,
              label_row_css = "background-color: #666; color: #fff;"),
  kable(df2) %>%
    kable_styling(bootstrap_options = c("striped", "bordered"),
                  full_width = F) %>%
    row_spec(0, bold = T) %>%
    pack_rows("Pack some rows here", 1, 2,
              label_row_css = "background-color: #666; color: #fff;") %>%
    pack_rows("Pack some rows here", 3, 4,
              label_row_css = "background-color: #666; color: #fff;")
)) %>%
  kable_styling()
```
</code>```

It'd be great if you have any idea on how to remove the line!

答:
You can add the css .kable_wrapper { border: hidden;}:

```<code>---
title: "Untitled"
author: "benson23"
output:
  html_document:
  theme: default
toc: yes
toc_depth: 4
toc_float: true
code_folding: hide
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
library(tidyverse)
library(rmarkdown)
library(kableExtra)
library(knitr)
```

<style>
.kable_wrapper {
  border: hidden;
}
</style>

## Help with Kable

As you can see, there's an extra line above the aligned tables, which I would like to remove.

```{r kable_extra_line}
    df1 <- tibble(Dummy = c(90748, 91006, 90748, 91006),
                           A = c(0, 6, 1, 110),
                           B = c(8, 0, 339, 0),
                           C = c(6, 7, 909, 309),
                           D = c(0, 0, 0, 0))

    df2 <- tibble(Dummy = c(90748, 91006, 90748, 91006),
                           A = c(0, 901, 0, 50),
                           B = c(0, 1, 122, 0),
                           C = c(1847, 1, 754, 625),
                           D = c(1, 0, 0, 0))
    kables(list(
      kable(df1) %>%
        kable_styling(bootstrap_options = c("striped", "bordered"),
                      full_width = F) %>%
        row_spec(0, bold = T) %>%
        pack_rows("Pack some rows here", 1, 2,
                  label_row_css = "background-color: #666; color: #fff;") %>%
        pack_rows("Pack some rows here", 3, 4,
                  label_row_css = "background-color: #666; color: #fff;"),
      kable(df2) %>%
        kable_styling(bootstrap_options = c("striped", "bordered"),
                      full_width = F) %>%
        row_spec(0, bold = T) %>%
        pack_rows("Pack some rows here", 1, 2,
                  label_row_css = "background-color: #666; color: #fff;") %>%
        pack_rows("Pack some rows here", 3, 4,
                  label_row_css = "background-color: #666; color: #fff;")
    )) %>%
      kable_styling()
```
</code>```

Body:
I'm using Rmarkdown to generate an HTML output, where two kables are placed side by side using kable::kables. However, there's an extra line above the side-by-side kables that I would like to remove. I've read the documentation of kableExtra, but seems like they didn't cover this issue.

The horizontal line that I would like to remove is indicated by the red arrow below.

Here's my Rmarkdown code for your reference:

---
title: "Untitled"
author: "benson23"
output:
  html_document:
    theme: default
    toc: yes
    toc_depth: 4
    toc_float: true
    code_folding: hide
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
library(tidyverse)
library(rmarkdown)
library(kableExtra)
library(knitr)
```

## Help with Kable

As you can see, there's an extra line above the aligned tables, which I would like to remove.

```{r kable_extra_line}
df1 <- tibble(Dummy = c(90748, 91006, 90748, 91006),
                       A = c(0, 6, 1, 110),
                       B = c(8, 0, 339, 0),
                       C = c(6, 7, 909, 309),
                       D = c(0, 0, 0, 0))

df2 <- tibble(Dummy = c(90748, 91006, 90748, 91006),
                       A = c(0, 901, 0, 50),
                       B = c(0, 1, 122, 0),
                       C = c(1847, 1, 754, 625),
                       D = c(1, 0, 0, 0))
kables(list(
  kable(df1) %>%
    kable_styling(bootstrap_options = c("striped", "bordered"),
                  full_width = F) %>%
    row_spec(0, bold = T) %>%
    pack_rows("Pack some rows here", 1, 2,
              label_row_css = "background-color: #666; color: #fff;") %>%
    pack_rows("Pack some rows here", 3, 4,
              label_row_css = "background-color: #666; color: #fff;"),
  kable(df2) %>%
    kable_styling(bootstrap_options = c("striped", "bordered"),
                  full_width = F) %>%
    row_spec(0, bold = T) %>%
    pack_rows("Pack some rows here", 1, 2,
              label_row_css = "background-color: #666; color: #fff;") %>%
    pack_rows("Pack some rows here", 3, 4,
              label_row_css = "background-color: #666; color: #fff;")
)) %>%
  kable_styling()
```

It'd be great if you have any idea on how to remove the line!

Body:
You can add the css .kable_wrapper { border: hidden;}:

---
title: "Untitled"
author: "benson23"
output:
  html_document:
  theme: default
toc: yes
toc_depth: 4
toc_float: true
code_folding: hide
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
library(tidyverse)
library(rmarkdown)
library(kableExtra)
library(knitr)
```

<style>
.kable_wrapper {
  border: hidden;
}
</style>

## Help with Kable

As you can see, there's an extra line above the aligned tables, which I would like to remove.

```{r kable_extra_line}
    df1 <- tibble(Dummy = c(90748, 91006, 90748, 91006),
                           A = c(0, 6, 1, 110),
                           B = c(8, 0, 339, 0),
                           C = c(6, 7, 909, 309),
                           D = c(0, 0, 0, 0))

    df2 <- tibble(Dummy = c(90748, 91006, 90748, 91006),
                           A = c(0, 901, 0, 50),
                           B = c(0, 1, 122, 0),
                           C = c(1847, 1, 754, 625),
                           D = c(1, 0, 0, 0))
    kables(list(
      kable(df1) %>%
        kable_styling(bootstrap_options = c("striped", "bordered"),
                      full_width = F) %>%
        row_spec(0, bold = T) %>%
        pack_rows("Pack some rows here", 1, 2,
                  label_row_css = "background-color: #666; color: #fff;") %>%
        pack_rows("Pack some rows here", 3, 4,
                  label_row_css = "background-color: #666; color: #fff;"),
      kable(df2) %>%
        kable_styling(bootstrap_options = c("striped", "bordered"),
                      full_width = F) %>%
        row_spec(0, bold = T) %>%
        pack_rows("Pack some rows here", 1, 2,
                  label_row_css = "background-color: #666; color: #fff;") %>%
        pack_rows("Pack some rows here", 3, 4,
                  label_row_css = "background-color: #666; color: #fff;")
    )) %>%
      kable_styling()
```


问:
Split String and Add and Return Double Value using stream api Java
I have String "ORIGINAL_PAYMODE:90.23,VOUCHRE:30.10,ORIGINAL_PAYMODE:80.30,VOUCHRE:33.10" i need to Split and add all ORIGINAL_PAYMODE value occurring in String using stream api

In this case I need result of 90.23+80.30 = 170.53

code :

```<code>public static void main(String[] args) {

String reftype
="ORIGINAL_PAYMODE:90.23,VOUCHRE:30.10,ORIGINAL_PAYMODE:80.30,VOUCHRE:33.10";
Pattern COMMA = Pattern.compile(",");

Pattern p = Pattern.compile(":");

String result = COMMA.splitAsStream(reftype)
            .filter(role -> role.contains("ORIGINAL_PAYMODE"))
            .map(String::trim)
            .collect(Collectors.joining(","));

System.out.println(result);
</code>```

答:
The easy step from what you have is to map it to a double stream, then sum().

I'm also splitting on the comma as it seems to be what separates your key/value pairs.

```<code>double sum = Arrays.stream(s.split(",")) //split as you wish or find efficient
                   .filter(v -> v.startsWith("ORIGINAL_PAYMODE"))
                   .map(v -> v.substring(v.indexOf(":") + 1).trim())
                   .mapToDouble(Double::parseDouble)
                   .sum()
</code>```

Body:
I have String "ORIGINAL_PAYMODE:90.23,VOUCHRE:30.10,ORIGINAL_PAYMODE:80.30,VOUCHRE:33.10" i need to Split and add all ORIGINAL_PAYMODE value occurring in String using stream api

In this case I need result of 90.23+80.30 = 170.53

code :

public static void main(String[] args) {

String reftype
="ORIGINAL_PAYMODE:90.23,VOUCHRE:30.10,ORIGINAL_PAYMODE:80.30,VOUCHRE:33.10";
Pattern COMMA = Pattern.compile(",");

Pattern p = Pattern.compile(":");

String result = COMMA.splitAsStream(reftype)
            .filter(role -> role.contains("ORIGINAL_PAYMODE"))
            .map(String::trim)
            .collect(Collectors.joining(","));

System.out.println(result);

Body:
The easy step from what you have is to map it to a double stream, then sum().

I'm also splitting on the comma as it seems to be what separates your key/value pairs.

double sum = Arrays.stream(s.split(",")) //split as you wish or find efficient
                   .filter(v -> v.startsWith("ORIGINAL_PAYMODE"))
                   .map(v -> v.substring(v.indexOf(":") + 1).trim())
                   .mapToDouble(Double::parseDouble)
                   .sum()


问:
Retrofit: multiple query parameters in @GET command?
I am using Retrofit and Robospice to make API calls in my android application. All @POST methods work great, and so do @GET commands without any parameters in the URL, but I can't get any @GET calls to work with parameters on the end!

For example, if my API path was "my/api/call/" and I wanted 2 parameters "param1" and "param2" in the URL, the get call would look like:

http://www.example.com/my/api/call?param1=value1&param2=value2

so I have setup my @GET interface like so:

```<code>@GET("/my/api/call?param1={p1}&param2={p2}")
Response getMyThing(@Path("p1")
String param1, @Path("p2")
String param2);
</code>```

but I get an error saying
"An exception occurred during request network execution : URL query string "/my/api/call?param1={p1}&param2={p2}" on method getMyThing may not have replaced block."

What am I doing wrong?

答:
This is the best way you can use:

```<code>@GET("api_link_put_here")
    apiResponse getMyData(
            @Query("first_param") String firstParam,
            @Query("second_param") String secondParam
    );
</code>```

this will automatically work like:
http://baseurl.com/api_link_put_here?first_param=firstParam&second_param=secondParam

Body:
I am using Retrofit and Robospice to make API calls in my android application. All @POST methods work great, and so do @GET commands without any parameters in the URL, but I can't get any @GET calls to work with parameters on the end!

For example, if my API path was "my/api/call/" and I wanted 2 parameters "param1" and "param2" in the URL, the get call would look like:

http://www.example.com/my/api/call?param1=value1&param2=value2

so I have setup my @GET interface like so:

@GET("/my/api/call?param1={p1}&param2={p2}")
Response getMyThing(@Path("p1")
String param1, @Path("p2")
String param2);

but I get an error saying
"An exception occurred during request network execution : URL query string "/my/api/call?param1={p1}&param2={p2}" on method getMyThing may not have replaced block."

What am I doing wrong?

Body:
This is the best way you can use:

@GET("api_link_put_here")
    apiResponse getMyData(
            @Query("first_param") String firstParam,
            @Query("second_param") String secondParam
    );

this will automatically work like:
http://baseurl.com/api_link_put_here?first_param=firstParam&second_param=secondParam