""" # COVID-19 Open Research Dataset (CORD-19) Analysis ![CORD-19.png](attachment:CORD-19.png) *An example of result snippet is shown below, For each tasks a sperate notebook is created with answers to each questions in a task added to a excel* """ """ **TASK 1:** Task Details What do we know about COVID-19 risk factors? What have we learned from epidemiological studies? Specifically, we want to know what the literature reports about: 1. Data on potential risks factors * Smoking, pre-existing pulmonary disease * Co-infections (determine whether co-existing respiratory/viral infections make the virus more transmissible or virulent) and other co-morbidities * Neonates and pregnant women * Socio-economic and behavioral factors to understand the economic impact of the virus and whether there were differences. 2. Transmission dynamics of the virus, including the basic reproductive number, incubation period, serial interval, modes of transmission and environmental factors 3. Severity of disease, including risk of fatality among symptomatic hospitalized patients, and high-risk patient groups 4. Susceptibility of populations 5. Public health mitigation measures that could be effective for control """ """ In the above Questions, Questions 1, 3 AND 5 are attempted using a combination of word2vec embedding to extract the keywords and search algorithm to extract the research articles related to it. Word2vec embedding provides the keywords that are closest to answer the questions """ """ # SOLUTION APPROACH ![covid.JPG](attachment:covid.JPG) """ """ # PROS and CONS PROS: 1. Good understanding and iference of the research papers through TSNE and LDA models 2. Well Trained word embedding models 3. Accurate keyword extraction with emamples demonstrated with results CONS: 1. Manual intervention to establish the relationship of search algorithm to derive answers to the questions. """ """ # VISUALIZATION """ """ Import libraries """ # This Python 3 environment comes with many helpful analytics libraries installed # It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python # For example, here's several helpful packages to load in import re import string import collections import pandas as pd import seaborn as sns import matplotlib.pyplot as plt from nltk import word_tokenize from nltk.stem import PorterStemmer from nltk.corpus import stopwords from sklearn.manifold import TSNE from sklearn.cluster import KMeans from sklearn.feature_extraction.text import HashingVectorizer from sklearn.feature_extraction.text import TfidfVectorizer from sklearn.model_selection import train_test_split from sklearn.cluster import MiniBatchKMeans from time import time %matplotlib inline import os import pandas as pd import nltk nltk.download('stopwords') nltk.download('punkt') import spacy import spacy.cli from spacy.matcher import Matcher from spacy.matcher import PhraseMatcher spacy.cli.download("en") spacy.cli.download("en_core_web_lg") nlp = spacy.load('en_core_web_lg') """ Cleaning of corpus helper functions """ ## set english loanguage stop_words = set(stopwords.words('english')) ## declaration of Porter stemmer. porter=PorterStemmer() ## Clean Null Record in dataframe def cleanEmptyData(columnName,df): return df[df[columnName].notnull()] ## Remove Punctuation def remove_punctuation(columnName,df): return df.loc[:,columnName].apply(lambda x: re.sub('[^a-zA-z\s]','',x)) ## Convert To Lower Case def lower_case(input_str): input_str = input_str.lower() return input_str ## Remove duplicate item in the dataframe def removeDuplicate(df,list): df.drop_duplicates(list, inplace=True) ## Remove nlp stop words def remove_stop_words(columnName,df): return df.loc[:,columnName].apply(lambda x: [word for word in x.split() if word not in stop_words]) ##Remove single character from the sentence def remove_one_character_word(columnName,df): return df.loc[:,columnName].apply(lambda x: [i for i in x if len(i) > 1]) ## Join as a single text with seperator def join_seperator(columnName,df): seperator = ', ' return df.loc[:,columnName].apply(lambda x: seperator.join(x)) ## apply stemmer to data frame fields def apply_stemmer(columnName,df): return df.loc[:,columnName].apply(lambda x: [porter.stem(word) for word in x]) ## Data Cleaning Process function def dataCleaningProcess(dataFrame): ## remove duplicate records removeDuplicate(dataFrame,['abstract', 'text_body']) ## clean null value records clean_data = cleanEmptyData('text_body',dataFrame) clean_data.loc[:,'text_body_clean'] = clean_data.loc[:,'text_body'].apply(lambda x: lower_case(x)) ## removing punctuation clean_data.loc[:,'text_body_clean'] = remove_punctuation('text_body_clean',clean_data) ## apply stop words clean_data.loc[:,'text_body_clean'] = remove_stop_words('text_body_clean',clean_data) ## apply stemmer for each tokens clean_data.loc[:,'text_body_clean'] = apply_stemmer('text_body_clean',clean_data) ## removing single charter word in the sentence clean_data.loc[:,'text_body_clean'] = remove_one_character_word('text_body_clean',clean_data) ## join as a single text from words token clean_data.loc[:,'text_body_clean'] = join_seperator('text_body_clean',clean_data) ## remove coma after join clean_data.loc[:,'text_body_clean'] = remove_punctuation('text_body_clean',clean_data) return clean_data """ Re usable Helper functions """ ## get words token from text def getWordsFromText(_text): words = [] for i in range(0,len(_text)): words.append(str(_text.iloc[i]['text_body']).split(" ")) return words # Read Excel data as Data Frame def readExcelToDataFrame(path): research_dataframe = pd.read_csv(path,index_col=False) research_dataframe.drop(research_dataframe.columns[research_dataframe.columns.str.contains('unnamed',case = False)],axis = 1, inplace = True) return research_dataframe ## basic scatter plot def showScatterPlot(_X,title): # sns settings sns.set(rc={'figure.figsize':(15,15)}) # colors palette = sns.color_palette("bright", 1) # plot sns.scatterplot(_X[:,0], _X[:,1], palette=palette) plt.title(title) # plt.savefig("plots/t-sne_covid19.png") plt.show() ## scatter plot with cluster def showClusterScatterPlot(_X, _y_pred, title): # sns settings sns.set(rc={'figure.figsize':(10,10)}) # colors palette = sns.color_palette("bright", len(set(_y_pred))) # plot sns.scatterplot(_X[:,0], _X[:,1], hue=_y_pred, legend='full', palette=palette) plt.title(title) # plt.savefig("plots/t-sne_covid19_label.png") plt.show() ## drop clumns def getTargetData(dataFrame): text_body = dataFrame.drop(["doc_id", "source", "title", "abstract"], axis=1) return getWordsFromText(text_body) ## train model for tSNE clustering visualization def trainEmbededData(_perplexity,dataFrame,total_cluster, _n_iter): ## convert text to word frequency vectors vectorizer = TfidfVectorizer(max_features=2**12) ## training the data and returning term-document matrix. _X = vectorizer.fit_transform(dataFrame['text_body_clean'].values) ## tsne declartion tsne = TSNE(verbose=1, perplexity=_perplexity,learning_rate=200, random_state=0, n_iter=_n_iter) _X_embeded = tsne.fit_transform(_X.toarray()) ## clusterring for tsne _kmeans = MiniBatchKMeans(n_clusters=total_cluster) return _X_embeded,_kmeans,_X ## predicting cluster centers and predict cluster index for each sample def predict(_kmeans,_X): return _kmeans.fit_predict(_X) ## reusable fucntion for TSNE K-Mean Clustering with TF-IDF def analyse(pplexity,data_frame,cluster,iter): ## train model for tSNE clustering visualization embeded,kmeans,x = trainEmbededData(pplexity,data_frame,cluster,iter) pred = predict(kmeans,x) ## visualized the scatter plot showClusterScatterPlot(embeded,pred,'t-SNE Covid-19 - Clustered(K-Means) - Tf-idf with Plain Text') return embeded,kmeans,x """ **Loading data(a)-Complete data by reading uploaded CSV** | CSV file is created by parsing the JSON data """ research_dataframe = readExcelToDataFrame('/kaggle/input/coviddata21/data.csv') research_dataframe.head() """ Data Cleaning """ clean_data =dataCleaningProcess(research_dataframe) """ Visualise Cleaned Data and removing columns """ clean_data.head() clean_process_data = clean_data.drop(["doc_id", "source", "title", "abstract"], axis=1) clean_process_data.head(20) """ Read the meta data from meta_data CSV """ meta_data = readExcelToDataFrame('/kaggle/input/metadata/meta.csv') meta_data.head() """ Data Preparation included the below process: * Load Research & Meta Data * Meta Data filter for published time from 2019 to 2020 on doc_id * Remove unused fields for the search inference. """ def prepare_search_data(_meta_data_frame,research_dataframe): ## add a field doc_id _meta_data_frame["doc_id"] = _meta_data_frame["sha"] ## clean NUll record _meta_data_frame = cleanEmptyData('doc_id', _meta_data_frame) _meta_data_frame = cleanEmptyData('publish_time', _meta_data_frame) ## select only 2019 & 2020 published records meta_data_filter = _meta_data_frame[_meta_data_frame['publish_time'].str.contains('2019') | _meta_data_frame['publish_time'].str.contains('2020')] ## clean NUll record research_dataframe_clean = cleanEmptyData('doc_id', research_dataframe) research_dataframe_clean = cleanEmptyData('text_body', research_dataframe_clean) ## merging of Research data and meta data on doc_id tmp_data_frame = research_dataframe_clean.merge(meta_data_filter, on='doc_id', how='right') ## remove un used fields clean_process_data = tmp_data_frame.drop(["source", "abstract_x", "abstract_x","sha","source_x","title_y","pmcid","pubmed_id","license","abstract_y","journal","Microsoft Academic Paper ID","WHO #Covidence"], axis=1) ## clean NUll record clean_process_data = cleanEmptyData('text_body', clean_process_data) clean_process_data = clean_process_data.rename(columns={'title_x': 'title'}) # reordering the column index columns = ["doc_id","doi", "publish_time", "authors","url","title", "text_body"] clean_process_data = clean_process_data.reindex(columns=columns) return clean_process_data def process_title(x): if not str(x['title_x']).lower() =='nan': return str(x['title_x']) + ' (' + str(x['url']) + ')' else: return str(x['url']) """ Filtered Data to be used to visualize the word cloud in the upcoming cells to understand the importance of topics and vocabs used. """ filter_data = prepare_search_data(meta_data,research_dataframe) filter_data.head() """ # EDA WORD CLOUD FOR THE TITLE OF ARTICLES FROM 2019 - 2020 """ """ Import libraries """ from wordcloud import WordCloud, STOPWORDS import matplotlib.pyplot as plt """ **WORD CLOUD VISUALIZATION ** We have filtered 2019-2020 research papers to understand COVID-19 data through the word cloud visualization. """ def show_WordCloud(filter_data): comment_words = ' ' stopwords = set(STOPWORDS) # iterate through the csv file for val in filter_data: # typecaste each val to string val = str(val) # split the value tokens = val.split() #print(val) # Converts each token into lowercase for i in range(len(tokens)): tokens[i] = tokens[i].lower() for words in tokens: comment_words = comment_words + words + ' ' #print(comment_words) wordcloud = WordCloud(width = 800, height = 800, background_color ='white', max_words = 200, stopwords = stopwords, min_font_size = 10).generate(comment_words) # plot the WordCloud image plt.figure(figsize = (10, 10), facecolor = None) plt.imshow(wordcloud) plt.axis("off") plt.tight_layout(pad = 0) plt.show() show_WordCloud(filter_data.title) """ The Visualization clearly talks about the : **coronavirus, transmission, infection, vaccine, ourbreak etc.. ** Giving a clear picture of the terminalogies and informations that can be retrived from the research papers. """ """ **Dimensionality Reduction with t-SNE** Using t-SNE we can reduce our high dimensional features vector to 2 dimensions. By using the 2 dimensions as x,y coordinates, the text_body_clean can be plotted. t-SNE will attempt to preserve the relations of the higher dimensional data as closely as possible when shrunk to 2D Analyse with perplexity of 5000, cluster 10, iteration : 15000 The optimal cluster parameter is decided after clustering the data from different preplexity and clusters and visualized. """ embeded,kmeans,x = analyse(5000,clean_process_data,10,15000) """ We tried understanding documents with different clusters with various perplexity and identified the optimal perplexity where the convergence of documents took place. """ """ # MODEL BUILDING AND INFERENCE """ """ LOAD the Research Dataset """ papers = research_dataframe['text_body'].astype('str') len(papers) papers.head() #meta_data_filter = _meta_data_frame[_meta_data_frame['publish_time'].str.contains('2019') | _meta_data_frame['publish_time'].str.contains('2020')] """ We perform some basic text wrangling or preprocessing before diving into topic modeling. We keep things simple here """ %%time import nltk import tqdm nltk.download('wordnet') stop_words = nltk.corpus.stopwords.words('english') wtk = nltk.tokenize.RegexpTokenizer(r'\w+') wnl = nltk.stem.wordnet.WordNetLemmatizer() def normalize_corpus(papers): norm_papers = [] for paper in tqdm.tqdm(papers): paper = paper.lower() paper_tokens = [token.strip() for token in wtk.tokenize(paper)] paper_tokens = [wnl.lemmatize(token) for token in paper_tokens if not token.isnumeric()] paper_tokens = [token for token in paper_tokens if len(token) > 1] paper_tokens = [token for token in paper_tokens if token not in stop_words] paper_tokens = list(filter(None, paper_tokens)) #if paper_tokens: norm_papers.append(paper_tokens) return norm_papers norm_papers = normalize_corpus(papers) print(len(norm_papers)) """ Build a Bi-gram Phrase Model """ import gensim bigram = gensim.models.Phrases(norm_papers, min_count=20, threshold=20, delimiter=b'_') # higher threshold fewer phrases. bigram_model = gensim.models.phrases.Phraser(bigram) print(bigram_model[norm_papers[0]][:50]) print(bigram_model[norm_papers[1]][:50]) norm_corpus_bigrams = [bigram_model[doc] for doc in norm_papers] # Create a dictionary representation of the documents. dictionary = gensim.corpora.Dictionary(norm_corpus_bigrams) print('Sample word to number mappings:', list(dictionary.items())[:15]) print('Total Vocabulary Size:', len(dictionary)) """ Looks like we have a lot of unique phrases in our corpus of research papers, based on the preceding output. Several of these terms are not very useful since they are specific to a paper. Hence, we will prune our vocabulary and start removing terms. """ # Filter out words that occur less than 20 documents, or more than 60% of the documents. dictionary.filter_extremes(no_below=20, no_above=0.6) print('Total Vocabulary Size:', len(dictionary)) """ **Transforming corpus into bag of words vectors** We can now perform feature engineering by leveraging a simple Bag of Words model. """ bow_corpus = [dictionary.doc2bow(text) for text in norm_corpus_bigrams] print(bow_corpus[1][:50]) print([(dictionary[idx] , freq) for idx, freq in bow_corpus[1][:50]]) print('Total number of papers:', len(bow_corpus)) """ **Topic Models with Latent Dirichlet Allocation (LDA)** """ """ ***Building model** > %%time > > > TOTAL_TOPICS = 10 > > lda_model = gensim.models.LdaModel(corpus=bow_corpus, id2word=dictionary, chunksize=1740, > alpha='auto', eta='auto', random_state=42, > iterations=500, num_topics=TOTAL_TOPICS, > passes=20, eval_every=None)* """ """ **Load the LDA Model ** """ import joblib lda_model = joblib.load('/kaggle/input/coviddata21/lda_model.jl') topics_assigned = lda_model[bow_corpus] len(topics_assigned) b= pd.DataFrame(topics_assigned,columns = ['T0','T1','T2','T3','T4','T5','T6','T7','T8','T9']) """ > TOPIC CSV CREATION """ d= pd.concat([research_dataframe['text_body'],b],axis=1) """ EXTRACT CSV """ d.to_csv("Topic_paper_07042020_v4.csv") """ **LDA TOPICS WITH TOPIC ID** """ for topic_id, topic in lda_model.print_topics(num_topics=50, num_words=20): print('Topic #'+str(topic_id+1)+':') print(topic) print() import numpy as np topics_coherences = lda_model.top_topics(bow_corpus, topn=20) avg_coherence_score = np.mean([item[1] for item in topics_coherences]) print('Avg. Coherence Score:', avg_coherence_score) """ **LDA TOPIC WITH WEIGHTS** """ topics_with_wts = [item[0] for item in topics_coherences] print('LDA Topics with Weights') print('='*50) for idx, topic in enumerate(topics_with_wts): print('Topic #'+str(idx+1)+':') print([(term, round(wt, 3)) for wt, term in topic]) print() """ **LDA TOPIC WITHOUT WEIGHTS** """ print('LDA Topics without Weights') print('='*50) for idx, topic in enumerate(topics_with_wts): print('Topic #'+str(idx+1)+':') print([term for wt, term in topic]) print() """ **Evaluating topic model:** Quality We can use perplexity and coherence scores as measures to evaluate the topic model. Typically, lower the perplexity, the better the model. Similarly, the lower the UMass score and the higher the Cv score in coherence, the better the model. """ cv_coherence_model_lda = gensim.models.CoherenceModel(model=lda_model, corpus=bow_corpus, texts=norm_corpus_bigrams, dictionary=dictionary, coherence='c_v') avg_coherence_cv = cv_coherence_model_lda.get_coherence() umass_coherence_model_lda = gensim.models.CoherenceModel(model=lda_model, corpus=bow_corpus, texts=norm_corpus_bigrams, dictionary=dictionary, coherence='u_mass') avg_coherence_umass = umass_coherence_model_lda.get_coherence() perplexity = lda_model.log_perplexity(bow_corpus) print('Avg. Coherence Score (Cv):', avg_coherence_cv) print('Avg. Coherence Score (UMass):', avg_coherence_umass) print('Model Perplexity:', perplexity) """ **Creation of Word Cloud Topic wise** """ from wordcloud import WordCloud, STOPWORDS import matplotlib.pyplot as plt stopwords = set(STOPWORDS) def show_wordcloud(data, title = None): wordcloud = WordCloud( background_color='white', stopwords=stopwords, max_words=2000, max_font_size=40, scale=3, random_state=1 # chosen at random by flipping a coin; it was heads ).generate(str(data)) fig = plt.figure(1, figsize=(15, 15)) plt.axis('off') if title: fig.suptitle(title, fontsize=20) fig.subplots_adjust(top=2.3) plt.imshow(wordcloud) plt.show() with open("/kaggle/input/coviddata21/Topic_paper_07042020_v4.csv",encoding = 'utf8', errors='ignore') as f: df_topic = pd.read_csv(f) #df = df.replace("\n"," ").dropna() """ **WORD CLOUD FOR THE DOMINANT TOPIC(S) ** Example: Topic 1 can have 20 documents while Topic 2 may have 200 documents in it.Hence a word cloud is drawn to visualize the intersting vocabs involving each Topic """ """ **Topic 1** """ show_WordCloud(df_topic.loc[df_topic['Dominant_topic'] == 0]['text_body']) #show_WordCloud(filter_data.title) """ In the above word cloud for the topics were mostly about genome of the covid-19 and possibilties of other corona virus genomes and carriers. """ """ **Topic 2** """ show_WordCloud(df_topic.loc[df_topic['Dominant_topic'] == 1]['text_body']) """ > In the above word cloud for the topics were mostly about infection samples and detection of virus in a patient """ """ **Inference of some topics ** * Topic 1&5: mainly deals with the virus structure and genome * Topic 3: deals with the origin and history * Topic 4&10: For public health risk information * Topic 9: Has lab research on various subjects """ """ **WORD2VEC MODEL ** WORD2VEC EMBEDDING IS USED TO PREDICT THE TARGET WORD(S) FORM THE CONTEXT WORDS(Questions from task) USING CBOW In this COVID-19 challenge the questions form the context words, we leverage the context words from the questions to predict the target words to form the keywords, which inturn will lead us to the answers for the respective context words(questions). we are using the CBOW approach in word2vec to acheive this. """ """ **Robust Word2Vec Model with Gensim** The __`gensim`__ framework, created by Radim Řehůřek consists of a robust, efficient and scalable implementation of the Word2Vec model. We will leverage the same on our covid-19 corpus. In our workflow, we will tokenize our normalized corpus and then focus on the following four parameters in the Word2Vec model to build it. - __`size`:__ The word embedding dimensionality : 100 - __`window`:__ The context window size : 20 - __`min_count`:__ The minimum word count : 1 - __`iter`:__ Iteration : 1000 - __`sg`:__ Training model, 1 for skip-gram otherwise CBOW : CBOW We have build a Word2Vec model on the corpus. """ """ Import libraries """ from __future__ import print_function __author__ = 'maxim' import numpy as np import gensim import string from gensim.models import Word2Vec from keras.callbacks import LambdaCallback from keras.layers.recurrent import LSTM from keras.layers.embeddings import Embedding from keras.layers import Dense, Activation from keras.models import Sequential from keras.utils.data_utils import get_file from gensim.models import Word2Vec import pandas as pd import numpy as np import re import json import pandas as pd from gensim.models.fasttext import FastText from os import listdir from os.path import isfile, join from tqdm import tqdm import nltk from nltk.corpus import stopwords from nltk.tokenize import word_tokenize import re import gensim import nltk """ word2vec model loading, we are using the training model by loading it. """ print('loading model') word_model = Word2Vec.load("/kaggle/input/coviddata21/word2vec_1000ITR.model") print('model loaded') """ **Model Evaluation:** Example 1: **" COVID-19 "** keyword when tested against our word embedding model gave the following results. [('ncp', 0.7167163491249084), ('wuhan', 0.7067078351974487), ('2019ncov', 0.7065909504890442), ('sarscov2', 0.6876667737960815), ('mers', 0.6260266304016113), ('sars', 0.6118236780166626), ('2020', 0.6101416349411011), ('hubei', 0.5763685703277588), ('mainland', 0.5460340976715088), ('china', 0.531872034072876)] notable keywords from the word2vec output are: 'wuhan','sars','china' relating to the orgin of the virus Example 2: For Multiple keywords **'treatment','option'** keyword when tested against our word embedding model gave the following results. [('treatments', 0.88392174243927), ('therapies', 0.7969704866409302), ('therapy', 0.7752498984336853), ('drugs', 0.7263898253440857), ('medications', 0.7093261480331421), ('antivirals', 0.6795921325683594), ('regimens', 0.6594418287277222), ('prophylaxis', 0.653971254825592), ('medication', 0.6348516941070557), ('antibiotics', 0.6138178706169128)] notable keywords from the word2vec outpur are : 'prophylaxis','therapies' relating to preventive medications . As shown below: """ #Severity of disease, including risk of fatality among symptomatic hospitalized patients, and high-risk patient groups a=['severity','disease','fatality','patients','symptomatic'] print(word_model.most_similar(positive=a,topn=30)) print("=================================================================") print(word_model.predict_output_word(a,topn=30)) #'populations','susceptibility','covid19' #Severity of disease, including risk of fatality among symptomatic hospitalized patients, and high-risk patient groups """ risk factor * Smoking, pre-existing pulmonary disease * Co-infections (determine whether co-existing respiratory/viral infections make the virus more transmissible or virulent) and other co-morbidities * Neonates and pregnant women * Socio-economic and behavioral factors to understand the economic impact of the virus and whether there were differences. """ #Data on potential risks factors #--Smoking, pre-existing pulmonary disease a=['covid19','risk','smoking','existing', 'pulmonary' ,'disease','comorbidity'] #comorbidities|mellitus|cardiovascular|vulnerability|hypertension|pneumonia|chronic|cvd print(word_model.most_similar(positive=a,topn=30)) print("=================================================================") print(word_model.predict_output_word(a,topn=30)) #'populations','susceptibility','covid19' #Severity of disease, including risk of fatality among symptomatic hospitalized patients, and high-risk patient groups """ As shown in the above steps, the context of the context words 'covid19' are well understood by the word2vec model in predicting the target words 'wuhan','sars','china' relating to the orgin of the virus which are very relavant in approaching the answers to the questions. """ #Co-infections (determine whether co-existing respiratory/viral infections make the virus more transmissible or virulent) and other co-morbidities a=['infection','coexisting','virus','transmissible', 'virulent' ,'comorbidity'] #severe|fatal|coinfection|contagious|susceptible|merscov|sftsv|highly|ibv|pedv|csfv|pathogenicity|eiav|prrsv|prv|h1n1pdm|hepatatis|lethalmhv| print(word_model.most_similar(positive=a,topn=30)) print("=================================================================") print(word_model.predict_output_word(a,topn=30)) # Risk of COVID-19 in Neonates and pregnant women a=['covid19','virus','risk','neonates','pregnant', 'women','postpartum','congenital'] #stillbirth|miscarriage|congenital|malformations|microcephaly|abortion|obstetric|lbw|covid19 print(word_model.most_similar(positive=a,topn=40)) print("=================================================================") print(word_model.predict_output_word(a,topn=40)) #Socio-economic and behavioral factors to understand the economic impact of the virus and whether there were differences. a=['covid19','virus','risk','economy','impact','behaviour','industry','global','socioeconomic','consequences'] #lessen productivity|macroeconomic affecting|vulnerability|devastating|reducing|recession|unemployment print(word_model.most_similar(positive=a,topn=40)) print("=================================================================") print(word_model.predict_output_word(a,topn=40)) print(word_model.most_similar(positive=['treatment','options' ])) """ As shown in the above steps, the context of the context words 'treatment','options' are well understood by the word2vec model in predicting the target words 'therapy','antivirals','regimens','prophylaxis' which are very relavant in finding the answers to the questions. """ """ **Answering Approach word embedding method** Word embedding for the following questions are extracted by passing a set of words from the question to the word embedding model and closest keywords are extracted to further find the closest keywords as shown in the example below: 1. Data on potential risks factors * Smoking, pre-existing pulmonary disease * Co-infections (determine whether co-existing respiratory/viral infections make the virus more transmissible or virulent) and other co-morbidities * Neonates and pregnant women * Socio-economic and behavioral factors to understand the economic impact of the virus and whether there were differences. 2. Severity of disease, including risk of fatality among symptomatic hospitalized patients, and high-risk patient groups 3. Public health mitigation measures that could be effective for control **Example:** **Question 3 : Persistence of virus on surfaces of different materials (e,g., copper, stainless steel, plastic).** ***Embedding Step to extract keywords* ** > print(word_model.predict_output_word(['covid19','surfaces'],topn=30)) > print(word_model.predict_output_word(['covid19','surfaces','nonporous','handrails','desks','floor','fomite'],topn=30)) > print(word_model.most_similar(['covid19','fomites','airborne','nonporous','handrails','desks','floor','fomite'],topn=30)) ***Search Algorithm to extracted related research articles:*** > searc_by_keys_as_excel('fomites|airborne|nonporous|handrails|desks|floor|fomite|hands|toilets|door|bedrails',search_data) > """ arr_most_similar = [['severity','disease','fatality','patients','symptomatic'], ['covid19','risk','smoking','existing', 'pulmonary' ,'disease','comorbidity'], ['infection','coexisting','virus','transmissible', 'virulent' ,'comorbidity'], ['covid19','virus','risk','neonates','pregnant', 'women','postpartum','congenital'], ['covid19','virus','risk','economy','impact','behaviour','industry','global','socioeconomic','consequence']] arr_predict=[['severity','disease','fatality','patients','symptomatic'], ['covid19','risk','smoking','existing', 'pulmonary' ,'disease','comorbidity'], ['infection','coexisting','virus','transmissible', 'virulent' ,'comorbidity'], ['covid19','virus','risk','neonates','pregnant', 'women','postpartum','congenital'], ['covid19','virus','risk','economy','impact','behaviour','industry','global','socioeconomic','consequence']] #EXAMPLE #Public health mitigation measures that could be effective for control #Intial input fed to word2vec model to extract related words that could lead to the answers of this question #['public','health','mitigation','measures', 'effective', 'control','covid19','disease']] #RESULT from first Query: [('interventions', 0.7447171807289124), ('prevention', 0.7438710927963257), ('preventive', 0.7241473197937012), ('policies', 0.7131460905075073), ('intervention', 0.7103219628334045), ('management', 0.7056314945220947), ('implementing', 0.6939526200294495), ('policy', 0.6897070407867432), ('quarantine', 0.6833001971244812), ('planning', 0.6516326069831848), ('implementation', 0.641217827796936), ('awareness', 0.6297034025192261), ('containment', 0.6278428435325623), ('preparedness', 0.622099757194519), ('epidemic', 0.6220937967300415), ('timely', 0.6204564571380615), ('community', 0.6184203624725342), ('government', 0.6134730577468872), ('outbreak', 0.6104072332382202), ('pandemic', 0.6079082489013672)]] #Updated input taken from first querying of word2vec model after choosing relevant keywords #['mitigation','measures','control','covid19','quarantine','containment','awareness','policies']] #RESULT from second Query: [[('interventions', 0.7805880308151245), ('intervention', 0.7159266471862793), ('policy', 0.690924346446991), ('preventive', 0.6865450143814087), ('implementing', 0.6757345795631409), ('implementation', 0.6531403064727783), ('planning', 0.6507831811904907), ('practices', 0.6400246620178223), ('prevention', 0.6383914947509766), ('management', 0.6378570199012756), ('government', 0.6369956731796265), ('preparedness', 0.6348874568939209), ('restrictions', 0.6139740943908691), ('campaigns', 0.6061151027679443), ('behaviors', 0.5989149808883667), ('plans', 0.5976110696792603), ('decisions', 0.5968020558357239), ('timely', 0.591980516910553), ('governmental', 0.5905696153640747), ('biosecurity', 0.5887465476989746)]] #Most Similar Keywrods Detection len(arr_most_similar) arrans=[] print(len(arr_most_similar)) count=0 for i in arr_most_similar: print('--------->',i) answers=word_model.most_similar(positive=i,topn=30) arrans.append(answers) count +=1 print('=========',count) print(arrans) print(len(arrans)) #Predicted Keywords Detection arr_predict len(arr_predict) arrans_arr_predict=[] print(len(arr_predict)) count=0 for j in arr_predict: print('--------->',j) answers1=word_model.predict_output_word(j,topn=30) arrans_arr_predict.append(answers1) count +=1 print('=========',count) print(arrans_arr_predict) print(len(arrans_arr_predict)) for q in arrans: print(q) #arrans for k in arrans_arr_predict: print(k) """ **KEYWORD EXTRACTION** * Public health mitigation measures that could be effective for control **--> 'interventions|policies|quarantine|awareness|preparedness|restrictions|campaigns|behaviors|biosecurity|governmental|search_data'** * Smoking, pre-existing pulmonary disease **--> comorbidities|mellitus|cardiovascular|vulnerability|hypertension|pneumonia|chronic** * Co-infections (determine whether co-existing respiratory/viral infections make the virus more transmissible or virulent) and other co-morbidities **--> severe|fatal|coinfection|contagious|susceptible|merscov|sftsv|highly|ibv|pedv|csfv|pathogenicity|eiav|prrsv|prv|h1n1pdm|hepatatis|lethal|mhv** * Neonates and pregnant women **--> stillbirth|miscarriage|congenital|malformations|microcephaly|abortion|obstetric|lbw|covid19** * Socio-economic and behavioral factors to understand the economic impact of the virus and whether there were differences. **--> lessen productivity|macroeconomic affecting|vulnerability|devastating|reducing|recession|unemployment** """ """ **SEARCH ALGORITHM ** To extract the relevant research documents from the keywords extarcted """ ## constant for spliting sentence alphabets= "([A-Za-z])" prefixes = "(Mr|St|Mrs|Ms|Dr)[.]" suffixes = "(Inc|Ltd|Jr|Sr|Co)" starters = "(Mr|Mrs|Ms|Dr|He\s|She\s|It\s|They\s|Their\s|Our\s|We\s|But\s|However\s|That\s|This\s|Wherever)" acronyms = "([A-Z][.][A-Z][.](?:[A-Z][.])?)" websites = "[.](com|net|org|io|gov)" ## spliting to sentence from text def split_into_sentences(text): text = " " + text + " " text = text.replace("\n"," ") text = re.sub(prefixes,"\\1<prd>",text) text = re.sub(websites,"<prd>\\1",text) if "Ph.D" in text: text = text.replace("Ph.D.","Ph<prd>D<prd>") text = re.sub("\s" + alphabets + "[.] "," \\1<prd> ",text) text = re.sub(acronyms+" "+starters,"\\1<stop> \\2",text) text = re.sub(alphabets + "[.]" + alphabets + "[.]" + alphabets + "[.]","\\1<prd>\\2<prd>\\3<prd>",text) text = re.sub(alphabets + "[.]" + alphabets + "[.]","\\1<prd>\\2<prd>",text) text = re.sub(" "+suffixes+"[.] "+starters," \\1<stop> \\2",text) text = re.sub(" "+suffixes+"[.]"," \\1<prd>",text) text = re.sub(" " + alphabets + "[.]"," \\1<prd>",text) if "”" in text: text = text.replace(".”","”.") if "\"" in text: text = text.replace(".\"","\".") if "!" in text: text = text.replace("!\"","\"!") if "?" in text: text = text.replace("?\"","\"?") text = text.replace(".",".<stop>") text = text.replace("?","?<stop>") text = text.replace("!","!<stop>") text = text.replace("<prd>",".") sentences = text.split("<stop>") sentences = sentences[:-1] sentences = [s.strip() for s in sentences] return sentences ## search inference text by key words and return all the matches sentence def search_inference_keys(text, keywords): sentences = split_into_sentences(text) txt = '' for sent in sentences: r = re.compile(keywords,flags=re.IGNORECASE) if len(r.findall(sent))>0: txt = str(txt) + str(sent) return txt ## check key words exist or not in a sentence def check_exist_multiple_keywords(text, keywords): r = re.compile(keywords, flags=re.IGNORECASE) if len(r.findall(text))>0: return True else: return False ## Search Inference and download results as excel def searc_by_keys_as_excel(keyword, src_data_frame): data_frame = src_data_frame ## check exist to slice down the related contents data_frame['search_key_status'] =data_frame.loc[:,'text_body'].apply(lambda x: check_exist_multiple_keywords(x,keyword)) ## select only target data process_data_frame = data_frame.query('search_key_status == True') ## filter on corona and covid 19 related data process_data_frame['search_covid_content'] =process_data_frame.loc[:,'text_body'].apply(lambda x: check_exist_multiple_keywords(x,'covid-19|sars-cov-2|2019-ncov|ncov-19|coronavirus')) ## get only covid-19|sars-cov-2|2019-ncov|ncov-19|coronavirus data process_data_frame = process_data_frame.query('search_covid_content == True') process_data_frame.loc[:,'inference'] = process_data_frame.loc[:,'text_body'].apply( lambda x: search_inference_keys(x,keyword)) ## remove unused fields final_data = process_data_frame.drop(["search_key_status","text_body"], axis=1) ## download as excel final_data.to_excel(str(keyword) + '_result.xlsx', sheet_name='keyword') return final_data # Search Inference for "incubation period" and download results as excel search_data = prepare_search_data(meta_data,research_dataframe) """ *The Keywords extracted through embedding are passed on to the below functions to extarct the list of research articles to further manually pick the articles relevant to the questions to answer the questions.* **Following are the keywords picked question wise:** * Public health mitigation measures that could be effective for control **--> 'interventions|policies|quarantine|awareness|preparedness|restrictions|campaigns|behaviors|biosecurity|governmental|search_data'** * Smoking, pre-existing pulmonary disease **--> comorbidities|mellitus|cardiovascular|vulnerability|hypertension|pneumonia|chronic** * Co-infections (determine whether co-existing respiratory/viral infections make the virus more transmissible or virulent) and other co-morbidities **--> severe|fatal|coinfection|contagious|susceptible|merscov|sftsv|highly|ibv|pedv|csfv|pathogenicity|eiav|prrsv|prv|h1n1pdm|hepatatis|lethal|mhv** * Neonates and pregnant women **--> stillbirth|miscarriage|congenital|malformations|microcephaly|abortion|obstetric|lbw|covid19** * Socio-economic and behavioral factors to understand the economic impact of the virus and whether there were differences. **--> lessen productivity|macroeconomic affecting|vulnerability|devastating|reducing|recession|unemployment** Seperate Excel is created in the Output section for each questions with its possible research articles that can prove the inference made using the word2vec answering approach. """ """ Question 1:* Public health mitigation measures that could be effective for control **--> 'interventions|policies|quarantine|awareness|preparedness|restrictions|campaigns|behaviors|biosecurity|governmental|search_data'** """ final_data =searc_by_keys_as_excel('interventions|policies|quarantine|awareness|preparedness|restrictions|campaigns|behaviors|biosecurity|governmental|search_data',search_data) #comorbidities|mellitus|cardiovascular|vulnerability|hypertension|pneumonia|chronic #'interventions','policies','quarantine','awareness''preparedness','restrictions','campaigns','behaviors', #'biosecurity','governmental' final_data.head() """ Question 2: * Smoking, pre-existing pulmonary disease **--> comorbidities|mellitus|cardiovascular|vulnerability|hypertension|pneumonia|chronic** """ final_data =searc_by_keys_as_excel('comorbidities|mellitus|cardiovascular|vulnerability|hypertension|pneumonia|chronic',search_data) """ Question 3: Co-infections (determine whether co-existing respiratory/viral infections make the virus more transmissible or virulent) and other co-morbidities --> severe|fatal|coinfection|contagious|susceptible|merscov|sftsv|highly|ibv|pedv|csfv|pathogenicity|eiav|prrsv|prv|h1n1pdm|hepatatis|lethal|mhv """ final_data =searc_by_keys_as_excel('severe|fatal|coinfection|contagious|susceptible|merscov|sftsv|highly|ibv|pedv|csfv|pathogenicity|eiav|prrsv|prv|h1n1pdm|hepatatis|lethalmhv|',search_data) """ Question 4: Neonates and pregnant women --> stillbirth|miscarriage|congenital|malformations|microcephaly|abortion|obstetric|lbw|covid19 """ final_data =searc_by_keys_as_excel('stillbirth|miscarriage|congenital|malformations|microcephaly|abortion|obstetric|lbw|covid19',search_data) """ Question 5: * Socio-economic and behavioral factors to understand the economic impact of the virus and whether there were differences. **--> lessen productivity|macroeconomic affecting|vulnerability|devastating|reducing|recession|unemployment** """ final_data =searc_by_keys_as_excel('lessen | productivity|macroeconomic | affecting|vulnerability|devastating|reducing|recession|unemployment',search_data) """ # CONCLUSION """ """ **Question 1: Public health mitigation measures that could be effective for control ** 492 f24d3b4b4af138be06b7452b7acefc8948bc1056 doi.org/10.1101/2020.01.28.923169 2020-01-28 Shao, P.; Shan, Y. Beware of asymptomatic transmission: Study on 2019-nCoV prevention and control measures based on extended SEIR model TRUE The purpose of this study is to reveal the role of the three most important current measures to control the spread of the epidemic, such as quarantine of infected persons, reduction of human mobility, and improvement of treatment. 508 754315299d847600d6c5d414665c728d40bf731d doi.org/10.1101/2020.01.27.922443 2020-01-30 Ming, W.-k.; Huang, J.; Zhang, C. J. P. Breaking down of healthcare system: Mathematical modelling for controlling the novel coronavirus (2019-nCoV) outbreak in Wuhan, China TRUE With the increasing incidence of confirmed cases, corresponding spread control policies and emergency actions are taking place.[6] Earlier studies on the effectiveness of spread control measures during infectious disease pandemic showed large-scale strategies, such as closure of school closure, case isolation, household quarantine, internal travel restrictions and border control, were able to delay the spread and/or reduce incidence rate at certain periods through the outbreak season.922443 doi: bioRxiv preprint As of 31 st January, it is estimated that there were 246,172 cases given a 10% diagnosis rate whilst being 88,075 and 52,094 cases given diagnosis rates of 50% and 90%, respectively, if no public health interventions were implemented ( Table 2) .If 70% efficacy rate could be achieved (Scenario 4), the forecasting number of cases would drop dramatically to 11,056 as of 10 th February compared to 115,355 without public health interventions (Scenario 2).922443 doi: bioRxiv preprint Therefore, the burdens on healthcare system would be substantial, particularly for the isolation wards and ICU, if no effective public health interventions were implemented.Second, classic SIR model assumes a constant infection rate, which is not likely to be true as interventions being implemented.Therefore, in this study, we constructed SIR models with multiple efficacy rates of public health interventions as proxy for the change of infection rate.To achieve higher efficacy of the public health interventions, efforts from individuals should not be neglected.All these are extremely important in raising awareness in the public as to personal preventive steps given the present situation (mild or subclinical symptoms observed in many cases and observed long incubation period).We believe that these volunteering activities can contribute to a successful delivery of public health principle and, in turn, efficacious interventions.To conclude, our estimates of the healthcare system burdens arising from the actual number of cases infected by the novel coronavirus appear to be considerable if no effective public health interventions were implemented.922443 doi: bioRxiv preprint -13 -public transport) and further effective large-scale interventions spanning all subgroups of populations (e.g., universal facemask wear) with an aim to obtain overall efficacy with at least 70%-90% to ensure the functioning of and avoid the breakdown of healthcare system. Highlight: Most important current measures to control the spread of the epidemic, such as quarantine of infected persons, reduction of human mobility, and improvement of treatment. =============================================================================================== **Question 2: Data on potential risks factors: Smoking, pre-existing pulmonary disease** Observation made from research papers published on 2020: 1572 f294f0df7468a8ac9e27776cc15fa20297a9f040 10.3390/v12020244 2020 Xu, Jiabao; Zhao, Shizhe; Teng, Tieshan; Abdalla, Abualgasim Elgaili; Zhu, Wan; Xie, Longxiang; Wang, Yunlong; Guo, Xiangqian Systematic Comparison of Two Animal-to-Human Transmitted Human Coronaviruses: SARS-CoV-2 and SARS-CoV TRUE The source of unexplained pneumonia was first discovered in Wuhan in Dec, 2019, and SARS-CoV-2, a new coronavirus, was isolated from the respiratory epithelium of patients.The following month, there were clusters of atypical pneumonia reported in other parts of mainland China, Hong Kong [21] , Canada [22] , and Singapore [23] .29th, 2019, the health departments of Hubei Province received a report that four employees of the South China Seafood Wholesale Market were diagnosed with unknown-caused pneumonia in a local hospital, which was the first report of SARS-CoV-2 [27] .The following month, there were clusters of atypical pneumonia reported in other parts of mainland China, Hong Kong [21] , Canada [22] , and Singapore [23] .29th, 2019, the health departments of Hubei Province received a report that four employees of the South China Seafood Wholesale Market were diagnosed with unknown-caused pneumonia in a local hospital, which was the first report of SARS-CoV-2 [27] .COVID-19 can be classified into light, normal, severe, and critical types based on the severity of the disease [31] : (1) Mild cases-the clinical symptoms were mild, and no pneumonia was found on the chest computed tomography (CT); (2) normal cases-fever, respiratory symptoms, and patients found to have imaging manifestations of pneumonia; (3) severe cases-one of the following three conditions: Respiratory distress, respiratory rate ≥ 30 times/min (in resting state, refers to oxygen saturation ≤ 93%), partial arterial oxygen pressure (PaO2)/oxygen absorption concentration (FiO2) ≤ 300 mmHg (1 mmHg = 0.However, severe cases have been documented in young adults who have unique factors, particularly those with chronic diseases, such as diabetes or hepatitis B. Those with a long-term use of hormones or immunosuppressants, and decreased immune function, are likely to get severely infected.COVID-19 can be classified into light, normal, severe, and critical types based on the severity of the disease [31] : (1) Mild cases-the clinical symptoms were mild, and no pneumonia was found on the chest computed tomography (CT); (2) normal cases-fever, respiratory symptoms, and patients found to have imaging manifestations of pneumonia; (3) severe cases-one of the following three conditions: Respiratory distress, respiratory rate ≥ 30 times / min (in resting state, refers to oxygen saturation ≤ 93%), partial arterial oxygen pressure (PaO2)/oxygen absorption concentration (FiO2) ≤ 300 mmHg (1 mmHg = 0.However, severe cases have been documented in young adults who have unique factors, particularly those with chronic diseases, such as diabetes or hepatitis B. Those with a long-term use of hormones or immunosuppressants, and decreased immune function, are likely to get severely infected.However, severe COVID-19 cases and deaths have mostly been in the middle-aged adults and the elderly with long smoking histories or other basic diseases, such as heart disease and hypertension [43, 44] .Viruses 2020, 12, x FOR PEER REVIEW 5 of 18 basic diseases, such as heart disease and hypertension [43, 44] .The early symptoms of SARS and COVID-19 are very similar to winter influenza, and the most important way to distinguish flu and pneumonia is to take throat swabs for viral testing [68] . Highlights: However, severe COVID-19 cases and deaths have mostly been in the middle-aged adults and the elderly with long smoking histories or other basic diseases, such as heart disease and hypertension ======================================================================================= **Question 3: Co-infections (determine whether co-existing respiratory/viral infections make the virus more transmissible or virulent) and other co-morbidities** 137 39a7144b3eb9ddf5b9076163aa61099d8b58f977 http://dx.doi.org/10.1093/ofid/ofz424 2019 Oct 3 ['Noyola, Daniel E', 'Hunsberger, Sally', 'Valdés Salgado, Raydel', 'Powers, John H', 'Galindo-Fraga, Arturo', 'Ortiz-Hernández, Ana A', 'Ramirez-Venegas, Alejandra', 'Moreno-Espinosa, Sarbelio', 'Llamosas-Gallardo, Beatriz', 'Guerrero, M Lourdes', 'Beigel, John H', 'Ruiz-Palacios, Guillermo', 'Perez-Patrigeon, Santiago', None] Open Forum Infectious Diseases Comparison of Rates of Hospitalization Between Single and Dual Virus Detection in a Mexican Cohort of Children and Adults With Influenza-Like Illness TRUE Current estimates indicate that lower respiratory tract infections (LRTIs) are the fifth leading cause of death in the world, accounting for 2.74 million deaths in 2015 [1] .The etiology of acute respiratory infections is diverse, and respiratory viruses are increasingly recognized as important causes of severe respiratory infections.Before the introduction of molecular detection methods, the etiology of a large proportion of acute respiratory infections could not be ascertained.The increasing use of reverse transcription polymerase chain reaction (RT-PCR) and other molecular methods has allowed for detection of respiratory viruses in a large proportion of cases.In addition, during the last 2 decades, previously unrecognized agents, such as human metapneumovirus (HMPV), human bocavirus (HBoV), rhinovirus C, and several coronaviruses, have been identified as new causes of respiratory infection [2] [3] [4] [5] .As such, the use of currently available diagnostic techniques allows detection of at least 1 pathogen in the majority of patients [6] .Additionally, the rates of hospitalization are significantly different based on the virus isolated.Mexican children 5 years of age and younger presenting with influenza-like illness (ILI) caused by human respiratory syncytial virus (RSV) and HMPV have been shown to be at greater risk of hospitalization compared with other viruses [7] .As a result of the increasing use of molecular detection of respiratory viruses and the frequent detection of some of these viruses in asymptomatic individuals, there is a need to clarify their role in the etiology of LRTI [8] .In addition, the availability of diagnostic platforms that allow for the simultaneous detection of many pathogens has resulted in the identification of ≥2 agents in a large number of patients with respiratory infections [9] [10] [11] .Before the use of molecular methods, detection of viral co-infections was relatively rare [12] .In contrast, most recent studies report detection of >1 virus in approximately one-fourth of patients (22.1%-22 .7%) [9] [10] [11] .This has created new opportunities for studying the contribution of each virus in the development, intensity, and duration of symptoms, as well as complications (eg, pneumonia) and death.To address this, many studies have sought to determine whether co-infection with ≥2 viruses contributes to the severity of an infection.Some of these studies have reported that the presence of >1 virus is associated with more severe infections, whereas others have not [9] [10] [11] [13] [14] [15] .In a systematic review and meta-analysis of studies carried out in children <5 years of age, no association between co-infection and increase in disease severity was found, but the need for further studies on this matter was identified [16] .Variability in the results of these studies might be a reflection of the populations included in each study, the definition of severity used, or the viruses that were compared.Of particular relevance is the definition of co-infection, as many studies compare those infected with a specific virus to infection with >1 virus [9] [10] [11] 15] .When assessing the effect of the presence of 2 viruses, various combinations may have differential effects on severity.In the present study, we investigated whether severity, defined as hospitalization with acute respiratory infections, increases with 2 viruses over that of each single virus.We analyzed data from a large prospective ILI cohort during 4 consecutive years in Mexico (ILI-002 study).This analysis is based on data from ILI-002, a hospital-based prospective observational cohort study of ILI [17] [18] [19] .The present analysis includes all participants enrolled in the ILI-002 study in whom 1 or 2 viruses were detected.The ILI-002 study was carried out at 6 public hospitals, 5 of them located in Mexico City and 1 in San Luis Potosí.Participating hospitals included 2 general hospitals (1 located in Mexico City and 1 in San Luis Potosí), 2 tertiary care pediatric hospitals, and 2 tertiary care hospitals (1 of them dedicated to the treatment of respiratory disorders, whereas the other provides medical care in a wide range of medical specialties).Adults and children seeking medical attention with ILI, defined as a respiratory symptom (eg, cough, dyspnea) plus a systemic symptom (eg, fever, malaise), were invited to participate in the ILI-002 protocol (ClinicalTrials.gov identifier: NCT01418287).For those enrolled, a follow-up telephone or face-to-face interview was performed at 14+/-3 days, and a visit happened 28+/-5 days after inclusion.The study protocol was approved by the ethics committee at all participating institutions, and all participants or guardians signed an informed consent or an assent form when pertinent.A nasopharyngeal swab for multiple PCR pathogen detection was obtained at enrollment.Samples were stored in transport media at 4°C at each site (for sites located in Mexico City) and sent daily to a central facility ( [HAdV] ) and 4 bacteria (Bordetella pertussis, Chlamydophila pneumoniae, Legionella pneumophila, and Mycoplasma pneumoniae).The 22-pathogen assay added CoV HKU1, HBoV, and influenza A (H1N1) pdm09 while removing influenza A H5N1.As reported by the manufacturer, the analytical limit of detection of the assay varies between 5 and 50 copies per reaction for most targets.Samples that were tested originally with the RespiFinder19 kit were subsequently tested for HBoV detection with the use of virus-specific primers.In addition, all samples were tested by real-time RT-PCR for influenza A following the Centers for Disease Control and Prevention (CDC) protocol [20] .Participants hospitalized during the 28 days of the study follow-up were considered to have severe disease.Hospitalization was defined as participants who were admitted to the hospital or remained in the emergency departments for at least 24 hours.Participants with a detected bacterial pathogen (Bordetella pertussis, Chlamydophila pneumoniae, Legionella pneumophila, and Mycoplasma pneumoniae), no virus, or >2 viruses were excluded.Comorbidities were defined as 1 of the following: chronic obstructive pulmonary disease, cardiovascular disease, diabetes mellitus, previous use of systemic steroids, obesity, overweight, and underweight.For this analysis, we grouped similar genera of viruses and examined the 8 most frequent groups of viruses isolated: influenza (A, A (H1N1)pdm09, and B grouped as influenza), HMPV, HPIV, RSV, RV, HAdV, CoV, and HBoV.Comparisons of baseline factors were made between hospitalization and nonhospitalization groups.All potential risk factors that were not categorical were grouped into categories.Chi-square statistics were used to make the univariate comparisons of the risk factors.Logistic regression models were used to compare hospitalization between all pairs of viruses and the combinations of the 2 viruses.Combinations with <10 participants were not analyzed.Each logistic regression model included sex, age (grouped into 3 categories), days since symptom onset (grouped into 3 categories), and comorbidity (yes/no).Odds ratios and 95% confidence intervals were calculated.From 2010 to 2014, 5662 participants were included in the ILI-002 study.From these, 96.89% had a 28-day interview, 1619 had no virus isolated, and 32 had no sample; additionally, 85 were excluded for other reasons such as bacterial infections (18 subjects), missing covariate information (11 subjects), and >2 viruses (56 subjects).The final data set had 3926 participants.Of the 3926 participants, 1856 (47.3%) were hospitalized, 1411 (35.9%) were <11 years old, and 308 (7.8%) were >60 years old; 1673 (42.6%) were males (Table 1) .Of the 1856 hospitalized cases, 65 died with 1 virus and 12 died with 2 viruses detected.Table 2 shows the distribution of participants with a single virus diagnosis across the covariates used in the logistic regression models.Influenza, HPIV, CoV, and RV were detected more frequently in participants 11-60 years old, whereas RSV, HMPV, HAdV, and HBoV were more common in children <11 years old.One virus was detected in 3285 and 2 viruses were detected in 641 participants (Table 3) .RV (n = 1433), influenza (n = 888), and CoV (n = 703) were the most frequently detected viruses (either alone or in co-infection).The most frequent combination was influenza+CoV (116 of 641 dual infections; 18%).Influenza was found in combination with other agents in 237 participants (37% of 641 dual infections).There were 52 subjects with a combination of >3 viruses.The numbers were too small to perform any statistical analyses but were analyzed descriptively.In the subjects with 3 viruses detected, 56% had RV, 54% had influenza, and 48% had CoV.The combination of 3 viruses that occurred the most often (in 6 subjects) was influenza, HMPV, and HBoV.The adjusted odds ratios (ORs) for hospitalization rates between each of the viruses included in the study are shown in Figure 1 .Participants infected with HBoV were more likely to be hospitalized than those infected with influenza, CoV, HPIV, and RV ( Figure 1A) .Those with HAdV, HMPV, and RSV had similar but not statistically significant results.Participants infected with RSV were more likely to be hospitalized compared with cases of influenza, CoV, RV, HPIV, HAdV, and, to a lesser extent (not statistically significant), HMPV ( Figure 1B ).Participants with HMPV were more likely to be hospitalized compared with cases of influenza, CoV, and RV, but were not statistically significantly different compared with HPIV and HAdV ( Figure 1C ).HPIV cases were more likely to be hospitalized compared with cases of influenza, CoV, and, to lesser extent, HAdV (as assessed by point estimates) ( Figure 1D ), although none of these comparisons were statistically significant.Participants with HAdV were more likely to be hospitalized compared with cases of influenza and CoV (as assessed by point estimates), although neither of these comparisons reached statistical significance ( Figure 1E ).Participants with RV were more likely to be hospitalized as compared with CoV cases ( Figure 1F ).Although not statistically significant, CoV cases were less likely to be hospitalized as compared with those with influenza ( Figure 1G ).Figure 1H shows comparisons with influenza cases (already described).When 2 viruses were isolated, those with combinations of RSV+HPIV, CoV+HMPV, and CoV+RSV were less likely to be hospitalized than participants infected with individual viruses (Figure 2A-C) .The point estimates for individual viruses in HMPV and RV demonstrated a higher likelihood of these patients being hospitalized than those with combinations, but all confidence intervals included 1 ( Figure 2D ).Participants with HBoV+RV were more likely to be hospitalized than those with RV, but were hospitalized as frequently as those with HBoV alone ( Figure 2J ).The point estimate for severity in CoV+RV was greater than that for individuals for CoV or RV alone ( Figure 2K ), but the confidence interval included 1.The confidence intervals for all other combinations included 1, but the point estimates indicated that some of the combinations could be more severe than 1 of the single agents ( Figure 2L -R).Reports of the impact of multiple viral infections have been more frequent with the availability of PCR assays that detect multiple pathogens [9] [10] [11] [12] [13] [14] [15] .Most reports analyze data by grouping all combinations and compare this group with different single viruses.This has resulted in variable interpretations.ILI-002 is a large study of those with ILI in which a multipathogen PCR assay was performed on samples from all participants.The size of this cohort allowed there to be a sufficient number of participants with various virus combinations to perform separate analyses for some virus combinations and examine whether virus combinations increase severity over individual viruses.Our results highlight the importance of carrying out these separate analyses.Our data demonstrate that the severity of diseases was higher with specific viruses (eg, HBoV, RSV, and HMPV).However, in no combination was the dual infection significantly worse than in both of the individual viruses.Furthermore, as a class effect, it does not appear that infection with >1 virus increases severity of disease.Many of the confidence intervals for the combinations include no difference for each of the comparisons with the individual viruses.Based solely on point estimates, which may change with increasing numbers of participants, there appear to be some patterns that indicate a leading or governing effect of 1 virus in the combination.An example of this is the combination of CoV+RSV ( Figure 2C , panel C); CoV cases and CoV+RSV cases were less severe than cases of only RSV.Moreover, the severity of CoV+RSV cases was no different than that of CoV cases.This could indicate that CoV is the leading/ governing agent in the combination.Similarly, HBoV may be governing RV in the combination of these 2 viruses (panel J), RV governing CoV (panel K), influenza governing HBoV (panel N), and RV governing RSV (panel P).The confidence intervals around the combinations are often large and include 1, so these interpretations are not conclusive and could differ if larger numbers of participants were studied.However, the pattern observed in the point estimates is suggestive of the governing effect described above and warrants further study.One explanation for the governing results is that virological testing was done only once in each participant, which does not distinguish sequential infections with 2 viruses from simultaneous infection with both viruses.It is possible that these results could reflect sequential infections rather than simultaneous infection and that symptoms (and hospitalization) could be the result of only 1 of the 2 viruses.The sample collection time with respect to the course of illness would then be an important factor.In a study carried out in participants with ILI, viral co-infections were detected more frequently in samples obtained during the first 2 days from symptom onset compared with those obtained after 3-7 days [21] .Thus, it is plausible that detection of co-infections might be the result of prolonged shedding of 1 virus with a subsequent infection with the second virus.Also, it could reflect a reduced ability of a second virus to replicate due to an already initiated host response and the production of interferon as a result of an initial viral infection.Interference of 1 virus with another virus has been shown to occur in vitro [22] , and epidemiological studies suggest that circulation of 1 virus might affect circulation of another virus in communities [23] .Overall, most previous studies have shown similar severity of single infections when compared with mixed infections [16] .However, some studies have shown significant differences between specific combinations of viruses and single viruses.For example, in cases of co-infection with RSV+RV and RSV+HBoV, the illness appeared to be more severe than in cases of RSV or HBoV infection alone [24] .Our results for RSV+RV follow the same pattern, but we did not have sufficient data to study the combination RSV+HBoV.Our analysis showed that participants with detection of only influenza virus were less likely to require hospitalization than participants in whom other viruses were detected.This was observed despite the fact that the study included the 2013-2014 winter season, when a severe wave of influenza A(H1N1) pdm09 was registered in Mexico [25] .This result could be derived from inclusion of all influenza subtypes in the analysis.The lower hospitalization rate in participants with influenza virus infection compared with those with other viruses may also be explained by influenza vaccination.Since 2009, influenza vaccination coverage in Mexico has been high [26] , and influenza vaccination has been reported to reduce influenza hospitalizations [27] .Unfortunately, it was not possible to obtain detailed data regarding influenza vaccination status for study participants to assess this.We also found that for single virus comparisons, HBoV, RSV, and HMPV were associated with severe infections.The RSV and HMPV finding is consistent with other reports that show these viruses to be leading causes of LRTI in children and adults [28] [29] [30] .In contrast, the role of HBoV as a cause of severe infections is less well established.Because HBoV infection is very common and frequently found in asymptomatic participants, the pathogenic role of this virus has been questioned [31, 32] .Children with higher viral loads tend to have more severe infections [33] , longer hospitalization duration [34, 35] , and are found to have co-infection by other viruses [36, 37] less frequently than those with lower viral loads.However, in children in daycare, viral load had no apparent association with severity of illness [31] , which also might reflect challenges in reproducibly measuring viral load in secretions.In all, these studies suggest that HBoV is frequently present in children as an asymptomatic or chronic infection with low viral loads, whereas some infections in which a high viral load is present may be associated with severe infections requiring hospitalization.Our study did not measure viral load, so it is unclear if our HBoV participants represented a sample of high-viral load participants, as we did not include asymptomatic participants with HBoV.The strengths of this study include the large numbers of participants and the consistent baseline testing and follow-up.Limitations include the definition of severity, as hospitalization was a surrogate for actual patient health status.Patients may be hospitalized for causes other than their respiratory illness, such as worsening of comorbidities, observation in high-risk participants, or social reasons.Further studies should be done using direct measures of patient health status such as intensity and duration of participants' symptoms or complications (eg, pneumonia).We have developed a symptom scale for influenza (FLU-PRO) as part of the study ILI-002 that could be used in future studies [38] .Adjusted odds ratio (on log scale) 20 The presence of chronic underlying conditions is an important factor that is associated with risk of developing severe respiratory infections.In addition, it is possible that chronic conditions may increase the risk of acquiring infections by multiple pathogens.A recent study reported that patients with coinfection caused by 2 or 3 different influenza virus strains were more likely to have underlying cardiovascular disorders than those with single influenza infections [39] ; however, no differences were observed in the prevalence of other underlying conditions.Although some other studies have found a higher prevalence of chronic disorders in patients with multiple viral pathogens, these appear to be limited to specific conditions, and no differences have been observed for other disorders [15, 40] .In addition, many studies have not found an association between the presence of chronic disorders and detection of multiple viruses [9, 14, 21, 24] .The main objective of our study was to determine if codetection of 2 viruses was associated with worse outcome, and we did not analyze which factors may have led to acquisition of ≥2 viruses; nevertheless, our analysis included the presence of chronic conditions as a covariate, in order to account for potential confounding.In a previous analysis of ILI-002 limited to children <5 years of age, the severity of single viruses was compared [7] , and the results were similar to our findings for single viruses in the present study.The majority of studies have focused on children, and the effect of mixed viral infections in adults is less clear [40] .One of the strengths of this analysis is that our study population included both pediatric and adult symptomatic participants.Although some comparisons resulted in estimates with wide confidence intervals, our interpretation based on point estimates revealed several patterns and could be hypothesis-generating for future studies.Our results suggest that, in general, having >1 virus detected by PCR on a respiratory sample does not increase the severity of disease from ILI.To assess the differences in hospitalization rates of mixed respiratory infections, it is necessary to carry out analyses between specific combinations of viruses.When 2 viruses are detected, it appears that the clinical severity of a respiratory infection, as defined by hospitalization, may be associated with 1 of these agents.Future studies with a larger number of participants designed to distinguish between sequential and simultaneous infection and using direct measures of patient health status should be of help in defining the role of each virus during the evolution of an acute respiratory episode. Highlight: In addition, many studies have not found