# Hyperparameter Estimation with openEBGM

## Background

DuMouchel used an “Empirical Bayes” (EB) method. That is, the data are a driving force behind the choice of the prior distribution (in contrast to typical Bayesian methods, where the prior is chosen without regard to the actual data). One outcome of this is a known posterior distribution that relies on a set of hyperparameters derived from the prior distribution. These hyperparameters are estimated by their most likely value in the context of Empirical Bayesian methods. One process by which this can occur involves maximizing the likelihood function of the marginal distributions of the counts (or in our case, minimizing the negative log-likelihood function).

### Optimization

Global optimization is a broad field. There are many existing R packages with minimization routines which may be used in the estimation of these hyperparameters. The hyperparameter estimation functions offered by openEBGM utilize the following:

• Optimization using PORT routines
• Newton-type algorithm
• A quasi-Newton method (also known as a variable metric method)
• A version of the Expectation-Maximization (EM) Algorithm known as the Expectation/Conditional Maximization (ECM) Algorithm (1)

openEBGM’s hyperparameter estimation functions use local optimization algorithms. The Newton-like approaches use algorithms implemented in R’s stats package and allow the user to choose multiple starting points to improve the chances of finding a global optimum. The user is encouraged to explore a variety of optimization approaches because the accuracy of a global optimization result is extremely difficult to verify and other approaches might work better in some cases.
{"ft_lang_label":"__label__en","ft_lang_prob":0.6421161,"math_prob":0.93812275,"size":18374,"snap":"2021-31-2021-39","text_gpt3_token_len":6600,"char_repetition_ratio":0.13091998,"word_repetition_ratio":0.09264854,"special_character_ratio":0.42734298,"punctuation_ratio":0.2042394,"nsfw_num_words":0,"has_unicode_error":false,"math_prob_llama3":0.96866363,"pos_list":[0],"im_url_duplicate_count":[null],"WARC_HEADER":"{\"WARC-Type\":\"response\",\"WARC-Date\":\"2021-09-27T15:22:27Z\",\"WARC-Record-ID\":\"<urn:uuid:9cd355be-492c-4af0-a0d7-8c1e8b2fd150>\",\"Content-Length\":\"76491\",\"Content-Type\":\"application/http; msgtype=response\",\"WARC-Warcinfo-ID\":\"<urn:uuid:b5ec8991-cb72-4e15-b8de-988be72637ec>\",\"WARC-Concurrent-To\":\"<urn:uuid:b487a01f-ce87-4b59-97a8-9f4336bf8f14>\",\"WARC-IP-Address\":\"165.229.11.55\",\"WARC-Target-URI\":\"https://cran.yu.ac.kr/web/packages/openEBGM/vignettes/x3_hyperParameterVignette.html\",\"WARC-Payload-Digest\":\"sha1:HSKSRPCIX6UI3AUXFBGMCCVRLTARSP3P\",\"WARC-Block-Digest\":\"sha1:A6TOSN46LUAKE2BR2T7TBQXNWORFI5OX\",\"WARC-Identified-Payload-Type\":\"text/html\",\"warc_filename\":\"/cc_download/warc_2021/CC-MAIN-2021-39/CC-MAIN-2021-39_segments_1631780058456.86_warc_CC-MAIN-20210927151238-20210927181238-00043.warc.gz\"}"}