# Stochastic/Markov Game §

Stochastic Games generalize MDPs to multiple interacting decision-makers.

Mentioned in F1TENTH Research Proposal.

Formalization

A Markov Game is a tuple

• where is a finite set of states for player
• where is a finite set of actions for player
• is a state transition probability matrix
• is a reward function, where
• is a discount factor,

Value Function

Nash Equilibrium A Nash Equilibrium of the Markov game is a joint policy , such that for any and

• where represents the indices of all agents in N except agent .

Resources
{"ft_lang_label":"__label__en","ft_lang_prob":0.87471294,"math_prob":0.79187304,"size":627,"snap":"2023-40-2023-50","text_gpt3_token_len":152,"char_repetition_ratio":0.11717496,"word_repetition_ratio":0.06185567,"special_character_ratio":0.215311,"punctuation_ratio":0.09821428,"nsfw_num_words":0,"has_unicode_error":false,"math_prob_llama3":0.9957842,"pos_list":[0],"im_url_duplicate_count":[null],"WARC_HEADER":"{\"WARC-Type\":\"response\",\"WARC-Date\":\"2023-12-07T11:15:53Z\",\"WARC-Record-ID\":\"<urn:uuid:e112ed09-65c9-44a2-888a-eeff45196ed3>\",\"Content-Length\":\"50344\",\"Content-Type\":\"application/http; msgtype=response\",\"WARC-Warcinfo-ID\":\"<urn:uuid:895192b9-4f74-4462-9870-ee9f78e5bc1e>\",\"WARC-Concurrent-To\":\"<urn:uuid:3cc740f7-93ea-4f63-90cd-7ac0c6798ebd>\",\"WARC-IP-Address\":\"185.199.109.153\",\"WARC-Target-URI\":\"https://stevengong.co/notes/Stochastic-Game\",\"WARC-Payload-Digest\":\"sha1:3YCK3J7DVBBBGM5VS2WAV3RAZ7LVDR6Y\",\"WARC-Block-Digest\":\"sha1:QX3B26P6HDHOOFD7LSYEPR4GQOYMNBGS\",\"WARC-Identified-Payload-Type\":\"text/html\",\"warc_filename\":\"/cc_download/warc_2023/CC-MAIN-2023-50/CC-MAIN-2023-50_segments_1700679100651.34_warc_CC-MAIN-20231207090036-20231207120036-00167.warc.gz\"}"}