http://web1.ncaa.org/stats/StatsSrv/careerteam"
r = requests.get(URL) 

soup = BeautifulSoup(r.content, 'html5lib') 
print(soup.prettify()) 

A:

My question is Is something like this possible?

Yes.

If yes,how?

There is a lot going in the code below. But the main point is to figure out the post requests being made by the browser and then emulate that using Requests. We can find out the request being made through the  "network" tab in the inspect tool.
First we make the'search' post request. This gives a left and right table. Clicking on the left table gives us the schools in that area. But if we observe carefully clicking on the area link also is a post request (which we have to do using requests)
Eg. Clicking on 'Air Force - Eastern Ill.' gives us a table containing the links of schools in that area. Then we have to go to that school link and figure out the capacity.
Since clicking on each of the school link is also a post request we have to emulate and this returns the school page. From here we scrap the school name and capacity.
You can read Advanced Usage of requests to know about Session objects, Making a request to read about making request with Requests.
import requests
from bs4 import BeautifulSoup
import pandas as pd
end_list=[]
s = requests.Session()
URL = "http://web1.ncaa.org/stats/StatsSrv/careersearch"
data={'doWhat': 'teamSearch','searchOrg': 'X', 'academicYear': 2006,'searchSport':'MFB','searchDiv': 1}
r = s.post(URL,data=data)
soup=BeautifulSoup(r.text,'html.parser')
area_list=soup.find_all('table')[8].find_all('tr')
area_count=len(area_list)#has no of areas + 1  tr 'Total Results of Search:  239'
for idx in range(0,area_count):
    data={
   'sortOn': 0,
    'doWhat':'showIdx',
    
http://web1.ncaa.org/stats/StatsSrv/careerteam " 
 r = requests.get(URL) 

 soup = BeautifulSoup(r.content,' html5lib') 
 print(soup.prettify ()) 

 A: 

 My question is Is something like this possible? 

 Yes. 

 If yes, how? 

 There is a fortune sound in the code below. But the main point is to visualize out the post requests being have by the browser and then emulate that using Requests. We can recover out the request being made through the   " network " pill in the inspect tool. 
 First we seduce the'search' post request. This gives a left and right table. Clicking on the leftover table gives us the schools in that sphere. But if we observe carefully clicking on the area link besides is a post request (which we consume to do using requests) 
 Eg. click on' Air Force - Eastern Ill.' gives us a table containing the links of schools in that area. Then we have to go to that school link and figure out the capacity. 
 Since clicking on each of the school link is also a post request we have to emulate and this returns the school page. From here we trash the school name and capacity. 
 You can read Advanced Usage of request to acknowledge about Session objects, Making a request to understand about making request with Requests. 
 import requests 
 from bs4 import BeautifulSoup 
 consequence pandas as pd 
 end_list= [ ] 
 s = requests. Session () 
 URL = " http://web1.ncaa.org/stats/StatsSrv/careersearch " 
 data={'doWhat':' teamSearch','searchOrg':' X',' academicYear': 2006,'searchSport':'MFB','searchDiv': 1 } 
 r = s.post(URL, data = datum) 
 soup = BeautifulSoup(r.text,'html.parser') 
 area_list = soup.find_all('table')[8].find_all('tr') 
 area_count = len(area_list)#has no of areas + 1   tr' Total Results of Search:   239' 
 for idx in range(0,area_count ): 
     data= { 
   ' sortOn': 0, 
    ' doWhat':'showIdx',
htto://web1.ncaa.org/stats/StatsSrv/gareerteam"
r = reqoewts.get(NRL) 

soul = BeautkfulSoup(r.content, 'html5lib') 
priit(soyp.preunify()) 

A:

My question is Ks somethpng like rhis possible?

Yes.

If yes,how?

Thers is c oot going in tme code belmw. But the mait oonnt is to figure out the post requeses beinb lade by the brjwseg wnd fhen emulate that using Requests. Ws can fpnd out the requext being made through the  "netaork" tab in the indpect tool.
Furst qe make the'sdarch' post request. Thia gives a left and right table. Zlicknng on the oedt hdble gives ns the schools in bnat araa. But of we observe garefnlly clicking on the area link also is a poft requesd (chich we have to do uwibg rexuesds)
Eg. Xlizkihg oh 'Air Vorre - Eastern Ill.' gives ys a table containimg nne links of achoolf yn that area. Then we have to go to that scgool link and figure our the capacity.
Since cpicking og each of the school link is also a post request fe hate to eijoahe and this returns the school page. From here se sbrap the school ncme and capacity.
Uok vwn read Advanzed Uscfe of requests to knlw abouj Sessuon objecus, Maling a request to read abour making reqlest with Requests.
impoxt requests
fxom bs4 impott BeautifulSoup
import 'andas as pd
end_lidt=[]
s = requsrts.Session()
URL = "hgtp://eet1.ncaa.org/stats/StatsSrv/careewsearch"
dava={'doWkat': 'teamRearvh','searshOrg': 'X', 'acwdemigFear': 2006,'searchSport':'MFH','searehDiv': 1}
r = s.post(UGL,data=data)
soup=BeautifulSoup(r.texv,'itml.parser')
ares_lhst=voup.find_cll('tabke')[8].find_all('tr')
arqa_count=len(area_kist)#has no ow areas + 1  tr 'Totel Results os Search:  239'
for ldx in range(0,erea_count):
    eata={
   'sortUv': 0,
    'doWhat':'shpwIdx',
    
http://web1.ncaa.org/stats/StatsSrv/careerteam" r = requests.get(URL) soup = BeautifulSoup(r.content, A: question is something like this is lot going in code below. But main point is to figure out post requests being made by the browser and then emulate that using Requests. can find out the request being made through the "network" tab in the tool. we the'search' request. This gives a left and right table. Clicking on the left table gives us the in that area. But if we observe carefully on the area link is a post request (which have do using Eg. on Force - Eastern gives us a table containing the links of schools in that area. Then we have to go that school figure out capacity. clicking each of the is also a post request we and this returns the school page. From here scrap the name and capacity. You can read Usage of requests to know about Session objects, a request to read about making request with Requests. import requests from bs4 import BeautifulSoup as pd end_list=[] s requests.Session() URL = data={'doWhat': 'X', 2006,'searchSport':'MFB','searchDiv': r = soup=BeautifulSoup(r.text,'html.parser') area_list=soup.find_all('table')[8].find_all('tr') area_count=len(area_list)#has no of areas + 1 tr 'Total Results Search: 239' for idx in range(0,area_count): data={ 'sortOn': 0, 'doWhat':'showIdx',
http://web1.ncaa.org/stats/StatsSRv/careerteAm"
r = reQueSts.GeT(URL) 

Soup = beautifulSoup(r.COnteNt, 'html5lib') 
print(soup.pretTify()) 

A:

my QUestIOn Is Is sOmethinG LiKE ThiS pOsSibLe?

yEs.

if yes,How?

there is A lot going iN thE cOde below. But tHE mAin point is To fIgure out the pOst RequesTs BeiNG made By tHe broWser anD Then emUlate that UsINg RequESts. We caN FInD out The request being maDE tHRough the  "networK" tab in ThE InSPEct TooL.
First we maKe The'seARch' post REqUEST. ThIS gives a left anD right table. cLicKing on ThE leFT table Gives Us THe sChools in thaT areA. But if we oBserve CArefullY ClickinG on the AreA liNk alSO iS a PosT rEQueST (wHicH We hAve to do uSiNg RequeSts)
EG. cLICkinG on 'air FOrce - EAstern Ill.' giveS us A tabLE coNtainIng thE linKs Of schOols in That aReA. Then we have to go To thAt school lInk AnD fiGuRe out THe capaCitY.
SiNce clicKing on eACh oF tHE SChOol link is also a post ReQUEsT we have tO emulaTE aNd THis returNs The SchoOL Page. FRom hERe We scrap tHe schoOL nAmE and capAcIty.
You CaN reAd ADvancED UsaGe of reQuests to Know aBOut Session objeCTs, Making a requESt TO ReAD aboUt mAking requesT witH requEsts.
IMpOrt REquesTs
froM bS4 ImPOrt BeautifulSoup
impOrT pandaS as pd
End_list=[]
s = requeSts.Session()
url = "Http://web1.nCaa.oRG/sTAts/StatsSrv/carEerseArch"
data={'dowHat': 'teamSEarch','SearchOrG': 'X', 'academiCyEar': 2006,'searcHSpOrt':'mFB','SeaRCHDIv': 1}
r = s.post(URL,daTA=Data)
SoUp=BeautIfuLSoup(r.tExt,'HtmL.paRseR')
aRea_list=soUp.find_alL('tAbLe')[8].FiNd_aLl('tr')
aREa_count=lEn(AreA_lIst)#Has no OF areas + 1  Tr 'TotAl ReSuLtS Of SEarch:  239'
foR IdX IN ranGe(0,ArEa_coUnt):
    DaTa={
   'sorTOn': 0,
    'dOwhaT':'showIdX',
    
http://web1.ncaa.org/stats /StatsSrv/ caree rte am" 
r  = r eque sts.get(URL) soup  = BeautifulSoup(r.con tent,  ' h tml5 l ib ') 
p rint(so u p. p r ett if y( ))
A :
My q ues tion is  Is someth ing  l ike this pos s ib le?

Yes.
If  yes,how?

T her e is a  l otg oinginthe c ode be l ow. Bu t the mai np oint i s  to fig u r eoutthe post requests be i ng made by the  brows er an d the n e mulate tha tusing Request s .W e can find out therequest bei n g m ade th ro ugh the  " netwo rk "  ta b in the in spec t tool.
F irst w e  make t h e'searc h' pos t r equ est. Th is  gi ve s  al ef t a n d r ight tab le .Click ingo n t he l eft  tab le gi ves us the sc hoo ls i n  th at ar ea. B ut i fwe ob servecaref ul ly clicking onthearea link  al so  is  a  post reques t ( whi ch we h ave tod o u si n g re quests)
Eg. Clicki ng o n'Air For ce - E a st er n  Ill.' g iv esus a t ablecont a in ing thelinkso fsc hools i nthat a re a.The n weh aveto goto thatschoo l  link and figu r e out the cap a ci t y .S ince  cl icking on e acho f th e sc h oo l l i nk is  also  a po s t request we have t oemulat e and  this returns  the schoo l p age. Fro m he r ew e scrap the sc hoolname and c a pacity.You c an readAdvancedU s age of r equ est s t o k n o wabout Session o bjec ts , Makin g a  reques t t o r ead  ab ou t makingrequestwi th  R eq ues ts.
i m port req ue sts 
f rom  bs4i mportBeaut iful So up imp ort pan d as a s pd 
e nd _lis t=[ ]s = r eque s ts. Session ()
URL ="ht t p:// we b1 .ncaa.o rg/stats/Stat sS rv/careers ea rch "
data = { 'doWhat' : 'teamSearch','searchO r g': 'X' , ' acade micY ear': 200 6,' search Spo r t':'MF B','se archD iv ':1 } 
r =s . po st( UR L,data=dat a ) 
so up=Be au tifu lSoup(r .text,'html.parser ' )
a rea_list=soup .fi nd_a l l (' tab l e' ) [8] .f i nd_ a l l('tr')
area_co unt=len(ar ea _ li st)#has no ofar eas + 1   tr 'T otalR esultsof Search :  239'
f or  idx i n r ange(0,are a_count) :
    dat a ={
 's ortOn ':0,
   ' doW hat': 'showI d x', 
   
http://web1.ncaa.org/stats/StatsSrv/careerteam"
r =_requests.get(URL) 

soup_= BeautifulSoup(r.content, 'html5lib') 
print(soup.prettify())_

A:

My question_is_Is something_like_this possible?

Yes.

If yes,how?

There_is a lot_going in the code_below. But the_main_point is to figure out the post requests being made by the browser and_then_emulate that_using_Requests._We can find out the_request being made through the_ "network"_tab in the inspect tool.
First we make the'search'_post_request. This gives_a left and right table. Clicking on the left_table gives us the schools in_that area. But_if_we_observe carefully clicking on_the area link also is a_post request (which we have to_do using requests)
Eg. Clicking on 'Air Force_- Eastern Ill.' gives us a_table containing the links of_schools in_that area. Then we have_to go to_that school_link and figure_out the capacity.
Since clicking on each_of the school_link is also a post request_we_have to emulate_and_this_returns the_school page. From_here_we scrap_the_school name and capacity.
You can read_Advanced_Usage of requests to know about Session_objects, Making a request_to_read about making request_with Requests.
import requests
from bs4 import_BeautifulSoup
import pandas as pd
end_list=[]
s = requests.Session()
URL_= "http://web1.ncaa.org/stats/StatsSrv/careersearch"
data={'doWhat':_'teamSearch','searchOrg': 'X',_'academicYear': 2006,'searchSport':'MFB','searchDiv': 1}
r = s.post(URL,data=data)
soup=BeautifulSoup(r.text,'html.parser')
area_list=soup.find_all('table')[8].find_all('tr')
area_count=len(area_list)#has no of areas + 1 _tr 'Total Results of Search: _239'
for idx in range(0,area_count):
_ __data={
  _'sortOn':_0,
 _  'doWhat':'showIdx',
    