title: Web locked CSV to dataframe in r
body: <p>I have a file on a private web server I am trying to access.  I must first go to a site and login with my credentials and then I can type a URL (there is no link) to access the file, which immediately downloads a csv file to the computer.  I am trying to get that csv file to automatically load into R either direct from online or have it automatically download and uploaded from my hard drive</p>
  
  <p>I am going to be refreshing this data 10-15 times a day which is why I need it automatic rather than manually downloading it every time.  </p>
  
  <p>I have tried a with several packages and have been impressed with Hadley's package rvest which has shown much easier than some things I have used in the past.  I am succeeding in downloading the data:</p>
  
  <pre><code>library(rvest)
  fs &lt;- html_session("somewebsite.org")
  fs.login &lt;- fs %&gt;% follow_link("Sign In")
  login.form &lt;- html_form(fs.login)[[1]]
  login.form &lt;-set_values(login.form, userName = "myusername", password =      "mypassword")
  active.session &lt;- submit_form(fs.login, login.form)
  my.data &lt;- jump_to(active.session, "somewebsite.org/report/groups")
  </code></pre>
  
  <p>I have ran it with a timer several times and it takes an average of 27 seconds which indicates it is downloading the file (roughly the same amount that it takes Google Chrome). The result is a variable class session with 7 elements 43.7 Mb</p>
  
  <pre><code>my.data
  </code></pre>
  
  <blockquote>
    <p> somewebsite/report/groups</p>
    
    <p>Status: 200</p>
    
    <p>Type:   text/csv</p>
    
    <p>Size:   45856046</p>
  </blockquote>
  
  <p>My question is how can I access the actual csv file or data in r?</p>
  
  <pre><code>str(my.data)
  
  List of 7  
   $ handle  :List of 2  
    ..$ handle:Class 'curl_handle' &lt;externalptr&gt;   
    ..$ url   : chr "somewebsite.org"  
    ..- attr(*, "class")= chr "handle"  
   $ config  :List of 7  
    ..$ method    : NULL  
    ..$ url       : NULL  
    ..$ headers   : NULL  
    ..$ fields    : NULL  
    ..$ options   :List of 1  
    .. ..$ autoreferer: int 1  
    ..$ auth_token: NULL  
    ..$ output    : NULL  
    ..- attr(*, "class")= chr "request"  
   $ url     : chr "https://somewebsite.org/report/groups"  
   $ back    : chr "https://somewebsite.org/report/groups"  
   $ forward : chr(0)   
   $ response:List of 10  
    ..$ url        : chr "https://somewebsite.org/report/groups"  
    ..$ status_code: int 200  
    ..$ headers    :List of 6  
    .. ..$ content-disposition: chr "attachment; filename=\"groups-2016-0318-063749.csv\""  
    .. ..$ content-type       : chr "text/csv"  
    .. ..$ date               : chr "Fri, 18 Mar 2016 18:37:49 GMT"  
    .. ..$ server             : chr "Apache-Coyote/1.1"  
    .. ..$ transfer-encoding  : chr "chunked"  
    .. ..$ connection         : chr "Close"  
    .. ..- attr(*, "class")= chr [1:2] "insensitive" "list"  
    ..$ all_headers:List of 1  
    .. ..$ :List of 3  
    .. .. ..$ status : int 200  
    .. .. ..$ version: chr "HTTP/1.1"  
    .. .. ..$ headers:List of 6  
    .. .. .. ..$ content-disposition: chr "attachment; filename=\"groups-2016-0318-063749.csv\""  
    .. .. .. ..$ content-type       : chr "text/csv"  
    .. .. .. ..$ date               : chr "Fri, 18 Mar 2016 18:37:49 GMT"  
    .. .. .. ..$ server             : chr "Apache-Coyote/1.1"  
    .. .. .. ..$ transfer-encoding  : chr "chunked"  
    .. .. .. ..$ connection         : chr "Close"  
    .. .. .. ..- attr(*, "class")= chr [1:2] "insensitive" "list"  
    ..$ cookies    :'data.frame': 6 obs. of  7 variables:  
    .. ..$ domain    : chr [1:6] "somewebsite.org" "#HttpOnly_.site.org" "signin.site.org" ".site.org" ...  
    .. ..$ flag      : logi [1:6] FALSE TRUE FALSE TRUE FALSE TRUE  
    .. ..$ path      : chr [1:6] "/" "/" "/" "/" ...  
    .. ..$ secure    : logi [1:6] FALSE TRUE FALSE FALSE TRUE TRUE  
    .. ..$ expiration: POSIXct[1:6], format: "2017-03-18 12:37:16" NA NA NA ...  
    .. ..$ name      : chr [1:6] "fs_experiments" "ObSSOCookie" "TS01289383" "TS01b89640" ...  
    .. ..$ value     : chr [1:6] "u%3D-anon-%2Ca%3Dshared-ui%2Cs%3Dac76fc702b255a493a5856b5432b92b4%2Cv%3D0100110011010000000111111111001110101101000000000001100"| __truncated__ "15yUK2dU%2B78GK7o587gtwh3i%2ByORXGD8ne5XJBiGkiHpDAJ3%2F7rQ4Gql6T5DqQIwCg%2FSwSioAMIzzaRxGEFKsCkc%2BGohi1fdWhbR0urah6%2BJikm9lA6"| __truncated__ "01999b7023d69473f53740d0f7f2969d9d79e1a18c7e259f6baf643ce642a330fc0a3604d7" "01999b7023960237ab42ec3f429e5a452fe3559d683a090b19a65cf66ce0c01bc21bdb29bf78f030d36d4eeff4dec21ff185c54b06" ...  
    ..$ content    : raw [1:45857717] 69 64 2c 6e ...  
    ..$ date       : POSIXct[1:1], format: "2016-03-18 18:37:49"  
    ..$ times      : Named num [1:6] 0 0 0.062 0.156 27.425 ...  
    .. ..- attr(*, "names")= chr [1:6] "redirect" "namelookup" "connect" "pretransfer" ...  
    ..$ request    :List of 7  
    .. ..$ method    : chr "GET"  
    .. ..$ url       : chr "https://somewebsite.org/report/groups"  
    .. ..$ headers   : Named chr "application/json, text/xml, application/xml, */*"  
    .. .. ..- attr(*, "names")= chr "Accept"  
    .. ..$ fields    : NULL  
    .. ..$ options   :List of 4  
    .. .. ..$ useragent    : chr "libcurl/7.43.0 r-curl/0.9.6 httr/1.0.0"  
    .. .. ..$ cainfo       : chr "C:/Users/Thisuser/Documents/R/win-library/3.2/httr/cacert.pem"  
    .. .. ..$ autoreferer  : int 1  
    .. .. ..$ customrequest: chr "GET"  
    .. ..$ auth_token: NULL  
    .. ..$ output    : list()  
    .. .. ..- attr(*, "class")= chr [1:2] "write_memory" "write_function"  
    .. ..- attr(*, "class")= chr "request"  
    ..$ handle     :Class 'curl_handle' &lt;externalptr&gt;   
    ..- attr(*, "class")= chr "response"  
   $ html    :&lt;environment: 0x000000001aad2f60&gt;   
   - attr(*, "class")= chr "session"  
  </code></pre>