Title: Limited Return of lines when HTTP-Requesting Bitbucket via R Package HTTR
Tags: r;http;httprequest;httpresponse
Question: Recently i've been using following approach to source code avaiable in an repository on a password secured Bitbucket:
https://stackoverflow.com/a/49480595/5068688
This has been working perfectly fine for me, just until now.
I will explain the problem in a short manner:
The code i am sourcing, is now longer then 500 lines. Unfortunately, the content of the HTTP-Response only provides me with the first 500 lines. Any lines after are not within the content.
The code i am using:
library(httr)
library(rjson)
url <- &quot;https://link.to/git/and/file.R&quot;
username <- &quot;my_git_username&quot;
token <- &quot;my_git_acess_token&quot;
get_response <- GET(url, authenticate(username, token))
get_content <- content(get_response, as = &quot;parse&quot;)
get_content$limit
Last line provides me with value: 500
When printing out get_content variable, it shows me the first 500 lines followed by:
$start
[1] 500
$isLastPage
[1] FALSE
$limit
[1] 500
$nextPageStart
[1] 500
I assume, the content is somehow limited to 500 lines. However, variable $isLastPage is FALSE which would imply, there is another Page following this one. So it could be that i just have to get to the next page? I couldt find anything about this either.
Thanks for any advice. Please tell me if you need further information.
Comment: That limit has nothing do to with the `httr` package itself. That limit is set by the API endpoint you you are interacting with. The [bitbucket API uses paging](https://docs.atlassian.com/bitbucket-server/rest/4.0.1/bitbucket-rest.html) to limit the size of the request. If you need more data, you will need to send another request for the next page. And keep doing so until isLastPage is TRUE. Since each API is different, `httr` can not automatically do this for you. You will need to build the requests youself. You'll need to add the value of `nextPageStart` to `?start=` in your URL.

Comment: Thanks a lot @MrFlick. I never thought it would be something coming from Bitbucket itself. Your comment enabled me to produce a viable solution.



{'source': 'StackExchangeDataset', 'subset': 'stackoverflow.com', 'metadata': {'AnswerCount': 0, 'CommentCount': 2, 'ContentLicense': 'CC BY-SA 4.0', 'CreationDate': '2022:00:08/09/22', 'FavoriteCount': None, 'PostTypeId': 1, 'Tags': '<r><http><httprequest><httpresponse>', 'Title': 'Limited Return of lines when HTTP-Requesting Bitbucket via R Package HTTR', 'ViewCount': 23, 'id': 73291069, 'user_AboutMe': '<p>I do python sometimes.</p>\n', 'user_AccountId': 6556014.0, 'user_DisplayName': 'Simon', 'user_Reputation': 3.0, 'user_Views': 28.0, 'user_id': 5068688.0}}