GitLab CI/CD: How do I generate a change log between pipeline runs?
Note: the answer is rather speculative, based on docs only, I didn't actually use GitLab pipelines yet.
You may be able to use the Pipelines API. More specifically you could get a pipeline's variables:

Get variables of a pipeline
GET /projects/:id/pipelines/:pipeline_id/variables

Attribute Type            Required    Description
id            integer/string  yes         The ID or URL-encoded path of the project owned by the authenticated user
pipeline_id   integer         yes         The ID of a pipeline

Example of response
[
  {
    &quot;key&quot;: &quot;RUN_NIGHTLY_BUILD&quot;,
    &quot;variable_type&quot;: &quot;env_var&quot;,
    &quot;value&quot;: &quot;true&quot;
  },
  {
    &quot;key&quot;: &quot;foo&quot;,
    &quot;value&quot;: &quot;bar&quot;
  }
]


I'm suspecting that in the response you'd find the variables shown in a pipeline creation response example, which include these which could be of interest to you:

  &quot;id&quot;: 61,
  &quot;sha&quot;: &quot;384c444e840a515b23f21915ee5766b87068a70d&quot;,

  &quot;before_sha&quot;: &quot;0000000000000000000000000000000000000000&quot;,


I'm uncertain if the before_sha for the current pipeline is what you're looking for, if not you'd just get the sha for the previous pipeline ID, which you could derive from the current pipeline ID (via the CI_PIPELINE_ID environment variable, see Predefined environment variables). I'm assuming that the pipeline IDs are consecutive.