Chapter 9. Viewing and exporting logs
Chapter 9. Viewing and exporting logs Activity logs are gathered for all repositories and namespace in Red Hat Quay. Viewing usage logs of Red Hat Quay. can provide valuable insights and benefits for both operational and security purposes. Usage logs might reveal the following information: Resource Planning : Usage logs can provide data on the number of image pulls, pushes, and overall traffic to your registry. User Activity : Logs can help you track user activity, showing which users are accessing and interacting with images in the registry. This can be useful for auditing, understanding user behavior, and managing access controls. Usage Patterns : By studying usage patterns, you can gain insights into which images are popular, which versions are frequently used, and which images are rarely accessed. This information can help prioritize image maintenance and cleanup efforts. Security Auditing : Usage logs enable you to track who is accessing images and when. This is crucial for security auditing, compliance, and investigating any unauthorized or suspicious activity. Image Lifecycle Management : Logs can reveal which images are being pulled, pushed, and deleted. This information is essential for managing image lifecycles, including deprecating old images and ensuring that only authorized images are used. Compliance and Regulatory Requirements : Many industries have compliance requirements that mandate tracking and auditing of access to sensitive resources. Usage logs can help you demonstrate compliance with such regulations. Identifying Abnormal Behavior : Unusual or abnormal patterns in usage logs can indicate potential security breaches or malicious activity. Monitoring these logs can help you detect and respond to security incidents more effectively. Trend Analysis : Over time, usage logs can provide trends and insights into how your registry is being used. This can help you make informed decisions about resource allocation, access controls, and image management strategies. There are multiple ways of accessing log files: Viewing logs through the web UI. Exporting logs so that they can be saved externally. Accessing log entries using the API. To access logs, you must have administrative privileges for the selected repository or namespace. Note A maximum of 100 log results are available at a time via the API. To gather more results that that, you must use the log exporter feature described in this chapter. 9.1. Viewing usage logs Logs can provide valuable information about the way that your registry is being used. Logs can be viewed by Organization, repository, or namespace on the v2 UI by using the following procedure. Procedure Log in to your Red Hat Quay registry. Navigate to an Organization, repository, or namespace for which you are an administrator of. Click Logs . Optional. Set the date range for viewing log entries by adding dates to the From and To boxes. Optional. Export the logs by clicking Export . You must enter an email address or a valid callback URL that starts with http:// or https:// . This process can take an hour depending on how many logs there are. 9.2. Viewing usage logs by using the API Logs can be viewed by Organization or repository by using the API. They can also be aggregated (grouped), or listed with more detailed. Logs can also be viewed by user, a specific date range, or by page. 9.2.1. Viewing aggregated logs Aggregated logs can be viewed by Organization, repository, a specific user, or the current user. You can also pass in optional commands like performer , starttime/endtime , and next_page to filter results. Prerequisites You have Created an OAuth access token . You have set BROWSER_API_CALLS_XHR_ONLY: false in your config.yaml file. Procedure Use the GET /api/v1/user/aggregatelogs API endpoint to return the aggregated (or grouped) logs for the current user: USD curl -X GET \ -H "Authorization: Bearer <bearer_token>" \ -H "Accept: application/json" \ "https://<quay-server.example.com>/api/v1/user/aggregatelogs" Example output {"aggregated": [{"kind": "create_tag", "count": 1, "datetime": "Tue, 18 Jun 2024 00:00:00 -0000"}, {"kind": "manifest_label_add", "count": 1, "datetime": "Tue, 18 Jun 2024 00:00:00 -0000"}, {"kind": "push_repo", "count": 2, "datetime": "Tue, 18 Jun 2024 00:00:00 -0000"}, {"kind": "revert_tag", "count": 1, "datetime": "Tue, 18 Jun 2024 00:00:00 -0000"}]} You can also pass in the performer and starttime/endtime queries to obtain aggregated logs for a specific user between a specific time period. For example: USD curl -X GET \ -H "Authorization: Bearer <bearer_token>" \ -H "Accept: application/json" \ "<quay-server.example.com>/api/v1/user/aggregatelogs?performer=<username>&starttime=<MM/DD/YYYY>&endtime=<MM/DD/YYYY>" Aggregated logs can also be viewed by Organization by using the GET /api/v1/organization/{orgname}/aggregatelogs . For example: USD curl -X GET \ -H "Authorization: Bearer <bearer_token>" \ -H "Accept: application/json" \ "<quay-server.example.com>/api/v1/organization/{orgname}/aggregatelogs" Aggregated logs can also be viewed by repository by using the GET /api/v1/repository/{repository}/aggregatelogs command. The following example includes the starttime/endtime fields: USD curl -X GET \ -H "Authorization: Bearer <bearer_token>" \ -H "Accept: application/json" \ "<quay-server.example.com>/api/v1/repository/<repository_name>/<namespace>/aggregatelogs?starttime=2024-01-01&endtime=2024-06-18"" 9.2.2. Viewing detailed logs Detailed logs can be viewed by Organization, repository, a specific user, or the current user. You can also pass in optional fields like performer , starttime/endtime , and next_page to filter results. Procedure Use the GET /api/v1/user/logs API endpoint to return a list of log entries for a user. For example: USD curl -X GET -H "Authorization: Bearer <bearer_token>" -H "Accept: application/json" "<quay-server.example.com>/api/v1/user/logs" You can also pass in the performer and startime/endtime queries to obtain logs for a specific user between a specific time period. For example: USD curl -X GET -H "Authorization: Bearer <bearer_token>" -H "Accept: application/json" "http://quay-server.example.com/api/v1/user/logs?performer=quayuser&starttime=01/01/2024&endtime=06/18/2024" Example output --- {"start_time": "Mon, 01 Jan 2024 00:00:00 -0000", "end_time": "Wed, 19 Jun 2024 00:00:00 -0000", "logs": [{"kind": "revert_tag", "metadata": {"username": "quayuser", "repo": "busybox", "tag": "test-two", "manifest_digest": "sha256:57583a1b9c0a7509d3417387b4f43acf80d08cdcf5266ac87987be3f8f919d5d"}, "ip": "192.168.1.131", "datetime": "Tue, 18 Jun 2024 18:59:13 -0000", "performer": {"kind": "user", "name": "quayuser", "is_robot": false, "avatar": {"name": "quayuser", "hash": "b28d563a6dc76b4431fc7b0524bbff6b810387dac86d9303874871839859c7cc", "color": "#17becf", "kind": "user"}}}, {"kind": "push_repo", "metadata": {"repo": "busybox", "namespace": "quayuser", "user-agent": "containers/5.30.1 (github.com/containers/image)", "tag": "test-two", "username": "quayuser", } --- Use the GET /api/v1/organization/{orgname}/logs endpoint to return logs for a specified organization: USD curl -X GET \ -H "Authorization: Bearer <bearer_token>" \ -H "Accept: application/json" \ "http://<quay-server.example.com>/api/v1/organization/{orgname}/logs" Use the GET /api/v1/repository/{repository}/logs endpoint to return logs for a specified repository: USD curl -X GET \ -H "Authorization: Bearer <bearer_token>" \ -H "Accept: application/json" \ "http://<quay-server.example.com>/api/v1/repository/{repository}/logs" 9.3. Exporting repository logs by using the UI You can obtain a larger number of log files and save them outside of the Red Hat Quay database by using the Export Logs feature. This feature has the following benefits and constraints: You can choose a range of dates for the logs you want to gather from a repository. You can request that the logs be sent to you by an email attachment or directed to a callback URL. To export logs, you must be an administrator of the repository or namespace. 30 days worth of logs are retained for all users. Export logs only gathers log data that was previously produced. It does not stream logging data. Your Red Hat Quay instance must be configured for external storage for this feature. Local storage does not work for exporting logs. When logs are gathered and made available to you, you should immediately copy that data if you want to save it. By default, the data expires after one hour. Use the following procedure to export logs. Procedure Select a repository for which you have administrator privileges. Click the Logs tab. Optional. If you want to specify specific dates, enter the range in the From and to boxes. Click the Export Logs button. An Export Usage Logs pop-up appears, as shown Enter an email address or callback URL to receive the exported log. For the callback URL, you can use a URL to a specified domain, for example, <webhook.site>. Select Confirm to start the process for gather the selected log entries. Depending on the amount of logging data being gathered, this can take anywhere from a few minutes to several hours to complete. When the log export is completed, the one of following two events happens: An email is received, alerting you to the available of your requested exported log entries. A successful status of your log export request from the webhook URL is returned. Additionally, a link to the exported data is made available for you to delete to download the logs. Note The URL points to a location in your Red Hat Quay external storage and is set to expire within one hour. Make sure that you copy the exported logs before the expiration time if you intend to keep your logs. 9.4. Exporting logs by using the API Detailed logs can be exported to a callback URL or to an email address. Prerequisites You have Created an OAuth access token . You have set BROWSER_API_CALLS_XHR_ONLY: false in your config.yaml file. Procedure Use the POST /api/v1/user/exportlogs endpoint to export logs for the current user: USD curl -X POST \ -H "Authorization: Bearer <bearer_token>" \ -H "Content-Type: application/json" \ -H "Accept: application/json" \ -d '{ "starttime": "<MM/DD/YYYY>", "endtime": "<MM/DD/YYYY>", "callback_email": "your.email@example.com" }' \ "http://<quay-server.example.com>/api/v1/user/exportlogs" Example output {"export_id": "6a0b9ea9-444c-4a19-9db8-113201c38cd4"} Use the POST /api/v1/organization/{orgname}/exportlogs endpoint to export logs for an Organization: USD curl -X POST \ -H "Authorization: Bearer <bearer_token>" \ -H "Content-Type: application/json" \ -H "Accept: application/json" \ -d '{ "starttime": "<MM/DD/YYYY>", "endtime": "<MM/DD/YYYY>", "callback_email": "org.logs@example.com" }' \ "http://<quay-server.example.com>/api/v1/organization/{orgname}/exportlogs" Use the POST /api/v1/repository/{repository}/exportlogs endpoint to export logs for a repository: USD curl -X POST \ -H "Authorization: Bearer <bearer_token>" \ -H "Content-Type: application/json" \ -H "Accept: application/json" \ -d '{ "starttime": "2024-01-01", "endtime": "2024-06-18", "callback_url": "http://your-callback-url.example.com" }' \ "http://<quay-server.example.com>/api/v1/repository/{repository}/exportlogs"
curl -X GET -H "Authorization: Bearer <bearer_token>" -H "Accept: application/json" "https://<quay-server.example.com>/api/v1/user/aggregatelogs" {"aggregated": [{"kind": "create_tag", "count": 1, "datetime": "Tue, 18 Jun 2024 00:00:00 -0000"}, {"kind": "manifest_label_add", "count": 1, "datetime": "Tue, 18 Jun 2024 00:00:00 -0000"}, {"kind": "push_repo", "count": 2, "datetime": "Tue, 18 Jun 2024 00:00:00 -0000"}, {"kind": "revert_tag", "count": 1, "datetime": "Tue, 18 Jun 2024 00:00:00 -0000"}]} curl -X GET -H "Authorization: Bearer <bearer_token>" -H "Accept: application/json" "<quay-server.example.com>/api/v1/user/aggregatelogs?performer=<username>&starttime=<MM/DD/YYYY>&endtime=<MM/DD/YYYY>" curl -X GET -H "Authorization: Bearer <bearer_token>" -H "Accept: application/json" "<quay-server.example.com>/api/v1/organization/{orgname}/aggregatelogs" curl -X GET -H "Authorization: Bearer <bearer_token>" -H "Accept: application/json" "<quay-server.example.com>/api/v1/repository/<repository_name>/<namespace>/aggregatelogs?starttime=2024-01-01&endtime=2024-06-18"" curl -X GET -H "Authorization: Bearer <bearer_token>" -H "Accept: application/json" "<quay-server.example.com>/api/v1/user/logs" curl -X GET -H "Authorization: Bearer <bearer_token>" -H "Accept: application/json" "http://quay-server.example.com/api/v1/user/logs?performer=quayuser&starttime=01/01/2024&endtime=06/18/2024" --- {"start_time": "Mon, 01 Jan 2024 00:00:00 -0000", "end_time": "Wed, 19 Jun 2024 00:00:00 -0000", "logs": [{"kind": "revert_tag", "metadata": {"username": "quayuser", "repo": "busybox", "tag": "test-two", "manifest_digest": "sha256:57583a1b9c0a7509d3417387b4f43acf80d08cdcf5266ac87987be3f8f919d5d"}, "ip": "192.168.1.131", "datetime": "Tue, 18 Jun 2024 18:59:13 -0000", "performer": {"kind": "user", "name": "quayuser", "is_robot": false, "avatar": {"name": "quayuser", "hash": "b28d563a6dc76b4431fc7b0524bbff6b810387dac86d9303874871839859c7cc", "color": "#17becf", "kind": "user"}}}, {"kind": "push_repo", "metadata": {"repo": "busybox", "namespace": "quayuser", "user-agent": "containers/5.30.1 (github.com/containers/image)", "tag": "test-two", "username": "quayuser", } --- curl -X GET -H "Authorization: Bearer <bearer_token>" -H "Accept: application/json" "http://<quay-server.example.com>/api/v1/organization/{orgname}/logs" curl -X GET -H "Authorization: Bearer <bearer_token>" -H "Accept: application/json" "http://<quay-server.example.com>/api/v1/repository/{repository}/logs" curl -X POST -H "Authorization: Bearer <bearer_token>" -H "Content-Type: application/json" -H "Accept: application/json" -d '{ "starttime": "<MM/DD/YYYY>", "endtime": "<MM/DD/YYYY>", "callback_email": "your.email@example.com" }' "http://<quay-server.example.com>/api/v1/user/exportlogs" {"export_id": "6a0b9ea9-444c-4a19-9db8-113201c38cd4"} curl -X POST -H "Authorization: Bearer <bearer_token>" -H "Content-Type: application/json" -H "Accept: application/json" -d '{ "starttime": "<MM/DD/YYYY>", "endtime": "<MM/DD/YYYY>", "callback_email": "org.logs@example.com" }' "http://<quay-server.example.com>/api/v1/organization/{orgname}/exportlogs" curl -X POST -H "Authorization: Bearer <bearer_token>" -H "Content-Type: application/json" -H "Accept: application/json" -d '{ "starttime": "2024-01-01", "endtime": "2024-06-18", "callback_url": "http://your-callback-url.example.com" }' "http://<quay-server.example.com>/api/v1/repository/{repository}/exportlogs"