How Gitoqlok Bypasses the Git API rate limit

User success story based on how Gitoqlok overcomes a large API request traffic.

The Challenge of Git Server API Rate Limits

Every action that delivers the data from Git repository to a Qlik application triggers a request to the connected Git Server's API. However, as versioned items (applications in our case) grow in size, encompassing numerous sheets, large data files, and user-uploaded attachments, Git Servers enforce rate limits to prevent abuse.

The problem with the limit is caused by the fact that for each individual object in the Qlik application a separate file is created in the repository (for convenience and to avoid conflicts). When receiving data from a repository, the result is the formula 1 file = 1 request to the repository, so for large applications rate limit creates a bottleneck that hampers the smooth versioning workflow.

Each Git Server Gitoqlok integrates with poses variable rate limits. Below you can find a summary table for limits as stated in the documentation:

Git Server Rate Limits:

Git Server	Rate Limit (Requests)	Limit Type	Reference
GitLab	300 per minute	Raw endpoint traffic (for a given project, commit, and file path)	https://docs.gitlab.com/ee/user/gitlab_com/index.html#gitlabcom-specific-rate-limits
	2,000 per minute	Authenticated API traffic (for a given user)	https://docs.gitlab.com/ee/user/gitlab_com/index.html#gitlabcom-specific-rate-limits
	Unlimited; Manual control limits	Self-hosted system: GitLab Enterprise	https://docs.gitlab.com/ee/administration/settings/files_api_rate_limits.html
GitHub	5,000 per hour	Primary rate limit for aythenticated users	https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api?apiVersion=2022-11-28
	15,000 per hour	GitHub Enterprise	https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api?apiVersion=2022-11-28
BitBucket	5,000 per hour	Access to raw files	https://support.atlassian.com/bitbucket-cloud/docs/api-request-limits/
	1,000 per hour	Access to repository data	https://support.atlassian.com/bitbucket-cloud/docs/api-request-limits/

Solution

Holistic File Retrieval

Rather than making individual requests for each file, Gitoqlok now downloads a comprehensive repository archive containing all necessary files and intelligently unzips them to extract the required version of a Qlik app. This innovative strategy minimizes the number of API requests, alleviating the risk of encountering rate limit errors from the Git Server.

Users can now confidently commit and update their applications without the fear of disruptions caused by rate limit restrictions. Its holistic file retrieval approach ensures that versioning large Qlik applications remains a seamless and efficient process, promoting a frustration-free experience for users across various Git Servers.