409 conflict when publishing many artifacts via maven-publish (gradle)

[glureau]

Select Topic Area

Bug

Body

* What went wrong:
For more on this, please refer to https://docs.gradle.org/8.8/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.
Execution failed for task ':aPrivateModule:publishIosSimulatorArm64PublicationToGitHubPackagesRepository'.
> Failed to publish publication 'iosSimulatorArm64' to repository 'GitHubPackages'
1098 actionable tasks: 759 executed, 339 from cache
   > Could not PUT 'https://maven.pkg.github.com/aPrivateCompany/aPrivateRepository/com/foo/bar/aPrivateModule-iossimulatorarm64/13.0.0-20250117090733/aPrivateModule-iossimulatorarm64-13.0.0-20250117090733.pom.md5'. Received status code 409 from server: Conflict

I've 2 main projects, each one containing more than 100 gradle modules. Using the standard maven-publish plugin, we were able to publish those packages for a while. Then a few months ago we notice that sometime there was this kind of error. There's clearly no conflict and restarting the build was "fixing" the problem. (Mostly snapshot builds which contains the date+time to avoid conflicts, ex: 13.0.0-20250117090733 from the error above.)

There days it's getting way worse, I've to restart the build multiple times, taking half a day or more of just checking CI and restarting this.

My hypothesis / uderstanding so far:

• maven-publish starts to publish
• github packages at some points times out (maybe a security feature to avoid DDOS?)
• maven-publish after a timeout attempt to upload again the same file
• then github packages returns the 409, because yes now there is a real duplicate

Thanks for your attention.

Similar topics

https://discuss.gradle.org/t/maven-publish-plugin-to-github-repo/39503/9

[github-actions[bot]]

💬 Your Product Feedback Has Been Submitted 🎉

Thank you for taking the time to share your insights with us! Your feedback is invaluable as we build a better GitHub experience for all our users.

Here's what you can expect moving forward ⏩

• Your input will be carefully reviewed and cataloged by members of our product teams. 
  • Due to the high volume of submissions, we may not always be able to provide individual responses.
  • Rest assured, your feedback will help chart our course for product improvements.
• Other users may engage with your post, sharing their own perspectives or experiences. 
• GitHub staff may reach out for further clarification or insight. 
  • We may 'Answer' your discussion if there is a current solution, workaround, or roadmap/changelog post related to the feedback.

Where to look to see what's shipping 👀

• Read the Changelog for real-time updates on the latest GitHub features, enhancements, and calls for feedback.
• Explore our Product Roadmap, which details upcoming major releases and initiatives.

What you can do in the meantime 💻

• Upvote and comment on other user feedback Discussions that resonate with you.
• Add more information at any point! Useful details include: use cases, relevant labels, desired outcomes, and any accompanying screenshots.

As a member of the GitHub community, your participation is essential. While we can't promise that every suggestion will be implemented, we want to emphasize that your feedback is instrumental in guiding our decisions and priorities.

Thank you once again for your contribution to making GitHub even better! We're grateful for your ongoing support and collaboration in shaping the future of our platform. ⭐

[jagerman777hx]

What specific configurations or changes have you already tried to mitigate this issue with the maven-publish plugin and GitHub Packages?

[glureau]

What configurations are you thinking about?

We are thinking about splitting the publishing to be able to delete and retry only the packages in errors, but it's not efficient at all (a poor workaround at best) and will involve a lot of work on our side...

[lppppu]

Have you tried implementing retry logic with exponential backoff in your build process to handle potential timeouts and conflicts, or would you like guidance on alternative approaches to mitigate this issue with the Maven Publish plugin and GitHub Packages?

[glureau]

I don't find a way to customize those maven-publish behaviors. Afaiu, a custom retry logic would require to change the maven-publish logic, as I'll need to retry with the same resolved snapshot name, which is basically cloning maven-publish and rewriting it.

A second option would be to operate on CI level. As I want 1 snapshot version for all packages, I need to have 1 successful upload of ALL packages. So basically it's a way to restart the entire publishing again and again. Yes it's saving some human time for sure, but right now I'm starting my 14th build, each one taking 20+ minutes and creating hundreds of half-deployments on GHP, that's far from great.

A third option is to not use github packages because upload is slow and buggy, but that's a company choice...

What I would love now is someone with gradle experience telling me "put that settings in this place and maven-publish will work better with GHP", or someone from GHP telling me "we have an identified issue for years and just found the fix".

[JMoserCricut]

@glureau We're also facing this same issue and i'm wondering if it might be related to the number of active versions that a package has. How many active versions do you have? on our failing repos it seems to be at 600+ active versions, which we honestly don't really need, but we shouldn't have to maintain a way of deleting old unneeded versions I wouldn't think.