After upgrading to Foreman 3.14 / Katello 4.16 I've had issues syncing ansible collections and jobs failed with the following error:
Error message: the server returns an error
HTTP status code: 502
Response headers: {"date"=>"Sun, 20 Apr 2025 14:30:42 GMT", "server"=>"Apache", "content-length"=>"341", "content-type"=>"text/html; charset=iso-8859-1"}
Response body: <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>502 Proxy Error</title>
</head><body>
<h1>Proxy Error</h1>
<p>The proxy server received an invalid
response from an upstream server.<br />
The proxy server could not handle the request<p>Reason: <strong>Error reading from remote server</strong></p></p>
</body></html>
Also, there's a Foreman Discourse Thread about this: PulpAnsibleClient::ApiError when syncing ansible collections.Keeping this short, these are the steps that solved the issue for me:
1. Set the 'Orphaned Content Protection Time' Value to a lower setting than 1440 minutes to be able to trigger a cleanup early:
[root@katello01 ~]# hammer settings set --name 'orphan_protection_time' --value '5'
2. Remove the Repository where all the ansible content is stored:
[root@katello01 ~]# hammer repository delete --organization-title 'archyslife' --product 'ansible-content' --name 'collections'
3. Remove the Product where all the ansible content was stored:
[root@katello01 ~]# hammer product delete --organization-title 'archyslife' --name 'ansible-content'
4. Wait a little longer than specified in the previous command where the 'Orphaned Content Protection Time' was set:
[root@katello01 ~]# sleep 360 && foreman-rake katello:delete_orphaned_content RAILS_ENV=production
5. Wait for the task to complete, I'd suggest using the WebUI for this. Alternatively, you can monitor '/var/log/foreman/production.log':
[root@katello01 ~]# tail -F /var/log/foreman/production.log
6. Stop all foreman services:
[root@katello01 ~]# foreman-maintain service stop
7. Run the installer to disable the ansible pulp plugin:
[root@katello01 ~]# foreman-installer --scenario katello --foreman-proxy-content-enable-ansible false
8. Check all 'foreman_proxy_content' settings in '/etc/foreman-installer/scenarios.d/katello-answers.yaml' to make sure they are applied correctly.
9. Afterwards, check the health of your installation:
[root@katello01 ~]# foreman-maintain service status
[root@katello01 ~]# foreman-maintain health check
10. Stop all foreman services again:
[root@katello01 ~]# foreman-maintain service stop
11. Rerun the installer to enable the ansible pulp plugin:
[root@katello01 ~]# foreman-installer --scenario katello --foreman-proxy-content-enable-ansible true
12. Double-Check '/etc/foreman-installer/scenarios.d/katello-answers.yaml' like mentioned above.
13. Recreate the product:
[root@katello01 ~]# hammer product create --organization-title 'archyslife' --name 'ansible-content'
14. Recreate the repository:
[root@katello01 ~]# hammer repository create --organization-title '15knetworks' --content-type 'ansible_collection' --name 'collections' --url 'https://galaxy.ansible.com/' --mirroring-policy 'mirror_content_only' --ansible-collection-requirements-file '/root/requirements.yml'
15. Sync the repository:
[root@katello01 ~]# hammer repository synchronize --organization-title '15knetworks.com' --product 'ansible-content' --name 'collections'
After completion of all these steps, I was able to sync ansible collections again.
Feel free to comment and / or suggest a topic.
Comments
Post a Comment