Ansible provisioning without internet access - ansible

I know that you can setup proxy in Ansible to provision behind corporate network:
https://docs.ansible.com/ansible/latest/user_guide/playbooks_environment.html
like this:
environment:
http_proxy: http://proxy.example.com:8080
Unfortunately in my case there is no access to internet from the server at all. Downloading roles locally and putting them under /roles folder seems solve the role issue, but roles still download packages from the internet when using:
package:
name: package-name
state: present
I guess there is no way to make dry/pre run so Ansible downloads all the packages, then push that into repo and run Ansible provision using locally downloaded packages?

This isn't really a question about Ansible, as all Ansible is doing is running the relevant package management system on the target host (i.e. yum, dnf or apt or whatever). So it is a question of what solution the specific package management tool provides, for this case.
There are a variety of solutions and for example in the Centos/RHEL world you can:
Create a basic mirror
Install a complete enterprise management system
There is another class of tool generally called an artefact repository. These started out life as tools to store binaries built from code, but have added a bunch of features to act as a proxy and cache packages from a wide variety of sources (OS Packages, PIP, NodeJS, Docker, etc). Two examples that have limited free offerings:
Nexus
Artifactory
They of course still need to collect those packages from a source, so at some point those are going to have to be downloaded to placed within these systems.

Like clockworknet pointed out this is more related to the RHEL package handling. Setting up local mirror somewhere inside the closed network can provide a solution in this situation. More info on "How to create a local mirror of the latest update for Red Hat Enterprise Linux 5, 6, 7 without using Satellite server?": https://access.redhat.com/solutions/23016

My solution:
Install Sonatype Nexus 3
create one or more yum proxy repositories
https://help.sonatype.com/repomanager3/formats/yum-repositories
use Ansible to add these proxies via yum_repository
https://docs.ansible.com/ansible/latest/modules/yum_repository_module.html
yum_repository:
name: proxy-repo
description: internal proxy repo
baseurl: https://your-nexus.server/url-to-repo```
note: did that for APT and works fine, would expect the same for yum

Related

Using Artifactory to proxy Cypress with CYPRESS_DOWNLOAD_MIRROR

Desired behavior
I would like to use CYPRESS_DOWNLOAD_MIRROR pointing toward my artifactory configuration for Cypress and just be able to do npm install and download library AND binary of Cypress
Current behavior
When setting Cypress in artifactory, and downloading it with CYPRESS_DOWNLOAD_MIRROR set toward this tool, the download script find binary files as X.Y.Z and not cypress.zip and fail. Apparently I can't rename binaries in artifactory. It seems to have made it available in a name X.Y.Z instead of cypress.zip
My artifactory admin tell me to do this command before but I can't since it's a post action of Cypress
curl –v « https://artifactory.mycompany.fr/artifactory/remote-download.cypress-generic/desktop/6.8.0?platform=win32&arch=x64 » > cypress.zip
Workaround
For now, I'm using CYPRESS_INSTALL_BINARY to point to a manually uploaded binary in artifactory but it's a pain because I have to separate Linux binary (for CI) and Windows binary (for dev) and if my package is configured with "cypress": "^6.2.1" the npm library will go to 6.2.1 and my binary will stuck to 6.2.0 for example...
Debug logs
Installing Cypress (version: 6.8.0)
× Downloading Cypress
→ Cypress Version: 6.8.0
Unzipping Cypress
Finishing Installation
The Cypress App could not be downloaded.
Does your workplace require a proxy to be used to access the Internet? If so, you must configure the HTTP_PROXY environment variable before downloading
Cypress. Read more: https://on.cypress.io/proxy-configuration
Otherwise, please check network connectivity and try again:
URL: https://artifactory.mycompany.fr/artifactory/remote-download.cypress.io/desktop/6.8.0?platform=win32&arch=x64
Error: self signed certificate in certificate chain
Download method
npm
Operating System
Linux
Windows
Other
I'm behind a proxy
I don't really know if it's an artifactory or a Cypress matter but I need help ^^
In addition to accepted answer, it is possible to replace pre-defined 'Query Params' with enabling 'Propagate Query Params'. If set, the query params passed with the request to Artifactory, will be passed on to the remote repo.
Please note, according to JFrog docs, this setting is only available for Generic type repositories.
I was able to make it work on Windows using the following:
I created a generic remote repository, making sure it is pointing to https://download.cypress.io, and under the advanced tab, added the query params: platform=win32&arch=x64 (notice there is a dedicated field for it).
The above is required in order to cache the correct binary based on the OS and arch (you might require a different remote repository with different query params).
I found it on Cypress doc that these query params control the binary type which will be downloaded (so we need to make sure it fits the client os and arch).
In the .npmrc I simply provided the following:
CYPRESS_DOWNLOAD_MIRROR=https://user:myverystrongpassword#myartifactory/artifactory/generic-cypress-windows
I've used this command (on MacOS) to directly pass path to downloaded Cypress.zip file
CYPRESS_INSTALL_BINARY=~/Downloads/cypress.zip yarn add cypress --D

Artifactory bundle install '/versions' file not found

We are running a local Artifactory Pro installation and have rubygems.org configured as a Remote Repository.
When running bundle install in a CI Job, the local Artifactory Instance does return a 404 File not found when querying for the /versions file. When doing a manual lookup in the Remote (&Cache) Repo, the file >is present<.
The path we pass to bundle install is provided by the Set me up Wizard and looks like this:
https://$rtf-instance.com/artifactory/api/gems/gems-remote/versions
This was mentioned in an issue here: https://www.jfrog.com/jira/browse/RTFACT-16005
and should have been fixed, but it is still not working in our installation.
Can't find any mention of RTFACT-16005 or the related RTFACT-19012 in the Release Notes.
Repo is setup with default values, no additional config done.
Are we missing something?
Environment:
debian 10 (buster)
nginx 1.14.2
artifactory-pro 7.15.4 / 7.15.3
To enable the gems compact index support you need to add the following system property (under $JFROG_HOME/var/etc/artifactory/artifactory.system.properties):
artifactory.gems.compact.index.enabled=true
You will need to restart Artifactory afterward.
This can be found in JFrog Wiki, here: https://www.jfrog.com/confluence/display/JFROG/RubyGems+Repositories#RubyGemsRepositories-RetrievingLatestRubyGemsPackageCompatiblewithYourRubyVersions

No ansible roles were found in Satellite 6.4 - system roles installed

I have an RHN Satellite 6.4 Server and have installed rhel-system-roles as per documentation.
yum install rhel-system-roles
But when I click inside the GUI on Configure->Ansible->Roles I get the error:
no ansible roles were found in satellite.
I have also copied some roles from the system-roles folder to /etc/ansible/roles/ and also made a test_role folder there but still cannot import or see them inside the GUI.
I have restarted the server. Can this be why I do not have a host that is connected ok without errors inside hosts?
Thanks in advance.

How to install additional software on a Netkit virtual machine?

I'm trying to use Netkit to test some of my C applications. In order to do so, I need to have gcc installed on my virtual machines. So I'm trying to install it following the instructions in the wiki. The second thing to do would be this:
Once vm has started, configure a name server inside its resolv.conf file
Here's what I find inside /etc/resolv.conf of the virtual machine:
#domain local.domain.nam
#nameserver w.x.y.z
#search suffix.for.unqualified.names
What should I write there? How to configure a name server? I tried to copy the resolv.conf of my host but it doesn't work.
If I try to run apt-get update here's the output I get:
You can look at /etc/resolv.conf on the host and add the nameserver lines found there to the guest file. Or you can use third-party recursive name servers. Here are some publicly accessible servers:
Google
Verisign
Note that Debian's APT configuration apparently contains unstable/sid repositories (based on the …/unstable/… part in the URLs). If the VM image was created a long time ago, this will make updates and installing additional software very difficult because the unstable/sid has evolved considerably since then, and upgrades for historic unstable to current versions does not always work.

where can I find source rpm for amzn repo?

I need to build a RPM package on my Centos6 EC2 instance, so I think it'll be best to use the "official" specs from amzn. Usually I did that with yumdownloader --source xxx but on the EC2 instance it cannot find any.
I checked /etc/yum.repo.d, which seems not to have any repo regarding src.
You can use the get_reference_source python script as described by Shadow Lau, but that needs the package being installed. And you need to run it on EC2 on an Amazon Linux AWS instance.
The script gets the URL to download from alami-source-request.amazonaws.com. Here is how you can use it:
https://alami-source-request.amazonaws.com/cgi-bin/source_request.cgi?instance_id=i&region=eu-west-1&version=2011-08-0&srpm_name=stunnel-4.29-3.6.amzn1.src.rpm
Unfortunately you need to know the exact package name. The version is as it's in the get_reference_source script. And it seems there is no validation done on instance_id.
The above URL will return another URL with an access key, where you can download the SRPM for a limited time. After that you have to generate another URL with the above source_request.cgi.
Look for Accessing Source Packages for Reference in
http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/AmazonLinuxAMIBasics.htm

Resources