Based on the AWS CodeArtifact FAQ https://aws.amazon.com/codeartifact/faq/ I do not see Ruby gems in the list of features. Is it possible to use AWS CodeArtifact instead of Gemfury?
I don't think it's possible, although I wish it were. I see only a handful of repo types when getting connection instructions:
npm
mvn
gradle
pip
twine
nuget
Related
I'm trying to deploy a Flask webapp to Heroku and I have seen conflicting information as to which files I need to include in the git repository.
My webapp is built within a virtual environment (venv), so I have a Pipfile and a Pipfile.lock. Do I also need a requirements.txt? Will one supersede the other?
Another related question I have is what would occur if a certain package failed to install in the virtual environment: can I manually add it to the requirements.txt or Pipfile? Would this effectively do the same thing as pipenv install ... or is that doing something else beyond adding the package to the list of requirements (considering Heroku is installing the packages upon deployment).
You do not need requirements.txt.
The Pipfile and Pipfile.lock that Pipenv uses are designed to replace requirements.txt. If you include all three files, Heroku will ignore the requirements.txt and just use Pipenv.
If you have build issues with a particular library locally I urge you to dig into that and get everything working on your local machine. But this isn't technically required... as long as the Pipfile and Pipfile.lock contain the right information (including hashes), Heroku will try to install your dependencies.
I have tried searching on various internet forums about the offline installation of Logstash plugins, but haven't found the solution that we can implement.
What I want to achieve is:
Install the following two logstash plugins:
logstash-filter-mutate-master
logstash-filter-aggregate-master
My Limitations Are:
At my company, there is strictly no Internet connectivity on Linux servers. Therefore, I need a solution where I could download some off-the-shelf package on my Windows machine, and then unpack it on the Linux machine, which would install the plugins I require.
I have looked at the Logstash documentation, which advises on how to install plugins offline, but that also needs a 'Staging' server which has logstash source-code and has Internet connectivity - the latter is non-existent in my organisation.
I would greatly appreciate if you could advise how to approach this. At present, when I try to install the 'mutate' plugin, here's the error I see:
$ bin/logstash-plugin install --local --no-verify
Installing...
Plugin not found, aborting
ERROR: Installation Aborted, message: Could not find gem 'ci_reporter_rspec (>= 0) java' in any of the gem sources listed in your Gemfile or installed on this machine.
Thank you.
If you want to install a logstash plugin in offline.
First way ;
You must change gemfile in logstash directory. As an example, download plugin source in somewhere of your logstash directory then modify gemfile in logstash directory with plugin name and path.
gem "logstash-input-example-plugin", :path => "./plugins/logstash-input-example-plugin"
After that just run the below command
bin/logstash-plugin install --no-verify
You can follow this link :
https://www.tutorialspoint.com/logstash/logstash_plugins.htm
Second way; using with gem file of the plugin. As an example;
bin/logstash-plugin install /path/to/logstash-output-kafka-1.0.0.gem
You can follow this link :
https://www.elastic.co/guide/en/logstash/current/working-with-plugins.html
Has anyone installed logstash-input-jmx (or any other "community" plugin for Logstash) in RHEL?
How did you manage the Ruby/gem dependencies?
After installing a recent version of Ruby, RubyGems, and Rails, the development/runtime gem dependencies have to be resolved, but attempting to install any of the gems mentioned in here, would lead to endless missing dependencies ...
Are there any packaged versions of Logstash JMX plugin around?
Thanks,
Babak.
Welcome to the house of fun. What I did was, which may be out of your scope but it might help....
$logstash_location/bin/logstash-plugin list
(as you might expect this shows what can be installed to your instance)
$logstash_location/bin/logstash-plugin install logstash-input-jmx
The plugin itself should resolve its own dependencies. I was having all sort of problems in Docker, but I had to build my Logstash tarball external to the machine. Make sure connectivity is ok too. It's a service, not local to your machine.
This post clearly describes the steps required to install a "community" logstash plugin, and can be used to install jmx plugin on RHEL. I've tried it on RHEL 7.3.
The main thing to consider is that in RHEL 7.3 the logstash can be found under /usr/share/logstash/ rather than /opt/logstash
Using the serverless framework v1.0.0, I have a 'requirements.txt' in my service root with the contents being the list of dependant python packages. (e.g. requests).
However my resulting deployed function fails as it seems these dependencies are not installed as part of the packaging
'Unable to import module 'handler': No module named requests'
I assume it is serverless that does the pip install, but my resulting zip file is small and clearly its not doing it, either by design or my fault as I am missing something? Is it because its Lambda that does this? If so what am I missing?)
Is there documentation on what is required to do this and how it works? Is it serverless that pip installs these or on aws lambda side?
You need to install serverless-python-requirements and docker
$ npm install serverless-python-requirements
Then add the following to your serverless.yml
plugins:
- serverless-python-requirements
custom:
pythonRequirements:
dockerizePip: non-linux
Make sure you have your python virtual environment active in CLI:
$ source venv/bin/activate
Install any dependencies with pip - note that in CLI you can tell if venv is active by the venv to the left of the terminal text
(venv) $ pip install <NAME>
(venv) $ pip freeze > requirements.txt
Make sure you have opened docker then deploy serverless as normal
$ serverless deploy
What will happen is that serverless-python-requirements will build you python packages in docker using a lambda environment, and then zip them up ready to be uploaded with the rest of your code.
Full guide here
Now you can use serverless-python-requirements. It works both for pure Python and libraries needing native compilation (using Docker):
A Serverless v1.x plugin to automatically bundle dependencies from requirements.txt and make them available in your PYTHONPATH.
Requires Serverless >= v1.12
The Serverless Framework doesn't handle the pip install. See https://stackoverflow.com/a/39791686/1111215 for the solution
I have a maven built docker image that was dependent on libssl1.0.2_1.0.2d-3_amd64.deb, but this has now a 404 and has been replaced by libssl1.0.2_1.0.2e-1_amd64.deb.
This is a problem because maven builds are meant to be durable - ie you can rebuild them at any point in the future. The main maven repo is durable, so artefacts taken from that will be there in the future. I could move the debs I need into the maven repo, but that is a bit of abuse of other peoples storage...
So is there a durable store of debian files that is guaranteed to exist... well at least until the revolution/meteor strike/Jurassic resurrection etc.
You can do this yourself with free, open-source tools. You can create your own APT repository for storing Debian packages. If you are interested in using GPG signature to sign repository metadata read this.
Once you've created the repository, you can create a configuration file in /etc/apt/sources.list.d/ pointing to your repository. You can run apt-get update to refresh your systems apt cache, and then run apt-get install to install the package of your choice.
BTW, you can install a particular version of a package by running: apt-get install packagename=version.
For example, to install version 1.0 of "test", run: apt-get install test=1.0.
Alternatively, if you don't want to deal with any of this yourself you can just use packagecloud.io for hosting Debian, RPM, RubyGem, and Python PyPI repositories. A lot of people who use our service have your exact use case, where Debian packages they depend on disappear from public repositories so they upload them to us.