If I run composer install from my host, I hit my local composer cache:
- Installing deft/iso3166-utility (1.0.0)
Loading from cache
Yet when building a container having in its Dockerfile:
RUN composer install -n -o --no-dev
I download all the things, e.g.:
- Installing deft/iso3166-utility (1.0.0)
Downloading: 100%
It's expected, yet I like to avoid it. As even on a rebuilt, it would also download everything again.
I would like to have a universal cache for composer that I could also reshare for other docker projects.
I looked into this and found the approach to define a volume in the Dockerfile:
ENV COMPOSER_HOME=/var/composer
VOLUME /var/composer
I added that to my Dockerfile, and expected to only download the files once, and hit the cache afterwards.
Yet when I modify my composer, e.g. remove the -o flag, and rerun docker build ., I expected to hit the cache on build, yet I still download the vendors again.
How are volumes supposed to work to have a data cache inside a docker container?
Use the experimental feature : Docker buildkit (Supported Since docker 18.09, docker-compose 1.25.4)
In your dockerfile
# syntax=docker/dockerfile:experimental
FROM ....
# ......
RUN --mount=type=cache,target=/var/composer composer install -n -o --no-dev
Now before building, make sure the env var is exported:
export DOCKER_BUILDKIT=1
docker build ....
If you are using docker-compose, make sure to export also COMPOSE_DOCKER_CLI_BUILD :
export COMPOSE_DOCKER_CLI_BUILD=1 DOCKER_BUILDKIT=1
docker-compose build ...
If it does not work with docker-compose, make sure your docker-compose version is above 1.25.4
docker-compose version
I found two ways of dealing with this problem, yet none deal with composer volumes anymore.
Fasten composer download process: Use hirak/prestissimo
composer global require "hirak/prestissimo:^0.3"
💡 With Composer 2.0, the above step is no longer required for faster downloads. In fact, it won't install on Composer 2.0 environments.
Force docker to use a cached composer install. Docker uses a cache on a RUN if the added files didn't change. If you only do COPY . /your-php-app, docker build will refresh all the cashes and re-run composer install even if only one unrelated file in the source tree changed. In order to make docker build to run composer install only install on package changes, one has to add composer.json and composer.lock file before adding the source files. Since one also needs the source files anyway, one has to use different folders for composer install and rsync the content back to the then added folder; furthermore one then has to run the post-install scripts manually. It should look something like this (untested):
WORKDIR /tmp/
COPY composer.json composer.lock ./
RUN composer install -n -o --no-dev --no-scripts
WORKDIR /your-php-app/
COPY . /your-php-app/
RUN rsync -ah /tmp/* /your/php-app/
RUN composer run-script post-install-cmd
or combine the two =)
I would like to have a universal cache for composer that I could also reshare for other docker projects.
Using a shared volume for the Composer cache works great when working with containers. If you want to go broader than just containers, and use a shared cache for e.g. local development as well, I've developed a solution for that called Velocita - how it works.
Basically, you use one global Composer plugin for local projects and inside and build containers. This not only speeds up downloads tremendously, it also helps with 3rd party outage for example.
I would consider utilizing the $HOME/.composer/cache/files directory. This is where composer reads/write to when using composer install.
If you are able to mount it from your host to your container that would work. Also you could just tar it up after each time your run composer install and then drop that in before you run composer install the next time.
This is loosely how Travis CI recommends doing this.
Also, consider using the --prefer-dist flag with your composer install command.
Info on that can be found here: https://getcomposer.org/doc/03-cli.md#install
--prefer-dist: Reverse of --prefer-source, composer will install from dist if possible. This can speed up installs substantially on build servers and other use cases where you typically do not run updates of the vendors. It is also a way to circumvent problems with git if you do not have a proper setup.
Some references on utilizing the composer cache for you:
https://blog.wyrihaximus.net/2015/07/composer-cache-on-travis/
https://github.com/travis-ci/travis-ci/issues/4579
Related
I need to run composer on my ddev project and don't have it on my Windows machine. For example, the project requires a composer install before startup. How can I use composer in this environment, especially on Windows?
Updated 2018-11-15 to show native ddev support (ddev composer command)
There are several ways to run composer for your project.
ddev v1.4.0 now has the ddev composer and ddev composer create commands. These run composer inside the container, so you're guaranteed to get composer behavior that matches the in-container hosting environment. (This matters most for Windows users.)
ddev composer require swiftmailer/swiftmailer
ddev composer update
ddev composer install
ddev composer create drupal-composer/drupal-project:8.x-dev --stability dev
Note that ddev composer create is not exactly the same as composer create-project so you don't have to understand complexities of the underlying filesystem. There are drupal and TYPO3 ddev composer create examples in the docs.
Nothing here prevents you from using any composer technique that you're comfortable with, but this is a great way to get predictable on-linux in-container composer builds. It should be hugely important for people using Windows OS, where composer is less available and has some unpredictable behavior.
Install on the host the old fashioned way: If composer is installed on your computer/host, just composer install. However, that only works on macOS and Linux, and only works if you have the right versions of php related components. It does not work well at all on Windows (NTFS) because the symlinks composer creates are not compatible with usage inside the (Linux) web container. (Composer is not hard to install on Windows: Use chocolatey and choco install -y composer. You'll want to enable the gd and curl extensions in c:\tools\php72\php.ini)
All the normal composer behavior has always been installed inside your web container, so you can use that whether or not you have composer on your host computer. For example: ddev exec composer install -d /var/www/html will do a composer install in the root of your repository, exactly the same as ddev composer install. You can also do ddev ssh and operate on the command line in the container.
Try this hooks approach to running composer install inside the container (on the mounted partition) every time your project starts:
hooks:
post-start:
- exec: composer install -d /var/www/html
For some older ideas on composer patterns (mostly obsoleted by ddev composer, See
How to: Use "composer create-project" and DDEV to start a new Drupal 8 site when Composer isn't installed on the host machine and
How to: Set up a D8/Composer site on Pantheon without CircleCI, custom upstreams
To expand on the accepted answer, DDEV now has a composer-specific hook.
hooks:
post-start:
- composer: install -d /var/www/html
The reason for using this instead of exec, I assume, is that there are also pre-composer and post-composer hooks, so maybe this also executes those hooks. I'm not sure of that or the actual difference, though.
I've installed Composer on a shared-host (hostgator) following this guide:
https://laravel.io/forum/02-13-2014-how-to-install-laravel-on-a-hostgator-shared-server
It was installed using this command:
curl -sS https://getcomposer.org/installer | php
When I try to run composer I get the message "/home2/georger/composer.phar: no such file or directory" although the file does exist, I can open it. Other questions that deal with this suggest moving it to /usr/local/bin but that's not an option for me.
As outlined in the installation instructions, you will need to pass it to the php interpreter:
$ php composer.phar
Alternatively, you might want to reconsider whether you really want to execute composer in a production environment or rather execute it during the deployment and then deploy the artifacts. In other words, you probably want to run composer install on the machine from which you want to deploy to the production system, and then synchronize files from there to the production system, for example, using rsync.
For reference, see https://getcomposer.org/doc/00-intro.md.
I have a new server, where multiple clients will host their webapps at. From Wordpress, to laravel, to simple html shizzle.
As you may know, Laravel requires Composer to be installed. This can be done locally, but also globaly. I am wondering (if there are any) about the pros and cons.
Of course, you can run the global installation from anywhere. But can this be a issue for other development projects on the server, or are there security for the global installation?
The disadvantage of using a globally installed composer is this
you're likely using different versions along the development pipeline
you may end up with different results
Just as an example, in a project, we had composer.phar checked in and updated regularly, but we ran into problems when the version we used was already updated to be able to use the ^ operator, however, a different binary was used during deployment, unaware of that operator, and the deployment failed.
The safest bet is to use the same version of composer.phar along the development pipeline. Alternatively, as mentioned before, keeping the globally installed composer regularly updated.
Since we usually use Makefiles in our projects, here's an example of what it looks like:
.PHONY: composer cs it test
it: cs test
composer:
composer self-update
composer validate
composer install
cs: composer
vendor/bin/php-cs-fixer fix --diff --verbose
test:
vendor/bin/phpunit --configuration=test/Unit/phpunit.xml
vendor/bin/phpunit --configuration=test/Integration/phpunit.xml
I would always suggest install globally, it will be easier for you to manage, and you could easily keep it up to date.
In the other projects they will not need to clutter their project with
composer.phar file
Quick explanation: I have a staging and a production server, both with the same deployment script (the only difference is the repo branch the clone). The deployment script runs bower install which is globally installed on both servers.
To install it globally I changed npm config set prefix to /home/forge/.npm-packages and afterwards ran npm install -g bower (note sudo wasn't needed, that's the point of changing the prefix). Once again, this was done in both servers.
When I ssh into each server, and run bower -v, which bower it is clear the command DO exist, and it IS added to PATH env. It is the same output for both servers.
Manually running bower install on the project root works for both server.
The issue is the forge deployment script, which only fails on production (IKR? I don't know what was I expecting).
The actual output is:
/home/forge/.forge/provision-2394191.sh: line 8: bower: command not found
The interesting part is, in my attempt to debug, I manually ran provision-2394191.sh and it worked.
What is wrong with my production server?
Seems like adding the new /home/forge/.npm-packages to the $PATH using export was not enough. To solve this I had to manually add it to the /etc/environment file.
I have Ubuntu 14.04 + Sublime text 3 and installed phpcs packages
additionalty I have installed phpcs and php-cs-fixer on my system
From this blog
I have found that phpmd (PHP Mess Detector) is also a required library, so installed phpmd as per given instructions on official php md page using alternative method From the github repository everything was finished.
:~/phpmd$ curl -s http://getcomposer.org/installer | php
#!/usr/bin/env php
All settings correct for using Composer
Downloading...
Composer successfully installed to: /home/keshav/phpmd/composer.phar
Use it: php composer.phar
but now when I write on terminal
phpmd /opt/lampp/htdocs/myproject, myfile.php
phpmd: command not found
There is phpmd folder on Home directory and everything without any error.
I have local project on core PHP create composer.json in project folder as per suggested on github .
Please tell me what means by
Then install Composer in your project (or download the composer.phar directly):
I think the problem is you've installed phpmd in a local directory, but you're trying to use it as if it was installed globally.
Installation instruction on the referenced sites can't really be made any clearer. Since you've already installed phpcs and php-cs-fixer, and those work for you, just follow similar instructions for phpmd. These are all PHP projects and are installed in a similar way.
Anyway, to use phpmd as a global command you have several options.
Github
Clone the github repository just like you did and add the phpmd bin directory to your PATH variable.
Global composer installation
Use the composer global command to install phpmd globally. You will also need to make sure that composer's bin directory is in the PATH. By default it's ~/.composer/vendor/bin.
composer global require phpmd/phpmd
This command will install phpmd globally, and as soon as ~/.composer/vendor/bin is in your PATH you'll be able to call it by simply invoking phpmd.
It's very well explained in composer's documentation: https://getcomposer.org/doc/03-cli.md#global
Download the phar archive
This is the simplest thing you can do. Simply go the phpmd releases, choose the latest and download the phar archive.
Put the phar file to whatever place you'd like. Just remember that it needs to be in your PATH. You can also rename it, to skip the .phar extension.
For example:
wget http://static.phpmd.org/php/2.1.3/phpmd.phar
sudo mv phpmd.phar /usr/bin/phpmd
sudo chmod +x /usr/bin/phpmd
Docker container
First, fetch the docker image with static analysis tools for PHP:
docker pull jakzal/phpqa
One of the tools provided by the image is phpmd. The command below will run phpmd in a docker container and mount the current working directory as a /project.
docker run -it --rm -v $(pwd):/project -w /project jakzal/phpqa \
phpmd src text cleancode,codesize,controversial,design,naming,unusedcode
When you use the composer-based install, it gets installed into the ./bin directory within the ./vendors directory. So for me, relative to my project's root directory, it was here:
./vendor/bin/phpmd
And I was able to run it from my project's root by running ./vendor/bin/phpmd . text codesize. (I'm not getting any useful output yet, but another issue)