Composer is a great tool for dependency management but for local development it can become time consuming to keep running it to update builds. There are ways round this such as symlinking, but with build tools like gulp it seems there should be better ways to monitor changes to local libraries.
Watching a local library makes sense but composer bases it's update decisions on the latest git commit, not on the current state of the working directory. How can I have a gulp watch task update my dependencies?
Turns out that you can simply watch the git reference, as long as your composer dependency is bound to a specific branch:
gulp.watch([
'/path/to/local/lib/.git/refs/heads/master',
'/path/to/other/lib/.git/refs/heads/dev',
],['composer']);
Where the composer task is defined
var composer = require('gulp-composer');
gulp.task('composer',function(){
composer('update',{bin:'php composer.phar'});
});
Solutions which also watch packagist.org would be greatly appreciated and may even beat out my own!
Related
Since composer was created, it made our life a lot easier. However, running composer install often ends up with so many files that can be useful for development, but just bloat the production server. Shared hosting often has limited inodes, or you must upload the "vendor" folder yourself since composer isn't on the server.
When the vendor folder has 6,000 files+ this is an issue, especially if you have multiple projects with 6,000 files each. And so many of these files are "README.MD" or "TODO". Production servers don't need the dev's "TODO" file.
I tried searching on Google but I can't find any clues, so anyone knows if there is a composer command that will install a production version?
There was a feature request for that, but it was rejected (several times). Right now the best what you can get from Composer is:
composer install --no-dev --prefer-dist
It will skip installation dev packages and prefer dist archives, which usually does not contain tests and other files not necessary to run package on production.
If this is still not enough, you may try to use octolab/cleaner plugin.
composer install --no-dev
The official link:
https://getcomposer.org/doc/03-cli.md#install
If you want to exclude something very specific, you want archive.
https://getcomposer.org/doc/04-schema.md#archive
"archive": {
"exclude": ["*.md", "vendor/**/tests", "/*.test"]
}
I have been using npm for a personal project and just recently stumbled across yarn. Would there be any harm or "intended side effects" to switching to yarn's package manager in the same project where I had been using npm?
Although a few commenters here say its ok to mix both yarn and npm on the same project, after using yarn and npm and then yarn again, this is what yarn has to say about it:
warning package-lock.json found. Your project contains lock files generated by tools
other than Yarn. It is advised not to mix package managers in order to avoid resolution
inconsistencies caused by unsynchronized lock files. To clear this warning, remove
package-lock.json.
Since to me it is not any harm to using both them into one project.
I use npm and yarn (50/50) in dev environment.
But on ci/di i use only yarn because it is faster, and i reduce build minutes thanks yarn.
Also they both create different .lock file names.
Nobody told about the lock files.
Imagine you use yarn on dev environment, and yarn on your build/production servers. When you install a package using yarn, and your project works on your computer, you probably would want to keep it working on a production environment (your server).
That being sad, you would commit you yarn.lock file, that "saves" the exact versions of each package you have, when the project ran on your computer.
On your buid/production server you should call yarn install, but asking to keep all the same versions with --frozen-lockfile parameter. Some even say "yarn install --frozen-lockfile should be the default behavior", and I agree.
Then... another dev jump in the project you are working and install a package using npm (other than yarn). That new package will not be included in your yarn.lock file, but, a new package-json.lock file would be created, telling the exact packages versions it is using.
When that commit arrives on your build/production server, it will crash, fail, because that new package doesn't exist on yarn.lock file. Someone would need to pull that changes, call a yarn to install the dependences and update the lock file with the new package dependences, and push it again to the repo.
A quick point about using the lock file or not. If you call a 'yarn install' on your build/production server some weeks after the last install on your machine, the server would have many other new versions than your last "stable" version. It already happened to me many times.
I published recently the package-locks-checks, which help ensure you have not just one lock file but also locked each package version on your project.
There will be a point that one or both will no longer work and your project will be stuck at only using the existing lock file. Meaning, the issue probably will involve installation fails if you opt to reinstall without a lock file. And that also means failure to create a new lock file, so you are stuck with the existing one that you are trying to get rid off in the first place. We are actually encountering this issue in one of our projects. Because it is so big, no one tries to fix the issue and just rely on the existing lock file.
So, even if we say it's a rare case that it won't cause harm. Mixing npm and yarn should be avoided.
Here https://classic.yarnpkg.com/en/docs/migrating-from-npm/ we may find a confirmation that Yarn's resolution algorithm is compatible with NPM resolution algorithm.
Inside a npm project (with package.json) if you run yarn it will read your node_modules folder (using the resolution algorithm) and create a yarn.lock file with your project's locked dependency tree.
Based on that I assume that they are compatible inside the same project.
Update 30/04/2021
My original reply refers to yarn 1 (classic), although I've just created a React app with create-react-app tool and it creates the project's repository with package.json + yarn.lock by default. Again, another demonstration that it's fine (even with the warning mentioned by Dave Pile).
At the end of the day this is a matter of putting both together to work and checking yourself...
Plus you get a warning from yarn as Dave Pile said because we have to push *-lock.json files changes you have to consider using npm version >= 7 to make sure whenever you install packages by npm it will update your yarn-lock.json file too.
Because whenever you install the packages either by npm or yarn depends on what you have chosen for updating a dependency in the package.json (Using tilde ( ~ ) which gives you bug fix releases and caret ( ^ ) gives you backward-compatible new functionality) it will update you.lock file and since you have to push it might happen that you have different version of lock files.
I know that we can always install a package via command:
composer require packageA
But I don't know if you guys ever have a situation like this:
You want to install a big size package "packageB" that your teammate added to composer.json and your wifi is slow so composer would take very very long to get the packageB. Then you have an idea:
"Maybe I try get the packageB zip from my teammate via flash drive and paste
it into my project."
And you did that, the package works as expected. Wonderful!
But then, you think again:
What if now I want to do the composer update other packages in my
project?
You try:
composer update
and then, what happen is composer will get the package again because you didn't use "composer install" or "composer update" to install packageB so composer doesn't know you have it.
(Sorry for the long explanation).
So my question is:
How do we let composer know that we have the package already so composer don't re-download the package again? Or this is the behavior of composer and I must always use "composer install/update", there is no other way?
And sorry, change to another wifi or find a faster internet connection is really not what I'm looking for. And I also know that we can install the package locally (see here: How to update a single composer package?).
Thanks in advance!
If we don't want to use repositories.
In my knowledge, the only option is to update you composer.json and composer.lock. Friend give you version 1.2 to vendor? Write in exactly version in composer.json and for composer.lock, you will need data from your friend too.
Run install then.
Should check, but not download any file. Still, problem is that all required libraries by this library, could be updated - you can only write down exactly version of them in file.
As default, I think, the didn't predict scenarios for that way.
This is the only solution for you, i know should work.
Composer does use caching heavily to reduce the amount of data to download. However this does not remove the need to download the package at least once.
Basically Composer has two modes to download: --prefer-dist will try to obtain a download URL for an archive file, and --prefer-source will try to obtain a copy of the version control system being used.
Both variants put the result into Composer's cache directory.
Over time you'll collect a couple of archive files locally, which allow for quick switches back and forth between existing version downloads, and newer versions will have to be downloaded once.
Also you can clone a git repository once, and Composer will try to reuse it when updating, by simply fetching new commits and checking out the appropriate tags. This still requires to clone the repository once.
You can work around cloning the repository by manually placing it at the correct spot, either by physically putting it there, or by symlinking the correct vendor directory. You can also make Composer aware of an official copy by adding the local copy as an entry to repositories. This will add this source to the existing collection of packages available from Packagist.
When I do composer install on the production system, should I use the flag --prefer-dist?
--no-dev is recommended, since it prevents the installation of packages that are only needed during development. But what is with --prefer-dist? This flag makes that the installed packages are without VCS stuff, as I read in this answer. I assume that I don't need this on a production machine. Am I right?
The dist file (e.g. tar archive) is usually quicker to download than cloning the repository (which is the case when using --prefer-source).
The main difference is that cloning the repository will give you everything, while a lib maintainer can create the dist itself. This means that they might not include the tests in the dist file for instance. That's why people suggest using --prefer-dist, as it might end up downloading less files.
Anyway, --prefer-dist isn't really needed as Composer always defaults to the dist file when downloading a stable dependency. As it's really a bad practice to have unstable dependencies running in production, you probably end up downloading the dist for all packages anyway (unless you use --prefer-source of course).
Me and my team are working on a project that uses Composer for dependency management. There seems to be a difference in how a composer update is handled on several machines (running the same latest build version of Composer), but we can't figure out why.
When my teammate runs a composer update on a dependency it tries to remove a lot of data/nodes from the composer.lock file (like the entire dist and support nodes):
When I run the same update, it tries to re-add all those keys again:
We can't figure out why this is happening. Is this a certain setting?
Update: On further inspection, it appears that the "(dis)appearing" nodes all contain https links, could it have something to do with a (missing) SSL library or something?
It seems that one Composer prefers dist and one prefers source.
Look into ~/.composer/config.json if you defined a preferred install in any of your two composers. Or if you defined different preferred installs in the composer.json.
"config": {
"preferred-install": "dist"
}
You can force composer to use either dist or source with
composer update --prefer-dist
or
composer update --prefer-source
--prefer-source: There are two ways of downloading a package: source and dist. For stable versions composer will use the dist by default. The source is a version control repository. If --prefer-source is enabled, composer will install from source if there is one. This is useful if you want to make a bugfix to a project and get a local git clone of the dependency directly.
--prefer-dist: Reverse of --prefer-source, composer will install from dist if possible. This can speed up installs substantially on build servers and other use cases where you typically do not run updates of the vendors. It is also a way to circumvent problems with git if you do not have a proper setup.