At the company I'm currently working we've recently started to move our code into different private repositories so that it's more maintainable and reusable (and also to make it easier to open-source it later).
Every PHP repository is also a Composer package that can be required in our project whenever we need it.
At the moment there's an issue with this approach: every time we need a package that depends on other packages we need to specify those also in the root composer.json.
For example, let's say that the in the root composer.json we need to require two packages company\b and company\c, and that the package company\c needs another package company\d. Then the resulting root composer.json will look like this:
{
"require": {
"company/b": "dev-master",
"company/c": "dev-master",
"company/d": "dev-master"
},
"autoload": {
"psr-4": {
"Company\\" : "src\Company"
}
},
"repositories": [
{
"type": "vcs",
"url": "git#bitbucket.org:company/b.git"
},
{
"type": "vcs",
"url": "git#bitbucket.org:company/c.git"
},
{
"type": "vcs",
"url": "git#bitbucket.org:company/d.git"
}
]
}
Is there a way to avoid specifying nested dependencies in the root composer.json and use the ones specified in the composer.json in every package?
Edit:
Everything I stated before is valid only for the private packages. If a package, let's say company\b, needs a public package that can be found on Packagist then that dependency CAN be specified in the company\b composer.json and it will be imported.
As you correctly found out, only the root package can add repository metadata to the collection of known packages.
I would suggest you take a look at Satis to create a local Composer repository. This would only require you to add this single repository to all your composer.json files of all packages, and it will be used as an updatable source of knowledge about all your private repositories. You no longer have to add a list of Git repos everywhere.
I am successfully hosting around 120 internal packages for our IT enterprise that way. Take this as a sign that once you start splitting isolated tasks into a package, you will get more of them pretty fast.
Also note that it is important to take versioning seriously. Stop depending on branches - tag your software, make releases, use semantic versioning. If you don't, things will break at some point, and people will curse you (correct) or Composer (incorrect) for not working or messing things up.
After a quick search and a look at the Composer documentation I discovered that the repositories can only be specified in the root composer.json.
Additionally it's possible to specify in the root composer.json whether to allow or not development versions of the packages using:
"minimum-stability": "dev",
"prefer-stable": true
Also this issue on GitHub was really useful.
Related
I'm splitting up some of my personal code to modularize and reuse it on different projects.
I've started using composer recently and have been using it for referencing these modules on my projects.
The following has worked for me so far:
First project composer.json
{
"name": "mpf/apimodule",
"version":"dev-main",
"autoload": {
"psr-4": {
"APIModule\\":"classes/"
}
}
}
Second project composer.json
{
"name": "mpf/crawler",
"version":"dev-main",
"autoload": {
"psr-4": {
"API\\": "classes/"
}
},
"repositories": [
{
"type": "vcs",
"url": "git#github.com:{User}/{Repo}.git"
}
],
"require": {
"fabpot/goutte": "^3.2",
"mpf/apimodule": "dev-main"
}
}
Both composers are compiled and the project works as intended.
But when I try to add a third layer
Third project composer.json
{
"autoload": {
"psr-4": {
"API\\": "classes/"
}
},
"repositories": [
{
"type": "vcs",
"url": "git#github.com:{User}/{Repo}.git"
}
],
"require": {
"mpf/crawler": "dev-main"
}
}
I get the following error when running the composer update command
Your requirements could not be resolved to an installable set of packages.
Problem 1
- Root composer.json requires mpf/crawler dev-main -> satisfiable by mpf/crawler[dev-main].
- mpf/crawler dev-main requires mpf/apimodule dev-main -> could not be found in any
version, there may be a typo in the package name.
I've found a similar issue, but my repositories are all public and that was the problem for them.
I've tried running composer -vvv / composer diagnose, but couldn't find any useful information with the results.
From the description given in your question, you add repositories into root composer.json files. This works fine as long as you're using the root composer.json file. That is the project having this file.
Now while this works on each per-project basis, when you put the third (or fourth, fifth etc. ) composer.json file into the mix, or as you word it "add a third layer", it stops working.
Technically it does not stop working, however Composer can not resolve that inherited repository any longer to resolve the root dependency:
- Root composer.json requires mpf/crawler dev-main \
-> satisfiable by mpf/crawler[dev-main].
- mpf/crawler dev-main requires mpf/apimodule dev-main \
-> could not be found in any version, there may be a typo in the package name.
As honest as Composer is here, it may be puzzling in your situation. You've certainly already double-checked there is no typo and still though, Composer can not find any version.
That is because Composer uses the composer.json#/repositories configuration only from the root composer.json - that is the file itself. Let's visualize this a bit:
composer.json
vendor/mpf/crawler/composer.json
My educated guess is that, albeit you've added the repository for mpf/crawler to composer.json the repository for mpf/apimodule has not been added to it but only to vendor/mpf/crawler/composer.json.
The fix is easy, add all repositories your root project requires to resolve all dependencies to that projects configuration file (composer.json in the project you install the dependencies in).
If you think this through, it might become more clear why that is so:
The moment you install from composer.json, all repositories should be defined already as otherwise the outcome of the install will be a pure game of luck. Packages would be able to overwrite your repositories configuration and you would not be in control any longer.
My recommendations to continue your journey:
Add repositories inside your root composer.json file, to ensure your projects' configuration is complete. (Goal: understanding root configuration, the project level)
When working with this, consider if you want to have a global configuration of repositories. That is you can on the level of your computer user configure a list of repositories shared across projects (on that host). (Goal: understanding global configuration, the level of your host, working with projects)
Take a Safari-Tour on the different types of repositories Composer offers (compare with the documentation) as you may find even more in there (e.g. path repositories, another central .json file you can share etc. - there are quite some options). (Goal: understanding of the different repository configuration types)
kuba points to an entry in the Composer FAQ for this topic (via):
Why can't Composer load repositories recursively? (Composer FAQ)
I need to clone a project of my company on my local machine.
Problem: There is a module required in the main composer.json from our old agency which does not exist anymore, so I don't have the access keys needed to download it, BUT I have the latest code of that module.
The composer.json of that module says its name is "randomagency/import", so I added the code at vendor/randomagency/import. Then I removed the module from the main composer.json.
But I get this error:
Class 'RandomAgency\ProcessPipelines\Helper\Condition\AbstractCondition' not found#0 /var/www/company/src/vendor/composer/ClassLoader.php(444): include()
#1 /var/www/company/src/vendor/composer/ClassLoader.php(322): Composer\Autoload\includeFile()
#2 [internal function]: Composer\Autoload\ClassLoader->loadClass()
#3 [internal function]: spl_autoload_call()
My colleague told me that I need to add the module in the main composer.json under the autoload section, but Im not sure how exactly it works.
The best approach to solve it would be to create a new composer package and replace the agency URL in composer.json with my own, but I need a quick & dirty method for now.
Don't put it into vendor (as it is expected to get removed in time and the name of the vendor-dir is a configuration detail), so technically you have already put it in the wrong location. However, this is easy to recover from, just use a different path within the project root directory, e.g. create a directory where you put packages into (pkg in the following). This works already with autoloader configuration only (if that library requires no other configuration to work):
PSR-4 Autoloader (only) from Project Directory
Then add that path to the projects autoload configuration (see autoload (Composer configuration schema) - Autoload mapping for a PHP autoloader.:
{
"autoload": {
"psr-4": {
"RandomAgency\\Import\\": "pkg/random-agency-import-4.2"
}
}
}
It's good practice you have a single folder for the package - not two as in <vendor>/<package> pattern - and append an identifier of the package version.
Require as Composer Package from Project Directory
Better thought, if that package still contains the composer.json configuration file, add it as a path repository and require it with a version constraint that matches the ./pkg/random-agency-import-4.2/composer.json#/version or * if the "version" member is missing:
{
"require": {
"random-agency/import": "*"
},
"repositories": [
{
"type": "path",
"url": "./pkg/random-agency-import-4.2"
}
]
}
You then don't need to configure the autoloader manually as it is taken from the libraries composer.json configuration. This is equally true for any other configuration part incl. the dependencies of the library.
The great benefit of the path repository is, that is accepts relative paths to your projects configuration file (composer.json).
Compare with your previous understanding:
The best approach to solve it would be to create a new composer package and replace the agency URL in composer.json with my own, but I need a quick & dirty method for now.
You either have solved it by now (the package you require has a composer.json file) or you still need to add the package information, but you don't need to create a new package or exchange URLs.
Package Repository: Inline a Composer Package from Project Directory
That is because you can inline the package as well in Composer with the package repository that now has the path and if you look at /repositories/0/package closely, you can see that this is another composer.json document, just inline at that place (and the /dist member is new):
{
"repositories": [
{
"type": "package",
"package": {
"name": "random-agency/import",
"version": "4.2.0",
"dist": {
"type": "path",
"url": "./pkg/random-agency-import-4.2"
},
"autoload": {
"psr-4": {
"RandomAgency\\Import\\": ""
}
}
}
}
],
"require": {
"random-agency/import": "4.2.0"
}
}
When you do updates etc., Composer then will install the packages files into the vendor folder (where-ever it may be, and regardless how often you remove it).
If - and only if - there is no ./pkg/random-agency-import-4.2/composer.json file, you would need to create a "package", as that is the composer.json in the end.
Alternative: compser.json within the package directory
Similar as you added the autoload part to composer.json and as an alternative to the package repository, you can create the ./pkg/random-agency-import-4.2/composer.json file your own, then the path repository already suffices and it is perhaps easier to configure / maintain as the roots project composer.json file does not expand all the package details.
But that is your preference, just showing that you have a couple of options within the project itself without creating outside / "new" packages.
Figured it out. Had to add it like this:
"autoload": {
"psr-4": {
"RandomAgency\\Import\\": "vendor/random-agency/import"
},
I am using the package teamtnt/laravel-scout-tntsearch-driver and I wish to make a very small change to one of the files within teamtnt/tntsearch which is one of the packages dependencies.
Usually I would:
Create a fork of the package.
Add the repository into my composer.json as follows:
"repositories": [
{"type": "vcs", "url": "https://github.com/user/packagefork"}
],
Require/Upgrade the package to the correct version (usually dev-master) keeping the original name spacing and it all works fine.
However, with a dependency which is not directly included in my composer.json file, this does not seem to work. Do I need to fork both the base package, and the dependency package even though I do not need to change anything within the base?
I am hoping there is a simple way to do this, without having to fork each level.
This was actually quite simple. Not too sure why it did not work originally! Instructions below for anyone wondering:
Fork the package (i.e. GitHub)
Add the repo from your username, to your projects main composer.json as follows:
"repositories": [
{"type": "vcs", "url": "https://github.com/youruser/tntsearch"}
],
Edit the composer.json file within the new fork you created in step 1 (youruser/tntsearch) and create/add to the extras key:
"extra": {
"branch-alias": {
"dev-master": "2.0.x-dev"
}
}
This effectively allows you to install your dev-master version, but allow other packages where this is a dependency to request the 2.0 version (in this case). You do need to be careful in this case that you have forked the correct version and any upgrades are correctly managed later down the line, or things may break!
More info on composer alias here
Require/Upgrade the package using original package namespace, at the dev-master version.
composer require teamtnt/tntsearch:dev-master
The name spacing and package versions will remain the same as before your fork, but the edits from your fork will be pulled into your project instead.
I want to achieve the following setup (I've omitted other dependencies and stuff for clarity):
Project's composer.json
{
"require": {
"vendor/masterpackage": "*"
}
}
Master package's composer.json
{
"require": {
"vendor/package-1": "dev-master",
"vendor/package-2": "dev-master"
}
}
The master package is versioned with releases, child packages are not, hence the dev-master requirement.
Since minimum-stability can only be set in the root composer.json and is applied to all dependencies (which is undesired in my case), is it possible to set a dev-stability flag for only the master package AND its own dependencies so that its child packages get installed properly? I've already tried "vendor/masterpackage": "*#dev" but to no avail...
The way you experiment with it, it will not work.
The dependencies of depended packages cannot be influenced directly. So if your project prohibits development packages, you are out of luck.
You'd can add these indirect packages into your projects composer.json, adding them with the necessary #dev or dev-X surroundings. That way the will be required directly, allowing for development stability, and the master package requirement will also be fulfilled.
However, be warned that depending on branches is a very very bad habit and makes your dependencies very brittle. The commit id that has been used is recorded, but once you update and it goes wrong, you'll have a hard time juggling all the moving parts, i.e. all repositories that are only providing a branch.
I've found a way to get this working and thought it would be beneficial to post it as an answer for future reference.
Project's composer.json
{
"require": {
"wikimedia/composer-merge-plugin": "dev-master",
"vendorname/masterpackage": "*"
},
"merge-plugin": {
"require": [
"vendor/vendorname/masterpackage/dev-composer.json"
]
}
}
Master package's dev-composer.json
{
"require": {
"vendorname/package-1": "dev-master#dev",
"vendorname/package-2": "dev-master#dev"
}
}
The merge plugin makes this working since it treats dependencies of specified JSON files as if they are in the root composer.json
I've probably misunderstood the docs (https://getcomposer.org) but is it possible to optimise the syntax for composer when including private repos. Ultimately; I want to step away from listing every repository and vendor pair when the vendor is the same...
In my projects composer.json I've got:
"repositories": [{
"type": "vcs",
"url": "git#bitbucket.org:{vendor}/{repo1}.git"
},{
"type": "vcs",
"url": "git#bitbucket.org:{vendor}/{repo2}.git"
},{
"type": "vcs",
"url": "git#bitbucket.org:{vendor}/{repo3}.git"
}]
...
"require": {
"{vendor}/{repo1}": "dev-master",
"{vendor}/{repo2}": "dev-master",
"{vendor}/{repo3}": "dev-master"
}
I figured; as the repositories is an array, then composer would search the repositories for a vendor/repo pair (or something) or if the vendor matches the vendor part of the repository url. It seems syntax heavy for a slight change...
But now I've confused myself by looking at package.json examples :s
Any kicks in the right direction would be great!
A repository does not have to have the same vendor/package name in every branch or tag - the name can change. So adding a repository simply extends the amount of knowledge Composer has about existing packages, while adding package names in require explicitly pinpoints them wherever they may be located.
If you use more than a handful of private repositories, I'd strongly suggest you create a packagist-like repository with either "Packagist", "Satis" or "Toran Proxy". That way you'd only add that one repo to all your composer.json files (not repeating all your private repos all over the place) and the packages you want to use. This greatly reduces the redundancy you feel, because you'd only deal with the package names everywhere, and with the private repository locations in the configuration file of your central repo solution.