Composer VCS repository not loading dependancies - composer-php

I'm including a private git repository via composer, and it's loading as expected from bitbucket, however I have codeception defined as a public dependency in my private package.
My private package is loaded, but none of it's dependencies are added. I have read that composer does not support recursive loading of dependancies when using repositories: https://getcomposer.org/doc/faqs/why-can%27t-composer-load-repositories-recursively.md
However my understanding of this is that my private repository can't define another private repository, but should still be able to make use of public repositories defined on packagist.org
Private repositories composer.json:
{
"name": "private/dependancy",
"description": "Private git dependency",
"type" : "library",
"require-dev": {
"codeception/codeception": "*"
}
}
Project's composer.json (Trimmed down to relevant sections)
{
"name": "primary/project",
"description": "Main project including a vcs dependancy",
"require": {
"private/dependancy" : "0.0.*"
},
"repositories":[
{
"type" : "vcs",
"url" : "some repo",
"options": {
"ssh2": "some crednetials"
}
}
]
}
Any guidance on this would be greatly appreciated.

Composer will not install dev-dependencies of the packages you require yourself.
Your primary/project requires private/dependency, which does not require anything else. Anything listed as require-dev is not installed, because that is considered to be used when developing private/dependency, not when using it.
Another thing that only get's evaluated when running Composer on the primary composer.json and not explicitly excluding dev dependencies is "autoload-dev".
composer install --no-dev
will not install ANY dev dependencies and not create autoloading for dev.
composer install
will install dev dependencies of the primary project, and create autoloading for dev - it will never install dev dependencies of any of the packages added via require or require-dev, and not add their autoload-dev.

Related

Composer error when trying to include multi level public personal repositories

I'm splitting up some of my personal code to modularize and reuse it on different projects.
I've started using composer recently and have been using it for referencing these modules on my projects.
The following has worked for me so far:
First project composer.json
{
"name": "mpf/apimodule",
"version":"dev-main",
"autoload": {
"psr-4": {
"APIModule\\":"classes/"
}
}
}
Second project composer.json
{
"name": "mpf/crawler",
"version":"dev-main",
"autoload": {
"psr-4": {
"API\\": "classes/"
}
},
"repositories": [
{
"type": "vcs",
"url": "git#github.com:{User}/{Repo}.git"
}
],
"require": {
"fabpot/goutte": "^3.2",
"mpf/apimodule": "dev-main"
}
}
Both composers are compiled and the project works as intended.
But when I try to add a third layer
Third project composer.json
{
"autoload": {
"psr-4": {
"API\\": "classes/"
}
},
"repositories": [
{
"type": "vcs",
"url": "git#github.com:{User}/{Repo}.git"
}
],
"require": {
"mpf/crawler": "dev-main"
}
}
I get the following error when running the composer update command
Your requirements could not be resolved to an installable set of packages.
Problem 1
- Root composer.json requires mpf/crawler dev-main -> satisfiable by mpf/crawler[dev-main].
- mpf/crawler dev-main requires mpf/apimodule dev-main -> could not be found in any
version, there may be a typo in the package name.
I've found a similar issue, but my repositories are all public and that was the problem for them.
I've tried running composer -vvv / composer diagnose, but couldn't find any useful information with the results.
From the description given in your question, you add repositories into root composer.json files. This works fine as long as you're using the root composer.json file. That is the project having this file.
Now while this works on each per-project basis, when you put the third (or fourth, fifth etc. ) composer.json file into the mix, or as you word it "add a third layer", it stops working.
Technically it does not stop working, however Composer can not resolve that inherited repository any longer to resolve the root dependency:
- Root composer.json requires mpf/crawler dev-main \
-> satisfiable by mpf/crawler[dev-main].
- mpf/crawler dev-main requires mpf/apimodule dev-main \
-> could not be found in any version, there may be a typo in the package name.
As honest as Composer is here, it may be puzzling in your situation. You've certainly already double-checked there is no typo and still though, Composer can not find any version.
That is because Composer uses the composer.json#/repositories configuration only from the root composer.json - that is the file itself. Let's visualize this a bit:
composer.json
vendor/mpf/crawler/composer.json
My educated guess is that, albeit you've added the repository for mpf/crawler to composer.json the repository for mpf/apimodule has not been added to it but only to vendor/mpf/crawler/composer.json.
The fix is easy, add all repositories your root project requires to resolve all dependencies to that projects configuration file (composer.json in the project you install the dependencies in).
If you think this through, it might become more clear why that is so:
The moment you install from composer.json, all repositories should be defined already as otherwise the outcome of the install will be a pure game of luck. Packages would be able to overwrite your repositories configuration and you would not be in control any longer.
My recommendations to continue your journey:
Add repositories inside your root composer.json file, to ensure your projects' configuration is complete. (Goal: understanding root configuration, the project level)
When working with this, consider if you want to have a global configuration of repositories. That is you can on the level of your computer user configure a list of repositories shared across projects (on that host). (Goal: understanding global configuration, the level of your host, working with projects)
Take a Safari-Tour on the different types of repositories Composer offers (compare with the documentation) as you may find even more in there (e.g. path repositories, another central .json file you can share etc. - there are quite some options). (Goal: understanding of the different repository configuration types)
kuba points to an entry in the Composer FAQ for this topic (via):
Why can't Composer load repositories recursively? (Composer FAQ)

How to add internal public repository to satis.json?

Need some help in understanding a scenario with satis.json , My question is we have already a package named as ""xamin/handlebars.php" used in our project as a composer dependency and recently this package is abandoned, so we have planned to fork the repository to our internal github and added patches to it. Now it's time to tell my satis.json to use the latest version from our github internal repo.
So I have updated repositories section as
"repositories": [
{"name": "xamin/handlebars.php",
"type": "git",
"url": "git#github.corp.test.com:BE/XaminProject-handlebars.php.git"}
],
and I just kept as it is in require section :
"require": {
"xamin/handlebars.php": "0.10.5",
}
Now if I try to do composer update from my local dev:
It says :
Loading composer repositories with package information
Warning: Accessing satis-new.test.com over http which is an insecure protocol.
Updating dependencies (including require-dev)
Your requirements could not be resolved to an installable set of packages.
Problem 1
- The requested package xamin/handlebars.php ^0.10.5 exists as xamin/handlebars.php[v0.10.4, v0.10.0, v0.10.1, v0.10.2, v0.10.3] but these are rejected by your constraint.

Nested dependencies and private repositories with composer

At the company I'm currently working we've recently started to move our code into different private repositories so that it's more maintainable and reusable (and also to make it easier to open-source it later).
Every PHP repository is also a Composer package that can be required in our project whenever we need it.
At the moment there's an issue with this approach: every time we need a package that depends on other packages we need to specify those also in the root composer.json.
For example, let's say that the in the root composer.json we need to require two packages company\b and company\c, and that the package company\c needs another package company\d. Then the resulting root composer.json will look like this:
{
"require": {
"company/b": "dev-master",
"company/c": "dev-master",
"company/d": "dev-master"
},
"autoload": {
"psr-4": {
"Company\\" : "src\Company"
}
},
"repositories": [
{
"type": "vcs",
"url": "git#bitbucket.org:company/b.git"
},
{
"type": "vcs",
"url": "git#bitbucket.org:company/c.git"
},
{
"type": "vcs",
"url": "git#bitbucket.org:company/d.git"
}
]
}
Is there a way to avoid specifying nested dependencies in the root composer.json and use the ones specified in the composer.json in every package?
Edit:
Everything I stated before is valid only for the private packages. If a package, let's say company\b, needs a public package that can be found on Packagist then that dependency CAN be specified in the company\b composer.json and it will be imported.
As you correctly found out, only the root package can add repository metadata to the collection of known packages.
I would suggest you take a look at Satis to create a local Composer repository. This would only require you to add this single repository to all your composer.json files of all packages, and it will be used as an updatable source of knowledge about all your private repositories. You no longer have to add a list of Git repos everywhere.
I am successfully hosting around 120 internal packages for our IT enterprise that way. Take this as a sign that once you start splitting isolated tasks into a package, you will get more of them pretty fast.
Also note that it is important to take versioning seriously. Stop depending on branches - tag your software, make releases, use semantic versioning. If you don't, things will break at some point, and people will curse you (correct) or Composer (incorrect) for not working or messing things up.
After a quick search and a look at the Composer documentation I discovered that the repositories can only be specified in the root composer.json.
Additionally it's possible to specify in the root composer.json whether to allow or not development versions of the packages using:
"minimum-stability": "dev",
"prefer-stable": true
Also this issue on GitHub was really useful.

Can I use Composer to clone git repo of a package unavailable in Packagist?

I would like to use jquery DataTables within my project. Since the package is not available in Packagist, I am trying to use composer to clone the git repo of DataTables but it fails. Please advise how to proceed:
{
"repositories": [
{
"type": "vcs",
"url": "https://github.com/DataTables/DataTables"
}
],
"require": {
"DataTables/DataTables": "master"
}
}
Then composer update returns:
Loading composer repositories with package information
Updating dependencies (including require-dev)
Your requirements could not be resolved to an installable set of packages.
Problem 1
- The requested package datatables could not be found in any version, there
may be a typo in the package name.
Potential causes:
- A typo in the package name
- The package is not available in a stable-enough version according to your min
imum-stability setting
see <https://groups.google.com/d/topic/composer-dev/_g3ASeIFlrc/discussion> f
or more details.
Read <http://getcomposer.org/doc/articles/troubleshooting.md> for further common
problems.
I would further like to clone/download a specific version of the repo.
Old question and answer, however:
The error is not in the name of the package (that is indeed DataTables/DataTables) but in the indication of the stability. If you want the 'master' branch, you need to write 'dev-master' in composer and the stability level is dev. Otherwise require a specific tag.
In your case:
{
"repositories": [
{
"type": "vcs",
"url": "https://github.com/DataTables/DataTables"
}
],
"require": {
"DataTables/DataTables": "dev-master"
}
}
will work as expected:
root#erme:/usr/local/munk_php/jquerydatatables# composer install
Loading composer repositorInstalling dependencies (including require-dev)
- Installing datatables/datatables (dev-master 96b7ef9)
Cloning 96b7ef9176543bbf1f1488c0f9538ad9dcc9bc01
Writing lock file
Generating autoload files
The package name in https://github.com/DataTables/DataTables/blob/master/component.json isn't DataTables/DataTables
Try
"require": {
"DataTables": "dev-master"
}

composer create-project from private repo

I have a private project hosted on Bit Bucket. I have an SSH key setup. Is there a way I can use the php composer create-project vendor/name path command in the same way as If it was on Packagist?
Well there are different ways to accomplish this one being the use of a composer repository that is used instead of packagist.org, which is a better more centralized way to manage your private composer packages. The other method is to use a composer.json that incorporates your private repos within your environments, per environment.
First
Composer allows you to use private repositories to create projects.
Like so...
composer create-project vendor/name path --repository-url=http://repo.yourcomposerrepo.com
Since you won't submit a private package to packagist. That url just needs a packages.json file at minimum, you could use satis or your own packagist if you want a more dynamic solution to the packages.json.
The method for using composer.json applies to already created projects that will use custom repositories for private packages, not for creating new projects from private repositories. Use the next method if you want to go down a similar route.
Second
Configure your private repository into your config.json globally for your environment. Then like normally..
composer create-project vendor/name path
Yes, Composer allows you to add private projects as 'repositories' to your composer.json file. So therefore you can include private projects into another project.
It provides support for GitHub and Bitbucket (as well as SVN and Mercurial).
You need to modify your composer.json file to look something like this:
{
"repositories": [ {
"type": "package",
"package": {
"name": "TheShiftExchange/test",
"version": "1.0.0",
"source": {
"url": "https://github.com/TheShiftExchange/test.git",
"type": "git",
"reference": "master"
}
}
}],
"require": {
"laravel/framework": "4.0.*",
"TheShiftExchange/test": "1.0.*"
},
}
The way I used to:
composer create-project vendor/name path --repository="{\"url\": \"https://bitbucket.org/user/project.git\", \"type\": \"vcs\"}" --stability=dev --remove-vcs
Reference: https://getcomposer.org/doc/03-cli.md#create-project
We have Toran Proxy (https://toranproxy.com/) installed as a private packagist, and for that we are able to create projects using command below
composer create-project vendor/framework --repository-url=http://your-toran-repo-url/repo/private/ --stability=dev project name
Stability version we use if the project is not tagged or you looking for bleeding edge version.
--stability=dev
Since this post has some traction, I thought I'd add another solution which I use.
Open up ~/.bash_profile
and add something like
function _cmsname {
composer create-project vendor/package --repository-url=http://private.repo.url.co.uk/ --stability=dev "$1"
}
alias cmsname=_cmsname
and the just type cmsname projectname in terminal.

Resources