On an existing project with a long list of packages and various feature branches where new dependencies are being added I want to mitigate and minimize merge conflicts by adding dependencies in alphabetical order.
To get this cleaned up, though, I'd like to be able to run the --sort-packages functionality on its own -- without adding or updating anything -- as just a single commit that cleans up the existing packages, and then add "sort-packages" : "true" to the "config" section of the composer.json file to ensure all new packages are added in alphabetical order going forward.
Is it possible to sort the packages listed in a messy composer.json file using composer's --sort-packages option on the CLI without actually adding or updating any dependencies?
The only workaround I've found so far is to run composer update some/package --sort-packages against a package that you're sure wont update because it is already at the latest version. This is not ideal.
I know I'm (really) late! But perhaps my answer can help others in the future ;-)
If you run
composer config sort-packages true
this will add the following in your composer.json file:
"config": {
"sort-packages": true
},
The next time you do a composer require (or update) it will automatically sort the whole list.
You can "re-require" a package you've already required. In my case, it's Symfony 3.4, so I did:
composer require symfony/symfony:3.4.*
If you don't have "sort-packages": true in your composer.json, you can do:
composer require --sort-packages symfony/symfony:3.4.*
From what I can tell, only the require command has the option for sorting packages so it seems you need to require a package for sorting to be applied.
It is since 1.0.0-alpha10 - released 2015-04-14 - that composer require has the --sort-packages option:
Added --sort-packages option to require command for sorting dependencies
This is not yet the composer.json#/config/sort-packages which is since 1.0.0-beta1 - released 2016-03-03, roughly a year later:
Added sort-packages config option to force sorting of the requirements when using the require command
It stems from pull-request #3549 and there is also a bit of backstory as well as more options discussed in Normalizing composer.json by
Andreas Möller (localheinz), Jan 2018.
First of all, +1 for using "sort-packages": true, for all the reasons you describe.
It is a bad idea to edit composer.lock directly, but I do not think the same applies to composer.json. I would edit the files in Vim, select everything inside the "require" and "require-dev" sections (one at a time) and :sort. Plus some fiddling to make sure that every line except the last has a comma.
Related
Using yarn 2's new plug n play (pnp) creates a long .pnp.js file. I get a bunch of merge conflicts while pulling, and these are not autofixed (unlike yarn.lock).
How are these conflicts solved? I'd rather not go them through manually as it's not clear what change to accept.
Example conflict
["virtual:844e49f9c8ad85b5809b347eb507fe8bfdc2d527102f53e0b4f78076a2ad5ea2556763170701137a2cafdc51d5a36d82e448010e65742a300748e0bc70028101#npm:11.2.7", {
"packageLocation": "./.yarn/$$virtual/#testing-library-react-virtual-2e67fd5293/0/cache/#testing-library-react-npm-11.2.7-3a0469c756-389c9f3e83.zip/node_modules/#testing-library/react/",
"packageDependencies": [
["#testing-library/react", "virtual:844e49f9c8ad85b5809b347eb507fe8bfdc2d527102f53e0b4f78076a2ad5ea2556763170701137a2cafdc51d5a36d82e448010e65742a300748e0bc70028101#npm:11.2.7"],
["#babel/runtime", "npm:7.13.10"],
<<<<<<< HEAD
["#testing-library/dom", "npm:7.30.4"],
["#types/react", "npm:17.0.3"],
["#types/react-dom", "npm:17.0.3"],
=======
["#testing-library/dom", "npm:7.31.0"],
["#types/react", "npm:17.0.8"],
["#types/react-dom", "npm:17.0.5"],
>>>>>>> d2bb5d9e537f9647e9757656de230e56282e0b15
["react", "npm:17.0.2"],
I would assume you can delete this file containing merge conflicts.
Next, you run yarn install which will generate this file again.
Or just run yarn install which will overwrite the the .pnp.cjs file and fix the merge conflicts (if any) in the yarn.lock file for you.
From the docs:
The generated .pnp.cjs file can be committed to your repository
as part of the Zero-Installs effort, removing the need to run yarn install in the first place.
As you can read, this file can - not must - be committed. However, if you commit it, you can use all your dependencies immediately after cloning the repo, switching branches, ... without need to run yarn install every time.
Note that the same does not count for yarn.lock file which you should never delete.
So I have a CakePHP 3 project and want to load FluentDOM, a PHP plugin not specifically written for CakePHP.
According to both software documentations, Composer is the way to go. In my understanding, all I would have to do is the following:
run composer require fluentdom/fluentdom in powershell
run composer require fluentdom/selectors-phpcss in powershell
OR
add the following to composer.json in the project's root directory:
"require": {
"fluentdom/fluentdom": "^7.0",
"fluentdom/selectors-phpcss": "^1.1"
}
run composer update in powershell
Both ways will install the desired plugins to vendor/fluentdom/{pluginname}/ as expected, but /vendor/cakephp-plugins.php won't include them, as implied by CakePHP's plugin installation manual.
The following attempt to load either plugin in a controller by writing
use Cake\Core\Plugin;
Plugin::load('fluentdom/fluentdom');
Plugin::load('fluentdom/selectors-phpcss');
would cause an exception that the desired plugins were not found in plugins/ :
Make sure your plugin fluentdom/fluentdom is in the {absolute project path}\plugins\ directory and was loaded
-- Which is already odd, because Composer wouldn't install anything there to begin with.
I found that I might get around this issue by manually extending vendor/cakephp-plugins.php to include the correct paths:
'fluentdom/fluentdom' => $baseDir . '/vendor/fluentdom/fluentdom/',
'fluentdom/selectors-phpcss' => $baseDir . '/vendor/fluentdom/selectors-phpcss/'
(However, that doesn't seem the way to go, because this file is auto-generated and overwritten by Composer after every update.)
And even then, the final issue still persists: although the plugins seem to be loaded successfully (confirmed by running Plugin::loaded()), I'd finally get the following exception when trying to access FluentDOM's classes as described in their wiki:
$document = new FluentDOM\DOM\Document();
Class 'App\Controller\FluentDOM\DOM\Document' not found
Does the plugin miss out on having its' autoload executed?
Even extending the line in my controller to Plugin::load('fluentdom/fluentdom', ['autoload' => true]);, but doesn't seem to help either; according to CakePHP's doc, that shouldn't be necessary anyway.
So what am I missing?
Found it! First of all, I had the false presumption that Plugins and Vendor Packages are more or less the same: they are not; thanks to Greg Schmidt for pointing this out in the question's comments.
The issue was in the line of how I tried to access FluentDOM's class. While
$document = new FluentDOM\DOM\Document();
worked in a standalone php file, it didn't within the Cake project; I was missing a backslash:
$document = new \FluentDOM\DOM\Document();
So, the entire path of actions to load a Vendor Package is merely:
run composer require fluentdom/fluentdom in powershell
run composer require fluentdom/selectors-phpcss in powershell
Use the new classes right away with $document = new \FluentDOM\DOM\Document();
No further steps required. Side note: Composer seems to refresh autoload config after installing a vendor file with composer require {vendor}/{package}, but in case it doesn't, or autoload config is messed up from earlier experiments, composer dumpautoload should fix it.
How can I switch an existing project easily to composer? This project is updated from 6.1 to 8.7 now and should run in composer. A fresh composer setup is not a problem. For the last project I created a new host, installed TYPO3 via composer, installed the extensions via composer and migrated the db, fileadmin and uploads. Is there an easier way?
Migrating TYPO3 from Classic Mode to Composer Mode requires at least the following steps:
Write down the current version of TYPO3 and all extensions
Remove all embedded TYPO3 and extension code incl. Git submodules
Add a Composer manifest
Add the Composer vendor-dir (and bin-dir if custom) to your .gitignore
Require TYPO3 and all extensions with the versions and --prefer-lowest, e.g.
composer require typo3/cms:^8.7.7 --prefer-lowest
This ensures that you don't accidentally perform updates before completing the switch.
Since no further changes to user files or database data is required you will be running TYPO3 in Composer Mode now.
Afterward you will most likely also need to adapt your deployment workflow to ensure at least one composer install is executed after deploying a new version.
There is no real other way, at least no automatic way, as you also upgrade maybe to newer versions or sometimes to the exact same version
This is now documented in the "Installation and Upgrade Guide": https://docs.typo3.org/m/typo3/guide-installation/master/en-us/MigrateToComposer/Index.html
The steps are already outlined in the accepted answer.
As an alternative, you might want to create an installation from scratch with Composer and then use the generated composer.json for your system.
It generally makes sense to have your docroot in a subdirectory before you start so you have, for example:
/var/www/mysite (here, the composer.json will be created)
└──public/
├── fileadmin
├── typo3
└── typo3_src
You can have a look at my extension migrate2composer. However, this will only take care of creating the composer.json file. You have to take care of the rest of the steps yourself.
What it basically does is:
generated a list of all extension with the existing version
dump a sample composer.json file
If you want to do this yourself in your source code, you can take a look at TYPO3\CMS\Core\Package\PackageManager. This worked for TYPO3 v9 and v10 but may change in later versions:
public function getInstalledPackages(string $versionConstraintType = self::VERSION_CONSTRAINT_CARET) : array
{
$packagesInfo = [];
$this->errors = [];
$this->setVersionConstraintType($versionConstraintType);
// collect information about active extensions
$packages = $this->packageManager->getAvailablePackages();
foreach ($packages as $package) {
$key = $package->getPackageKey();
if (!$this->packageManager->isPackageActive($key)) {
// ignore inactive packages
continue;
}
if ($package->getValueFromComposerManifest('type') === 'typo3-cms-framework') {
$type = 'system';
} else {
$type = 'local';
}
$name = $package->getValueFromComposerManifest('name');
// ....
By now the procedure for migrating to Composer is well documented in the official documentation.
Additional steps you must perform yourself:
mv public/typo3conf/sites config/sites
mv public/typo3conf/l10n var/labels
I have tried to generate a composer.json file from the PackageState.php successfully for a docker instance.
I loop through the PackageState.php, then parse each extension for its version and generate a composer.json from this aggregated information.
This is my script:
https://github.com/geri777/typo3-composerize
Like most people writing (and reading) the question about whether to keep composer.lock in version-control, we keep ours there.
However, this causes us trouble every time the file is independently updated in different code-branches. Even when the changes are unrelated and affect the sections of the file afar from each other, the "content-hash" line is causing a conflict every time. Worse, neither "side" is correct and whoever is doing the merging must regenerate the file by hand...
Maybe, the line is not really necessary? Before asking, whether (the current version of) composer will work without it, what functionality would be missing? The hash seems to guard against the file itself changing -- but the source-control system is already doing that...
Can I simply remove the line? If it can not be done today, would it be a desirable feature for composer?
Purpose of the content hash
As you can see in Composer\Package\Locker::getContentHash(), the content hash takes into account the following fields of composer.json:
$relevantKeys = array(
'name',
'version',
'require',
'require-dev',
'conflict',
'replace',
'provide',
'minimum-stability',
'prefer-stable',
'repositories',
'extra',
);
The only reason for the content hash to change is a change of one of the values of the corresponding properties in composer.json.
Composer uses the content hash to determine whether relevant fields in composer.json are in sync with composer.lock. You can run
$ composer validate
to find out if they are in sync.
If composer.json and composer.lock are not in sync, a message similar to this will be shown
The lock file is not up to date with the latest changes in composer.json, it is recommended that you run composer update.
For reference, see https://getcomposer.org/doc/03-cli.md#validate:
You should always run the validate command before you commit your composer.json file, and before you tag a release. It will check if your composer.json is valid.
Resolving conflicts in composer.lock
If you have trouble resolving conflicts in composer.lock, maybe this helps:
Step 1: Accept upstream changes
Usually, you will probably attempt to rebase a branch on top of the upstream changes. When already in conflict, use your IDE, or run
$ git checkout --theirs composer.lock
to accept the upstream changes to composer.lock. Since this is a generated file, you really don't want to resolve conflicts in it.
Step 2: Re-apply changes to composer.json and composer.lock
As pointed out earlier, there are a range of the relevant keys in composer.json. Some of them can be modified by corresponding commands, others cannot.
For example, if one of the changes is a newly added or removed package, run
$ composer require foo/bar:^1.2.3
or
$ composer remove foo/bar
to apply the changes.
If the changes cannot be applied by running a command, manually modify composer.json, then run
$ composer update --lock
This will update the content hash.
For reference, see https://getcomposer.org/doc/03-cli.md#update:
--lock: Only updates the lock file hash to suppress warning about the lock file being out of date.
TL;DR version from #localheinz:
Just resolve the conflict and then regenerate the lock file with:
$ composer update --lock
This will update the content hash.
For reference, see https://getcomposer.org/doc/03-cli.md#update:
--lock: Only updates the lock file hash to suppress warning about the lock file being out of date.
I searched the net and handbook, but I only managed to learn what is the masked package, and not how to install it. I did find some commands, but they don't seem to work on 2008 (looking at it, it seems those are for earlier versions). I have something like this:
localhost ~ # emerge flamerobin
Calculating dependencies
!!! All ebuilds that could satisfy "dev-db/flamerobin" have been masked.
!!! One of the following masked packages is required to complete your request:
- dev-db/flamerobin-0.8.6 (masked by: ~x86 keyword)
- dev-db/flamerobin-0.8.3 (masked by: ~x86 keyword)
I would like to install version 0.8.6, but don't know how? I found some instructions, but they tell me to edit or write to some files under /etc/portage. However, I don't have /etc/portage on my system:
localhost ~ # ls /etc/portage
ls: cannot access /etc/portage: No such file or directory
There are two different kinds of masks in gentoo. Keyword masks and package masks. A keyword mask means that the package is either not supported (or untested) by your architecture, or still in testing. A package mask means that the package is masked for another reason (and for most users it is not smart to unmask). The solutions are:
Add a line to /etc/portage/package.keywords (Check man portage in the package.keywords section). This is for the keyword problems.
Add a line to /etc/portage/package.unmask for "package.mask" problems (you can also use package.mask for the converse). This is in the same man file, under the section package.unmask. I advise to use versioned atoms here to avoid shooting in your own foot with really broken future versions a couple of months down the line.
These days there's also a more 'automated' solution, called "autounmask". No more file editing needed to unmask!
The great benefit of the package is, it also unmasks / handles keywords of dependencies if needed. It's provided in the package app-portage/autounmask.
/etc/portage/package.keywords and
/etc/portage/package.unmask
can be directories as well nowadays (but autounmask handles single files as well). In those directories, multiple can place multiple "autounmask" files, one file in each dir per "unmask"-package. If you use single files instead of dirs, 'autounmask' will place some kind of header / footer, and this way it becomes easy to remove "unmasks" if wanted.
Simply mkdir /etc/portage and edit as mentioned here: http://gentoo-wiki.com/TIP_Dealing_with_masked_packages#But_you_want_to_install_the_package_anyway...