Switch to composer mode in running instance - composer-php

How can I switch an existing project easily to composer? This project is updated from 6.1 to 8.7 now and should run in composer. A fresh composer setup is not a problem. For the last project I created a new host, installed TYPO3 via composer, installed the extensions via composer and migrated the db, fileadmin and uploads. Is there an easier way?

Migrating TYPO3 from Classic Mode to Composer Mode requires at least the following steps:
Write down the current version of TYPO3 and all extensions
Remove all embedded TYPO3 and extension code incl. Git submodules
Add a Composer manifest
Add the Composer vendor-dir (and bin-dir if custom) to your .gitignore
Require TYPO3 and all extensions with the versions and --prefer-lowest, e.g.
composer require typo3/cms:^8.7.7 --prefer-lowest
This ensures that you don't accidentally perform updates before completing the switch.
Since no further changes to user files or database data is required you will be running TYPO3 in Composer Mode now.
Afterward you will most likely also need to adapt your deployment workflow to ensure at least one composer install is executed after deploying a new version.

There is no real other way, at least no automatic way, as you also upgrade maybe to newer versions or sometimes to the exact same version

This is now documented in the "Installation and Upgrade Guide": https://docs.typo3.org/m/typo3/guide-installation/master/en-us/MigrateToComposer/Index.html
The steps are already outlined in the accepted answer.
As an alternative, you might want to create an installation from scratch with Composer and then use the generated composer.json for your system.

It generally makes sense to have your docroot in a subdirectory before you start so you have, for example:
/var/www/mysite (here, the composer.json will be created)
└──public/
├── fileadmin
├── typo3
└── typo3_src
You can have a look at my extension migrate2composer. However, this will only take care of creating the composer.json file. You have to take care of the rest of the steps yourself.
What it basically does is:
generated a list of all extension with the existing version
dump a sample composer.json file
If you want to do this yourself in your source code, you can take a look at TYPO3\CMS\Core\Package\PackageManager. This worked for TYPO3 v9 and v10 but may change in later versions:
public function getInstalledPackages(string $versionConstraintType = self::VERSION_CONSTRAINT_CARET) : array
{
$packagesInfo = [];
$this->errors = [];
$this->setVersionConstraintType($versionConstraintType);
// collect information about active extensions
$packages = $this->packageManager->getAvailablePackages();
foreach ($packages as $package) {
$key = $package->getPackageKey();
if (!$this->packageManager->isPackageActive($key)) {
// ignore inactive packages
continue;
}
if ($package->getValueFromComposerManifest('type') === 'typo3-cms-framework') {
$type = 'system';
} else {
$type = 'local';
}
$name = $package->getValueFromComposerManifest('name');
// ....
By now the procedure for migrating to Composer is well documented in the official documentation.
Additional steps you must perform yourself:
mv public/typo3conf/sites config/sites
mv public/typo3conf/l10n var/labels

I have tried to generate a composer.json file from the PackageState.php successfully for a docker instance.
I loop through the PackageState.php, then parse each extension for its version and generate a composer.json from this aggregated information.
This is my script:
https://github.com/geri777/typo3-composerize

Related

SFTP Namespace with phpseclib installed manually

I am trying to connect using SFTP (phpseclib) with a manual installation. That's the very first time I'm using namespaces so I don't know if what I'm doing is the way I have to.
I did download phpseclib from GitHub as ZIP and put it in /home/libs_web/php/class/phpseclib3
After that I used this code (just as example) :
require('/home/libs_web/php/class/phpseclib3/Net/SFTP.php');
use phpseclib3\Net\SFTP;
$sftp = new SFTP('localhost');
$sftp->login('username', 'password');
As mentioned here but with an include on top : https://phpseclib.com/docs/sftp
Here is my error : Fatal error: Uncaught Error: Class 'phpseclib3\Net\SSH2' not found in /home/libs_web/php/class/phpseclib3/Net/SFTP.php:52
It seems like my Namespace doesn't work correctly. I do not use autoloader, composer, and tried to set the working directory to /home/libs_web/php/class/phpseclib3/ & /home/libs_web/php/class/.
Don't know what to do more. If I include the file /Net/SSH2.php I'll have another error about another file. I think this isn't the proper way to work.
Could you please provide some help ?
phpseclib3 is best installed with Composer. eg. on the CLI do composer init; composer require phpseclib/phpseclib:~3.0. You'd also put require __DIR__ . '/vendor/autoload.php'; at the top of your file
If you wanted to do something like make your source code available for people to download and make it work on shared hosts were CLI access might not be available I guess you could do Composer and then upload the vendor/ directory by itself or include it with your zip file or whatever.

CakePHP 3.5: Cannot access Plugin Classes after installing it via Composer

So I have a CakePHP 3 project and want to load FluentDOM, a PHP plugin not specifically written for CakePHP.
According to both software documentations, Composer is the way to go. In my understanding, all I would have to do is the following:
run composer require fluentdom/fluentdom in powershell
run composer require fluentdom/selectors-phpcss in powershell
OR
add the following to composer.json in the project's root directory:
"require": {
"fluentdom/fluentdom": "^7.0",
"fluentdom/selectors-phpcss": "^1.1"
}
run composer update in powershell
Both ways will install the desired plugins to vendor/fluentdom/{pluginname}/ as expected, but /vendor/cakephp-plugins.php won't include them, as implied by CakePHP's plugin installation manual.
The following attempt to load either plugin in a controller by writing
use Cake\Core\Plugin;
Plugin::load('fluentdom/fluentdom');
Plugin::load('fluentdom/selectors-phpcss');
would cause an exception that the desired plugins were not found in plugins/ :
Make sure your plugin fluentdom/fluentdom is in the {absolute project path}\plugins\ directory and was loaded
-- Which is already odd, because Composer wouldn't install anything there to begin with.
I found that I might get around this issue by manually extending vendor/cakephp-plugins.php to include the correct paths:
'fluentdom/fluentdom' => $baseDir . '/vendor/fluentdom/fluentdom/',
'fluentdom/selectors-phpcss' => $baseDir . '/vendor/fluentdom/selectors-phpcss/'
(However, that doesn't seem the way to go, because this file is auto-generated and overwritten by Composer after every update.)
And even then, the final issue still persists: although the plugins seem to be loaded successfully (confirmed by running Plugin::loaded()), I'd finally get the following exception when trying to access FluentDOM's classes as described in their wiki:
$document = new FluentDOM\DOM\Document();
Class 'App\Controller\FluentDOM\DOM\Document' not found
Does the plugin miss out on having its' autoload executed?
Even extending the line in my controller to Plugin::load('fluentdom/fluentdom', ['autoload' => true]);, but doesn't seem to help either; according to CakePHP's doc, that shouldn't be necessary anyway.
So what am I missing?
Found it! First of all, I had the false presumption that Plugins and Vendor Packages are more or less the same: they are not; thanks to Greg Schmidt for pointing this out in the question's comments.
The issue was in the line of how I tried to access FluentDOM's class. While
$document = new FluentDOM\DOM\Document();
worked in a standalone php file, it didn't within the Cake project; I was missing a backslash:
$document = new \FluentDOM\DOM\Document();
So, the entire path of actions to load a Vendor Package is merely:
run composer require fluentdom/fluentdom in powershell
run composer require fluentdom/selectors-phpcss in powershell
Use the new classes right away with $document = new \FluentDOM\DOM\Document();
No further steps required. Side note: Composer seems to refresh autoload config after installing a vendor file with composer require {vendor}/{package}, but in case it doesn't, or autoload config is messed up from earlier experiments, composer dumpautoload should fix it.

Sort packages without adding/updating dependencies

On an existing project with a long list of packages and various feature branches where new dependencies are being added I want to mitigate and minimize merge conflicts by adding dependencies in alphabetical order.
To get this cleaned up, though, I'd like to be able to run the --sort-packages functionality on its own -- without adding or updating anything -- as just a single commit that cleans up the existing packages, and then add "sort-packages" : "true" to the "config" section of the composer.json file to ensure all new packages are added in alphabetical order going forward.
Is it possible to sort the packages listed in a messy composer.json file using composer's --sort-packages option on the CLI without actually adding or updating any dependencies?
The only workaround I've found so far is to run composer update some/package --sort-packages against a package that you're sure wont update because it is already at the latest version. This is not ideal.
I know I'm (really) late! But perhaps my answer can help others in the future ;-)
If you run
composer config sort-packages true
this will add the following in your composer.json file:
"config": {
"sort-packages": true
},
The next time you do a composer require (or update) it will automatically sort the whole list.
You can "re-require" a package you've already required. In my case, it's Symfony 3.4, so I did:
composer require symfony/symfony:3.4.*
If you don't have "sort-packages": true in your composer.json, you can do:
composer require --sort-packages symfony/symfony:3.4.*
From what I can tell, only the require command has the option for sorting packages so it seems you need to require a package for sorting to be applied.
It is since 1.0.0-alpha10 - released 2015-04-14 - that composer require has the --sort-packages option:
Added --sort-packages option to require command for sorting dependencies
This is not yet the composer.json#/config/sort-packages which is since 1.0.0-beta1 - released 2016-03-03, roughly a year later:
Added sort-packages config option to force sorting of the requirements when using the require command
It stems from pull-request #3549 and there is also a bit of backstory as well as more options discussed in Normalizing composer.json by
Andreas Möller (localheinz), Jan 2018.
First of all, +1 for using "sort-packages": true, for all the reasons you describe.
It is a bad idea to edit composer.lock directly, but I do not think the same applies to composer.json. I would edit the files in Vim, select everything inside the "require" and "require-dev" sections (one at a time) and :sort. Plus some fiddling to make sure that every line except the last has a comma.

composer and site configuration files

Is it possible to install a file out of the 'vendor' directory when doing a composer install/update?
Let me elaborate a bit more if your not sure what i mean.
I have a config file(s) that are stored in /config/ini/<filename>.ini and lots of vendor modules in the vendor directory. Would it be possible to package the ini files with the vendor packages so upon installation they are written to the correct directory?
Ideally I need to be able to achieve this because i have an Authentication vendor module that will need to be installed in various different applications. Being able to do this will mean that the private key and other shared configuration options can be stored with the vendor module (in a private repo ofc).
Thanks Mike
Yes, you can. You need to create a script which is attached to post-install-cmd or post-update-cmd. That script will look in the package directories, select the issues and dump them in the correct dir.
It'll be somewhere around these lines:
use Composer\Script\CommandEvent;
class ScriptHandler
{
public function bundleConfigs(CommandEvent $event)
{
$homeDir = $event->getComposer()->getConfig()->get('home');
$vendorDir = $event->getComposer()->getConfig()->get('vendor-dir');
$files = glob($vendorDir, '/*Module/config/*.ini');
foreach ($files as $file) {
copy($file, $homeDir.'/config/ini/'.basename($file));
}
}
}

Manual installation of Magento extension turns every file into a folder

I am completely baffled by this. I packaged up an extension and manually installed it on a fresh Magento instance. (Both the packaging and installing machine were running Magento 1.7). The installation went smoothly, except every single file I installed was turned into a folder, named after the file. Every single file. Has anyone run into this? Could it be a Magento bug?
I used to see this problem when I manually created a tar archive to use as a Magento Connect archive. Unfortunately, I don't have a solution, but here's what I understand about the problem.
While Magento Connect tgz packages are technically gzip compressed tar archives — the code that creates and extracts these archives in not the standard *nix tar tool. Instead, Magento implemented its own packing and unpacking tar code for Magento Connect
downloader/lib/Mage/Archive/Tar.php
Unfortunately, this packing and unpacking code hasn't been robustly tested across operating systems or against tar archives created with standard *nix tools. My problem with this code was archives created on my Mac OS system via tar wouldn't unpack correctly with Magento Connect's code on a system running linux.
Hard to track down, hard to report, hard to reproduce means hard to fix.
These directories are being created when Magento Connect unpacks the tgz file. I'm be 99% sure your directories are being created by this bit of code
#File: downloader/lib/Mage/Archive/Tar.php
if (in_array($header['type'], array("0",chr(0), ''))) {
if(!file_exists($dirname)) {
$mkdirResult = #mkdir($dirname, 0777, true);
if (false === $mkdirResult) {
throw new Mage_Exception('Failed to create directory ' . $dirname);
}
}
$this->_extractAndWriteFile($header, $currentFile);
$list[] = $currentFile;
} elseif ($header['type'] == '5') {
if(!file_exists($dirname)) {
$mkdirResult = #mkdir($currentFile, $header['mode'], true);
if (false === $mkdirResult) {
throw new Mage_Exception('Failed to create directory ' . $currentFile);
}
}
$list[] = $currentFile . DS;
These are the two locations where Magento unpacks the archives and creates a folder. For some reason, there's a certain condition on your two systems where the data is being packed, or unpacked, incorrectly in/out of the archive file. Try un-archiving the tgz file manually with a command line tool or your operating system's built in un-archive program. If weird things happen then at least you know it's the packing code that's the problem.
It's definitely a bug, and while I'd report it, the only "solution" would be to not create your archive on your local machine (which I realize is an awful solution, but Ours is not to question why and all that)
This is a bug that has been present since 1.7, due to an if comparison never evaluating to false when reading the #././LongLink header. I answered it more on this question:
https://magento.stackexchange.com/questions/5585/long-file-names-and-magento-connect-extension-packager/45187#45187
I found that issue happening when packing a Magento Extension on OS X that is linked (modman) into magento folders. Folder creation only occured on Windows systems.
Might that happen here too?
Rico
I encountered it, when for some reason my plugin file was set with suffix .gz
so it was plugin.tgz.gz
unzip it to plugin.tgz solved my issue
I think the issue is because of PHP version. I faced the same issue while installing extension on Magento 1.8.1 , but I found a fix by changing _getFormatParseHeader() function in /downloader/lib/Mage/Archive/Tar.php file.
Originally the function was :
protected static final function _getFormatParseHeader()
{
return 'a100name/a8mode/a8uid/a8gid/a12size/a12mtime/a8checksum/a1type/a100symlink/a6magic/a2version/'
. 'a32uname/a32gname/a8devmajor/a8devminor/a155prefix/a12closer';
}
I changed it to :
protected static final function _getFormatParseHeader()
{
if (version_compare(phpversion(), '5.5.0', '<') === true) {
return 'a100name/a8mode/a8uid/a8gid/a12size/a12mtime/a8checksum/a1type/a100symlink/a6magic/a2version/'
. 'a32uname/a32gname/a8devmajor/a8devminor/a155prefix/a12closer';
}
return 'Z100name/Z8mode/Z8uid/Z8gid/Z12size/Z12mtime/Z8checksum/Z1type/Z100symlink/Z6magic/Z2version/'
. 'Z32uname/Z32gname/Z8devmajor/Z8devminor/Z155prefix/Z12closer';
}
Really nasty bug.
For me it renaming my manually packed file from *.tar.gz to *.tgz solved it.
At least it worked on my ubuntu 15.04
Tested with magento 1.8
it's more likely that you choose the wrong path when adding content to your extension.
For me the bug happened when I added (non existing) files from layout/base instead from layout/base/default.

Resources