Laravel blank projects.. so many files! Can it be reduced somehow? - laravel

Last time I used Laravel was a long time ago and I decided to get back to it.
Now coming from CodeIgniter, which was a powerful framework in its own time, I was happy to upload projects to websites as the "system" folder containing the framework files contained only 121 files.
The problem with composer-based solutions however is that a tiny project can become huge, much bigger than a very large scale CodeIgniter project back in the days. All the dependencies have test folders, documentation, and tons of modules when sometimes only one method is used.
I gasped when creating an empty Laravel project using the instructions from the official documentation and seeing the "vendor" folder containing over 8,000 files!! (Not counting the folders) And it's doing nothing yet.. That is when using the --prefer-dist flag by the way. And I know about the --no-dev argument, which still has 5,000+ files. My point was that there is no way all these files are used, especially when using the distribution channel.
So my question is if there is a way to have a more selective empty Laravel project, as servers often have limited Inodes and 8,000 files + folders for every project makes you reach the limit really quickly (and upload takes forever if you can't install composer on your server).

Composer can remove extraneous files.
In your project's composer.json, specify the files you don't want with either the archive and/or exclude-files-from-classmaps configuration values, then use composer's archive command to create a zip. Upload the zip and expand on the server, or expand locally and transfer the now smaller package.
$ cat composer.json
...
{
"archive": {
"exclude": ["!vendor", "/test/*", "/*.jpg" ]
}
}
$ php composer.phar archive --format=zip --file=<filename-without-extension>
Those files matched by archive will not be present, at all, in your zip. Those files matched by exclude-files-from-classmaps will be present in the filesystem, but invisible to the autoloader.

I had a Same Situation Before few Days so i have created the console command to delete the unused files in the vendor Directory
Step :1
php artisan make:command CleanVendorFolderCommand
Step: 2
Copy the current code and paste int in the Command Class
<?php
namespace App\Console\Commands;
use Illuminate\Console\Command;
use RecursiveIteratorIterator;
use RecursiveDirectoryIterator;
use FilesystemIterator;
class CleanVendorFolderCommand extends Command
{
/**
* The name and signature of the console command.
*
* #var string
*/
protected $signature = 'clean:vendor {--o : Verbose Output} {--dry : Runs in dry mode without deleting files.}';
/**
* The console command description.
*
* #var string
*/
protected $description = 'Cleans up useless files from vendor folder.';
protected $patterns =
[
'test',
'tests',
'.github',
'README',
'CHANGELOG',
'FAQ',
'CONTRIBUTING',
'HISTORY',
'UPGRADING',
'UPGRADE',
'demo',
'example',
'examples',
'.doc',
'readme',
'changelog',
'composer',
'.git',
'.gitignore',
'*.md',
'.*.yml',
'*.yml',
'*.txt',
'*.dist',
'LICENSE',
'AUTHORS',
'.eslintrc',
'ChangeLog',
'.gitignore',
'.editorconfig',
'*.xml',
'.npmignore',
'.jshintrc',
'Makefile',
'.keep',
];
/**
* List of File and Folders Patters Going To Be Excluded
*
* #return void
*/
protected $excluded =
[
/**List of Folders*/
'src',
/**List of Files*/
'*.php',
'*.stub',
'*.js',
'*.json',
];
/**
* Create a new command instance.
*
* #return void
*/
public function __construct()
{
parent::__construct();
}
/**
* Execute the console command.
*
* #return mixed
*/
public function handle()
{
$patterns = array_diff($this->patterns, $this->excluded);
$directories = $this->expandTree(base_path('vendor'));
$isDry = $this->option('dry');
foreach ($directories as $directory)
{
foreach ($patterns as $pattern)
{
$casePattern = preg_replace_callback('/([a-z])/i', [$this, 'prepareWord'], $pattern);
$files = glob($directory . '/' . $casePattern, GLOB_BRACE);
if (!$files)
{
continue;
}
$files = array_diff($files, $this->excluded);
foreach ($this->excluded as $excluded)
{
$key = $this->arrayFind($excluded, $files);
if ($key !== false)
{
$this->warn('SKIPPED: ' . $files[$key]);
unset($files[$key]);
}
}
foreach ($files as $file)
{
if (is_dir($file))
{
$this->warn('DELETING DIR: ' . $file);
if (!$isDry)
{
$this->delTree($file);
}
} else
{
$this->warn('DELETING FILE: ' . $file);
if (!$isDry)
{
#unlink($file);
}
}
}
}
}
$this->warn('Folder Cleanup Done!');
}
/**
* Recursively traverses the directory tree
*
* #param string $dir
* #return array
*/
protected function expandTree($dir)
{
$directories = [];
$files = array_diff(scandir($dir), ['.', '..']);
foreach ($files as $file)
{
$directory = $dir . '/' . $file;
if (is_dir($directory))
{
$directories[] = $directory;
$directories = array_merge($directories, $this->expandTree($directory));
}
}
return $directories;
}
/**
* Recursively deletes the directory
*
* #param string $dir
* #return bool
*/
protected function delTree($dir) {
if (!file_exists($dir) || !is_dir($dir))
{
return false;
}
$iterator = new RecursiveIteratorIterator(new RecursiveDirectoryIterator($dir, FilesystemIterator::SKIP_DOTS), RecursiveIteratorIterator::CHILD_FIRST);
foreach ($iterator as $filename => $fileInfo)
{
if ($fileInfo->isDir())
{
#rmdir($filename);
} else {
#unlink($filename);
}
}
#rmdir($dir);
}
/**
* Prepare word
*
* #param string $matches
* #return string
*/
protected function prepareWord($matches)
{
return '[' . strtolower($matches[1]) . strtoupper($matches[1]) . ']';
}
protected function arrayFind($needle, array $haystack)
{
foreach ($haystack as $key => $value)
{
if (false !== stripos($value, $needle))
{
return $key;
}
}
return false;
}
protected function out($message)
{
if ($this->option('o') || $this->option('dry'))
{
echo $message . PHP_EOL;
}
}
}
TESTED ON
OS Name Microsoft Windows 10 Pro
Version 10.0.16299 Build 16299
Processor Intel(R) Core(TM) i3-7100U CPU # 2.40GHz, 2400 Mhz, 2 Core(s), 4 Logical Processor(s)
NOW THE TESTING PART
Before Size of vendor Folder
Size 57.0 MB (5,98,29,604 bytes)
Size on disk 75.2 MB (7,88,80,768 bytes)
Contains 12,455 Files, 2,294 Folders
Now Run the Command
php artisan clean:vendor
Size of vendor Folder after Running the command
Size 47.0 MB (4,93,51,781 bytes)
Size on disk 59.7 MB (6,26,76,992 bytes)
Contains 8,431 Files, 1,570 Folders
Hope it helps

Related

How to queue upload to s3 using Laravel?

I'm dispatching a job to queue my video file, the files are being stored on s3.
Everything is working except if I upload a video file for example that's 20mb, when I look in my bucket it says the file is 120b. So this makes me think that I'm uploading the path and filename as a string instead of the file object.
And for some reason, when I try getting the file using the Storage::get() or File::get() and dd the result, it shows a bunch or random and crazy characters.
It seems like I can only get these weird characters, or a string, I can't get the file object for some reason.
In my controller I'm also storing it in the public disk (I will delete the file later in my Jobs/UploadVideos.php file).
CandidateProfileController.php:
$candidateProfile = new CandidateProfile();
$candidateProfile->disk = config('site.upload_disk');
// Video One
if($file = $request->file('video_one')) {
$file_path = $file->getPathname();
$name = time() . $file->getClientOriginalName();
$name = preg_replace('/\s+/', '-', $name);
$file->storePubliclyAs('videos', $name, 'public');
$candidateProfile->video_one = $name;
}
if($candidateProfile->save()) {
// dispatch a job to handle the image manipulation
$this->dispatch(new UploadVideos($candidateProfile));
return response()->json($candidateProfile, 200);
} else {
return response()->json([
'message' => 'Some error occurred, please try again.',
'status' => 500
], 500);
}
Jobs/UploadVideos.php:
use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
protected $candidateprofile;
public $timeout = 120;
public $tries = 5;
/**
* Create a new job instance.
*
* #param CandidateProfile $candidateProfile
*/
public function __construct(CandidateProfile $candidateProfile)
{
$this->candidateprofile = $candidateProfile;
}
/**
* Execute the job.
*
* #return void
*/
public function handle()
{
$disk = $this->candidateprofile->disk;
$filename = $this->candidateprofile->video_one;
$original_file = storage_path() . '/videos/' . $filename;
try {
// Video One
Storage::disk($disk)
->put('videos/'.$filename, $original_file, 'public');
// Update the database record with successful flag
$this->candidateprofile->update([
'upload_successful' => true
]);
} catch(\Exception $e){
Log::error($e->getMessage());
}
}
File Storage docs
The 2nd parameter for put() should be the contents of the file not the path to the file. Also, unless you've updated the public disk in your config/filesystem.php, the video isn't going to be stored in storage_path() . '/videos/...'.
To get this to work you should just need to update your Job code:
$filename = 'videos/' . $this->candidateprofile->video_one;
Storage::disk($this->candidateprofile->disk)
->put($filename, Storage::disk('public')->get($filename), 'public');
$this->candidateprofile->update([
'upload_successful' => true,
]);
Also, wrapping your code in a try/catch will mean that the Job won't retry as it will technically never fail.

I get different random values for the same file

I am trying to store photo in DB, and I use the below way to generate a random file name and store it in the DB.
$path = $request->file('profile_photo')->store('public/profiles');
$profile = ltrim($path,"public/profiles/");
However, sometimes I get different values in
DB
and in my folder
I am using laravel 6.
ltrim(), rtrim(), trim() remove by character mask, not full string.
$profile = ltrim($path,"public/profiles/");
It means remove all "p", "u", "b", "l", "i", "c", "/", etc. from the left side of $path.
If you want to get filename without path, you could use basename() function.
$profile = basename($path);
Alright take a look at this src/Illuminate/Http/UploadedFile.php
/**
* Store the uploaded file on a filesystem disk.
*
* #param string $path
* #param array|string $options
* #return string|false
*/
public function store($path, $options = [])
{
return $this->storeAs($path, $this->hashName(), $this->parseOptions($options));
}
/**
* Store the uploaded file on a filesystem disk.
*
* #param string $path
* #param string $name
* #param array|string $options
* #return string|false
*/
public function storeAs($path, $name, $options = [])
{
$options = $this->parseOptions($options);
$disk = Arr::pull($options, 'disk');
return Container::getInstance()->make(FilesystemFactory::class)->disk($disk)->putFileAs(
$path, $this, $name, $options
);
}
You can do this to get ride of the collision and confusion
// cache the file
$file = $request->file('profile_photo');
// generate a new filename. getClientOriginalExtension() for the file extension
$filename = 'profile-photo-' . time() . '.' . $file->getClientOriginalExtension();
// save to public/photos as the new $filename
$path = $file->store('public/photos', $filename);
dd($path); // Check if you get the correct file path or not

codeigniter load external config file

I'm working with CodeIgniter and I’d like to load one or more config files located in an external folder, shared by different CI installation.
Is it possible?
I tried to extend the Loader Class, and call the new method:
$this -> load -> external_config(‘MY\EXTERNAL\PATH’);
Load is successful, but i can’t retrieve config items in my controller, because in MY_Loader class the core\Loader config property is not visible, and i can’t merge it with the new loaded values.
This is my code:
<?php if ( ! defined('BASEPATH')) exit('No direct script access allowed');
class MY_Loader extends CI_Loader {
/**
* List of all loaded config files
*
* #var array
*/
var $is_config_loaded = array();
/**
* List of all loaded config values
*
* #var array
*/
//var $ext_config = array();
function __construct(){
parent::__construct();
}
/**
* Loads an external config file
*
* #param string
* #param bool
* #param bool
* #return void
*/
public function external_config($file = '', $use_sections = FALSE, $fail_gracefully = FALSE)
{
$file = ($file == '') ? 'config' : str_replace('.php', '', $file);
$found = FALSE;
$loaded = FALSE;
$check_locations = defined('ENVIRONMENT')
? array(ENVIRONMENT.'/'.$file, $file)
: array($file);
foreach ($check_locations as $location)
{
$file_path = $file.'.php';
if (in_array($file_path, $this->is_config_loaded, TRUE))
{
$loaded = TRUE;
continue 2;
}
if (file_exists($file_path))
{
$found = TRUE;
break;
}
}
if ($found === FALSE)
{
if ($fail_gracefully === TRUE)
{
return FALSE;
}
show_error('The configuration file '.$file.'.php does not exist.');
}
else{
include($file_path);
if ( ! isset($config) OR ! is_array($config))
{
if ($fail_gracefully === TRUE)
{
return FALSE;
}
show_error('Your '.$file_path.' file does not appear to contain a valid configuration array.');
}
if ($use_sections === TRUE)
{
if (isset($this->ext_config[$file]))
{
$this->ext_config[$file] = array_merge($this->ext_config[$file], $config);
}
else
{
$this->ext_config[$file] = $config;
}
}
else
{
$this->ext_config = array_merge($this->ext_config, $config);
}
$this->is_config_loaded[] = $file_path;
unset($config);
$loaded = TRUE;
log_message('debug', 'Config file loaded: '.$file_path);
}
return TRUE;
}
}
I found in core\Config.php a property
var $_config_paths = array(APPPATH);
With the paths in wich look for config files.
How can i add paths to the array without chance the core classe code?
Any ideas?
Thank you very much!!!
CodeIgniter makes assumptions about the config file being loaded, its location, its file ext etc. You may load other files that is: from the same directory (support seems to be limited to this usage scenario).
However, you can (looking at the chapter https://ellislab.com/codeigniter/user-guide/libraries/config.html (Setting a Config Item)) iterate your file values and set them like so
(todo: add some checks if file exists, is readable, no decoding errors etcetera)
$myConfigValues = json_decode(file_get_contents($configKeyValuesFromOuterSpace));
foreach($myConfigValues as $key => $value){
$CI->config->set_item('item_name', 'item_value');
}
As for the exposure of the config object, in the loader you may anytime examine its public parts in more or less subtle manners, by using the code igniter singleton instance retrieval method:
$CI =& get_instance();
die(print_r($CI->config,true));
This will be a CI_Config object. You may also create a MY_Config extension.

Component Install: DB function reports no errors

When trying to install a component I am getting this error.
Component Install: DB function reports no errors
Error installing component
I am getting this error quite often on a test system when trying to install a component which uses SQL updates and causes first time when installing an error (even not SQL related, such a missing file from the manifest file).
Here are some steps on how to fix this, by manually uninstalling the component, as from the Extension manager the installation may / will fail.
Find the id of your extension (you may also find multiple entries)
SELECT *
FROM `#__extensions`
WHERE `name` LIKE '%myextensionname%'
LIMIT 0 , 30
Remove from #__schemas the entries for extension, where extension_id is the previous found id. Remove also any entries for non existing extensions:
Remove any assets for your extension:
SELECT *
FROM `#__assets`
WHERE `name` LIKE '%myextensionname%'
LIMIT 0 , 30
Remove any menu entries:
SELECT *
FROM #__menu
WHERE link LIKE '%myextensionname%'
LIMIT 0 , 30
Reinstall.
Not tested but this is the general idea I would use.
<?php
/**
* A JApplicationCli application built on the Joomla Platform
*
* To run this place it in the cli folder of your Joomla CMS installation (or adjust the references).
*
* #package Joomla.CleanupFailedInsall
* #copyright Copyright (C) 2013 Open Source Matters. All rights reserved.
* #license GNU General Public License version 2 or later; see LICENSE
*/
/*
* This application cleans up database leftovers from a failed install
*
* To run from the command line type
* php cleanupfailedinstall.php -e='extensionname'
*/
if (!defined('_JEXEC'))
{
// Initialize Joomla framework
define('_JEXEC', 1);
}
#ini_set('zend.ze1_compatibility_mode', '0');
error_reporting(E_ALL);
ini_set('display_errors', 1);
// Load system defines
if (file_exists(dirname(__DIR__) . '/defines.php'))
{
require_once dirname(__DIR__) . '/defines.php';
}
if (!defined('JPATH_BASE'))
{
define('JPATH_BASE', dirname(__DIR__));
}
if (!defined('_JDEFINES'))
{
require_once JPATH_BASE . '/includes/defines.php';
}
// Get the framework.
require_once JPATH_LIBRARIES . '/import.php';
// Get the framework.
require_once JPATH_LIBRARIES . '/import.legacy.php';
// Bootstrap the CMS libraries.
require_once JPATH_LIBRARIES . '/cms.php';
// Import the configuration.
require_once JPATH_CONFIGURATION . '/configuration.php';
// Uncomment this if you want to log
/*
// Include the JLog class.
jimport('joomla.log.log');
// Add the logger.
JLog::addLogger(
// Pass an array of configuration options
array(
// Set the name of the log file
'text_file' => 'test.log.php',
// (optional) you can change the directory
'text_file_path' => 'logs'
)
);
// start logging...
JLog::add('Starting to log');
*/
/**
* Cleanup Failed Install
*
* #package Joomla.Shell
*
* #since 1.0
*/
class CleanupFailedInstall extends JApplicationCli
{
public function __construct()
{
// Note, this will throw an exception if there is an error
// System configuration.
$config = new JConfig;
// Creating the database connection.
$this->db = JDatabase::getInstance(
array(
'driver' => $config->dbtype,
'host' => $config->host,
'user' => $config->user,
'password' => $config->password,
'database' => $config->db,
'prefix' => $config->dbprefix,
)
);
// Call the parent __construct method so it bootstraps the application class.
parent::__construct();
require_once JPATH_CONFIGURATION . '/configuration.php';
}
/**
* Entry point for the script
*
* #return void
*
* #since 1.0
*/
public function doExecute()
{
// Long args
$extensionname = $this->input->get('extensionname', null,'STRING');
// Short args
if (!$extensionname)
{
$extensionname = $this->input->get('e', null, 'STRING');
}
$extensionTable = new JTableExtension();
$extensionId = $extensionTable->find(array('name', $extensionname));
// This block taken from the platform component install adapter with minor moifications
// Remove the schema version
$query = $db->getQuery(true)
->delete('#__schemas')
->where('extension_id = ' . $extensionId);
$db->setQuery($query);
$db->execute();
// Remove the component container in the assets table.
$asset = JTable::getInstance('Asset');
if ($asset->loadByName($extensionname))
{
$asset->delete();
}
$extenstionTable->delete($extensionId);
$this->removeAdminMenus($extensionId);
// Remove categories for this component
$query->clear()
->delete('#__categories')
->where('extension=' . $db->quote($exensionname), 'OR')
->where('extension LIKE ' . $db->quote($extensionname . '.%'));
$db->setQuery($query);
$db->execute();
// Clobber any possible pending updates
$update = JTable::getInstance('update');
$uid = $update->find(array('element' => $row->element, 'type' => 'component', 'client_id' => 1, 'folder' => ''));
if ($uid)
{
$update->delete($uid);
}
}
/**
* Taken from the core installer component adapter
* Method to remove admin menu references to a component
*
* #param object &$row Component table object.
*
* #return boolean True if successful.
*
* #since 3.1
*/
protected function _removeAdminMenus($extensionId)
{
$db = JFactory::getDbo();
$table = JTable::getInstance('menu');
// Get the ids of the menu items
$query = $db->getQuery(true)
->select('id')
->from('#__menu')
->where($db->quoteName('client_id') . ' = 1')
->where($db->quoteName('component_id') . ' = ' . (int) $extensionId);
$db->setQuery($query);
$ids = $db->loadColumn();
// Check for error
if (!empty($ids))
{
// Iterate the items to delete each one.
foreach ($ids as $menuid)
{
if (!$table->delete((int) $menuid))
{
$this->setError($table->getError());
return false;
}
}
// Rebuild the whole tree
$table->rebuild();
}
return true;
}
}
JApplicationCli::getInstance('CleanupFailedInstall')->execute();
This an error / typo free version of what #Elin proposed (in case somebody wants to take this further). This solution did NOT work for me, but I think it has something in it.
<?php
/**
* A JApplicationCli application built on the Joomla Platform
*
* To run this place it in the cli folder of your Joomla CMS installation (or adjust the references).
*
* #package Joomla.CleanupFailedInsall
* #copyright Copyright (C) 2013 Open Source Matters. All rights reserved.
* #license GNU General Public License version 2 or later; see LICENSE
*/
/*
* This application cleans up database leftovers from a failed install
*
* To run from the command line type
* php cleanupfailedinstall.php -e='extensionname'
*/
if (!defined('_JEXEC'))
{
// Initialize Joomla framework
define('_JEXEC', 1);
}
#ini_set('zend.ze1_compatibility_mode', '0');
error_reporting(E_ALL);
ini_set('display_errors', 1);
// Load system defines
if (file_exists(dirname(__DIR__) . '/defines.php'))
{
require_once dirname(__DIR__) . '/defines.php';
}
if (!defined('JPATH_BASE'))
{
define('JPATH_BASE', dirname(__DIR__));
}
if (!defined('_JDEFINES'))
{
require_once JPATH_BASE . '/includes/defines.php';
}
// Get the framework.
require_once JPATH_LIBRARIES . '/import.php';
// Get the framework.
//require_once JPATH_LIBRARIES . '/import.legacy.php';
// Bootstrap the CMS libraries.
require_once JPATH_LIBRARIES . '/cms.php';
// Import the configuration.
require_once JPATH_CONFIGURATION . '/configuration.php';
// Uncomment this if you want to log
/*
// Include the JLog class.
jimport('joomla.log.log');
// Add the logger.
JLog::addLogger(
// Pass an array of configuration options
array(
// Set the name of the log file
'text_file' => 'test.log.php',
// (optional) you can change the directory
'text_file_path' => 'logs'
)
);
// start logging...
JLog::add('Starting to log');
*/
/**
* Cleanup Failed Install
*
* #package Joomla.Shell
*
* #since 1.0
*/
class CleanupFailedInstall extends JApplicationCli
{
public function __construct()
{
// Note, this will throw an exception if there is an error
// System configuration.
$config = new JConfig;
// Creating the database connection.
$this->db = JDatabase::getInstance(
array(
'driver' => $config->dbtype,
'host' => $config->host,
'user' => $config->user,
'password' => $config->password,
'database' => $config->db,
'prefix' => $config->dbprefix,
)
);
// Call the parent __construct method so it bootstraps the application class.
parent::__construct();
require_once JPATH_CONFIGURATION . '/configuration.php';
}
/**
* Entry point for the script
*
* #return void
*
* #since 1.0
*/
public function execute()
{
// Long args
$extensionname = $this->input->get('extensionname', 'urlaubsrechner', 'STRING');
// Short args
if (!$extensionname)
{
$extensionname = $this->input->get('e', null, 'STRING');
}
require_once JPATH_LIBRARIES . '/joomla/database/table/extension.php';
$extensionTable = new JTableExtension($this->db);
$extensionId = $extensionTable->find(array('name' => $extensionname));
if (! $extensionId)
{
throw new Exception('Could not find extension with name: ' . $extensionname);
}
// This block taken from the platform component install adapter with minor modifications
// Remove the schema version
$query = $this->db->getQuery(true)
->delete('#__schemas')
->where('extension_id = ' . $extensionId);
$this->db->setQuery($query);
$this->db->execute();
// Remove the component container in the assets table.
$asset = JTable::getInstance('Asset');
if ($asset->loadByName($extensionname))
{
$asset->delete();
}
$extensionTable->delete($extensionId);
$this->_removeAdminMenus($extensionId);
// Remove categories for this component
$query->clear()
->delete('#__categories')
->where('extension=' . $this->db->quote($extensionname), 'OR')
->where('extension LIKE ' . $this->db->quote($extensionname . '.%'));
$this->db->setQuery($query);
$this->db->execute();
// Clobber any possible pending updates
$update = JTable::getInstance('update');
$uid = $update->find(array('element' => $extensionTable->element, 'type' => 'component', 'client_id' => 1, 'folder' => ''));
if ($uid)
{
$update->delete($uid);
}
}
/**
* Taken from the core installer component adapter
* Method to remove admin menu references to a component
*
* #param object &$row Component table object.
*
* #return boolean True if successful.
*
* #since 3.1
*/
protected function _removeAdminMenus($extensionId)
{
$db = JFactory::getDbo();
$table = JTable::getInstance('menu');
// Get the ids of the menu items
$query = $db->getQuery(true)
->select('id')
->from('#__menu')
->where($db->quoteName('client_id') . ' = 1')
->where($db->quoteName('component_id') . ' = ' . (int) $extensionId);
$db->setQuery($query);
$ids = $db->loadColumn();
// Check for error
if (!empty($ids))
{
// Iterate the items to delete each one.
foreach ($ids as $menuid)
{
if (!$table->delete((int) $menuid))
{
$this->setError($table->getError());
return false;
}
}
// Rebuild the whole tree
$table->rebuild();
}
return true;
}
}
JApplicationCli::getInstance('CleanupFailedInstall')->execute();

fine-uploader PHP Server Side Merge

I'm been experimenting with Fine Uploader. I am really interested in the chunking and resume features, but I'm experiencing difficulties putting the files back together server side;
What I've found is that I have to allow for a blank file extension on the server side to allow the upload of the chunks, otherwise the upload will fail with unknown file type. It uploads the chunks fine with file names such as "blob" and "blob63" (no file extension) however is does not merge them back at completion of upload.
Any help or pointers would be appreciated.
$('#edit-file-uploader').fineUploader({
request: {
endpoint: 'upload.php'
},
multiple: false,
validation:{
allowedExtentions: ['stl', 'obj', '3ds', 'zpr', 'zip'],
sizeLimit: 104857600 // 100mb * 1024 (kb) * 1024 (bytes)
},
text: {
uploadButton: 'Select File'
},
autoUpload: false,
chunking: {
enabled: true
},
callbacks: {
onComplete: function(id, fileName, responseJSON) {
if (responseJSON.success) {
/** some code here **??
}
}
});
And this is the server side script (PHP):
// list of valid extensions, ex. array("stl", "xml", "bmp")
$allowedExtensions = array("stl", "");
// max file size in bytes
$sizeLimit = null;
$uploader = new qqFileUploader($allowedExtensions, $sizeLimit);
// Call handleUpload() with the name of the folder, relative to PHP's getcwd()
$result = $uploader->handleUpload('uploads/');
// to pass data through iframe you will need to encode all html tags
echo htmlspecialchars(json_encode($result), ENT_NOQUOTES);
/******************************************/
/**
* Handle file uploads via XMLHttpRequest
*/
class qqUploadedFileXhr {
/**
* Save the file to the specified path
* #return boolean TRUE on success
*/
public function save($path) {
$input = fopen("php://input", "r");
$temp = tmpfile();
$realSize = stream_copy_to_stream($input, $temp);
fclose($input);
if ($realSize != $this->getSize()){
return false;
}
$target = fopen($path, "w");
fseek($temp, 0, SEEK_SET);
stream_copy_to_stream($temp, $target);
fclose($target);
return true;
}
/**
* Get the original filename
* #return string filename
*/
public function getName() {
return $_GET['qqfile'];
}
/**
* Get the file size
* #return integer file-size in byte
*/
public function getSize() {
if (isset($_SERVER["CONTENT_LENGTH"])){
return (int)$_SERVER["CONTENT_LENGTH"];
} else {
throw new Exception('Getting content length is not supported.');
}
}
}
/**
* Handle file uploads via regular form post (uses the $_FILES array)
*/
class qqUploadedFileForm {
/**
* Save the file to the specified path
* #return boolean TRUE on success
*/
public function save($path) {
return move_uploaded_file($_FILES['qqfile']['tmp_name'], $path);
}
/**
* Get the original filename
* #return string filename
*/
public function getName() {
return $_FILES['qqfile']['name'];
}
/**
* Get the file size
* #return integer file-size in byte
*/
public function getSize() {
return $_FILES['qqfile']['size'];
}
}
/**
* Class that encapsulates the file-upload internals
*/
class qqFileUploader {
private $allowedExtensions;
private $sizeLimit;
private $file;
private $uploadName;
/**
* #param array $allowedExtensions; defaults to an empty array
* #param int $sizeLimit; defaults to the server's upload_max_filesize setting
*/
function __construct(array $allowedExtensions = null, $sizeLimit = null){
if($allowedExtensions===null) {
$allowedExtensions = array();
}
if($sizeLimit===null) {
$sizeLimit = $this->toBytes(ini_get('upload_max_filesize'));
}
$allowedExtensions = array_map("strtolower", $allowedExtensions);
$this->allowedExtensions = $allowedExtensions;
$this->sizeLimit = $sizeLimit;
$this->checkServerSettings();
if(!isset($_SERVER['CONTENT_TYPE'])) {
$this->file = false;
} else if (strpos(strtolower($_SERVER['CONTENT_TYPE']), 'multipart/') === 0) {
$this->file = new qqUploadedFileForm();
} else {
$this->file = new qqUploadedFileXhr();
}
}
/**
* Get the name of the uploaded file
* #return string
*/
public function getUploadName(){
if( isset( $this->uploadName ) )
return $this->uploadName;
}
/**
* Get the original filename
* #return string filename
*/
public function getName(){
if ($this->file)
return $this->file->getName();
}
/**
* Internal function that checks if server's may sizes match the
* object's maximum size for uploads
*/
private function checkServerSettings(){
$postSize = $this->toBytes(ini_get('post_max_size'));
$uploadSize = $this->toBytes(ini_get('upload_max_filesize'));
if ($postSize < $this->sizeLimit || $uploadSize < $this->sizeLimit){
$size = max(1, $this->sizeLimit / 1024 / 1024) . 'M';
die(json_encode(array('error'=>'increase post_max_size and upload_max_filesize to ' . $size)));
}
}
/**
* Convert a given size with units to bytes
* #param string $str
*/
private function toBytes($str){
$val = trim($str);
$last = strtolower($str[strlen($str)-1]);
switch($last) {
case 'g': $val *= 1024;
case 'm': $val *= 1024;
case 'k': $val *= 1024;
}
return $val;
}
/**
* Handle the uploaded file
* #param string $uploadDirectory
* #param string $replaceOldFile=true
* #returns array('success'=>true) or array('error'=>'error message')
*/
function handleUpload($uploadDirectory, $replaceOldFile = FALSE){
if (!is_writable($uploadDirectory)){
return array('error' => "Server error. Upload directory isn't writable.");
}
if (!$this->file){
return array('error' => 'No files were uploaded.');
}
$size = $this->file->getSize();
if ($size == 0) {
return array('error' => 'File is empty');
}
if ($size > $this->sizeLimit) {
return array('error' => 'File is too large');
}
$pathinfo = pathinfo($this->file->getName());
$filename = $pathinfo['filename'];
//$filename = md5(uniqid());
$ext = #$pathinfo['extension']; // hide notices if extension is empty
if($this->allowedExtensions && !in_array(strtolower($ext), $this->allowedExtensions)){
$these = implode(', ', $this->allowedExtensions);
return array('error' => 'File has an invalid extension, it should be one of '. $these . '.');
}
$ext = ($ext == '') ? $ext : '.' . $ext;
if(!$replaceOldFile){
/// don't overwrite previous files that were uploaded
while (file_exists($uploadDirectory . DIRECTORY_SEPARATOR . $filename . $ext)) {
$filename .= rand(10, 99);
}
}
$this->uploadName = $filename . $ext;
if ($this->file->save($uploadDirectory . DIRECTORY_SEPARATOR . $filename . $ext)){
return array('success'=>true);
} else {
return array('error'=> 'Could not save uploaded file.' .
'The upload was cancelled, or server error encountered');
}
}
}
In order to handle chunked requests, you MUST store each chunk separately in your filesystem.
How you name these chunks or where you store them is up to you, but I suggest you name them using the UUID provided by Fine Uploader and append the part number parameter included with each chunked request. After the last chunk has been sent, combine all chunks into one file, with the proper name, and return a standard success response as described in the Fine Uploader documentation. The original name of the file is, by default, passed in a qqfilename parameter with each request. This is also discussed in the docs and the blog.
It doesn't look like you've made any attempt to handle chunks server-side. There is a PHP example in the Widen/fine-uploader-server repo that you can use. Also, the documentation has a "server-side" section that explains how to handle chunking in detail. I'm guessing you did not read this. Have a look.) in the Widen/fine-uploader-server repo that you can use. Also, the documentation has a "server-side" section that explains how to handle chunking in detail. I'm guessing you did not read this. Have a look.
Note that, starting with Fine Uploader 3.8 (set to release VERY soon) you will be able to delegate all server-side upload handling to Amazon S3, as Fine Uploader will provide tight integration with S3 that sends all of your files directly to your bucket from the browser without you having to worry about constructing a policy document, making REST API calls, handling responses from S3, etc. I mention this as using S3 means that you never have to worry about handling things like chunked requests on your server again.

Resources