Custom chunk implementation - filepond

Question

Is it possible to customize the chunk configuration in Filepond such that the chunk information is provided to the upload server:
as query parameters instead of headers
with custom query parameter names instead of Upload-Length, Upload-Name, and Upload-Offset
I am trying to fit Filepond's chunk implementation to a third party upload endpoint that I don't have control over.
I have found the Advanced configuration where you provide a process function which I've played with a little bit to see what comes through the options param -- however that appears (I think) to make the chunking calculations my responsibility. My original thought was to manipulate the options.chunkServer.url to include the query params I need but I don't believe this processes individual chunks.
In case it makes a difference, this is being done in React using the react-filepond package.

Answer 1

I made and implementation in Laravel 6 using Traits and some "bad practices" (I didn't have time because ... release in prod) to join chunks into a file
Basically:
post to get unique id folder to storage
get chunks and join together
profit!
Here's the full code:
<?php
namespace App\Http\Traits\Upload;
use Closure;
use Faker\Factory as Faker;
use Illuminate\Http\Request;
use Illuminate\Http\UploadedFile;
use Illuminate\Support\Facades\Storage;
trait Uploadeable
{
public function uploadFileInStorage(Request $request, Closure $closure)
{
// get the nex offset for next chunk send
if (($request->isMethod('options') || $request->isMethod('head')) && $request->has('patch')) {
//get the temp dir
$dir = $request->patch . DIRECTORY_SEPARATOR;
// reead all chunks in directory
$patch = collect(Storage::files($dir))
->sortBy(function ($file) {
return Storage::lastModified($file);
});
// read offsets for calculate
$offsets = array();
$size = 0;
$last_offset = 0;
foreach ($patch as $filename) {
$size = Storage::size($filename);
list($dir, $offset) = explode('file.patch.', $filename, 2);
array_push($offsets, $offset);
if ($offset > 0 && !in_array($offset - $size, $offsets)) {
$last_offset = $offset - $size;
break;
}
// last offset is at least next offset
$last_offset = $offset + $size;
}
// return offset
return response($last_offset, 200)
->header('Upload-Offset', $last_offset);
}
// chunks
if ($request->isMethod('patch') && $request->has('patch')) {
// get the temp dir
$dir = $request->patch . DIRECTORY_SEPARATOR;
// read headers
$offset = $request->header('upload-offset');
$length = $request->header('upload-length');
// should be numeric values, else exit
if (!is_numeric($offset) || !is_numeric($length)) {
return response('', 400);
}
// get the file name
$name = $request->header('Upload-Name');
// sleep server for get a diference between file created to sort
usleep(500000);
// storage the chunk with name + offset
Storage::put($dir . 'file.patch.' . $offset, $request->getContent());
// calculate total size of patches
$size = 0;
$patch = Storage::files($dir);
foreach ($patch as $filename) {
$size += Storage::size($filename);
}
// make the final file
if ($size == $length) {
// read all chunks in directory
$files = collect(Storage::files($dir))
->sortBy(function ($file) {
return Storage::lastModified($file);
});
// create output file
//Log::info(storage_path('app'));
$new_file_name = $final_name = trim(storage_path('app') . DIRECTORY_SEPARATOR . $dir . $name);
$file_handle = fopen($new_file_name, 'w');
// write patches to file
foreach ($files as $filename) {
// get offset from filename
list($dir, $offset) = explode('.patch.', $filename, 2);
// read chunk
$patch_handle = fopen(storage_path('app') . DIRECTORY_SEPARATOR . trim($filename), 'r');
$patch_contents = fread($patch_handle, filesize(storage_path('app') . DIRECTORY_SEPARATOR . trim($filename)));
fclose($patch_handle);
// apply patch
fseek($file_handle, $offset);
fwrite($file_handle, $patch_contents);
}
// done with file
fclose($file_handle);
// file permission (prefered 0755)
chmod($final_name, 0777);
// remove patches
foreach ($patch as $filename) {
$new_file_name = storage_path('app') . DIRECTORY_SEPARATOR . trim($filename);
unlink($new_file_name);
}
// simple class (no time to explain)
$file = new UploadedFile(
$final_name,
basename($final_name),
mime_content_type($final_name),
filesize($final_name),
false
);
$dir = $request->patch . DIRECTORY_SEPARATOR;
$object = new \stdClass();
$object->full_path = (string)$file->getPathname();
$object->directory = (string)($dir);
$object->path = (string)($dir . basename($final_name));
$object->name = (string)$file->getClientOriginalName();
$object->mime_type = (string)$file->getClientMimeType();
$object->extension = (string)$file->getExtension();
$object->size = (string)$this->formatSizeUnits($file->getSize());
// exec closure
$closure($file, (object)$object, $request);
}
// response
return response()->json([
'message' => 'Archivo subido correctamente.',
'filename' => $name
], 200);
}
// get dir unique id folder temp
if ($request->isMethod('post')) {
$faker = Faker::create();
$unique_id = $faker->uuid . '-' . time();
$unique_folder_path = $unique_id;
// create directory
Storage::makeDirectory($unique_folder_path);
// permisos directorio
chmod(storage_path('app') . DIRECTORY_SEPARATOR . $unique_folder_path . DIRECTORY_SEPARATOR, 0777);
// response with folder
return response($unique_id, 200)
->header('Content-Type', 'text/plain');
}
}
private function formatSizeUnits($bytes)
{
if ($bytes >= 1073741824) {
$bytes = number_format($bytes / 1073741824, 2) . ' GB';
} elseif ($bytes >= 1048576) {
$bytes = number_format($bytes / 1048576, 2) . ' MB';
} elseif ($bytes >= 1024) {
$bytes = number_format($bytes / 1024, 2) . ' KB';
} elseif ($bytes > 1) {
$bytes = $bytes . ' bytes';
} elseif ($bytes == 1) {
$bytes = $bytes . ' byte';
} else {
$bytes = '0 bytes';
}
return $bytes;
}
}
´´´