Generating Drupal image derivatives via a curl call - image

Question

We have a content type that uses a number of image styles to re-purpose images for a variety of different sections of our website, and have a large number of derivatives that need to be generated.
I want to use a script to pre-generate the necessary image derivatives before we go live after a major upgrade.
My thought was to write a script that uses Curl to call the URLs for which image derivatives will be created.
If in a browser I go to a specific URL that will cause generation of a derivative the image gets generated as expected. This is default Drupal behavior.
However, if I make a call to Curl on the command line for another URL that will cause generation of a derivative, the image does not get generated as expected.
I suspect it's because Curl is not actually downloading images. I also tried with Lynx and the result was the same.
Can anyone advise if there is a way to force Curl or Lynx to automatically download images so that the derivatives get created?
Thanks,
Pablo

Answer 1

you want to download all <img src="url" /> 's ?
easy, parse out the src attributes with DOMDocument and make an individual curl request for each image, kinda like this:
function downloadAllImagesFromUrl(string $url):int{
$imagesDownloaded=0;
$ch=curl_init();
if(!curl_setopt_array($ch,array(
CURLOPT_AUTOREFERER => true,
CURLOPT_BINARYTRANSFER => true,
CURLOPT_FOLLOWLOCATION => true,
CURLOPT_HTTPGET => true,
CURLOPT_SSL_VERIFYPEER => false,
CURLOPT_CONNECTTIMEOUT => 4,
CURLOPT_TIMEOUT => 8,
CURLOPT_COOKIEFILE => "", // <<makes curl save/load cookies across requests..
CURLOPT_ENCODING => "", // << makes curl post all supported encodings, gzip/deflate/etc, makes transfers faster
CURLOPT_URL=>$url,
CURLOPT_RETURNTRANSFER=>true
))){
throw new Exception(curl_error($ch));
}
$html=curl_exec($ch);
$domd=#DOMDocument::loadHTML($html);
foreach($domd->getElementsByTagName("img") as $img){
$src=$img->getAttribute("src");
if(!$src){
continue;
}
//Warning: you might want to parse_url PHP_URL_HOST / PHP_URL_PORT / PHP_URL_PATH
// if the urls are not absolute but relative.
curl_setopt($ch,CURLOPT_URL,$src);
curl_exec($ch);
++$imagesDownloaded;
}
curl_close($ch);
return $imagesDownloaded;
}
It is probably much faster to use get_headers() instead of curl_exec, but since PHP by default use ignore_user_abort, drupal may abort image generation if you dont actually download them but only get their headers. warning, the code above assumes all image src's are absolute. you need additional coding with parse_url & PHP_URL_HOST / PHP_URL_PORT / PHP_URL_PATH if you want to handle relative urls.. and note: this can be made much faster by using multithreading with curl_multi interface, but that requires much more complex coding..