I can't figure out how I should send scroll_id to ElasticSearch using Curl.
This is what I have tried so far, but it doesn't seem to work.
$url = "http://distribution.virk.dk/cvr-permanent/virksomhed/_search?scroll=2m&_scroll_id=".$_POST["scroll_id"];
$data = array(
"_scroll_id" => $_POST["scroll_id"],
"scroll_id" => $_POST["scroll_id"],
"size" => 10,
"_source" => array(
"Vrvirksomhed.cvrNummer",
"Vrvirksomhed.elektroniskPost",
"Vrvirksomhed.livsforloeb",
"Vrvirksomhed.hjemmeside",
"Vrvirksomhed.virksomhedMetadata.nyesteNavn.navn",
"Vrvirksomhed.hovedbranche",
"Vrvirksomhed.penheder",
"Vrvirksomhed.telefonnummer",
"Vrvirksomhed.virksomhedMetadata.nyesteBeliggenhedsadresse"
),
"query" => array (
"bool" => array (
"must_not" => array (
"exists" => array (
"field" => "Vrvirksomhed.livsforloeb.periode.gyldigTil"
)
)
)
)
);
The ElasticSearch returns the same 10 posts every time, so I think it doesn't get the scroll_id right.
Updated code after trying Val's suggestion. Using the setHosts I get a timeout after a long time. Leaving out the setHosts, I get the error saying that No alive nodes found in your cluster.
use Elasticsearch\ClientBuilder;
require 'vendor/autoload.php';
$username = "MY_USERNAME";
$password = "MY_PASSWORD";
$hosts = [
'host' => 'distribution.virk.dk',
'scheme' => 'http',
'path' => '/cvr-permanent',
'port' => '80',
'user' => $username,
'pass' => $password
];
$client = ClientBuilder::create()->setHosts($hosts)->build();
$params = [
'scroll' => '30s',
'size' => 50,
'type' => '/cvr-permanent/virksomhed',
'index' => 'virksomhed',
'body' => [
'query' => [
'match_all' => new \stdClass()
]
]
];
// Execute the search
// The response will contain the first batch of documents
// and a scroll_id
$response = $client->search($params);
// Now we loop until the scroll "cursors" are exhausted
while (isset($response['hits']['hits']) && count($response['hits']['hits']) > 0) {
// **
// Do your work here, on the $response['hits']['hits'] array
// **
// When done, get the new scroll_id
// You must always refresh your _scroll_id! It can change sometimes
$scroll_id = $response['_scroll_id'];
// Execute a Scroll request and repeat
$response = $client->scroll([
'body' => [
'scroll_id' => $scroll_id, //...using our previously obtained _scroll_id
'scroll' => '30s' // and the same timeout window
]
]);
}
There are two steps for using the scroll API.
In the first step you need to send the query and the duration of the scroll context.
In the second step, you don't need to send the query again, but only the scroll id you got from the previous scroll search.
You can find a full-blown example here
Related
i'm running graphql query through array in Laravel postAsyn. any solution for that. below i added graphql query.
$client = new Client([
// Base URI is used with relative requests
'base_uri' => $url,
// You can set any number of default request options.
'timeout' => 300.0,
]);
foreach($arr as $value){
$graphQLquery = '{'.
'"query": "query viewer {'.
'repositories(last: '. $value . ') {'.
'nodes {'.
'name'.
'id'.
'isPrivate'.
'nameWithOwner'.
'}'.
'}'.
'}",'.
'"variables": { "name": "'.$name.'", "id": "1", "isPrivate ": "True", "nameWithOwner": "VimDiesel"}'.
'}';'
}
$params = [
'query' => [
'access_token' => $token,
'query' => $query[0]
]
];
$promises = [
'image' => $client->postAsync('POST', 'graphql', json_encode($params))
];
$responses = Promise\Utils::unwrap($promises);
if anyone have solution for that please add. i need run this all queries concurrent processing. i'm using guzzelhttp.
Facing cURL Error: Operation timed out after 15001 milliseconds with 0 bytes received issues with Woocomerce API to create products.
I am using the Laravel package i.e https://github.com/Codexshaper/laravel-woocommerce
It was working fine and creating products but suddenly it stopped working and start throwing PHP errors.
Below are the method that I am using to create a book on Woocomerce from laravel Controller:
public function addProductToWC(Request $request)
{
set_time_limit(0);
$response = '';
if ($request->isMethod('post')){
if(!empty($request->get('book_id'))){
$book = Book::find($request->get('book_id'));
$coverImgPath = base_path('public/customize_book/'.Session::get('cover_image'));
if (file_exists($coverImgPath)) {
$imageurl = url('/public/customize_book/'.Session::get('cover_image'));
} else {
$imageurl = url('/images/'.$book->bookimage);
}
if(!empty($book->id)){
$data = [
'name' => $book->title,
'type' => 'simple',
'regular_price' => number_format($request->get('book_price')),
'description' => (!empty($book->description) ? $book->description :''),
'short_description' => 'Simple product short description.',
'categories' => [
[
'id' => 1
]
],
'images' => [
[
'src' => 'http://demo.woothemes.com/woocommerce/wp-content/uploads/sites/56/2013/06/T_2_front.jpg'
],
[
'src' => 'http://demo.woothemes.com/woocommerce/wp-content/uploads/sites/56/2013/06/T_2_back.jpg'
]
]
];
$product = Product::create($data);
if($product['id']){
$response = array('error' => false,'code' => '200', 'data' => array('product_id' => $product['id'], 'message' => 'Product created successfully.'));
}else{
$response = array('error' => true,'code' => '401', 'data' => array('product_id' => $product['id'], 'message' => 'Product syncing failed please try again later.'));
}
}else{
$response = array('error' => true,'code' => '401','message' => 'Invalid book detail please try again.');
}
}else{
$response = array('error' => true,'code' => '401','message' => 'Invalid book detail please try again.');
}
}else{
$response = array('error' => true,'code' => '401','message' => 'Invalid method please try again.');
}
// return response
return response()->json($response);
}
Looking at the composer.json at https://github.com/Codexshaper/laravel-woocommerce/blob/master/composer.json, I can see that they are using the woocommerce client "automattic/woocommerce": "^3.0" This defaults to a request timeout of 15 seconds, hence why set_time_limit(0); didn't fix the issue.
When using it directly you'd set the timeout in the options
$woocommerce = new Client(
env('MGF_WOOCOMMERCE_API_URL'), // Your store URL
env('MGF_WOOCOMMERCE_API_KEY'), // Your consumer key
env('MGF_WOOCOMMERCE_API_SECRET'), // Your consumer secret
[
'timeout' => 120, // SET TIMOUT HERE
'wp_api' => true, // Enable the WP REST API integration
'version' => 'wc/v3' // WooCommerce WP REST API version
]
);
Looking at the library source https://github.com/Codexshaper/laravel-woocommerce/blob/master/src/WooCommerceApi.php
$this->client = new Client(
config('woocommerce.store_url'),
config('woocommerce.consumer_key'),
config('woocommerce.consumer_secret'),
[
'version' => 'wc/'.config('woocommerce.api_version'),
'wp_api' => config('woocommerce.wp_api_integration'),
'verify_ssl' => config('woocommerce.verify_ssl'),
'query_string_auth' => config('woocommerce.query_string_auth'),
'timeout' => config('woocommerce.timeout'),
]
);
It looks like the timeout is coming from woocommerce.timeout in your Laravel config file.
I've been trying to work with Guzzle and learn my way around it, but I'm a bit confused about using a request in conjunction with empty or null values.
For example:
$response = $client->request('POST',
'https://www.testsite.com/coep/public/api/donations', [
'form_params' => [
'client' => [
'web_id' => NULL,
'name' => 'Test Name',
'address1' => '123 E 45th Avenue',
'address2' => 'Ste. 1',
'city' => 'Nowhere',
'state' => 'CO',
'zip' => '80002'
],
'contact' => [],
'donation' => [
'status_id' => 1,
'amount' => $donation->amount,
'balance' => $donation->balance,
'date' => $donation->date,
'group_id' => $group->id,
],
]
]);
After running a test, I found out that 'web_id' completely disappears from my request if set to NULL. My question is how do I ensure that it is kept around on the request to work with my conditionals?
At this point, if I dd the $request->client, all I get back is everything but the web_id. Thanks!
I ran into this issue yesterday and your question is very well ranked on Google. Shame that it has no answer.
The problem here is that form_params uses http_build_query() under the hood and as stated in this user contributed note, null params are not present in the function's output.
I suggest that you pass this information via a JSON body (by using json as key instead of form_params) or via multipart (by using multipart instead of form_params).
Note: those 2 keys are available as constants, respectively GuzzleHttp\RequestOptions::JSON and GuzzleHttp\RequestOptions::MULTIPART
Try to define anything like form_params, headers or base_uri before creating a client, so:
// set options, data, params BEFORE...
$settings = [
'base_uri' => 'api.test',
'headers' => [
'Accept' => 'application/json',
'Authorization' => 'ajustneedatokenheredontworry'
],
'form_params' => [
'cash' => $request->cash,
'reason' => 'I have a good soul'
]
];
// ...THEN create your client,
$client = new GuzzleClient($settings);
// ...and finally check your response.
$response = $client->request('POST', 'donations');
If you check $request->all() at the controller function that you are calling, you should see that were sent successfully.
For those using laravel, use this:
Http::asJson()
I am implementing an API that needs to receive data from my database. The data comes from my 'Products' table. I organized the products that I want to send through Guzzle.
public function post()
{
$product = Product::where('erp_status', '=', 1)->limit(5)->offset(0)->get();
$client = new Client([
// Base URI is used with relative requests
'base_uri' => 'http://httpbin.org',
// You can set any number of default request options.
'timeout' => 2.0,
]);
And right after I created a foreach to send the data through the URL.
foreach($product as $prod){
set_time_limit(0);
$r = $client->request('POST', 'https://api.mercadolibre.com/items?access_token=XXXXXXXXXXXXXXXXXXX', [
'json' => [
'title' => $prod->description->erp_name,
'category_id' => 'MLB46511',
'price' => 10,
'currency_id' => 'BRL',
'available_quantity' => 10,
'buying_mode' => 'buy_it_now',
'listing_type_id' => 'gold_special',
'condition' => 'new',
'description' => 'Teste',
'pictures' => [
['source' => $prod->image->erp_image]
]
]
]);
}
return view('home');
}
But every time I run. It returns the message:
cURL error 28: Operation timed out after 2012 milliseconds with 0 out of 0 bytes received (see http://curl.haxx.se/libcurl/c/libcurl-errors.html)
My variable $product have 5 thousand rows.
Does Guzzle support this amount? Is it possible to use the guzzle in this case or what is the pure PHP script?
Any suggestion?
This is my code :
$source_hosts = [
'http://azure-xxxxxxxxxxx.west-eu.azr.facetflow.io'
];
$source_client = \Elasticsearch\ClientBuilder::create()
->setHosts($source_hosts)->build();
$params = [
'index' => 'dir',
'type' => 'article',
'id' => '1',
'body' => ['id' => 1,'name' => "article"]
];
$response = $source_client->index($params);
print_r($response);
testing from localhost
i have the error bellow :
No alive nodes found in your cluster
Must disable sslCertificate :
$source_client = \Elasticsearch\ClientBuilder::create()
->setSSLVerification(false)
->setHosts($hosts)->build();
thanks every body