Hello, it is a very important project for me. I wrote an api and I want to pull data from 2 different sites with this api. While you can pull healthy data from one site, empty results are returned due to captcha from the other site. Now I scraped data from the same site before using pyhton selenium, bs4, but I can't scrape it with laravel api. I'm testing this info from postman.
The solution of the captcha is to combine my pyhton file, which I have provided, with my laravel api file, if possible, by converting the captcha to audio and then to text, which skips this part.
Another method is to write a pyhton service that can work with the captcha(pyhton) file.
The last and actually the most useful method for me if I can provide action is to skip captcha with my laravel service without entering the captcha event. I tried skipping by changing the cookie, but it was a very tiring and unhealthy method. I want to trick some kind of captcha here.
<?php
namespace App\Services\Mde;
use App\Core\ServiceResponse;
use App\Interfaces\IPricePredictionService;
class PricePredictionService extends BaseMDeService implements IPricePredictionService
{
/**
* #param mixed $brand
* #param mixed $model
* #param mixed $kilometerFrom
* #param mixed $kilometerTo
* #param mixed $yearFrom
* #param mixed $yearTo
* #param mixed $fuelTypes
* #param mixed $gearBoxes
* #param mixed $powerFrom
* #param mixed $powerTo
*
* #return ServiceResponse
*/
public function getByParameters(
$brand,
$model,
$kilometerFrom,
$kilometerTo,
$yearFrom,
$yearTo,
$fuelTypes,
$gearBoxes,
$powerFrom,
$powerTo
): ServiceResponse
{
$endpoint = $this->baseUrl . '&ms=' . $brand . '%3B' . $model;
$priceList = [];
for ($pageCounter = 1; $pageCounter <= 50; $pageCounter++) {
$parameters = [
'ml' => $kilometerFrom,
'ml%3A' => $kilometerTo, // duzelt
'fr' => $yearFrom, // duzelt
'fr%3A' => $yearTo, // duzelt
'fuel' => implode(' ', $fuelTypes), //ft=DIESEL
'gear' => implode(' ', $gearBoxes), //tr=MANUAL_GEAR
'powertype' => 'kw',
'pw' => $powerFrom, // duzelt
'pw%3A' => $powerTo, // duzelt
'page' => $pageCounter,
];
$response = $this->client->get($endpoint . '?' . http_build_query($parameters), [
'headers' => [
'Accept' => 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9',
//'Accept' => 'application/json',
'Accept-Encoding' => 'gzip, deflate, br',
'Accept-Language' => 'tr-TR,tr;q=0.9,en-US;q=0.8,en;q=0.7',
'User-Agent' => 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36' //,
],
]);
$clean1 = str_replace(["\n", "\t", "\r", " "], null, $response->getBody()->getContents());
$clean2 = str_replace(["""], null, $clean1);
$clean3 = preg_replace('~{(.*?)}~', null, $clean2);
$cleanResult = preg_replace('~{(.*?)}~', null, $clean3);
preg_match_all('~<a class="link--muted no--text--decoration result-item" (.*?)</a>~', $cleanResult, $articles);
//return $lists; // bos geliyor.
if ($articles[1]) {
foreach ($articles[1] as $article) {
preg_match('~<span class="h3 u-block" .*?>(.*?)</p>~', $article, $priceObject);
$priceWithCurrency = str_replace(',-', null, $priceObject[1]);
$priceWithoutCurrency = explode(' ', $priceWithCurrency)[1];
$price = intval(str_replace('.', null, $priceWithoutCurrency));
$priceList[] = $price;
}
}
}
$averagePrice = array_sum($priceList) / count($priceList);
return new ServiceResponse(
true,
'Average price calculated successfully.',
200,
intval($averagePrice)
);
}
}
I don't insist that these methods are just the ones that come to my mind, I want to proceed in this way, I just want to provide fast and practical transactions whenever possible. I'm sorry if I said something illogical and wrong. But I desperately need the ideas and guidance you will convey to me. Thank you from now. I'm also adding my api codes in case you want to take a look.
The controller class is below.
```
<?php
namespace App\Http\Controllers\Api\M;
use App\Core\Controller;
use App\Core\HttpResponse;
use App\Http\Requests\Api\M\PricePredictionController\CheckRequest;
use App\Interfaces\IPricePredictionService_M;
class PricePredictionController extends Controller
{
use HttpResponse;
/**
* #var $mService
*/
private $mService;
public function __construct(IPricePredictionService_M $mService)
{
$this->mService = $mService;
}
/**
* #param CheckRequest $request
*/
public function check(CheckRequest $request)
{
$response = $this->mService->getByParameters(
$request->brand_and_model,
$request->kilometerFrom_kilometerTo,
$request->yearFrom_yearTo,
$request->fuelTypes ?? [],
$request->gearBoxes ?? [],
$request->powerFrom_powerTo,
$request->country,
$request->bodyType,
$request->doors
);
return $this->httpResponse(
$response->getMessage(),
$response->getStatusCode(),
$response->getData(),
$response->isSuccess()
);
}
}
> The interfaces class is below.
```
<?php
namespace App\Interfaces;
use App\Core\ServiceResponse;
interface IPricePredictionService_M
{
/**
* #param mixed $brand_and_model
* #param $kilometerFrom_kilometerTo
* #param mixed $yearFrom_yearTo
* #param mixed $fuelTypes
* #param mixed $gearBoxes
* #param mixed $powerFrom_powerTo
* #param mixed $country
* #param mixed $bodyType
* #param mixed $doors
*
* #return ServiceResponse
*/
public function getByParameters(
$brand_and_model,
$kilometerFrom_kilometerTo,
$yearFrom_yearTo,
$fuelTypes,
$gearBoxes,
$powerFrom_powerTo,
$country,
$bodyType,
$doors
): ServiceResponse;
}
Related
Trying to run an API which will give me the updated temperature and humidity values but the curl function is not working as it gives a NULL reponse and throws error. running the from the terminal to test it
code:
class updateTempHumHourly extends Command
{
/**
* The name and signature of the console command.
*
* #var string
*/
protected $signature = 'update:temphum';
/**
* The console command description.
*
* #var string
*/
protected $description = 'Update temperature and humidity readings hourly';
/**
* Create a new command instance.
*
* #return void
*/
public function __construct()
{
parent::__construct();
}
/**
* Execute the console command.
*
* #return mixed
*/
public function handle()
{
$options = array(
'cluster' => 'ap2',
'useTLS' => true
);
$pusher = new \Pusher\Pusher(
'osdifnerwion4iownfowinf',
'dofhewofhewofnowenfid',
'7asdkland',
$options
);
$thinkers = t::where('user_id', '!=' , NULL)->where('thinker_status',1)->get();
foreach($thinkers as $t)
{
$temp = [
'action'=>'list_slave',
'mac'=> $t->thinker_MAC,
'type' => 'all',
'appkey' => 'nope'
];
json_encode($temp);
$response = Curl::to('http://103.31.82.46/open/open.php')
->withContentType('application/json')
->withData($temp)
->asJson(true)
->withHeader('Postman-Token: d5988618-676e-430c-808e-7e2f6cec88fc')
->withHeader('cache-control: no-cache')
->post();
foreach($response['slaves'] as $s)
{
if(array_key_exists("temp",$s) && array_key_exists("hum",$s))
{
$slave = sd::where("connected_thinker_MAC",$response['mac'])->where("device_id",$s['slave_id'])->first();
$slave->temperature = $s['temp'];
$slave->humidity = $s['hum'];
$slave->save();
$array = [];
$array['temperature'] = $s['temp'];
$array['humidity'] = $s['hum'];
$array['id'] = $s['slave_id'];
$pusher->trigger($t->user_id."-channel","s-event",$array);
\App\helper\log::instance()->tempHumLog($s);
}
}
}
}
}
the foreach loop throws an error that $response is equal to null. the curl function is not working from here but working fine regularly. help. i need to run this task every hour to get the average temperature and humidity.
Since you said that your code works somewhere that means that there is something wrong with your server and not the code itself probably (since it's a CURL issue).
Inside your php.ini file there is this line : extension=php_curl.dll to enable curl. Uncomment it on your server that you have the crons to make curl work. This should solve your problem.
Testing localy the api you provide us returns a 500 but if you say that it works for you i assume that this is just an example and the problem is in CURL itself.
I'm stuck I want to create a Laravel Nova Metrics Trend where it goes in the future instead of back in time.
This code will output: 15-March-2019, 14-March-2019, 13-March-2019,
public function calculate(Request $request)
{
return $this->sumByDays($request, Payment::class, 'hours_per_month', 'upcoming_invoice_date')->showLatestValue();
}
/**
* Get the ranges available for the metric.
*
* #return array
*/
public function ranges()
{
return [
3 => '3 Days',
];
}
I've already made some changes to Trend.php but it gave me nothing than errors.
/**
* Return a value result showing a sum aggregate over days.
*
* #param \Illuminate\Http\Request $request
* #param \Illuminate\Database\Eloquent\Builder|string $model
* #param string $column
* #param string $dateColumn
* #return \Laravel\Nova\Metrics\TrendResult
*/
public function sumByDays($request, $model, $column, $dateColumn = null)
{
return $this->aggregate($request, $model, Trend::BY_DAYS, 'sum', $column, $dateColumn);
}
Is this even possible?
Thanks,
The trend range is defined in Trend.php so you were on the right track!
Check out the protected function aggregate.
protected function aggregate($request, $model, $unit, $function, $column, $dateColumn = null)
{
The $endingDate can be changed to whatever you want!
$possibleDateResults = $this->getAllPossibleDateResults(
$startingDate = $this->getAggregateStartingDate($request, $unit),
$endingDate = Chronos::now();
The Chronos API has nice documentation but try something like this
$endingDate = Chronos::tomorrow()->addWeeks(4),
Note: I have done very minimal testing on this so use at your own risk. It does break the showLatestValue() method for trend metrics, but that could be addressed in TrendResult.php if you really needed it.
1) You can change the range dates to the future with a negative number (of days) as key:
public function ranges()
{
return [
-3 => 'Next 3 Days',
];
}
The problem is that it doesn't work because the order of datetimes will be wrong in the query as the endDate is always set to now() in Trend.php:
$startingDate = $this->getAggregateStartingDate($request, $unit),
$endingDate = Chronos::now(),
...
->whereBetween($dateColumn, [$startingDate, $endingDate])
2) Quick/dirty fix. In Trend.php change the whereBetween() second argument array (of datetimes) comparing and setting datetimes in ascending order.
->whereBetween(
$dateColumn,
[ min($startingDate, $endingDate), max($startingDate, $endingDate) ]
)
There is a complex form with a lot of nested fieldsets. Some fields need to be validated depending on field(-s) in another fieldset. So I cannot define all the validation rules directly in the getInputFilterSpecification() of the Fieldset, since there I cannot access other fieldsets (only the sub-fieldsets). The only way to do this is to extend the Form validation. Right? If so, how to do this?
MyForm extends Form
{
public function isValid()
{
$isFormValid = parent::isValid();
$isFieldXyzValid = // my additional validation logic
if(! $isFieldXyzValid) {
$fieldXyz->setMessages(...);
}
return $isFormValid && $isFieldXyzValid;
}
}
Like this? Or is a cleaner way to solve this problem?
I've already developed something similar in my previous project.
For doing this i used a service which take my form and set dynamic fieldset, and obviously custom validation rules.
In your controller, get your form (via the dependancy injection formManager (polyfilled or not).
$form = $this->formManager->get('{your form}');
Call your service and give it your form.
And in your service you can do anything you want like :
get your stuff (from DB or others) to determine wich fields are mandatory
Foreach on your form
Add or remove fieldset
Add or remove validationGroup fields
Add or remove filters
Add or remove validators
I performed those via (sample) in a foreach where $stuff is an element of doctrine collection
$nameFieldset = 'my_fieldset-'.$stuff->getId();
$globalValidator = new GlobalValidator();
$globalValidator->setGlobalValue($gloablValue);
$uoValidator = new UcValidator();
$uoValidator->setOptions(array(
'idToValidate' => $stuff->getId(),
'translator' => $this->translator
));
$globalValidator->setOptions(array(
'idToValidate' => $stuff->getId(),
'translator' => $this->translator
));
$name = 'qty-'.$stuff->getId();
$form = $this->setFilters($form, $name, $nameFieldset);
$globalValidator->setData($data);
$form = $this->setValidators($form, $name, $globalValidator, $uoValidator, $nameFieldset);
Where setFilters and setValidators are custom methods wich add filters and validator to my fields (also custom)
/**
* #param myForm $form
* #param $name
* #param string $nameFieldset
* #return myForm
*/
public function setFilters($form, $name, $nameFieldset)
{
$form->getInputFilter()->get('items')->get($nameFieldset)
->get($name)
->getFilterChain()
->getFilters()
->insert(new StripTags())
->insert(new StringTrim());
return $form;
}
/**
* #param myForm $form
* #param $name
* #param $globalValidator
* #param $uoValidator
* #param $nameFieldset
* #return myForm
*/
public function setValidators($form, $name, $globalValidator, $uoValidator, $nameFieldset)
{
$optionsSpace = [
'translator' => $this->translator,
'type' => NotEmpty::SPACE
];
$optionsString = [
'translator' => $this->translator,
'type' => NotEmpty::STRING
];
$optionsDigits = [
'translator' => $this->translator,
];
$form->getInputFilter()->get('items')
->get($nameFieldset)
->get($name)
->setRequired(true)
->getValidatorChain()
->attach($uoValidator, true, 1)
->attach($globalValidator, true, 1)
// We authorize zéro but not space nor strings
->attach(new NotEmpty($optionsSpace), true, 2)
->attach(new NotEmpty($optionsString), true, 2)
->attach(new Digits($optionsDigits), true, 2);
return $form;
}
I want to validate a form which holds a dropdown menu and a text input field.
The user can choose a project from the dropdown menu. If he wants to create a new project he can use the text input field next to the dropdown menu.
Here is my upload type:
public function buildForm(FormBuilderInterface $builder, array $options)
{
$builder
->setAction('upload')
->setMethod('POST')
// project name dropdown menu
->add('projectname', 'choice' , array(
'label' => 'upload_project_label',
'choices' => $this->projects,
'attr' => array(
'class' => 'form-control some',
'required' => 'true'
)
))
// newprojectname text input
->add('newprojectname', 'text', array(
'label' => false,
'attr' => array(
'class' => 'form-control',
'required' => false,
'placeholder' => 'upload_newprojectname_placeholder'
)
)
)
...
And this is a snippet from my upload entity:
/**
* #ORM\Column(type="text")
*
* #var string $projectname
* #Assert\NotBlank()
*/
protected $projectname;
/**
* #ORM\Column(type="text")
*
* #var string $newprojectname
* #Assert\Length(
* min = 3,
* max = 7,
* minMessage = "min message",
* maxMessage = "max message"
* )
*/
protected $newprojectname;
My question is is there a possibility of a query to check if the field newproject is set (i.e. a string is entered)? And if so let the Assert annotation do its job.
This can be done a number of ways, all of which will likely satisfy your requirements.
Use a custom callback - this is the quickest and most straightforward
Use an expression validator - a lot of people have issues with embedding a meta-language within PHP which is perfectly valid but this is another quick-ish way of doing things
Use group sequences, specifically the group sequence provider functionality
Which one you choose is up to you but callbacks are a quick and easy starting point that you can build on if your validation constraints become more complex.
This is the code block for advised solution as custom callback validation.
I had to add another function in my upload entity which looks like this:
use Symfony\Component\Validator\Context\ExecutionContextInterface;
/**
* Function holds custom validation for fields in import/upload form
* #Assert\Callback
* #param ExecutionContextInterface $context
*/
public function validate(ExecutionContextInterface $context)
{
// check new project name
$newProjectName = $this->getNewprojectname();
if(!empty($newProjectName)) {
// letters only
$pattern = '/[a-zA-Z]/';
if($this->isPatternWrong($newProjectName, $pattern)) {
$context
->buildViolation('Please use letters only.')
->atPath('newprojectname')
->addViolation();
}
// text max. 7 digits
$maxlength = 7;
if($this->isStringTooLong($newProjectName, $maxlength)) {
$context
->buildViolation('Max. length 7 digits.')
->atPath('newprojectname')
->addViolation();
}
}
}
private function isPatternWrong($string, $pattern)
{
$result = preg_match($pattern, $string);
if($result === 0) {
return true;
}
return false;
}
private function isStringTooLong($string, $length)
{
if(strlen($string) > $length) {
return true;
}
return false;
}
Hi programatically i add mail in magento. Default mail system works well. but my mail through this bug.
2013-10-08T05:06:17+00:00 ERR (3):
exception 'Exception' with message 'This letter cannot be sent.' in /home/domain/public_html/fashion/app/code/core/Mage/Core/Model/Email/Template.php:398
Stack trace:
#0 /home/mall140/public_html/fashion/app/code/local/seller/Marketplaceseller/controllers/SendmailController.php(18): Mage_Core_Model_Email_Template->send('amia.1901#...', 'victor', Array)
#1 /home/app/code/core/Mage/Core/Controller/Varien/Action.php(419): Webkul_Marketplacepartner_SendmailController->indexAction()
#2 /home/fashion/app/code/core/Mage/Core/Controller/Varien/Router/Standard.php(250): Mage_Core_Controller_Varien_Action->dispatch('index')
#3 /home/app/code/core/Mage/Core/Controller/Varien/Front.php(176): Mage_Core_Controller_Varien_Router_Standard->match(Object(Mage_Core_Controller_Request_Http))
#4 /home/app/code/core/Mage/Core/Model/App.php(354): Mage_Core_Controller_Varien_Front->dispatch()
#5 /home/app/Mage.php(683): Mage_Core_Model_App->run(Array)
#6 /home/index.php(87): Mage::run('', 'store')
This is error code. please help me
This exception means Mage_Core_Model_Email_Template::isValidForSend() returned false:
public function isValidForSend()
{
return !Mage::getStoreConfigFlag('system/smtp/disable')
&& $this->getSenderName()
&& $this->getSenderEmail()
&& $this->getTemplateSubject();
}
Without seeing the code setting up your email template before sending it's hard to say which of the four conditions fails.
My wild guess would be that SMTP mails are disabled in your backend.
Check whether System -> Configuration -> System -> Mail sending settings -> Disable Email Communications is set to "No".
I was also getting same error during calling sendTransactional() function. I was thinking that there is some issue in email template in subject. To find exact issue as mentioned by Jurgen Thalen in his answer I inspected Mage_Core_Model_Email_Template::isValidForSend() function like below
public function isValidForSend()
{
//remove this after checking, never change core files
Mage::log("Sender Name:".$this->getSenderName());
Mage::log("Sender email:".$this->getSenderEmail());
Mage::log("Template subject:".$this->getTemplateSubject());
Mage::log("Email disabled:".Mage::getStoreConfigFlag('system/smtp/disable'));
return !Mage::getStoreConfigFlag('system/smtp/disable')
&& $this->getSenderName()
&& $this->getSenderEmail()
&& $this->getTemplateSubject();
}
Now check your system.log for me template subject was coming fine but sender name and sender email was not coming.
Below was my code:
sendTransactional(
$template,
"myemail#gmail.com",
$recipient['email'],
$recipient['name'],
array(
'myvar1' => $myvar1,
'myvar2' => $myvar2,
'myvar3' =>$myvar3,
'myvar4' =>$myvar4
)
)
Below is send transaction function in core file:
/**
* Send transactional email to recipient
*
* #param int $templateId
* #param string|array $sender sneder informatio, can be declared as part of config path
* #param string $email recipient email
* #param string $name recipient name
* #param array $vars varianles which can be used in template
* #param int|null $storeId
* #return Mage_Core_Model_Email_Template
*/
public function sendTransactional($templateId, $sender, $email, $name, $vars=array(), $storeId=null)
As you can see second parameter $sender should be String or array, as I was passing string but was not accepting.
I changed my sendTransactional() like below: second parameter passing as array
sendTransactional(
$template,
array(
'email' => "myemail#gmail.com",
'name' => "satish sojitra",
),
$recipient['email'],
$recipient['name'],
array(
'myvar1' => $myvar1,
'myvar2' => $myvar2,
'myvar3' =>$myvar3,
'myvar4' =>$myvar4
)
)
It works like charm! Don't forget to remove Mage::log statements from core files.
Below is complete code snippet to send transactional email
$translate = Mage::getSingleton('core/translate');
//#var $translate Mage_Core_Model_Translate
$translate->setTranslateInline(false);
$mailTemplate = Mage::getModel('core/email_template');
//add bcc
$mailTemplate->addBcc("bccemail#gmail.com");//accept array also
//add cc: $Cc_email_ids can be array or string
$mailTemplate->getMail()->addCc($Cc_email_ids);//accept array also
/* #var $mailTemplate Mage_Core_Model_Email_Template */
//template path
$templateConfigPath = "configuration/product_settings/failure_template";
$template = Mage::getStoreConfig($templateConfigPath, Mage::app()->getStore()->getId());
$to = array(
array("email"=> "to_1#gmail.com","name"=>"to_name"),
array("email"=> "to_2#gmail.com","name"=>"to_name")
);
$sendTo = array();
foreach ($to as $recipient)
{
if (is_array($recipient))
{
$sendTo[] = $recipient;
}
else
{
$sendTo[] = array(
'email' => $recipient,
'name' => null,
);
}
}
$myvar1 = 0;
$myvar2 = 0;
$myvar3 = 0;
$myvar4 = "transactional email testing";
foreach ($sendTo as $recipient) {
$mailTemplate->setDesignConfig(array('area'=>'frontend', 'store'=>Mage::app()->getStore()->getId()))
->sendTransactional(
$template,
array(
'email' => "myemail#gmail.com",
'name' => "satish sojitra",
),
$recipient['email'],
$recipient['name'],
array(
'myvar1' => $myvar1,
'myvar2' => $myvar2,
'myvar3' =>$myvar3,
'myvar4' =>$myvar4
)
);
}
$translate->setTranslateInline(true);
I hope this will help someone.
If your code is like following
$emailTemplate->sendTransactional($emailTemplateId, $sender, $emailAddress, $name, $data, $store);
and Mage_Core_Model_Email_Template::isValidForSend() returned false. Then most probabbly you are not passing sender information correct.
Check app\code\core\Mage\Core\Model\Email\Template.php
/**
* Send transactional email to recipient
*
* #param int $templateId
* #param string|array $sender sender information, can be declared as part of config path
* #param string $email recipient email
* #param string $name recipient name
* #param array $vars variables which can be used in template
* #param int|null $storeId
*
* #throws Mage_Core_Exception
*
* #return Mage_Core_Model_Email_Template
*/
public function sendTransactional($templateId, $sender, $email, $name, $vars=array(), $storeId=null)
Pass sender data as an array
$sender = array(
'name' => $fromEmail,
'email' => $fromName
);
$emailTemplate->sendTransactional($emailTemplateId, $sender, $emailAddress, $fullName, $data, $store);
then after check if it works.