Laravel to zapier webhook: How to avoid resource exhaustion? - laravel

My project requires me to retrieve inbound & outbound mails, retrieve the recipient emails, cc emails, sender email, store the retrieved emails in the database, do some computation, and update or create a row in google sheets.
My steps to doing the task:
In zapier, I have create zaps that triggers when receiving, and sending mails.
In my laravel, I have a function that receives and stores the data in the database from zapier when the receive/sent zap triggers.
After storing all the data, the function will then return the email, email domains of the inserted data, which will then be passed to the computation function.
I also have a zap that updates the google sheet. (My api -> Webhooks by Zapier -> Google Sheet)
After the computation, the computed data, in a form of multidimensional array, loop through it, and call the Update sheet webhook endpoint in each iteration.
A sample code, let's do example for sending mails:
// Controller
public function sent(Request $request)
{
$recipients = explode(",", $request->recipients); // Need to explode since the recipients is being received as comma delimited string.
$ccs = explode(",", $request->ccs); // Need to explode since the cc emails is being received as comma delimited string.
ProcessMail::dispatch($recipients, $ccs);
}
// Service class responsible for doing the logic
public function processSent($recipients, $ccs)
{
$new_recipients = [];
$new_ccs = [];
foreach ($recipients as $recipient) {
$domain = explode('#', $recipient)[1];
$recipientModel = tap(Mail::updateOrCreate(
['email' => $recipient, 'domain' => $domain],
))->increment('count_column');
$new_recipients[] = [
'email' => $recipient,
'domain' => $domain,
'count_column' => $recipientModel->count_column,
];
}
// Same process for the cc emails
return array_merge($new_recipients, $new_ccs);
}
// ProcessMail Job
$recipients_and_ccs = $service->processSent($recipients, $ccs);
Http::pool(function (Pool $pool) use ($service, $recipients_and_ccs ) {
foreach ($recipients_and_ccs as $key => $rec_cc) {
$addtl_rows = $service->getAddtlRows($rec_cc['email'], $rec_cc['domain']);
// The purpose of this to append 5 additional key value pair in the array.
for ($i = 0; $i < 5; $i++) {
$rec_cc["addt_row_{$i}"] = isset($addt_rows[$i]) ? 'this' : 'that';
}
$pool->as("pool-{$key}")->withBody(json_encode($rec_cc), 'json')->post('ZAPIER_ENDPOINT');
return $pool;
}
});
Now, when I send a mail to 50 recipients, some of the task in the zap will stop with error saying: The app returned "Resource has been exhausted (e.g. check quota)."..
One solution that I can think of is to chunk the $recipients_and_ccs by desirable amount of rows, say 10 records per chunk.
But before I start with my solution. It would be really great to hear some more pro/optimized solution.

Related

How can I check email validation in cypress

I want to check validation for input element. Can I check if typing wrong or valid Email format in my input.
Like this.
cy.get('#email_signup').type(validateEmail())
var email = "";
var possible = "abcd#.gh";
for (var i = 0; i < 10; i++)
email += possible.charAt(Math.floor(Math.random() * possible.length));
return email;
}
cy.get('.nextBtn').click();
cy.get('.error-message').should('be.visible');
According to what you expect to do, you need two external functions to create emails and to get a valid state of emails. Then you need to loop the it hook throughout all the emails.
//Create Emails
//Did a small modification so that you can decied the email length you need
const emails = (val) => {
var email = "";
var possible = "abcd#.gh";
for (var i = 0; i < val; i++){
email += possible.charAt(Math.floor(Math.random() * possible.length));}
return email;
}
//validate emails
//I have used a general Regex here, use the regex you have used in your website insted
const validateEmail = (email) => {
var re = /^\w+([\.-]?\w+)*#\w+([\.-]?\w+)*(\.\w{2,3})+$/;
return re.test(email);
}
//Test Cases (I have added 10 loops so it will create 10 test cases)
//Change the test case count as much as you need
for (let index = 0; index < 10; index++) {
const TestEmail = emails(10)
const EmailState = validateEmail(TestEmail)
it("EmailTest -"+ TestEmail +" - " + EmailState,() => {
cy.get('#email_signup').type(TestEmail)
cy.get('.nextBtn').click();
if(!EmailState){
cy.get('.error-message').should('be.visible');
}else{
cy.get('.error-message').should('not.be.visible');
}
})
}
Your method of creating emails is awesome. But make sure you add a separate test to check specific and valid scenarios as random emails might not cover them all
This is how the tests are going to look like.
Note: As I have mentioned earlier. It's always better to know your test data. So Without a random generation. Try to have a list of valid, invalid email scenarios with true false conditions And loop through them.
First of all it's good to generate random emails, but the best way to do is have a set of emails in a array. (may be in a JSON) and loop through them and check for the email validity.
Eg:
{
"Email_1":"['robot#mail.com','valid']",
"Email_2":"['robotmail.com','InValid']",
}
Because then you know the email conditions you are testing. But if you want to go with a random email generation method. I totally agree with the Muditha Perera's answer. It works perfectly.

Laravel generate password and insert user in database by hand

I have few users with no email and password, and I would like to update users by adding email from freemail column, and setup password from random string.
This is my sample code:
public function updateUserProfil()
{
//find list user to be update(juste one to test)
$users = User::where('isfree', '1')->first();
//Random string generate
$motdepasse = str_random(6);
//find user
$updateUser= User::find($users->id);
//setup fields
$updateUser->email = $users->freemail;
$updateUser->remember_token = str_random(10);
$updateUser->motdepasse = $motdepasse;
$updateUser->password = bcrypt($motdepasse);
$updateUser->isfree = 0;
$updateUser->save();
}
The problem is that when I try to connect with email($users->freemail) and password($motdepasse), which is not encrypted random string, I get error that:
my credential is not valid
what did I miss ?
You can use update() to update rows. So your code must be
public function updateUserProfil()
{
//find list user to be update(juste one to test)
$users = User::where('isfree', '1')->first();
//Random string generate
$motdepasse = str_random(6);
//find user
$updateUser= User::find($users->id);
//setup fields
$updateUser->update([
'email'=>$users->freemail,
'remember_token'=>str_random(10),
'motdepasse'=>$motdepasse,
'password'=>bcrypt($motdepasse),
'isfree'=>0,
]);
}

jquery datatable with ajax based pagination

I have javascript function that populates datatable using Ajax. My javascript code looks like :
$('#results').dataTable({
// Ajax load data
"ajax": {
"url": "get_intl_tickets",
"type": "POST",
"data": {
"user_id": 451,
"csrfmiddlewaretoken" : csrftoken,
}
}
})
My server side script in django has a function that loads around 500 data rows. Now the problem is that I don't want to load whole data at a time. Instead I want to have first 10 data rows. Then with pagination, another 10 rows like that.
I read the page server side processing documentation of datatables. I tried with "serverSide": true option as well. I am not understanding server side script. There is given an example of PHP. It seems that they are not using any parameters like draw, recordsFiltered, recordsTotal there. There they have used php SSP class. And it is unknown what does it do. I am trying to implement it in django.
But I am not finding proper good documentation to implement. Any help will be appreciated.
Old question but one I also had a surprisingly difficult time finding an answer to, so in case anyone else ends up here... :P
I found this 2020 article very helpful, specifically part 6 showing the "complete code" that includes getting the correct variables, building the SQL query, and how to build/structure the data object that it responds with:
https://makitweb.com/datatables-ajax-pagination-with-search-and-sort-php/
Their example posted below:
<?php
## Database configuration
include 'config.php';
## Read value
$draw = $_POST['draw'];
$row = $_POST['start'];
$rowperpage = $_POST['length']; // Rows display per page
$columnIndex = $_POST['order'][0]['column']; // Column index
$columnName = $_POST['columns'][$columnIndex]['data']; // Column name
$columnSortOrder = $_POST['order'][0]['dir']; // asc or desc
$searchValue = mysqli_real_escape_string($con,$_POST['search']['value']); // Search value
## Search
$searchQuery = " ";
if($searchValue != ''){
$searchQuery = " and (emp_name like '%".$searchValue."%' or
email like '%".$searchValue."%' or
city like'%".$searchValue."%' ) ";
}
## Total number of records without filtering
$sel = mysqli_query($con,"select count(*) as allcount from employee");
$records = mysqli_fetch_assoc($sel);
$totalRecords = $records['allcount'];
## Total number of record with filtering
$sel = mysqli_query($con,"select count(*) as allcount from employee WHERE 1 ".$searchQuery);
$records = mysqli_fetch_assoc($sel);
$totalRecordwithFilter = $records['allcount'];
## Fetch records
$empQuery = "select * from employee WHERE 1 ".$searchQuery." order by ".$columnName." ".$columnSortOrder." limit ".$row.",".$rowperpage;
$empRecords = mysqli_query($con, $empQuery);
$data = array();
while ($row = mysqli_fetch_assoc($empRecords)) {
$data[] = array(
"emp_name"=>$row['emp_name'],
"email"=>$row['email'],
"gender"=>$row['gender'],
"salary"=>$row['salary'],
"city"=>$row['city']
);
}
## Response
$response = array(
"draw" => intval($draw),
"iTotalRecords" => $totalRecords,
"iTotalDisplayRecords" => $totalRecordwithFilter,
"aaData" => $data
);
echo json_encode($response);
Nice exemple:
https://datatables.net/examples/server_side/defer_loading.html
But you need edit server side.
Response demo
{
draw:2,
recordsFiltered:57,
recordsTotal:57
}

Observables and fetching paged data?

I need to create an observable, which I can "pull" data from, to work with a pageable api. I can only fetch 100 items per request, I want to be able to use observable as a generator function (on which I can call .next() to issue a request to get next 100 items.
I can't unfortunately find a way to do it with Rx. I suppose it's possible using controlled observable or a subject. Can you guys show me an example.
this is what I've gotten so far:
function list(entityType, viewName, fetchAll = false) {
var skip = 0,
total = 0;
const subject = new Rx.Subject(),
response$ = subject
.takeWhile(() => skip <= total)
.startWith(skip)
.flatMap((skip) => fetchPagePromise(skip)),
next = () => subject.onNext(skip);
if (fetchAll) {
Rx.Observable.timer(100, 100).subscribe(() => next());
}
return {
data$: response$.map(response => response),
next: fetchAll === true ? undefined : next
};
function fetchPagePromise() {
let limit = 100,
obj = {
viewName, limit, skip
},
qs = objectToQueryString(obj);
return $http.get(`${apiBase}/api/data/${entityType}${qs}`).then((res) => {
total = res.data.Total;
skip += limit;
return res.data.Rows;
});
}
}
this kinda works like a generator. it returns an Observable and next handler. Whenever next is called it pulls next 100 items from api and pushes into the Observable. Also if there’s a third parameter fetchAll passed, then it will keep fetching data until there’s no more. What scares me though that there are 2 mutating vars in function's closure - skip and total, and I don't know if managing them like this in asynchronous/unpredictable environment is ok.
One of the things you generally want to avoid is trying to make Rx into a plain old event emitter. Usually it is an indicator when you try and just trigger Observables manually by passing around a Subjects observer interface.
You should ask yourself, where is my data coming from? What calls next(), what calls that, etc. After enough of these you will generally find that this will lead you to something that can be wrapped by an Observable directly rather than explicitly calling next(). Also, I think the fetchAll flag should really be kept externally. You are only making the interface confusing by essentially turning it into a void method just by passing in a flag.
So I would recommend refactoring like so:
Rx.Observable.prototype.lazyRequest = function(entityType, viewName, limit = 100) {
var source = this;
return Rx.Observable.create(obs => {
var response = source
//Skip is really just the (limit * index)
.map((x, i) => i * limit)
.flatMap((skip) => {
let obj = {viewName, skip, limit},
qs = objectToQueryString(obj);
//Handle promises implicitly
return $http.get(`${apiBase}/api/data/${entityType}${qs}`);
},
//Return this with our skip information
(skip, res) => {skip, res})
//Publish it so the stream get shared.
.publish();
//This will emit once once you are out of data
var stop = response.first(x => x.skip >= x.res.data.Total);
return new CompositeDisposable(
//Complete this stream when stop emits
response.takeUntil(stop)
//Downstream only cares about the data rows
.map(x => x.res.data.Rows)
.subscribe(obs),
//Hook everything up
response.connect());
});
}
Then you can use it like so:
//An example of a "starting point", a button click
//Update the rows every time a new event comes through
Rx.Observable.fromEvent($button, 'click')
.startWith(0) //Inject some data into the pipeline
.lazyRequest(entityType, viewName)
.subscribe(/*Do something with the returned rows*/);
//Get all of the rows, will keep hitting the endpoint until it completes
Rx.Observable.interval(100)
.lazyRequest(entityType, viewName)
//Gather all the values into an array and emit that.
.toArray()
.subscribe();

Subscription start date is required (recurring)

i am trying to do Recurring Payments on my site. but i gettin a problem here.
its is giving ack=Fail
[TIMESTAMP] => 2012-06-21T04:36:01Z
[CORRELATIONID] => fe0a920b9dee5
[ACK] => Failure
[VERSION] => 65.1
[BUILD] => 3067390
[L_ERRORCODE0] => 11549
[L_SHORTMESSAGE0] => Start Date is required
[L_LONGMESSAGE0] => Subscription start date is required
[L_SEVERITYCODE0] => Error
while my code for the reoccuring which i am passing is
function DoRecuringPayment($paymentInfo=array()){
/**
* Get required parameters from the web form for the request
*/
$paymentType =urlencode('Sale');
$firstName =urlencode($paymentInfo['Member']['first_name']);
$lastName =urlencode($paymentInfo['Member']['last_name']);
$creditCardType =urlencode($paymentInfo['CreditCard']['credit_type']);
$creditCardNumber = urlencode($paymentInfo['CreditCard']['card_number']);
$expDateMonth =urlencode($paymentInfo['CreditCard']['expiration_month']);
$padDateMonth = str_pad($expDateMonth, 2, '0', STR_PAD_LEFT);
$expDateYear =urlencode($paymentInfo['CreditCard']['expiration_year']);
$cvv2Number = urlencode($paymentInfo['CreditCard']['cv_code']);
$address1 = urlencode($paymentInfo['Member']['billing_address']);
$address2 = urlencode($paymentInfo['Member']['billing_address2']);
$country = urlencode($paymentInfo['Member']['billing_country']);
$city = urlencode($paymentInfo['Member']['billing_city']);
$state =urlencode($paymentInfo['Member']['billing_state']);
$zip = urlencode($paymentInfo['Member']['billing_zip']);
$bperiod=urlencode( $paymentInfo['Bill']['month']); ;
$bfrequency=urlencode($paymentInfo['bill']['frequency']);
$amount = urlencode($paymentInfo['Order']['theTotal']);
$currencyCode="USD";
$time22= date('Y-m-d',time());
$startdate=urlencode(date('Y-m-d',time()));
$paymentType=urlencode('Sale');
$ip=$_SERVER['REMOTE_ADDR'];
/* Construct the request string that will be sent to PayPal.
The variable $nvpstr contains all the variables and is a
name value pair string with & as a delimiter */
$nvpstr="&AMT=$amount&CREDITCARDTYPE=$creditCardType&ACCT=$creditCardNumber&EXPDATE=".$padDateMonth.$expDateYear."&CVV2=$cvv2Number&FIRSTNAME=$firstName&LASTNAME=$lastName&STREET=$address1&CITY=$city&STATE=$state".
"&ZIP=$zip&COUNTRYCODE=US&CURRENCYCODE=$currencyCode&PROFILESTARTDATE=$startdate&BILLINGPERIOD=$bperiod&BILLINGFREQUENCY=$bfrequency";
/* Make the API call to PayPal, using API signature.
The API response is stored in an associative array called $resArray */
$resArray=$this->hash_call("CreateRecurringPaymentsProfile",$nvpstr);
/* Display the API response back to the browser.
If the response from PayPal was a success, display the response parameters'
If the response was an error, display the errors received using APIError.php.
*/
return $resArray;
//Contains 'TRANSACTIONID,AMT,AVSCODE,CVV2MATCH, Or Error Codes'
}
I geting the error Subscription start date is required.

Resources