Magento - Customer import slow - looking to optimise script - magento

I have a cron that imports/updates 3000 customers everyday. This however takes approx 5 hours... I need to do this everyday so I'm trying to optimise my code for speed.
This is my current code:
// ~3000 customers
foreach($results as $customerRecord) {
// customer general data:
$customer = $customerModel;
$customer->setWebsiteId($websiteId);
$email = $customerRecord['cc_email'];
$password = $email; // set password same as email
$customer->loadByEmail($email);
$customer->setEmail($email);
$nameAr = explode(" ", $customerRecord['cc_name']);
$customer->setFirstname($nameAr[0]);
$customer->setLastname($nameAr[1]);
$customer->setPassword($password);
// custom attributes
$customer->setCustomerFromOw("Yes");
$customer->setCdOwAccount($customerRecord['cd_ow_account']);
$customer->setDiscountStructureName($customerRecord['discount_structure_name']);
$customer->setPriceListName($customerRecord['price_list_name']);
$customer->setCcdCreditLimit($customerRecord['ccd_credit_limit']);
$customer->setCcdtBalance($customerRecord['ccdt_balance']);
$customer->setAdditionalPriceLists($customerRecord['additional_price_lists']);
$customer->setOwDeliveryRates($customerRecord['ow_delivery_rates']);
// Netzarbeiter CustomerActivation - ensure set to true
$customer->setCustomerActivated(true);
try {
$customer->setConfirmation(null);
// Netzarbeiter CustomerActivation - ensure set to true
$customer->setCustomerActivated(true);
$customer->save();
}
catch (Exception $ex) {
Zend_Debug::dump($ex->getMessage());
Mage::log('FAIL: Customer with email '.$email.' - '.$ex->getMessage(), null, 'sulman.log');
}
// customer delivery address:
$_delivery_address = array (
'firstname' => $nameAr[0],
'lastname' => $nameAr[1],
'street' => array (
'0' => $customerRecord['cd_invoice_address1'],
'1' => $customerRecord['cd_invoice_address2']." ".$customerRecord['cd_invoice_address3'],
),
'city' => $customerRecord['cd_invoice_town'],
'region_id' => '',
'region' => $customerRecord['cd_invoice_county'],
'postcode' => $customerRecord['cd_invoice_postcode'],
'country_id' => $customerRecord['cd_invoice_country_code'],
'telephone' => $customerRecord['cd_invoice_telephone'],
'company'=> $customerRecord['cd_invoice_name'],
);
$customAddress = Mage::getModel('customer/address');
$customAddress->setData($_delivery_address)
->setCustomerId($customer->getId())
->setIsDefaultBilling('1')
->setIsDefaultShipping('1')
->setSaveInAddressBook('1');
try {
$customAddress->save();
}
catch (Exception $ex) {
Zend_Debug::dump($ex->getMessage());
}
}
I really can't see any other way of doing this - but if anyone can see a way to optimise this I would appreciate it!
Thanks

Use AvS_FastSimpleImport module for import customer fast below is like for module.
https://github.com/avstudnitz/AvS_FastSimpleImport
You need to pass data in array format for import customers.
require_once 'src/app/Mage.php';
umask(0);
Mage::app()->setCurrentStore(Mage_Core_Model_App::ADMIN_STORE_ID);
$data[] = array('firstname' => "test",
.....
);
try {
/** #var $import AvS_FastSimpleImport_Model_Import */
$import = Mage::getModel('fastsimpleimport/import');
$import->processCustomerImport($data);
} catch (Exception $e) {
print_r($import->getErrorMessages());
}

Related

Payment failed but order is Placed in Laravel with Stripe

I have integrated stripe payment in flutter app but After payment got failed even then order placed in Laravel database so please check, what I have done wrong.
Please check at save method, May be I am wrong and can't validate purchase response.
payment controller
public function makePayment(Request $request)
{
try{
$data = $request->input('cartItems');
$cartItems = json_decode($data, true);
$orderData = $request->input('order');
$selectPaymentOption = json_decode($orderData, true);
$totalAmount = 0.0;
foreach ($cartItems as $cartItem){
$order = new Order();
$order->order_date = Carbon::now()->toDateString();
$order->product_id = $cartItem['productId'];
$order->payment_type = $selectPaymentOption['paymentType'];
$order->user_id = $request->input('userId');
$order->quantity = $cartItem['productQuantity'];
$order->amount = ($cartItem['productPrice'] - $cartItem['productDiscount']);
$totalAmount+= $order->amount * $order->quantity;
$order->save();
}
if($selectPaymentOption['paymentType'] == 'Card'){
\Stripe\Stripe::setApiKey('sk_test_hJUgYYzeXtitxxxx71lK8nE00MELJJS8c');
$token = \Stripe\Token::create([
'card' => [
'number' => $request->input('cardNumber'),
'exp_month' => $request->input('expiryMonth'),
'exp_year' => $request->input('expiryYear'),
'cvc' => $request->input('cvcNumber')
]
]);
$charge = \Stripe\Charge::create([
'amount' => $totalAmount * 100,
'currency' => 'inr',
'source' => $token,
'receipt_email' => $request->input('email'),
]);
}
return response(['result' => true]);
} catch (\Exception $exception){
return response(['result' => $exception]);
}
}
and my Flutter's Post request is here.
I want to POST _makePayment method after complete payment successful.
void _makePayment(BuildContext context, Payment payment) async {
PaymentService _paymentService = PaymentService();
var paymentData = await _paymentService.makePayment(payment);
var result = json.decode(paymentData.body);
print(paymentData);
CartService _cartService = CartService();
this.widget.cartItems!.forEach((cartItem) {
_cartService.makeTheCartEmpty();
});
if (result['result'] == true) {
_showPaymentSuccessMessage(context);
Timer(Duration(seconds: 4), () {
Navigator.pop(context);
Navigator.push(
context, MaterialPageRoute(builder: (context) => HomeScreen()));
});
}
}
Referring to my comment above, this is the rough solution I suggested in your controller you have to switch the logic
public function makePayment(Request $request)
{
try{
$data = $request->input('cartItems');
$cartItems = json_decode($data, true);
$orderData = $request->input('order');
$selectPaymentOption = json_decode($orderData, true);
##Change your frontend logic to pass total amount as variable
$totalAmount = $request->totalAmount;
if($selectPaymentOption['paymentType'] == 'Card'){
##Never have any sk or pk in your controller, switch this to config('common.sk_test')
\Stripe\Stripe::setApiKey(config('common.sk_test'));
$token = \Stripe\Token::create([
'card' => [
'number' => $request->input('cardNumber'),
'exp_month' => $request->input('expiryMonth'),
'exp_year' => $request->input('expiryYear'),
'cvc' => $request->input('cvcNumber')
]
]);
$charge = \Stripe\Charge::create([
'amount' => $totalAmount * 100,
'currency' => 'inr',
'source' => $token,
'receipt_email' => $request->input('email'),
]);
}
##After the stripe transaction is finished you can foreach your cart and do what you need to your database
foreach ($cartItems as $cartItem){
$order = new Order();
$order->order_date = Carbon::now()->toDateString();
$order->product_id = $cartItem['productId'];
$order->payment_type = $selectPaymentOption['paymentType'];
$order->user_id = $request->input('userId');
$order->quantity = $cartItem['productQuantity'];
$order->amount = ($cartItem['productPrice'] - $cartItem['productDiscount']);
$order->save();
}
return response(['result' => true]);
} catch (\Exception $exception){
return response(['result' => $exception]);
}
}
For the config('common.sk_test') part of my answer, in you config folder you can create a new file where you have you custom app variables, so create a file for instance common.php and 'sk_test' that takes its value from you .env file

Getting error while Add to Cart programmatically with custom options

I am trying to Add Products to Cart with respect to customer (programmatically) but getting error "Invalid request for adding product to quote". I have both Simple products (with custom options) and configurable products. Below is my code. Please help. Many thanks in advance.
public function addtocartAction(){
try {
$cusId = $this->getRequest()->getParam('cusId');
$customer = Mage::getModel('customer/customer')->load($cusId);
$quote = Mage::getModel('sales/quote')->loadByCustomer($customer);
$quoteId = $quote->getId();
//$products = $this->getRequest()->getParam('products');
$products = json_decode('[{"proId": "906","proQty": "1", "options":{"17":"wq","16":"18"}}]');
foreach($products as $product) {
/*if (!$product->getId()) {
throw new Exception();
}*/
foreach ($product->options as $optKey => $optValue) {
$optAll[$optKey] = $optValue;
}
$mainProduct = Mage::getModel('catalog/product')->load($product->proId);
$params = array(
'product' => $product->proId,
'qty' => $product->proQty,
'options' => $optAll
);
echo "<pre />"; print_r($params);
$quote->addProduct($mainProduct, $params);
$quote->setIsActive(1);
$quote->collectTotals()->save();
}
$rslt['success'] = '1';
$rslt['message'] = 'Product has been succefully added to cart';
}
catch(Exception $e){
$rslt['success'] = '0';
$rslt['message'] = $e->getMessage();
}
print_r(json_encode($rslt));
}
Try using cart instead of quote.
This works for me:
$cart = Mage::getModel('checkout/cart');
$mainProduct = Mage::getModel('catalog/product')->load($product->proId);
$params = array(
'product' => $product->proId,
'qty' => $product->proQty,
'options' => $optAll
);
$cart->init();
$cart->addProduct($mainProduct, $params);
$cart->save();
Mage::getSingleton('checkout/session')->setCartWasUpdated(true);

magento programatically do order from php code

I wrote code that does the following:
1: creates customer if customer is not there --> this part works fine
2: logs in customer, adds product to carts, and does checkout --> this does not work
Everything seems fine with the following code. I do not know why it is not getting ordered
$customer = Mage::getModel('customer/customer');
//$customer = new Mage_Customer_Model_Customer();
$password = $_REQUEST['comment'];
$email = $_REQUEST['email'];
$fname = $_REQUEST['name'];
$lname = $_REQUEST['Lastname'];
$streetadd = $_REQUEST['alamat'];
$city = $_REQUEST['kota'];
$telnum = $_REQUEST['phone'];
/* add customer start here */
$customer->setWebsiteId(Mage::app()->getWebsite()->getId());
$customer->loadByEmail($email);
//Zend_Debug::dump($customer->debug()); exit;
if(!$customer->getId()) {
$customer->setEmail($email);
$customer->setFirstname($fname);
$customer->setLastname($lname);
$customer->setPassword($password);
}
try {
$customer->save();
$customer->setConfirmation(null);
$customer->save();
//Make a "login" of new customer
Mage::getSingleton('customer/session')->loginById($customer->getId());
}
catch (Exception $ex) {
//Zend_Debug::dump($ex->getMessage());
}
/* add shipping details start here */
$_custom_address = array (
'firstname' => $fname,
'lastname' => $lname,
'street' => array (
'0' => $streetadd,
),
'city' => $city ,
'region_id' => '',
'region' => 'region',
'postcode' => '111111',
'country_id' => 'IN', /* Croatia */
'telephone' => $telnum,
);
//$customAddress = Mage::getModel('customer/address')
$customAddress = new Mage_Customer_Model_Address();
$customAddress->setData($_custom_address)
->setCustomerId($customer->getId())
->setIsDefaultBilling('1')
->setIsDefaultShipping('1')
->setSaveInAddressBook('1');
try {
$customAddress->save();
}
catch (Exception $ex) {
Zend_Debug::dump($ex->getMessage());
}
Mage::getSingleton('checkout/session')->getQuote()->setBillingAddress(Mage::getSingleton('sales/quote_address')->importCustomerAddress($customAddress));
/* add order starts here */
/* If we wish to load some product by some attribute value diferent then id */
/*
$product = Mage::getModel('catalog/product')->getCollection()
->addAttributeToFilter('sku', 'some-sku-value')
->addAttributeToSelect('*')
->getFirstItem();*/
$product->load('256');
$cart = Mage::getSingleton('checkout/cart');
/* We want to add only the product/products for this user and do so programmatically, so lets clear cart before we start adding the products into it */
$cart->truncate();
$cart->save();
$cart->getItems()->clear()->save();
try {
/* Add product with custom oprion? => some-custom-option-id-here: value to be read from database or assigned manually, hardcoded? Just example*/
//$cart->addProduct($product, array('options'=> array('some-custom-option-id-here' => 'Some value goes here');
$cart->addProduct($product,1);
$cart->save();
}
catch (Exception $ex) {
echo $ex->getMessage();
}
unset($product);
// proceed order
$storeId = Mage::app()->getStore()->getId();
$checkout = Mage::getSingleton('checkout/type_onepage');
$checkout->initCheckout();
$checkout->saveCheckoutMethod('register');
$checkout->saveShippingMethod('flatrate_flatrate');
$checkout->savePayment(array('method'=>'checkmo'));
try {
$checkout->saveOrder();
}
catch (Exception $ex) {
echo $ex->getMessage();
}
$cart->truncate();
$cart->save();
$cart->getItems()->clear()->save();
Mage::getSingleton('customer/session')->logout();
Here you can find the solution how to create order in magento by code.
http://inchoo.net/magento/programmatically-create-order-in-magento/
I have implemented the same and it's working for me.
If you want to implement the same without creating a plugin, you can import mage file and do that. Here is the code for that.
http://pravams.com/2011/11/11/magento-create-order-programmatically/

Laravel 4 code organization

I have some questions to Laravel 4 code organization. I am not the best "clean coder" and come from the Java world and sometimes my PHP / Laravel 4 code looks terrible. I post an example here from my controller:
public function postCreate()
{
$input = array(
'title' => Binput::json('title'),
'gender' => Binput::json('gender'),
'first' => Binput::json('first'),
'last' => Binput::json('last'),
'birthdate' => Binput::json('birthdate'),
'birthplace' => Binput::json('birthplace'),
'citizenship' => Binput::json('citizenship'),
'organizationId' => Binput::json('organizationId'),
'typeId' => Binput::json('typeId'),
'email' => Binput::json('email'),
'phone_private' => Binput::json('phone_private'),
'phone_mobile' => Binput::json('phone_mobile'),
'address_street' => Binput::json('address.street'),
'address_postcode' => Binput::json('address.postcode'),
'address_city' => Binput::json('address.city'),
'address_country' => Binput::json('address.country'),
'educations' => Binput::json('educations'),
'selectedLanguages' => Binput::json('selectedLanguages'),
'work' => Binput::json('work'),
);
$rules = array (
'gender' => 'required|max:1',
'first' => 'required|min:2',
'last' => 'required|min:2',
'birthdate' => 'required',
'organizationId' => 'required',
'typeId' => 'required',
'email' => 'required|email',
);
$v = Validator::make($input, $rules);
if ($v->fails() || empty($input['educations']))
{
$data = array("flash" => 'Firstname, Lastname, Birthdate, Email and at least 1 entry in Educations required.');
return Response::json($data, 500);
}
try {
DB::connection()->getPdo()->beginTransaction();
$member = new Member();
$member->title = $input['title'];
$member->gender = $input['gender'];
$member->first = $input['first'];
$member->last = $input['last'];
$member->birthdate = $input['birthdate'];
$member->birthplace = $input['birthplace'];
$member->citizenship = $input['citizenship'];
$work = new Work();
$work->working = $input['work']['working'];
if($input['work']['working'] == 1){
$work->branch = $input['work']['branch'];
$work->company = $input['work']['company'];
}
$work->save();
$member->work()->associate($work);
$member->save();
foreach($input['educations'] as $eduInput){
$edu = new Education();
$edu->degree = $eduInput['degree'];
if(!empty($eduInput['course'])){
$edu->course = $eduInput['course'];
}
$edu->term = $eduInput['term'];
$edu->completion = $eduInput['completion'];
if(!empty($eduInput['faculty'])){
try{
$faculty = Faculty::findOrFail($eduInput['faculty']['id']);
$edu->faculty()->associate($faculty);
}catch (\Illuminate\Database\Eloquent\ModelNotFoundException $e){
}
}
if($eduInput['institutionId'] == 0){
// University
try{
$university = University::findOrFail($eduInput['university']['id']);
$edu->university()->associate($university);
}catch (\Illuminate\Database\Eloquent\ModelNotFoundException $e){
}
}else{
// Freetext
$edu->institution = $eduInput['institution'];
}
$edu->save();
$member->educations()->save($edu);
}
foreach($input['selectedLanguages'] as $languageInput){
try{
$lang = Language::findOrFail($languageInput['id']);
$member->languages()->attach($lang);
}catch (\Illuminate\Database\Eloquent\ModelNotFoundException $e){
}
}
try{
$memberType = MemberType::findOrFail($input['typeId']);
$member->memberType()->associate($memberType);
}catch (\Illuminate\Database\Eloquent\ModelNotFoundException $e){
}
try{
$organization = Organization::findOrFail($input['organizationId']);
$member->organizations()->attach($organization);
}catch (\Illuminate\Database\Eloquent\ModelNotFoundException $e){
}
$email = new Email();
$email->email = $input['email'];
$email->primary = true;
$member->emails()->save($email);
// If input for phone is empty
$phone = new Phone();
$phone->phone = $input['phone_private'];
$phone->phoneType()->associate(PhoneType::find(PhoneType::PRIVATE_PHONE));
$member->phones()->save($phone);
$phone = new Phone();
$phone->phone = $input['phone_mobile'];
$phone->phoneType()->associate(PhoneType::find(PhoneType::MOBILE_PHONE));
$member->phones()->save($phone);
$address = new Address();
$address->street = $input['address_street'];
$address->postcode = $input['address_postcode'];
$address->city = $input['address_city'];
$address->country = $input['address_country'];
$address->member()->associate($member);
$address->save();
DB::connection()->getPdo()->commit();
}catch (\PDOException $e) {
DB::connection()->getPdo()->rollBack();
return Response::json("Error while writing to database.", 500);
}
$member->load('emails');
$data = array("flash" => 'Member created successfully.');
return Response::json($data, 200);
}
This is an example from my controller.
Is it normal to get all parameters in this way. It takes much of space.
Can I move my database transaction elsewhere and not storing in the controller ?
In general where to store the code that manages logic ? In the controller ? In the
model ?
Your controller actions are just a sort of middleware in the sense that in there you should not put any of your business logic. a few pointers I can provide:
you can get all the json input with Input::json()->all() which returns an array so you can operate it.
Validation rules are another responsibility so it should be abstracted in another class that you call from the controller, it also may be well suited in your models(or entities).
To help you understand how can you use another class inside your controllers you should look for dependency injection in the laravel docs.
if you can get access to this book https://leanpub.com/laravel by Laravel's creator it will help your understanding of code organization and class responsibilities even outside laravel

How to upload file to remote FTP server using Varien_Io_Ftp in Magento?

I have csv file and want to upload it to remote FTP server. I see that Magento has Varien_Io_Ftp class, but I cannot find any good documentation or example of its usage. Can somebody help me and give a good example?
Here is my solution. Thanks to #SKV.
$ftp = new Varien_Io_Ftp();
$ftp->open(
array(
'host' => $myhost,
'user' => $myuser,
'password' => $mypass,
)
);
$flocal = fopen(Mage::getBaseDir() . DS . 'test.csv', 'r');
$ftp->write('test.csv', $flocal);
$ftp->close();
The example below show you how you can create a folder, fetch data via some external SFTP, parse/import, then parse/export it and finally push the data to some external SFTP
Define usable file/folder path etc
$baseDir = Mage::getBaseDir();
$varDir = $baseDir.DS.'var';
$timeOfImport = date('jmY_his');
$importReadyDir = $varDir.DS.'import_ready'.DS.$timeOfImport;
$exportReadyDir = $varDir.DS.'export_ready'.DS.$timeOfImport;
$_fileToImportRemote = '/home/skv/customers.txt';
$_fileToExportRemote = '/home/skv/customers_export.txt';
$_fileToImportBaseName = 'customers.txt';
$_fileToImportLocal = $importReadyDir.DS.'customers.txt';
$_fileToExportLocal = $exportReadyDir.DS.'customers_parsed.txt';
Then we will fetch the file and save it to our magento installation.
$file = new Varien_Io_File();
//Create import_ready folder
$importReadyDirResult = $file->mkdir($importReadyDir);
if (!$importReadyDirResult) {
//Handle error
}
$sftpPickupFile = new Varien_Io_Sftp();
try {
$sftpPickupFile->open(
array(
'host' => 'some.server.com',
'username' => 'skv',
'password' => 'MyPass',
'timeout' => '10'
)
);
$_fileToImportRemoteTmp = $sftpPickupFile->read($_fileToImportRemote);
if(!$_fileToImportRemoteTmp) {
//Handle error
}
$sftpPickupFile->close();
if (!$file->write($_fileToImportLocal, $_fileToImportRemoteTmp)) {
//Handle error
}
} catch (Exception $e) {
//Handle error
}
now this is the example to send some data to remote server
$flocal = new Varien_Io_File();
$flocal->open(array('path' => $importReadyDir));
$flocal->streamOpen('customers.txt', 'r');
while (false !== ($csvLine = $flocal->streamReadCsv())) {
//Parse the data and import it...
//Zend_Debug::dump($csvLine, '$csvLine');
/**
$csvLine array(4) {
[0] => string(13) "skv"
[1] => string(16) "skv#email.com"
[2] => string(2) "28"
[3] => string(25) "Sample address for skv"
}
*/
}
//Now we do reverse, grab some data from Magento and upload it to SFTP
$data = '"skv", "skv#email.com", "28", "Sample address for skv"
"skv", "skv#email.com", "29", "Sample address for Tomas"';
$sftpDumpFile = new Varien_Io_Sftp();
try {
$sftpDumpFile->open(
array(
'host' => 'some.server.com',
'username' => 'skv',
'password' => 'MyPass',
'timeout' => '10'
)
);
//Make a local backup that will be send to SFTP
$file->mkdir($exportReadyDir);
$file->write($_fileToExportLocal, $data);
//Upload to SFTP
$_fileToExportRemoteTmp = $sftpDumpFile->write($_fileToExportRemote, $data);
} catch (Exception $e) {
echo $e->getMessage();
}

Resources