How to build a dynamic doctrine query? - doctrine

I am trying to build a dynamic doctrine query. When I try like this, it works
$qb->andWhere($qb->expr()->orX(
$qb->expr()->andX(
$qb->expr()->eq('t.width', '245'),
$qb->expr()->eq('t.height', '45'),
),
$qb->expr()->andX(
$qb->expr()->eq('t.width', '225'),
$qb->expr()->eq('t.height', '65'),
)
));
But I will pass the key and value from the array.
My array looks like this:
[
0 => [
"width" => "245"
"height" => "45"
]
1 => [
"width" => "225"
"height" => "65"
]
]
Now, I tried following code.
$conditions = $qb->expr()->orX(
$qb->expr()->andX()
);
foreach ($wheres as $outerKey => $outerValue) {
foreach ($outerValue as $innerKey => $innerValue) {
$conditions->add("te.$innerKey = :innerValue");
$qb->setParameter('innerValue', $innerValue);
}
}
$qb->andWhere($conditions);
dd($qb->getDQL());
But the SQL returned is not the same as when I tried with static value.

So you can do this in 2 ways (that I could think of). First, using expressions like you showed above but with and extra private function:
private function getExpressions(QueryBuilder $qb, array $fields)
{
$andX = $qb->expr()->andX();
foreach ($fields as $field => $value) {
$andX->add($qb->expr()->eq('t.' . $field, $value));
}
return $andX;
}
public function getDQL($conditions)
{
$qb = $this->createQueryBuilder('t');
$orX = $qb->expr()->orX();
foreach ($conditions as $i => $fields) {
$orX->add($this->getExpressions($qb, $fields));
}
$qb->add('where', $orX);
dump($qb->getDQL());
exit;
}
For me it was a bit time consuming to figure this out. I actually did it a lot faster the way mentioned above (building the where clause manually):
$i = 0;
foreach ($conditions as $fields) {
$j = 0;
foreach ($fields as $field => $value){
$whereStr .= " " . $field . " = " . $value;
$j++;
if ($j < count($fields)){
$whereStr .= " AND";
}
}
$i++;
if ($i < count($conditions)){
$whereStr .= " OR";
}
}
If I understand your logic correctly this should work. You can switch the orX/andX expressions if I misunderstood your requirements.

Related

How to insert big data on the laravel?

I am using laravel 5.6
My script to insert big data is like this :
...
$insert_data = [];
foreach ($json['value'] as $value) {
$posting_date = Carbon::parse($value['Posting_Date']);
$posting_date = $posting_date->format('Y-m-d');
$data = [
'item_no' => $value['Item_No'],
'entry_no' => $value['Entry_No'],
'document_no' => $value['Document_No'],
'posting_date' => $posting_date,
....
];
$insert_data[] = $data;
}
\DB::table('items_details')->insert($insert_data);
I have tried to insert 100 record with the script, it works. It successfully insert data
But if I try to insert 50000 record with the script, it becomes very slow. I've waited about 10 minutes and it did not work. There exist error like this :
504 Gateway Time-out
How can I solve this problem?
As it was stated, chunks won't really help you in this case if it is a time execution problem. I think that bulk insert you are trying to use cannot handle that amount of data , so I see 2 options:
1 - Reorganise your code to properly use chunks, this will look something like this:
$insert_data = [];
foreach ($json['value'] as $value) {
$posting_date = Carbon::parse($value['Posting_Date']);
$posting_date = $posting_date->format('Y-m-d');
$data = [
'item_no' => $value['Item_No'],
'entry_no' => $value['Entry_No'],
'document_no' => $value['Document_No'],
'posting_date' => $posting_date,
....
];
$insert_data[] = $data;
}
$insert_data = collect($insert_data); // Make a collection to use the chunk method
// it will chunk the dataset in smaller collections containing 500 values each.
// Play with the value to get best result
$chunks = $insert_data->chunk(500);
foreach ($chunks as $chunk)
{
\DB::table('items_details')->insert($chunk->toArray());
}
This way your bulk insert will contain less data, and be able to process it in a rather quick way.
2 - In case your host supports runtime overloads, you can add a directive right before the code starts to execute :
ini_set('max_execution_time', 120 ) ; // time in seconds
$insert_data = [];
foreach ($json['value'] as $value)
{
...
}
To read more go to the official docs
It makes no sense to use an array and then convert it to a collection.
We can get rid of arrays.
$insert_data = collect();
foreach ($json['value'] as $value) {
$posting_date = Carbon::parse($value['Posting_Date']);
$posting_date = $posting_date->format('Y-m-d');
$insert_data->push([
'item_no' => $value['Item_No'],
'entry_no' => $value['Entry_No'],
'document_no' => $value['Document_No'],
'posting_date' => $posting_date,
....
]);
}
foreach ($insert_data->chunk(500) as $chunk)
{
\DB::table('items_details')->insert($chunk->toArray());
}
Here is very good and very Fast Insert data solution
$no_of_data = 1000000;
$test_data = array();
for ($i = 0; $i < $no_of_data; $i++){
$test_data[$i]['number'] = "1234567890";
$test_data[$i]['message'] = "Test Data";
$test_data[$i]['status'] = "Delivered";
}
$chunk_data = array_chunk($test_data, 1000);
if (isset($chunk_data) && !empty($chunk_data)) {
foreach ($chunk_data as $chunk_data_val) {
DB::table('messages')->insert($chunk_data_val);
}
}
I used the code below to check the update or insert data of 11 thousand rows. I hope it useful for you.
$insert_data = [];
for ($i=0; $i < 11000; $i++) {
$data = [
'id' =>'user_'.$i,
'fullname' => 'Pixs Nguyen',
'username' => 'abc#gmail.com',
'timestamp' => '2020-03-23 08:12:00',
];
$insert_data[] = $data;
}
$insert_data = collect($insert_data); // Make a collection to use the chunk method
// it will chunk the dataset in smaller collections containing 500 values each.
// Play with the value to get best result
$accounts = $insert_data->chunk(500);
// In the case of updating or inserting you will take about 35 seconds to execute the code below
for ($i=0; $i < count($accounts); $i++) {
foreach ($accounts[$i] as $key => $account)
{
DB::table('yourTable')->updateOrInsert(['id'=>$account['id']],$account);
}
}
// In the case of inserting you only take about 0.35 seconds to execute the code below
foreach ($accounts as $key => $account)
{
DB::table('yourTable')->insert($account->toArray());
}

Batch insert in codeigniter

Hi I am quite new to cpdeignoter I just wanted to ask if I am doing the insert batch in ci right because it seemed not to work here is my controller
foreach($dat as $key => $val){
$data[] = array('ModelNumber' => $_POST['txt_model_num'][$key],
'Access' => $_POST['txt_accessories'][$key],
'SerialNumber' => $_POST['txt_snum'][$key],
'Charges' => $_POST['txt_charges'][$key],
'OtherRemarks' => $_POST['txt_rem'][$key],
'RequirementID' => $id1);
}
$cmd3 = $this->Software_model->add_type($data);
if($cmd3){
foreach($sql->result_array() as $row){
$id2 = $row['ID'];
}
$data2s = array();
foreach($dat as $key => $val){
$data2s[] = array('EquipmentName' => $_POST['txt_equipb'][$key],
'EquipmentType' => $_POST['txt_equiptype'][$key],
'RequirementID' => $id2);
}
$cmd2 = $this->Software_model->add_equip($data2s);
and this is the model
public function add_type($data)
{
return $this->db->insert_batch('jobtype', $data);
}
public function add_equip($data2s)
{
return $this->db->insert_batch('equipment', $data2s);
}
and the way I duplicate the textboxes in the views is like this
function second function(){
var etype = document.createElement('input');
etype.type = 'text';
etype.setAttribute("name", "txt_equiptype[]");
etype.setAttribute("class", "form-control");
etype.setAttribute("id", "etype");
etype.setAttribute("placeholder", "Enter Equipment Type");
document.getElementById('third').appendChild(etype);
}
thanks in advance
Hope this will help you :
Try out this alternate method :
foreach($dat as $key => $val)
{
$data[$key]['ModelNumber'] = $_POST['txt_model_num'][$key];
$data[$key]['Access'] = $_POST['txt_accessories'][$key];
$data[$key]['SerialNumber'] = $_POST['txt_snum'][$key];
$data[$key]['Charges'] = $_POST['txt_charges'][$key];
$data[$key]['OtherRemarks'] = $_POST['txt_rem'][$key];
$data[$key]['RequirementID'] = $id1;
}
/* Note : remove auto increment primary key from above array if you are including*/
user it for both the foreach

How move a save model outside a loop in Yii?

I need to move outside a loop the $model->save() sentence, by example:
foreach ($nodes as $i => $node) {
$model = new Singer();
$model->name = $node['name'];
$model->birthDate = $node['birthDate'];
$model->save();
}
There is no way to save the model outside the loop? I mean if there is 100000 records in $nodes the save method will be executed 100000 times??
Thank you!
$values = array();
foreach ($nodes as $node) {
$values[]=array(
'name' => $node['name'],
'birthDate' => $node['birthDate']
);
}
Yii::app()->db->schema->getCommandBuilder()->createMultipleInsertCommand(Singer::model()->tableName(), $values)->execute();
There isn't any other solution. If you have 100,000 records, you have to iterate 100,00 times!
You can also use CDbCommand for faster response in this way:
$values = "";
foreach ($nodes as $i => $node) {
$values.= "(". $node['name'] . "," . $node['birthDate'] . ")" . ",";
}
$values = rtrim($values, ",") //remove last "," from $values
$sql = 'ISERT INTO "singer"(name, birthDate) VALUES '.$values;
$connection = Yii::app() -> db;
$command = $connection -> createCommand($sql);
$command -> execute();

Yii: save multiple records in one query

I'm trying to save a lot of CActiveRecord model objects in a loop.
I have something like this:
foreach ($array_of_items as $item) {
$values = array(
"title" => $item->title,
"content" => $item->content,
);
$object = new MyModel;
$object->attributes = $values;
$object->save();
}
In my case, this creates about 400 CActiveRecord objects. The saving process is really slow, because each save() queries the database.
Is there a way to save all those objects in one go?
Something like:
$objects = array();
foreach ($array_of_items as $item) {
$values = array(
"title" => $item->title,
"content" => $item->content,
);
$object = new MyModel;
$object->attributes = $values;
$objects[] = $object;
}
save_all_objects($objects);
I could not find anything on the subject. Anyone?
you can validate() your model, and if it was ok you can append it so a sql text for insert,
and after your loop, just use databases commandBuilder() and execute your prepared text
$sql = '';
if($object->validate())
{
$sql .= ',("' . $object->attr1 . '")'// append to script,(you get the idea, you need to also make a correct values)
}
...
if(!empty($sql))
{
$sql = 'INSERT INTO table (attr1) Values' . $sql;// make complete script
// execute that command
}
Since v1.1.14, the method createMultipleInsertCommand() of CDbCommandBuilder class is available.
For insert multi rows, Put this code in components folder under GeneralRepository.php file name.
<?php
class GeneralRepository
{
/**
* Creates and executes an INSERT SQL statement for several rows.
* By: Nabi K.A.Z. <www.nabi.ir>
* Version: 0.1.0
* License: BSD3
*
* Usage:
* $rows = array(
* array('id' => 1, 'name' => 'John'),
* array('id' => 2, 'name' => 'Mark')
* );
* GeneralRepository::insertSeveral(User::model()->tableName(), $rows);
*
* #param string $table the table that new rows will be inserted into.
* #param array $array_columns the array of column datas array(array(name=>value,...),...) to be inserted into the table.
* #return integer number of rows affected by the execution.
*/
public static function insertSeveral($table, $array_columns)
{
$connection = Yii::app()->db;
$sql = '';
$params = array();
$i = 0;
foreach ($array_columns as $columns) {
$names = array();
$placeholders = array();
foreach ($columns as $name => $value) {
if (!$i) {
$names[] = $connection->quoteColumnName($name);
}
if ($value instanceof CDbExpression) {
$placeholders[] = $value->expression;
foreach ($value->params as $n => $v)
$params[$n] = $v;
} else {
$placeholders[] = ':' . $name . $i;
$params[':' . $name . $i] = $value;
}
}
if (!$i) {
$sql = 'INSERT INTO ' . $connection->quoteTableName($table)
. ' (' . implode(', ', $names) . ') VALUES ('
. implode(', ', $placeholders) . ')';
} else {
$sql .= ',(' . implode(', ', $placeholders) . ')';
}
$i++;
}
$command = Yii::app()->db->createCommand($sql);
return $command->execute($params);
}
}
And usage anywhere:
$rows = array(
array('id' => 1, 'name' => 'John'),
array('id' => 2, 'name' => 'Mark')
);
GeneralRepository::insertSeveral(User::model()->tableName(), $rows);
https://www.yiiframework.com/extension/yii-insert-multi-rows

How to add a tfoot row to table generated with codeigniter table class

I want to add footer row to the table generated by CI table class. There are classes available which extend the table class and add this functionality. I would prefer to use native feature if available without extending the table class.
Is it possible in CI table class?
I ended up putting a div under the table with similar formatting to offer an illusion of footer row. Hopefully in future the table class will include the tfoot feature.
I am aware of extended classes which will add this feature to CI table class.
This is a old question but I just ran into the same issue, here's the modified Table library that allows for table footers
https://github.com/rhspeer/TableWithTableFooter
I did extend the table class, I also set the template to work with jquery ui by default, and unlike the example above, I don't recreate the base methods that don't need updating.
In application/libraries add a file named MY_table.php
<?php if ( ! defined('BASEPATH')) exit('No direct script access allowed');
class MY_Table extends CI_Table{
var $footer = array();
function __construct()
{
parent::__construct();
$this->template = array(
'table_open' => '<table class="ui-corner-top ui-widget" >',
'thead_open' => '<thead class="ui-widget-header">',
'thead_close' => '</thead>',
'heading_row_start' => '<tr>',
'heading_row_end' => '</tr>',
'heading_cell_start' => '<th>',
'heading_cell_end' => '</th>',
'tbody_open' => '<tbody class="ui-widget-content">',
'tbody_close' => '</tbody>',
'row_start' => '<tr class="table_row_odd">',
'row_end' => '</tr>',
'cell_start' => '<td>',
'cell_end' => '</td>',
'row_alt_start' => '<tr class="table_row_even">',
'row_alt_end' => '</tr>',
'cell_alt_start' => '<td>',
'cell_alt_end' => '</td>',
'tfoot_open' => '<tfoot class="ui-widget-header ui-priority-secondary">',
'footer_row_start' => '<tr>',
'footer_row_end' => '</tr>',
'footer_cell_start' => '<th>',
'footer_cell_end' => '</th>',
'tfoot_close' => '</tfoot>',
'table_close' => '</table>'
);
$this->empty_cells = ' ';
}
function set_template($template)
{
// extends the normal method so that only the required elements have to be entered. the remainder stay as defaults.
if ( ! is_array($template))
{
return FALSE;
}
else
{
foreach($template as $param => $value)
{
$this->template[$param] = $value;
}
}
}
function set_footer()
{
$args = func_get_args();
$this->footer = $this->_prep_args($args);
}
function clear()
{
$this->footer = array();
parent::clear();
}
// extend the generate table method. just adds the bit in to handle tfoot.
function generate($table_data = NULL)
{
// The table data can optionally be passed to this function
// either as a database result object or an array
if ( ! is_null($table_data))
{
if (is_object($table_data))
{
$this->_set_from_object($table_data);
}
elseif (is_array($table_data))
{
$set_heading = (count($this->heading) == 0 AND $this->auto_heading == FALSE) ? FALSE : TRUE;
$this->_set_from_array($table_data, $set_heading);
}
}
// Is there anything to display? No? Smite them!
if (count($this->heading) == 0 AND count($this->rows) == 0)
{
return 'Undefined table data';
}
// Compile and validate the template date
$this->_compile_template();
// set a custom cell manipulation function to a locally scoped variable so its callable
$function = $this->function;
// Build the table!
$out = $this->template['table_open'];
$out .= $this->newline;
// Add any caption here
if ($this->caption)
{
$out .= $this->newline;
$out .= '<caption>' . $this->caption . '</caption>';
$out .= $this->newline;
}
// Is there a table heading to display?
if (count($this->heading) > 0)
{
$out .= $this->template['thead_open'];
$out .= $this->newline;
$out .= $this->template['heading_row_start'];
$out .= $this->newline;
foreach ($this->heading as $heading)
{
$temp = $this->template['heading_cell_start'];
foreach ($heading as $key => $val)
{
if ($key != 'data')
{
$temp = str_replace('<th', "<th $key='$val'", $temp);
}
}
$out .= $temp;
$out .= isset($heading['data']) ? $heading['data'] : '';
$out .= $this->template['heading_cell_end'];
}
$out .= $this->template['heading_row_end'];
$out .= $this->newline;
$out .= $this->template['thead_close'];
$out .= $this->newline;
}
if (count($this->footer) > 0)
{
$out .= $this->template['tfoot_open'];
$out .= $this->newline;
$out .= $this->template['footer_row_start'];
$out .= $this->newline;
foreach ($this->footer as $footer)
{
$temp = $this->template['footer_cell_start'];
foreach ($footer as $key => $val)
{
if ($key != 'data')
{
$temp = str_replace('<th', "<th $key='$val'", $temp);
}
}
$out .= $temp;
$out .= isset($footer['data']) ? $footer['data'] : '';
$out .= $this->template['footer_cell_end'];
}
$out .= $this->template['footer_row_end'];
$out .= $this->newline;
$out .= $this->template['tfoot_close'];
$out .= $this->newline;
}
// Build the table rows
if (count($this->rows) > 0)
{
$out .= $this->template['tbody_open'];
$out .= $this->newline;
$i = 1;
foreach ($this->rows as $row)
{
if ( ! is_array($row))
{
break;
}
// We use modulus to alternate the row colors
$name = (fmod($i++, 2)) ? '' : 'alt_';
$out .= $this->template['row_'.$name.'start'];
$out .= $this->newline;
foreach ($row as $cell)
{
$temp = $this->template['cell_'.$name.'start'];
foreach ($cell as $key => $val)
{
if ($key != 'data')
{
$temp = str_replace('<td', "<td $key='$val'", $temp);
}
}
$cell = isset($cell['data']) ? $cell['data'] : '';
$out .= $temp;
if ($cell === "" OR $cell === NULL)
{
$out .= $this->empty_cells;
}
else
{
if ($function !== FALSE && is_callable($function))
{
$out .= call_user_func($function, $cell);
}
else
{
$out .= $cell;
}
}
$out .= $this->template['cell_'.$name.'end'];
}
$out .= $this->template['row_'.$name.'end'];
$out .= $this->newline;
}
$out .= $this->template['tbody_close'];
$out .= $this->newline;
}
$out .= $this->template['table_close'];
// Clear table class properties before generating the table
$this->clear();
return $out;
}
}
I think this what are you search for
https://github.com/EllisLab/CodeIgniter/wiki/Table
Not necessary extends table library, you would to use set_template function, passing array template.

Resources