I am struggling to migrate a complex SQL query to the Doctrine DBAL in TYPO3. My old repository query looks like this:
$enableFields = $GLOBALS['TSFE']->sys_page->enableFields('tx_test');
// calculate distance between geo coordinates
$distance = '';
if ($geoData) {
$distance = ', 3956 * 2 * ASIN( SQRT (POWER ( SIN((' . $geoData['latitude'] . ' - abs(tx_test.latitude)) * pi() / 180 / 2), 2) + COS(' . $geoData['latitude'] . ' * pi() / 180) * ' . 'COS(abs(tx_test.latitude)' .
' * pi() / 180) * POWER (SIN((' . $geoData['longitude'] .
' - tx_test.longitude)' .
' * pi() / 180 / 2), 2))) AS distance';
}
// General query
$sql = 'SELECT * ' . $distance .
' FROM tx_test ' .
' WHERE DATE_FORMAT(FROM_UNIXTIME(start), "%Y") = '.(int)$settings['flexformYear'].' AND published = 1 ' . $enableFields;
if($headline) {
$sql .= ' AND headline like '.$GLOBALS['TYPO3_DB']->quoteStr($headline).'%';
}
// ... and some more ...
$query = $this->createQuery();
$results = $query->statement($sql)->execute();
To migrate to Doctrine
Now I could easily remove the $GLOBALS['TYPO3_DB']->quoteStr() and replace the last two lines of the code above with this:
$connection = GeneralUtility::makeInstance(ConnectionPool::class)->getConnectionForTable('tx_test');
$results = $connection->executeQuery($sql)->fetchAll();
But this returns an array of the results and not the objects with the attached sub objects, eg. a FileReference object or another attached model.
Is there another way to achieve the desired result? If not, how can I secure/sanitize the user input for $headline? And do I need to write the SQL for the joined tables myself?
Here is a part of some distance calculations that should fit your use case regarding the use of Doctrine and the quoting.
Using Doctrine you'll get records as array, too. See below if you want Extbase objects.
Get the TYPO3 flavor of the Doctrine DBAL QueryBuilder:
$q = GeneralUtility::makeInstance(ConnectionPool::class)
->getQueryBuilderForTable($table = 'tx_....');
$q->select(
...
'a.lat',
'a.lng'
)
->from($table /*, 'alias if you want' */);
I am not using the fluent interface much when doing calculations like that (although it is of course possible to do so by using any MySQL function with $q->selectLiteral()).
Parameters
To prevent SQL injection you should quote all possible user input with $q->quote() / $q->quoteIdentifier() or use parameters $q->createNamedParameter().
Constraints example
This is just a part of the constraints. It contains an example how to combine them based on conditions as is common in a search function.
if ((float)$searchObject->radiusKm > .5) {
$_radiusOrs = [
'IF (
' . $q->quoteIdentifier('lat') . ' = 0,
100000,
12742 * ASIN(
SQRT(
POWER(
SIN(
( ' . $q->quote((float)$searchObject->lat) . '
- ABS( ' . $q->quoteIdentifier('lat') . ' )
) * 0.0087266
),
2
)
+
COS( ' . $q->quote((float)$searchObject->lng) . ' * 0.01745329 ) * COS(
ABS( ' . $q->quoteIdentifier('lat') . ' ) * 0.01745329
) * POWER(
SIN(
( ' . $q->quote((float)$searchObject->lng) . '
- ' . $q->quoteIdentifier('lng') . '
) * 0.0087266
),
2
)
)
)
) < ' . $q->quote((float)$searchObject->radiusKm),
];
$q->andWhere(
$q->expr()->orX(...$_radiusOrs)
);
}
...
$aRes = $q->execute()->fetchAll();
(If you want to debug: You get the SQL with $q->getSQL(), $q->getParameters())
Mapping to Extbase objects
$dataMapper = $this->objectManager->get(\TYPO3\CMS\Extbase\Persistence\Generic\Mapper\DataMapper::class);
$objects = $dataMapper->map(YourExtbaseModel::class, $aRes);
I would suggest: Use Extbase objects only if you have few objects or if you have enough memory and you don't care about performance. You should mostly get away with plain arrays just as good.
The Extbase query builder allowed for a little optimization when outputting into a Fluid template: passing the \TYPO3\CMS\Extbase\Persistence\Generic\QueryResult allowed to use the f:paginate ViewHelper on it which did not have to instantiate all the objects but only those shown on current page. That possiblity is not there (yet?) when using the Doctrine QueryBuilder. So using Extbase models should rather be last resort now than being the default. It seemed to be "best practice" - that is not true any more IMHO.
Removed text is '% 33', '% 40'. (single quote is not included)
$text = "% 33";
$xss_text = $this->input->post($text, TRUE);
echo $xss_text;
// result is 3 and #
Is their any command in Foxpro that convert the DBF into a particular excel sheet.
I have three DBF(dbf_1, dbf_2, dbf_3). My current program convert the file using copy to "filename.xls" type fox2x and then I will manually copy the consolidate all the sheet into one excel. For me, this method I using is alright but what if their are 20 or more dbf that I will consolidate. Is their any command in foxpro that convert the dbf's into one excel file. I already use the foxpro Automation but it is to slow.
No there isn't.
Also "copy to ... type fox2x". although better than many other type selections (such as csv and xls) should not be chosen when there are better ways.
You are saying automation is slow, but don't know if you are really finding automation slow, or if you have tried it in the ways that you shouldn't use to transfer data to Excel. The sample below, use one of the variations of my "vfp2excel" function and automation. It transfers sample Customer, Employee, Orders, OrdItems and Products data in 2.5 seconds on my machine. If you really meant it as slow then no dice, otherwise here is the sample:
* These represent complex SQL as a sample
Select emp_id,First_Name,Last_Name,;
Title,Notes ;
from (_samples+'\data\employee') ;
into Cursor crsEmployee ;
readwrite
Replace All Notes With Chrtran(Notes,Chr(13)+Chr(10),Chr(10))
Select cust_id,company,contact,Title,country,postalcode ;
from (_samples+'\data\customer') ;
into Cursor crsCustomer ;
nofilter
Select * ;
from (_samples+'\data\orders') ;
into Cursor crsOrders ;
nofilter
Select * ;
from (_samples+'\data\orditems') ;
into Cursor crsOrderDetail ;
nofilter
Select * ;
from (_samples+'\data\products') ;
into Cursor crsProducts ;
nofilter
* Now we want to get these on 3 sheets
* Sheet1: Employees only
* Sheet2: Customers only
* Sheet3: Orders, ordItems, Products layed out horizontally
Local oExcel
oExcel = Createobject("Excel.Application")
With oExcel
.DisplayAlerts = .F.
.Workbooks.Add
.Visible = .T.
With .ActiveWorkBook
For ix = 1 To 3 && We want 3 Sheets
If .sheets.Count < m.ix
.sheets.Add(,.sheets(.sheets.Count)) && Add new sheets
Endif
Endfor
* Name the sheets
.WorkSheets(1).Name = "Employees"
.WorkSheets(2).Name = "Customers"
.WorkSheets(3).Name = "Order, OrderDetail, Products" && max sheetname is 31 chars
* Start sending data
* First one has headers specified
VFP2Excel('crsEmployee', .WorkSheets(1).Range("A1"), ;
"Id,First Name,Last Name,Employee Title,Comments about employee" ) && To sheet1, start at A1
VFP2Excel('crsCustomer', .WorkSheets(2).Range("A1") ) && To sheet2, start at A1
VFP2Excel('crsOrders', .WorkSheets(3).Range("A1") ) && To sheet3, start at A1
* Need to know where to put next
* Leave 2 columns empty - something like 'G1'
lcRange = _GetChar(.WorkSheets(3).UsedRange.Columns.Count + 3) + '1'
* To sheet3, start at next to previous
VFP2Excel('crsOrderDetail', .WorkSheets(3).Range(m.lcRange) )
lcRange = _GetChar(.WorkSheets(3).UsedRange.Columns.Count + 3) + '1'
* To sheet3, start at next to previous
VFP2Excel('crsProducts', .WorkSheets(3).Range(m.lcRange) )
#Define xlJustify -4130
#Define xlTop -4160
* I just happen to know notes in at column 5 from SQL
* No need to query from excel to keep code simple
* Lets format that column specially instead of leaving
* at the mercy of Excel's autofitting
.WorkSheets(1).UsedRange.VerticalAlignment = xlTop && set all to top
With .WorkSheets(1).Columns(5)
.ColumnWidth = 80 && 80 chars width
.WrapText = .T.
* .HorizontalAlignment = xlJustify && doesn't work good always
Endwith
* Finally some cosmetic stuff
For ix=1 To 3
With .WorkSheets(m.ix)
.Columns.AutoFit
.Rows.AutoFit
Endwith
Endfor
.WorkSheets(1).Activate
Endwith
Endwith
* Author: Cetin Basoz
* This is based on earlier VFP2Excel function codes
* that has been published on the internet, at various sites
* since 2001. Not to be messed with others' code who named the same but has
* nothing to do with the approaches taken here (unless copy & pasted and claimed
* to be their own work, < s > that happens).
Procedure VFP2Excel(tcCursorName, toRange, tcHeaders, tnPrefferredWidthForMemo)
* tcCursorName
* toRange
* tcHeaders: Optional. Defaults to field headers
* tnPrefferredWidthForMemo: Optional. Default 80
* Function VFP2Excel
tcCursorName = Evl(m.tcCursorName,Alias())
tnPrefferredWidthForMemo = Evl(m.tnPrefferredWidthForMemo,80)
Local loConn As AdoDB.Connection, loRS As AdoDB.Recordset,;
lcTemp,lcTempDb, oExcel,ix, lcFieldName, lcHeaders
lnSelect = Select()
lcTemp = Forcepath(Sys(2015)+'.dbf',Sys(2023))
lcTempDb = Forcepath(Sys(2015)+'.dbc',Sys(2023))
Create Database (m.lcTempDb)
Select * From (m.tcCursorName) Into Table (m.lcTemp) Database (m.lcTempDb)
Local Array aMemo[1]
Local nMemoCount
nMemoCount = 0
lcHeaders = ''
For ix = 1 To Fcount()
lcFieldName = Field(m.ix)
If Type(Field(m.ix))='M'
nMemoCount = m.nMemoCount + 1
Dimension aMemo[m.nMemoCount]
aMemo[m.nMemoCount] = m.ix
Replace All &lcFieldName With Chrtran(&lcFieldName,Chr(13)+Chr(10),Chr(10))
Endif
lcHeaders = m.lcHeaders + Iif(Empty(m.lcHeaders),'',',')+Proper(m.lcFieldName)
Endfor
tcHeaders = Evl(m.tcHeaders,m.lcHeaders)
Use In (Juststem(m.lcTemp))
Close Databases
Set Database To
loStream = Createobject('AdoDb.Stream')
loConn = Createobject('ADODB.Connection')
loRS = Createobject("ADODB.Recordset")
loConn.ConnectionString = "Provider=VFPOLEDB;Data Source="+m.lcTempDb
loConn.Open()
loRS = loConn.Execute("select * from "+m.lcTemp)
loRS.Save( loStream )
loRS.Close
loConn.Close
Erase (m.lcTemp)
* Use first row for headers
Local Array aHeader[1]
loRS.Open( loStream )
toRange.Offset(1,0).CopyFromRecordSet( loRS ) && Copy data starting from headerrow + 1
Set Safety Off
Delete Database (m.lcTempDb) Deletetables
Select (m.lnSelect)
For ix=1 To Iif( !Empty(m.tcHeaders), ;
ALINES(aHeader, m.tcHeaders,1,','), ;
loRS.Fields.Count )
toRange.Offset(0,m.ix-1).Value = ;
Iif( !Empty(m.tcHeaders), ;
aHeader[m.ix], ;
Proper(loRS.Fields(m.ix-1).Name) )
toRange.Offset(0,m.ix-1).Font.Bold = .T.
Endfor
#Define xlJustify -4130
#Define xlTop -4160
* This part is cosmetic
toRange.WorkSheet.Activate
With toRange.WorkSheet.UsedRange
.VerticalAlignment = xlTop && set all to top
For ix=1 To m.nMemoCount
With .Columns(aMemo[m.ix])
.ColumnWidth = m.tnPrefferredWidthForMemo && 80 chars width
.WrapText = .T.
Endwith
Endfor
.Columns.AutoFit
.Rows.AutoFit
Endwith
Endproc
* Return A, AA, BC etc noation for nth column
Function _GetChar
Lparameters tnColumn && Convert tnvalue to Excel alpha notation
If m.tnColumn = 0
Return ""
Endif
If m.tnColumn <= 26
Return Chr(Asc("A")-1+m.tnColumn)
Else
Return _GetChar(Int(Iif(m.tnColumn % 26 = 0,m.tnColumn - 1, m.tnColumn) / 26)) + ;
_GetChar((m.tnColumn-1)%26+1)
Endif
Endfunc
This is what I was looking for :-) I was trying with my knowledge of Excel Automation programming in Visual FoxPro but always got errors. My task was to create "n" Sheets from one big cursors which I want to parse regarding customer selection of attribute name from cursor to get also "n" Sheets. This sample is for 3 cursors and 3 Sheets and it is generic. But I need this for "n" cursors and one attribute which customer select to distinct and get "n" Sheets in one Excel file. So now I have dynamic procedure. I customized this code and solve my problem which I am trying to end for about 4 days. So again thank you for this code and off course I will not modify VFP2Excel procedure and wrote somewhere else my name. Thanks for help !
There is no native VFP function to do this, BUT, there is an awesome open source project which has a class that will make this very easy:
VFPx Workbook Xlsx - See it here on Github: XLSX Workbook for FoxPro
It has 3 magical functions that will do exactly what you asked for:
CreateWorkbook()
AddSheet()
SaveTableToWorkbook()
(Repeat commands 2 and 3 above for each DBF/Sheet you want to create)
It is well documented with a 54-page PDF and code sample that explains everything you'll need to know.
A huge file was located on the url .
http://gensho.acc.umu.se/debian-cd/8.2.0/amd64/iso-cd/debian-8.2.0-amd64-lxde-CD-1.iso
Now i want to download 10000 bytes between 10000th byte and 19999th byte on the file .
HOW to write the php command?
This would read 8K of binary data from the URL:
$filename = "http://gensho.acc.umu.se/debian-cd/8.2.0/amd64/iso-cd/debian-8.2.0-amd64-lxde-CD-1.iso";
$handle = fopen($filename, "rb");
$contents = fread($handle, 8192);
fclose($handle);
see http://php.net/manual/en/function.fread.php
In order to read from the 10,000nd byte up to the 19,999th you need to skip the first 10,000 bytes, as there is no "filepointer" which can be positioned w/o reading like fseek on normal files:
...
$skip= fread($handle, 10000);
$contents = fread($handle, 9999);
...
I have an Apache server running a web application. In this webapplication I show a video using JWPlayer. JWPlayer uses http pseudostreaming to fetch the video from a PHP script which serves up this video. All this works well and the video is streamed well.
The problem I am having is while the video is streaming I also use AJAX calls to fetch some XML files which are used by Adobe Flash files on the same page. While streaming these XML file fetches are kept 'pending' until the entire video is loaded. Using Chrome I can see that the video gets loaded byte by byte. When the video is entirely loaded, then the XML files are fetched. Also if I open another tab in my browser while a video is streaming and try to load the web application again, it will also not show until the video is entirely loaded.
This seems te be an Apache setting of some sort. The MPM settings for apache are:
ThreadsPerChild 150
MaxRequestsPerChild 0
This seems to be correct. Any ideas what could be wrong?
If you are using PHP sessions then this is probably what is causing the IO blocking.
php blocking when calling the same file concurrently
I was making a system with private video streaming. So, i was needing a streaming via php, because using php programming i was able to restringe user access.
I was having troubles to streaming video and execute other script on the server.
Using the session_write_close() solve the problem to open another scripts and i found that script on web that helps me sooooo much.
I want to share, because that script makes a real streaming.
I found it on http://www.tuxxin.com/php-mp4-streaming/ website.
All thanks to the author of this code =D
ENJOY !
<?php
$file = 'video360p.mp4';
$fp = #fopen($file, 'rb');
$size = filesize($file); // File size
$length = $size; // Content length
$start = 0; // Start byte
$end = $size - 1; // End byte
header('Content-type: video/mp4');
//header("Accept-Ranges: 0-$length");
header("Accept-Ranges: bytes");
if (isset($_SERVER['HTTP_RANGE'])) {
$c_start = $start;
$c_end = $end;
list(, $range) = explode('=', $_SERVER['HTTP_RANGE'], 2);
if (strpos($range, ',') !== false) {
header('HTTP/1.1 416 Requested Range Not Satisfiable');
header("Content-Range: bytes $start-$end/$size");
exit;
}
if ($range == '-') {
$c_start = $size - substr($range, 1);
}else{
$range = explode('-', $range);
$c_start = $range[0];
$c_end = (isset($range[1]) && is_numeric($range[1])) ? $range[1] : $size;
}
$c_end = ($c_end > $end) ? $end : $c_end;
if ($c_start > $c_end || $c_start > $size - 1 || $c_end >= $size) {
header('HTTP/1.1 416 Requested Range Not Satisfiable');
header("Content-Range: bytes $start-$end/$size");
exit;
}
$start = $c_start;
$end = $c_end;
$length = $end - $start + 1;
fseek($fp, $start);
header('HTTP/1.1 206 Partial Content');
}
header("Content-Range: bytes $start-$end/$size");
header("Content-Length: ".$length);
$buffer = 1024 * 8;
while(!feof($fp) && ($p = ftell($fp)) <= $end) {
if ($p + $buffer > $end) {
$buffer = $end - $p + 1;
}
set_time_limit(0);
echo fread($fp, $buffer);
flush();
}
fclose($fp);
exit();
?>