I am studying Foxpro to create a simple application for manipulating data from two tables A and B (size of tableB >> size of tableA). The data from an Excel spreadsheet is imported into these two tables.
tableA
id balance load state
1 10 null l
2 22 null l
3 31 null l
tableB
Load id id ord fact type 1st value rounded value state
1 1 1 0.09 1 null null l
2 1 2 0.02 0 null null l
3 1 3 0.13 1 null null l
4 1 4 -0.05 0 null null l
5 2 1 0.01 1 null null l
6 2 2 0.092 1 null null l
7 2 3 0.03 0 null null l
8 3 1 0.14 1 null null l
9 3 2 0.12 0 null null l
10 3 3 -0.02 0 null null l
My friend wants me to write a Foxpro code to do the following things: first, create empty tableA and tableB containing the columns shown above. Each columns will be loaded by (hundreds of thousands) of data from an excel spreadsheet everyday. Second, for each unique id, the code updates the 3 columns 1st value, rounded value and load with given formulas:
1st value[i] = If(Type[i]=0, load[i-1]*fact[i], load[i-1]*fact[i]/(1-fact[i]))
1st value[1] = If(Type[1]=0, balance[1]*fact[1], balance[1]*fact[1]/(1-fact[1]))
rounded value[i] = If(1st value[i]>0, rounddown(1st value[i], 1), roundup(1st value[i],2)
load[i+1] = load[i] + rounded value[i+1] (i >= 1)
load[1] = balance[1] + rounded value[1]
I think I have to create a table like the following to store the calculation above for this step:
Calculation Table
balance id ord 1st value rounded value load
10 1 1 0.989 0.90 10.9 (= 10 + 0.9)
10.9 1 2 0.218 0.20 11.1 (= 10.9 + 0.2)
11.1 1 3 1.658 1.60 12.7 (= 11.1 + 1.6)
11.06 1 4 -0.635 -0.64 11.06 (=12.7 + (-0.64))
Desired output
Using results in Calculation Table, we update the original tableA and tableB as follows:
tableB
Load id id ord 1st value rounded value state
1 1 1 0.989 0.90 calculated
2 1 2 0.218 0.20 calculated
3 1 3 1.658 1.60 calculated
4 1 4 -0.635 -0.64 calculated
5 2 1 ... .... calculated
6 2 2 ... .... calculated
tableA (Note: for each value in `load id`, the `load` column only stores the **last** value in the `calculation` table which corresponds to maximum `ord`)
id balance load state
1 10 9.5 calculated
2 22 ... calculated
3 31 ... calculated
Can anyone please help me with the syntax for creating tableB, computing and store results for columns 1st value, rounded value and load into a calculation table with Inner Join function on id column between tableA and tableB , and update tableB?
My attempt:
First step (Creating two tables A and B with column fields shown above)
CREATE TABLE tableA;
( id int, ;
balance double, ;
load C(240), ;
state C(240), ;)
CREATE TABLE tableB;
( Load id int, ;
id int, ;
ord int, ;
fact double, ;
type binary (not sure....) ;
1st value C(240),;
rounded value C(240), ;
state C(240), ;)
(adding as another answer just because others got too long to read)
can you try your code with this dataset
(drive.google.com/open?id=1uCWwt5ubd2_F8w2gsh3v4VDpibWz7PAz) to see if
you will get the two output tables from your code, each similar to the
one shown in the previous Excel worksheet I uploaded for you?
I downloaded that spreadsheet and here is what I needed to change:
Your ranges were C8:F35 and H8:O62 for tableA and B. Also your "balance" was named "base". New code (downloaded to d:\temp\workbook2.xlsx) edited to match ranges and "balance" to "base":
* Get the data from given excel filename and ranges
* first range is tableA, second one is tableB
GetDataFromExcel("d:\temp\WorkBook2.xlsx", "Sheet1$C8:F35", "Sheet1$H8:O62")
* Now data is in cursors csrA and crsB do the calculation in these
DoCalculation()
* Done. Show the results selecting and browsing the crsA and B
Select crsA
Browse
Select crsB
Browse
* Get specific fields only from crsB
Select loadId, id, ord, firstVal, roundedVal, state ;
from crsB ;
into cursor crsBCustom ;
nofilter
browse
* Check data from both cursors (join)
* I chose the fields as I see fit
* ta and tb are local aliases for crsA and crsB
* helping to write shorter SQL in this case
Select tb.LoadId, tb.Id, ta.base, ta.load, ;
tb.firstValue, tb.roundVal, ;
ta.State as StateA, tb.State as StateB ;
from crsA ta ;
inner join crsB tb on ta.Id = tb.Id ;
order by tb.Id, tb.Ord ;
into cursor crsBoth ;
NoFilter
browse
* Does the specific calculations on specific data
Procedure DoCalculation
*1st value[1] = If(Type[1]=0, Base[1]*fact[1], Base[1]*fact[1]/(1-fact[1]))
*rounded value[i] = If(1st value[i]>0, rounddown(1st value[i], 1), roundup(1st value[i],2)
*rounded value[1] = If(1st value[1]>0, rounddown(1st value[1], 1), roundup(1st value[1],2)
*load[1] = Base[1] + rounded value[1]
* i > 1 - ord > 1
*1st value[i] = If(Type[i]=0, load[i-1]*fact[i], load[i-1]*fact[i]/(1-fact[i]))
*rounded value[i] = If(1st value[i]>0, rounddown(1st value[i], 1), roundup(1st value[i],2)
*load[i+1] = load[i] + rounded value[i+1] (i >= 1)
*declare local variable
Local lnBase
* select crsB and create an index there
Select CrsB
Index On Padl(Id,10,'0')+Padl(ord,10,'0') Tag ALinkB
* select crsA as parent and link to crsB
* using the "id" part of index
Select crsA
Set Relation To Padl(Id,10,'0') Into CrsB
* start looping the rows
Scan
* working with a new Id (1, 2, ...)
* save base value to m.lnBase
lnBase = crsA.Base
* select crsB and start looping the rows there
* because of the index in effect and the relation created
* pointer would be on the first crsB row with a matching Id
* and since Ord is also part of the index the first row of
* given Id
* Limit the looping in crsB (child table) to Id in crsA
* using WHILE clause
Select CrsB
Scan While Id = crsA.Id
* do replacing starting on first row of this Id (Ord=1)
* we don't have any scope clauses in replace, thus
* we are doing "single row" updates
Replace ;
firstValue With m.lnBase*fact / Iif(!Type, 1, 1-fact), ;
roundVal With Iif(firstValue > 0, ;
roundDown(firstValue,1), ;
roundUp(firstValue, 2))
* after each replace update m.lnBase value
* to use in next row
lnBase = m.lnBase + CrsB.roundVal
Endscan
* completed updating crsB
* select crsA and also update crsA.base with final 'load' value
Select crsA
Replace Load With m.lnBase
Endscan
* Update state to 'Calculated'
Update crsA set state = 'Calculated'
Update crsB set state = 'Calculated'
Endproc
* Get data from excel with given filename and ranges
* This code is not generic and expects the
* data to be in a specific format.
* Does not do any error check
Procedure GetDataFromExcel(tcExcelFileName, tcTableARange, tcTableBRange)
* declare and define the connection string to excel
Local lcConStr
lcConStr = ;
'Provider=Microsoft.ACE.OLEDB.12.0;'+;
'Data Source='+Fullpath(m.tcExcelFileName)+';'+;
'Extended Properties="Excel 12.0;HDR=Yes"'
* Declare and define the 2 SQL needed to get data for A and B
* rename the fields in SQL for easier handling
Local lcSQLA, lcSQLB
TEXT to lcSQLA textmerge noshow
Select [id], [base], [load], [state]
from [<< m.tcTableARange >>]
ENDTEXT
TEXT to m.lcSQLB textmerge noshow
select
[Load Id] as LoadId,
[Id], [Ord], [Fact], [Type],
[1st value] as firstValue,
[Rounded value] as roundVal,
[State]
from [<< m.tcTableBRange >>]
ENDTEXT
* Execute the queries and place results in given cursors
ADOQuery(m.lcConStr, m.lcSQLA, "crsTableA")
ADOQuery(m.lcConStr, m.lcSQLB, "crsTableB")
* Sanitize the cursors a bit
* (OledB query would assign rather generic datatypes)
Select Cast(Id As Int) As Id, Cast(Base As Double) As Base, ;
Cast(Load As Double) As Load, Cast(State As c(50)) As State ;
from crsTableA ;
into Cursor crsA ;
readwrite
Select Cast(LoadId As Int) As LoadId, ;
Cast(Id As Int) As Id, Cast(ord As Int) As ord, ;
Cast(fact As Double) As fact, Cast(Type As logical) As Type, ;
Cast(firstValue As Double) As firstValue, ;
Cast(roundVal As Double) As roundVal, ;
Cast(State As c(50)) As State From crsTableB ;
into Cursor CrsB ;
readwrite
Use In (Select('crsTableA'))
Use In (Select('crsTableB'))
Endproc
* roundUp and down custom functions
* RoundUp and Down excel style
* Not correct math wise IMHO
Procedure roundUp(tnValue, tnPlaces)
Local lnResult, lnValue
lnValue = Abs(m.tnValue)
If Round(m.lnValue, m.tnPlaces) != m.lnValue
lnValue = Round(m.lnValue+((10^-(m.tnPlaces+1))*5), m.tnPlaces)
Endif
Return Sign(m.tnValue) * m.lnValue
Endproc
Procedure roundDown(tnValue, tnPlaces)
Local lnResult, lnValue
lnValue = Abs(m.tnValue)
If Round(m.lnValue, m.tnPlaces) != m.lnValue
lnValue = Round(m.lnValue-((10^-(m.tnPlaces+1))*5), m.tnPlaces)
Endif
Return Sign(m.tnValue) * m.lnValue
Endproc
* Generic function to query a given data source
* and place results in a cursor
Procedure ADOQuery(tcConStr,tcQuery,tcCursorName)
Local oConn As 'ADODB.Connection'
Local oRS As ADODB.RecordSet
oConn = Createobject('ADODB.Connection')
oConn.Mode= 1 && adModeRead
oConn.Open( m.tcConStr )
oRS = oConn.Execute(m.tcQuery)
RS2Cursor(oRS,m.tcCursorName)
oRS.Close
oConn.Close
Endproc
* Helper function to ADOQuery to convert
* an ADODB.Recordset to a VFP cursor
Procedure RS2Cursor(toRS, tcCursorName) && simple single cursor - not intended for complex ones
tcCursorName = Iif(Empty(m.tcCursorName),'ADORs',m.tcCursorName)
Local xDOM As 'MSXML.DOMDocument'
xDOM = Createobject('MSXML.DOMDocument')
toRS.Save(xDOM, 1)
Xmltocursor(xDOM.XML, m.tcCursorName)
Endproc
This is the whole code. Just changing the filepath and name to yours, select all the code, right click and execute selection to see results. Or save it as a prg, say ImportMyExcel.prg and run it:
ImportMyExcel()
You could see the results I have so I didn't upload any results.
Also, is Procedure RS2Cursor(toRS, tcCursorName) intended to generate
the 2 output tables? Why do we need this procedure though: Procedure
ADOQuery(tcConStr,tcQuery,tcCursorName)?
Well those procedures are a little tricky for a newcomer (maybe not). I think you should know the history of VFP, cursors, cursor adapters, converting ADO recordset to a cursor etc (probably advanced level). I don't know, those were the procedures I came up with and published also on the foxite link that I gave to you. Just think they are black boxed (like a built-in one) functions doing they are work. ADOQuery's work is to simply query an OLEDB source and return the result as a cursor. With a cursorAdapter you might not need such a procedure but that procedure was designed before CursorAdapter existence.
Two more questions please: 1) where does the m come from in
m.lnBalance?
m. explicitly notifies the compiler that it is a memory variable. It is referred to as MDOT. There are developers who claim it is not needed and generally it leads to long running discussions (and likely you would find my name in those discussions). Up until today nobody could show and\or demonstrate me why we shouldn't or we don't need to use it. If you believe me it is not a preference but a thing that you should use.
2) Don't we need to define crsTableA? Or you meant we can use the
CREATE Table tableA in your previous code to make crsTableA valid?
No. There is no table in that code. We read the data from excel into a cursor (crsTableA and crsTableB initially) and then sanitize into 2 cursors crsA and crsB. All of them are cursors. Cursors are like tables but are not persisted on disk. They may even spend all their life in memory and are gone when you close them. Here I preferred cursors because without harming any real data you could run N times and check your results. When you are satisfied persisting the data is as simple as a "Select ... into" or "insert into ..." (there are more ways too) a table. Even in the case of a table you don't need to use "Create Table ...". A "select Into ..." command can select the data from a source and save it to a table by creating it (like a combined 'create table ...' and then 'insert into ...').
Also, I saw that B9:E12 does not match the range of tableA or tableB
in the Excel spreadsheet I uploaded for you before. Am I missing
something here?
It matched your original samples if you think data starts at B9 and G9 respectively.
I have another question: can you please clarify on what these lines
do: Select CrsB Index On Padl(Id,10,'0')+Padl(ord,10,'0') Tag
ALinkB Select crsA Set Relation To Padl(Id,10,'0') Into CrsB.
I think I explained this part in the previous question. I will soon comment the code itself.
Adding as another answer to prevent clutter. I can do further explanations if you need to. Here I used the Excel ranges that would match to sample data. You would replace the range with the actual one (as well as the excel filename):
GetDataFromExcel("c:\myFolder\myExcel.xlsx", "B9:E12", "G9:N19")
DoCalculation()
Select crsA
Browse
Select crsB
Browse
Procedure DoCalculation
*1st value[1] = If(Type[1]=0, balance[1]*fact[1], balance[1]*fact[1]/(1-fact[1]))
*rounded value[i] = If(1st value[i]>0, rounddown(1st value[i], 1), roundup(1st value[i],2)
*rounded value[1] = If(1st value[1]>0, rounddown(1st value[1], 1), roundup(1st value[1],2)
*load[1] = balance[1] + rounded value[1]
* i > 1 - ord > 1
*1st value[i] = If(Type[i]=0, load[i-1]*fact[i], load[i-1]*fact[i]/(1-fact[i]))
*rounded value[i] = If(1st value[i]>0, rounddown(1st value[i], 1), roundup(1st value[i],2)
*load[i+1] = load[i] + rounded value[i+1] (i >= 1)
Local lnBalance
Select CrsB
Index On Padl(Id,10,'0')+Padl(ord,10,'0') Tag ALinkB
Select crsA
Set Relation To Padl(Id,10,'0') Into CrsB
Scan
lnBalance = crsA.Balance
Select CrsB
Scan While Id = crsA.Id
Replace ;
firstValue With m.lnBalance*fact / Iif(!Type, 1, 1-fact), ;
roundVal With Iif(firstValue > 0, ;
roundDown(firstValue,1), ;
roundUp(firstValue, 2))
lnBalance = m.lnBalance + CrsB.roundVal
Endscan
Select crsA
Replace Load With m.lnBalance
Endscan
Endproc
Procedure GetDataFromExcel(tcExcelFileName, tcTableARange, tcTableBRange)
Local lcConStr
lcConStr = ;
'Provider=Microsoft.ACE.OLEDB.12.0;'+;
'Data Source='+Fullpath(m.tcExcelFileName)+';'+;
'Extended Properties="Excel 12.0;HDR=Yes"'
Local lcSQLA, lcSQLB
TEXT to lcSQLA textmerge noshow
Select [id], [balance], [load], [state]
from [Sheet1$<< m.tcTableARange >>]
ENDTEXT
TEXT to m.lcSQLB textmerge noshow
select
[Load Id] as LoadId,
[Id], [Ord], [Fact], [Type],
[1st value] as firstValue,
[Rounded value] as roundVal,
[State]
from [Sheet1$<< m.tcTableBRange >>]
ENDTEXT
ADOQuery(m.lcConStr, m.lcSQLA, "crsTableA")
ADOQuery(m.lcConStr, m.lcSQLB, "crsTableB")
Select Cast(Id As Int) As Id, Cast(Balance As Double) As Balance, ;
Cast(Load As Double) As Load, Cast(State As c(1)) As State ;
from crsTableA ;
into Cursor crsA ;
readwrite
Select Cast(LoadId As Int) As LoadId, ;
Cast(Id As Int) As Id, Cast(ord As Int) As ord, ;
Cast(fact As Double) As fact, Cast(Type As logical) As Type, ;
Cast(firstValue As Double) As firstValue, ;
Cast(roundVal As Double) As roundVal, ;
Cast(State As c(1)) As State From crsTableB ;
into Cursor CrsB ;
readwrite
Use In (Select('crsTableA'))
Use In (Select('crsTableB'))
Endproc
Procedure roundUp(tnValue, tnPlaces)
If Round(m.tnValue, m.tnPlaces) = m.tnValue
Return m.tnValue
Else
Return Round(m.tnValue+((10^-(m.tnPlaces+1))*5), m.tnPlaces)
Endif
Endproc
Procedure roundDown(tnValue, tnPlaces)
If Round(m.tnValue, m.tnPlaces) = m.tnValue
Return m.tnValue
Else
Return Round(m.tnValue-((10^-(m.tnPlaces+1))*5), m.tnPlaces)
Endif
Endproc
Procedure ADOQuery(tcConStr,tcQuery,tcCursorName)
Local oConn As 'ADODB.Connection'
Local oRS As ADODB.RecordSet
oConn = Createobject('ADODB.Connection')
oConn.Mode= 1 && adModeRead
oConn.Open( m.tcConStr )
oRS = oConn.Execute(m.tcQuery)
RS2Cursor(oRS,m.tcCursorName)
oRS.Close
oConn.Close
Endproc
Procedure RS2Cursor(toRS, tcCursorName) && simple single cursor - not intended for complex ones
tcCursorName = Iif(Empty(m.tcCursorName),'ADORs',m.tcCursorName)
Local xDOM As 'MSXML.DOMDocument'
xDOM = Createobject('MSXML.DOMDocument')
toRS.Save(xDOM, 1)
Xmltocursor(xDOM.XML, m.tcCursorName)
Endproc
EDIT: I edited the other answer for the comments beneath it. Now for your questions:
Shouldn't GetDataFromExcel("c:\myFolder\myExcel.xlsx", "B9:E12", "G9:N19") get called after the Procedure Procedure
GetDataFromExcel(tcExcelFileName, tcTableARange, tcTableBRange)??
No. Procedures are always placed after normal execution code in a prg file. IOW if your PRG has:
Do Something
* ...
Procedure SomeProcedure
* ...
endproc
Procedure Something
endproc
Code starts with calling Something and executes the lines after that up until it sees the first Procedure call (or FUNCTION, DEFINE CLASS). Something might be a procedure (as in the sample) or a separate prg.
Shouldn't Procedure roundUp and Procedure roundDown get called before roundDown(firstValue,1), ; roundUp(firstValue, 2))??
No, same as the above. What you say more looks like the rules of core C.
Does the left ID on this line Scan While Id = crsA.Id come from CrsB?? Also, why is there the change from crsA to CrsA? Is this a
typo? – user177196 5 mins ago
Yes. it comes from crsB. But in a sense, you are right I should be explicit and include the alias there as:
Scan while crsB.Id = crsA.Id
In VFP if you don't include an alias, then the one that is current is assumed.
We are scanning crsA in outer loop. Then we are switching to crsB and scanning there, after we are done switching back to crsA (actually scan command remembers the alias it is associated and does this switch when it hits endscan implicitly but I prefer to be explicit).
EDIT:
Select CrsB
Index On Padl(Id,10,'0')+Padl(ord,10,'0') Tag ALinkB
Select crsA
Set Relation To Padl(Id,10,'0') Into CrsB
On first two lines we are selecting crsB cursor and creating an index on it. Index expression contains both the Id and Old fields. VFP doesn't support multiple column names in an index key, but it supports expressions. Padding both fields with 10 zeros we are creating keys like:
Id, Ord: 2,3 as an example has a key 00000000020000000003
We could make it smaller but anyway since not knowing how much big the Id,Ord could be made it 10 in length to fit any 32 bits integer value.
Then on 3rd, 4th lines we are selecting cursor crsA and then setting relation from crsA into crsB via the expression Padl(Id,10,'0') - Id padded with 10 zeros. From crsA Id:1 has a relation key of 0000000001 then (matching all index keys that start with 0000000001 whatever the Ord part is - BTW having Ord in index too makes sure that they are ordered by Ord).
In effect, when the record pointer points to Id:1 in crsA, in crsB automatically those with Id:1 are matched (best observed with a browse - browse crsB then select crsA and browse. As you navigate in crsA, you would see the browse window for crsB would show only the rows with matching Id). Conceptually it looks like this controlling the record pointer in both cursors:
crsA (id) crsB (Id, Ord)
1 ----+------- 1,1
+------- 1,2
+------- 1,3
+------- 1,4
2 ----+------- 2,1
+------- 2,2
+------- 2,3
I used that because it is a powerful feature of VFP was an easier way to express what you want. The same could be achieved by using SQL Update too, however, VFP's SQL is not that much powerful and would be much more complex to write (For [1] easy but for > 1 case it gets complex - it was also not so easy in other backends too in distant past but in time, backends like postgreSQL, MS SQL server ... etc have gained much more support for such queries).
Well you have a long question, containing multiple questions within. I will try to reply in pieces (editing my answer in between), since it would be a long answer (might even be good to divide into multiple answers).
First, your create table syntax was close but incorrect. VFP (it is not VFB but V FP by the way), does not support spaces in field names (unless it is a long fieldname). Using field names with spaces would just be asking for trouble. So prefer not using them. It would look like:
CREATE TABLE tableA;
( id int, ;
balance double, ;
load C(240), ;
state C(240))
CREATE TABLE tableB;
( Load id int, ;
id int, ;
ord int, ;
fact double, ;
type int ;
firstValue C(240),;
roundedVal C(240), ;
state C(240))
Note that after final field you don't have comma and ; in VFP means continue the command on next line (so removed in last field definition lines). I also changed the 2 field names to be compatible with a free table's field naming (max 10 in length and must start with a letter, no spaces). Easier to use the tables this way.or cursors provided you do it in one shot and do not try to change the structure later.
If you want to use longfieldnames then you can do that just as you do with free tables but the table needs to be part of a database. It would also work for cursors provided you do that in one shot and do not attempt to alter the structure afterwards.
While I added code there to create TableA, TableB, you are saying those tables' data would come from Excel. You didn't really give detailed information about the Excel part of it (how data is represented-is that as a data ranges?). There is a great probability that you create these two tables simply by selecting the data from Excel using ODBC/OLEDB directly.
For getting data from Excel I posted some detailed information on Foxite, you can check the post in this link. I am not giving any sample code here as I don't yet know the Excel part really.
Assuming we got the data from Excel let's check other parts (BTW in table B id is called a Foreign Key, not primary. It links the rows in TableB top TableA).
1st value[i] = If(Type[i]=0, balance[i]*fact[i], balance[i]*fact[i]/(1-fact[i]))
We can use either REPLACE command (xBase command) or SQL Update command to accomplish this. Let's do not think about the differences here (not worth really) and choose SQL Update to do job (the syntax would be reusable in other databases too - say MS SQL server, postgreSQL, mySQL ...).
Update tableB ;
set firstValue = iif( type = 0, ;
tableA.balance * fact, ;
tableA.balance * fact/(1-fact)) ;
from tableA ;
where tableA.Id = tableB.Id
Or slightly simplified:
Update tableB ;
set firstValue = tableA.balance * fact / ;
iif( type = 0, 1, (1-fact)) ;
from tableA ;
where tableA.Id = tableB.Id
Note that VFP would execute this expression per row so we don't need the [i] (array identifier) that you have in your pseudocode.
Next one:
rounded value[i] = If(Type[i]>0, rounddown(1st value[i], 1), roundup(1st value[i],2)
Would be translated in the same manner:
Update tableB ;
set roundVal = iif(type > 0, ;
rounddown(firstValue,1), ;
roundup(firstValue,2)) ;
from tableA ;
where tableA.Id = tableB.Id
However, VFP doesn't have roundup and rounddown functions, I only wrote these as a conceptual translation. What you can do is to create two custom functions that does RoundUp and RoundDown. There are multiple ways to write these functions and IMHO the easiest would be to write them as 2 separate .prg files where those prg files are in your search path when you execute the above SQL command:
RoundUp.prg
Lparameters tnValue, tnPlaces
If Round(m.tnValue, m.tnPlaces) = m.tnValue
Return m.tnValue
Else
Return Round(m.tnValue+((10^-(m.tnPlaces+1))*5), m.tnPlaces)
Endif
RoundDown.prg
Lparameters tnValue, tnPlaces
If Round(m.tnValue, m.tnPlaces) = m.tnValue
Return m.tnValue
Else
Return Round(m.tnValue-((10^-(m.tnPlaces+1))*5), m.tnPlaces)
Endif
The functions in the link you provided doesn't seem right to me for the job (but was not easy to understand and test so didn't spend time on checking thoroughly).
I am not sure if one sheet containing both tables is good. I don't remember off the top of my head, if Tables collection was a member of the WorkSheet or WorkBook. If WorkSheet then that would do. I can check and write sample code for that later (possibly tomorrow).
You could use datatype LOGICAL (l) for Type. In MS SQL server and other backends it correspond to bit (1 or 0). Internally stored as boolean but in expressions used as .T./.F. (true\false symbolic representation in VFP. On code you could simply use it as:
iif( type, ...
same as saying iif(type = .T., ...) - as in Type > 0. And:
iif( !type, ...
same as saying iif( type = .F., ...) or iif( type NOT equal to .T., ... - as in Type = 0.
I didn't use inner join in this case, because it is sufficient to use a from TableA where here (same in other backends, although general tendency is to write that using join).
EDIT: Added the code as another answer.
As per your questions:
Inner join is not needed to be explicitly defined, there is an implicit join there. Instead of writing an SQL update, I preferred to utilize VFP's xBase capabilities and used scan...endscan instead (could do with SQL but would be more complex).
Yes it means putting those 2 RoundUp.prg and RoundDown.prg files into the same directory path of our main file code above BUT only if main file code is in current directory or in search path. To make it more clear, consider:
c:\SomeFolder\RoundUp.prg
c:\SomeFolder\RoundDown.prg
c:\ANOTHERFolder\Main.prg
and you are in:
c:\YetAnotherFolder
If you call main.prg like this:
do ('c:\ANOTHERFolder\Main.prg')
It needs to find RoundUp, RoundDown and it can if c:\Somefolder is included in SET('PATH') - ie:
Set path to c:\SomeFolder;c:\VFPHomeFolderMaybe
Or if you don't want to think of pathing you could include those RoundUp\Down code as procedure in the code (as I did in the code in the other answer - note that in VFP there is no difference between a PROCEDURE and a FUNCTION. You are free to choose either one. Some developers prefer to use FUNCTION for those that return a value - but in fact any PROCEDURE\FUNCTION returns a value so let's say those that are used for a return value.)
I don't think logical type mean "1" or "0" automatically, correct? If
that's the case, I would have to leave it as int type, because the
input is always defined as 1 or 0 for type column.
Well, that is hard to answer formally. In VFP boolean data
type is defined by literals .F. and .T. You can cast(aBoolean to int) and you get 0 and 1 respectively. Or you can cast(1 as logical) to get .T. IOW 1\0 and .T..F. are interchangeable in a sense. It all depends where you want to use it. If data is coming from external source, it would come in as 1\0. Just by casting or getting it into column of datatype logical (implicit cast) it is treated as .T..F. Or you are sending data from a logical to an external source (say an XML, MS SQL server, postgreSql, other OLEDB\ODBC datasource) then .T..F. is casted as 1\0.
In SQL there are aggregation operators, like AVG, SUM, COUNT. Why doesn't it have an operator for multiplication? "MUL" or something.
I was wondering, does it exist for Oracle, MSSQL, MySQL ? If not is there a workaround that would give this behaviour?
By MUL do you mean progressive multiplication of values?
Even with 100 rows of some small size (say 10s), your MUL(column) is going to overflow any data type! With such a high probability of mis/ab-use, and very limited scope for use, it does not need to be a SQL Standard. As others have shown there are mathematical ways of working it out, just as there are many many ways to do tricky calculations in SQL just using standard (and common-use) methods.
Sample data:
Column
1
2
4
8
COUNT : 4 items (1 for each non-null)
SUM : 1 + 2 + 4 + 8 = 15
AVG : 3.75 (SUM/COUNT)
MUL : 1 x 2 x 4 x 8 ? ( =64 )
For completeness, the Oracle, MSSQL, MySQL core implementations *
Oracle : EXP(SUM(LN(column))) or POWER(N,SUM(LOG(column, N)))
MSSQL : EXP(SUM(LOG(column))) or POWER(N,SUM(LOG(column)/LOG(N)))
MySQL : EXP(SUM(LOG(column))) or POW(N,SUM(LOG(N,column)))
Care when using EXP/LOG in SQL Server, watch the return type http://msdn.microsoft.com/en-us/library/ms187592.aspx
The POWER form allows for larger numbers (using bases larger than Euler's number), and in cases where the result grows too large to turn it back using POWER, you can return just the logarithmic value and calculate the actual number outside of the SQL query
* LOG(0) and LOG(-ve) are undefined. The below shows only how to handle this in SQL Server. Equivalents can be found for the other SQL flavours, using the same concept
create table MUL(data int)
insert MUL select 1 yourColumn union all
select 2 union all
select 4 union all
select 8 union all
select -2 union all
select 0
select CASE WHEN MIN(abs(data)) = 0 then 0 ELSE
EXP(SUM(Log(abs(nullif(data,0))))) -- the base mathematics
* round(0.5-count(nullif(sign(sign(data)+0.5),1))%2,0) -- pairs up negatives
END
from MUL
Ingredients:
taking the abs() of data, if the min is 0, multiplying by whatever else is futile, the result is 0
When data is 0, NULLIF converts it to null. The abs(), log() both return null, causing it to be precluded from sum()
If data is not 0, abs allows us to multiple a negative number using the LOG method - we will keep track of the negativity elsewhere
Working out the final sign
sign(data) returns 1 for >0, 0 for 0 and -1 for <0.
We add another 0.5 and take the sign() again, so we have now classified 0 and 1 both as 1, and only -1 as -1.
again use NULLIF to remove from COUNT() the 1's, since we only need to count up the negatives.
% 2 against the count() of negative numbers returns either
--> 1 if there is an odd number of negative numbers
--> 0 if there is an even number of negative numbers
more mathematical tricks: we take 1 or 0 off 0.5, so that the above becomes
--> (0.5-1=-0.5=>round to -1) if there is an odd number of negative numbers
--> (0.5-0= 0.5=>round to 1) if there is an even number of negative numbers
we multiple this final 1/-1 against the SUM-PRODUCT value for the real result
No, but you can use Mathematics :)
if yourColumn is always bigger than zero:
select EXP(SUM(LOG(yourColumn))) As ColumnProduct from yourTable
I see an Oracle answer is still missing, so here it is:
SQL> with yourTable as
2 ( select 1 yourColumn from dual union all
3 select 2 from dual union all
4 select 4 from dual union all
5 select 8 from dual
6 )
7 select EXP(SUM(LN(yourColumn))) As ColumnProduct from yourTable
8 /
COLUMNPRODUCT
-------------
64
1 row selected.
Regards,
Rob.
With PostgreSQL, you can create your own aggregate functions, see http://www.postgresql.org/docs/8.2/interactive/sql-createaggregate.html
To create an aggregate function on MySQL, you'll need to build an .so (linux) or .dll (windows) file. An example is shown here: http://www.codeproject.com/KB/database/mygroupconcat.aspx
I'm not sure about mssql and oracle, but i bet they have options to create custom aggregates as well.
You'll break any datatype fairly quickly as numbers mount up.
Using LOG/EXP is tricky because of numbers <= 0 that will fail when using LOG. I wrote a solution in this question that deals with this
Using CTE in MS SQL:
CREATE TABLE Foo(Id int, Val int)
INSERT INTO Foo VALUES(1, 2), (2, 3), (3, 4), (4, 5), (5, 6)
;WITH cte AS
(
SELECT Id, Val AS Multiply, row_number() over (order by Id) as rn
FROM Foo
WHERE Id=1
UNION ALL
SELECT ff.Id, cte.multiply*ff.Val as multiply, ff.rn FROM
(SELECT f.Id, f.Val, (row_number() over (order by f.Id)) as rn
FROM Foo f) ff
INNER JOIN cte
ON ff.rn -1= cte.rn
)
SELECT * FROM cte
Not sure about Oracle or sql-server, but in MySQL you can just use * like you normally would.
mysql> select count(id), count(id)*10 from tablename;
+-----------+--------------+
| count(id) | count(id)*10 |
+-----------+--------------+
| 961 | 9610 |
+-----------+--------------+
1 row in set (0.00 sec)
Oracle behaves really extrange with the next query:
I am trying to evaluate these three records, one of them should show the column digitado = 1 because it accomplishes all the conditions, which are, NUM_DOCUMENTO_ENCABEZADO iS NOT NULL and ORIGEN_PLANILLA = 2
NUM_DOCUMENTO NUM_DOCUMENTO_ENCABEZADO ORIGEN_PLANILLA
8220568059 8220568059 2
8220681644 2
940723593097 1
select x.num_documento,
x.origen_planilla,
x.num_documento_encabezado,
case
when x.num_documento_encabezado > '0' and x.origen_planilla = 2 then
1
else
0
end digitado
from (
select h.num_documento,
h.num_documento_encabezado,
h.origen_planilla
from (
select a.num_documento,
c.num_documento num_documento_encabezado,
case when NVl(UPPER(a.txt_observacion),'X') like '%SGP%' THEN 1 ELSE 2 END origen_planilla
from epsis.ryc_recaudo a,
epsis.ryc_recaudo_unificado b,
epsis.ryc_documento_encabezado c
where a.fec_pago >= to_date('28082013','ddmmyyyy') ---aca se coloca el dia del ultimo proceso,
and a.fec_pago < to_date('25092013','ddmmyyyy')-- el cecaudo viene un dia atrasados
and b.num_documento(+) = a.num_documento
and c.num_documento(+) = b.num_documento --80595
and a.num_documento in ( '940723593097', '8220681644','8220568059')
) h,
epsis.ryc_divide_documento f,
epsis.ryc_documento_encabezado g
where f.num_documento(+) = h.num_documento
and g.num_documento(+) =f.num_division
group by h.num_documento,
h.num_documento_encabezado,
h.origen_planilla
) x
This is the result:
NUM_DOCUMENTO ORIGEN_PLANILLA NUM_DOCUMENTO_ENCABEZADO DIGITADO
8220568059 2 8220568059 0
8220681644 2 0
940723593097 1 0
The column DIGITADO should be "1" for the first record.
Oracle can not evaluate this "CASE" properly:
case
when x.num_documento_encabezado > '0' and x.origen_planilla = 2 then
1
else
0
end digitado
I have tried diferent things, for example if I change the previous code for this:
case
when length(x.num_documento_encabezado||x.origen_planilla) > 1 then
1
else
0
end digitado
This is the result:
NUM_DOCUMENTO ORIGEN_PLANILLA NUM_DOCUMENTO_ENCABEZADO DIGITADO
8220568059 2 8220568059 1
8220681644 2 0
940723593097 1 0
It works for every record, but that is not the point, the point is that oracle is not able to evaluate the "AND" expression, and the ortiginal query is much longer than the example displayed.
Now, another extrange this is that, when I execute the query only for the record that is ok, I mean this
and a.num_documento in ('8220568059')
the "AND" expression in the case sentence works properly with the original "CASE".
Result:
NUM_DOCUMENTO ORIGEN_PLANILLA NUM_DOCUMENTO_ENCABEZADO DIGITADO
8220568059 2 8220568059 1
Another thing is that, and here is where i believe the problem is, when no outer join in the second subquery, then the query runs ok, but I need that outer join, I am talking about this:
where f.num_documento(+) = h.num_documento
and g.num_documento(+) =f.num_division
I really don't want to rewrite the full query, does anyone know why this happen?
Create and insert statements, these ones reproduce the issue
create table tmp_origin
(
doc varchar2(30),
val number,
obs varchar2(30)
);
create table tmp_uni
(
doc varchar2(30),
doc_origin varchar2(30)
);
create table tmp_div
(
doc varchar2(30),
doc_div varchar2(30)
);
insert into tmp_origin values ('8220568059',100000, 'NORMAL');
insert into tmp_origin values ('8220681644',200000, 'NORMAL');
insert into tmp_origin values ('940723593097',300000, 'SGP');
commit;
insert into tmp_uni values ('8220568059','8220568059');
commit;
This is the query adapted to the above lines, I have also added some others cases, so you can compare and identify that the first one is not working
select x.num_documento,
x.origen_planilla,
x.num_documento_encabezado,
case
when x.num_documento_encabezado is not null and x.origen_planilla = 2 then
1
else
0
end digitado,
case
when length(x.num_documento_encabezado||x.origen_planilla) > 1 then
1
else
0
end digitado2,
case
when x.origen_planilla = 2 then
case
when x.num_documento_encabezado is not null then
1
else
0
end
else
0
end digitado3
from (
select h.num_documento,
h.num_documento_encabezado,
h.origen_planilla
from (
select a.doc num_documento,
b.doc num_documento_encabezado,
case when NVl(UPPER(a.obs),'X') like '%SGP%' THEN 1 ELSE 2 END origen_planilla
from tmp_origin a,
tmp_uni b
where a.doc in ( '940723593097', '8220681644','8220568059')
and b.doc(+) = a.doc
) h,
tmp_div f
where f.doc(+) = h.num_documento
group by h.num_documento,
h.num_documento_encabezado,
h.origen_planilla
) x
You should almost never use the comparison operators with VARCHAR2, it is almost never useful (except if you are writing a sorting algorithm). In your case especially, it doesn't do what you expect.
When you compare VARCHAR2s, the result will depend upon character ordering (for instance '2' is "greater" than '10' because 2 comes after 1 in the character table).
Consider:
SQL> select * from dual where '8220568059' > '0';
DUMMY
-----
X
SQL> select * from dual where ' 8220568059' > '0';
DUMMY
-----
Always use the right datatype for the right task. There is almost always only one datatype that will work correctly. You should always use NUMBER and explicit datatype conversion when working with numbers:
SQL> select * from dual where to_number('8220568059') > 0;
DUMMY
-----
X
Also if you just want to know if a value is NULL, please use the IS NOT NULL operator:
SQL> WITH DATA AS (
2 SELECT '8220568059' num_documento_encabezado,
3 2 origen_planilla FROM dual UNION ALL
4 SELECT '', 2 FROM dual UNION ALL
5 SELECT '', 1 FROM dual)
6 SELECT x.origen_planilla,
7 x.num_documento_encabezado,
8 CASE
9 WHEN x.num_documento_encabezado IS NOT NULL
10 AND x.origen_planilla = 2 THEN
11 1
12 ELSE
13 0
14 END digitado
15 FROM DATA x;
ORIGEN_PLANILLA NUM_DOCUMENTO_ENCABEZADO DIGITADO
--------------- ------------------------ ----------
2 8220568059 1
2 0
1 0