I have a situation where I have following csv file(say file.csv) with following data:
AcctId,Name,OpenBal,closingbal
1,abc,1000,
2,,0,
3,xyz,,
4,,,
how can I loop through this file using unix shell and say for example for column $2 (Name) , I want to get all occurances of Name column accept null values and pass it to for example following oracle query with single quotes '','' format?
select * from account
where name in (collection of values from csv file column name
but excluding null values)
and openbal in
and same thing for column 3 (collection of values from csv file column Openbal
but excluding null values)
and same thing for column 4 (collection of values from csv file column
closingbal but excluding null values)
In short what I want is pass the csv column values as input parameter to oracle sql query and update query too ? but again I dont want to include null values in it. If a column is entirely null for all rows I want to exclude it too?
Not sure why you'd want to loop through this file in a unix shell script: perhaps because you can't think of any better approach? Anyway, I'm going to skip that and offer a pure Oracle solution.
We can expose data in CSV files to the database using external tables. These are like regular tables except their data comes from files in OS directories on the database server (rather than the database's storage). Find out more.
Given this approach it is easy to write the query you want. I suggest using sub-query factoring to select from the external table once.
with cte as ( select name, openbal, closingbal
from your_external_tab )
select *
from account a
where a.name in ( select cte.name from cte )
and a.openbal in ( select cte.openbal from cte )
and a.closingbal in ( select cte.closingbal from cte )
The behaviour of the IN clause is to exclude NULL from consideration.
Incidentally, that will return a different (larger) result set from this:
select a.*
from account a
, your_external_table e
where a.name = e.name
and a.openbal= e.openbal
and a.closingbal = e.closingbal
Related
I am trying to replace all of the NULL values to 0 in a column of a big table in HIVE.
However, every time I try to implement some code I end up generating a new column to the table. The column I am trying to change/modify still exists and still has the NULL values but the new column that is automatically generated (i.e. _c1) is what I want the column I am trying to modify, to look like.
I tried to run a COALESCE but that also ended up generating a new column. I also tried to implement a CASE WHEN, but the same results ensued.
Select *,
CASE WHEN columnname IS NULL THEN 0
ELSE columnname
END
from tablename;
Also tried
SELECT coalesce(columnname, CAST(0 AS BIGINT)) FROM tablename
I would just like to update the table with the other columns being as is but the column I want to modify still has its original name but instead of NULL values it has 0's that replaced them.
I don't want to generate a new column but modify an existing one.
How should I do that?
Use insert overwrite .. option.
insert overwrite table tablename
select c1,c2,...,coalesce(columnname,0) as columnname
from tablename
Note that you have to specify all the other column names required in select.
How can I use a column value as a column name. I've tried this:
SELECT TableX.(
SELECT OdTable.columnamecell
from OdTable
where 1 =1
AND OdTable.KeyValue = TableX.SomeValue
) as MyValue
,TableX.OtherValue as OtherValue
, TableX.SomeValue
from TableX
WHERE 1 = 1
Or to say it another way: Can I use a table column value as a column name for another query or sub-query?
To clarify: The table: OdTable has a column with values that are the column name in another table.
No, and Yes. You can't do this with "standard" SQL; all table and column names must be known, as literals, when the query is compiled; they can't be provided at runtime. What you want is called "dynamic SQL"; sometimes it is the only solution to a problem, but most of the time it is used when it is not necessary. It has several disadvantages (security risk, performance penalty, difficulty to maintain, ...)
I have data in a file like below(comma(,) separated).
ID,Name,Sal
101,Ramesh,M,1000
102,Prasad,K,500
I want the output table to be like below
101, Ramesh M, 1000
102, Prasad K, 500
i.e Name and Surname in a single column in the output
In Hive if I give row format delimited fields terminated by ',' it will not work. Do we need to write a serde?
Solution can be in MR or PIG also.
Why you dont use concat function, if you dont want process data and just query the raw data, think about creating a view on it :
select ID,concat(Name ,' ' ,Surname),Sal from table;
You can use concat function.
First, You can create the table(i.e. table1) with raw data having 4 columns delimited by comma :
ID, first_name,last_name, salary
Then concat the first_name and last_name using select query and store the results in another table using CTAS(Create TABLE AS SELECT) feature
CREATE TABLE EMP_TABLE AS SELECT ID, CONCAT(first_name,' ','last_name) as NAME, salary from table1
I am loading data from a csv file into a table using sqlldr. There is one column which is not present in every row of the csv file. The data needed to populate this column is present in one of the other columns of the row. I need to split (split(.) )that column's data and populate into that column.
Like:-
column1:- abc.xyz.n
So the unknown column(column2) should be
column2:- xyz
Also, there is another column which is present in the row but it's not what I want to input into the table. It is also needed to be populated from column1. But there are around 50 if-else cases in that. Is decode preferable to do this?
column1:- abc.xyz.n
Then,
column2:- hi if(column1 has 'abc')
if(column1 has 'abd' then 'hello')
like this there are around 50 if-else cases.
Thanks for help.
For the first part of your question, define the column1 data in the control file as BOUNDFILLER with a name that does not match a table column name which tells sqlldr to remember it but don't use it. If you need to load it into a column, use the column name plus the remembered name. For column2, use the remembered BOUNDFILLER name in an expression where it returns the part you need (in this case the 2nd field, allowing for NULLs):
x boundfiller,
column1 EXPRESSION ":x",
column2 EXPRESSION "REGEXP_SUBSTR(:x, '(.*?)(\\.|$)', 1, 2, NULL, 1)"
Note the double backslash is needed else it gets removed as it gets passed to the regex engine from sqlldr and the regex pattern is altered incorrectly. A quirk I guess.
Anyway after this column1 ends up with "abc.xyz.n" and column2 gets "xyz".
For the second part of your question, you could use an expression as already shown but call a custom function you create where you pass the extracted value and it would return the searched value from a lookup table. You certainly don't want to hardcode your 50 lookup values. You could do the same thing basically in a table level trigger too. Note I show a select statement for an example only but this should be encapsulated in a function for reusability and maintainability:
Just to show you can do it:
col2 EXPRESSION "(select 'hello' from dual where REGEXP_SUBSTR(:x, '(.*?)(\\.|$)', 1, 2, NULL, 1) = 'xyz')"
The right way:
col2 EXPRESSION "(myschema.mylookupfunc(REGEXP_SUBSTR(:x, '(.*?)(\\.|$)', 1, 2, NULL, 1)))"
mylookupfunc returns the result of looking up 'xyz' in the lookup table, i.e. 'hello' as per your example.
I have SSRS report and I need to filter a static table that I created inside the report based on parameter. There is no data source to this table and I'm entering the data manually.
The tablix contain 3 columns.
How can I filter the columns based on parameter?
I tried in the expression =#param1 for example but it doesn't work.
For now I only manage to filter if the expression is on data source fields.
Do you literally have a table with a number of values in it written directly into the report? If so I don't think you will be able to perform any filtering on it as effectively all you've done it write data into textboxes that are displayed.
I would imagine your best option would be to instead create a new dataset and populate this with your static data, e.g.
SELECT 'A' AS Letter, 'English' AS Language
UNION
SELECT 'B' AS Letter, 'French' AS Language
UNION
SELECT 'A' AS Letter, 'German' AS Language
To give you a table as follows
Letter | Language
-------+----------
A | English
B | French
A | German
That you could then filter on Letter = A
So essentially you have a Tablix that has 3 columns pre-populated with information you have manually entered into the text boxes themselves? Since you've already entered that data, I don't believe there is a way to filter that at run time. That data is hard coded in essence. The Filter ability in SSRS is used as a WHERE clause so it restricts what is brought forth into the report from the query.
I would create a data source connection to a dummy database, create a DataSet, and create a query that fills a temporary table will all the information that you've manually entered. Once you create the temporary table and inserted values into it, you can then perform a SELECT with a parameter. Your Tablix will only be populated with information that matches the parameter. Something to the effect of this:
CREATE TABLE #TempTable (
ID INT
,Name VARCHAR(MAX)
,Email VARCHAR(MAX)
)
INSERT INTO #TempTable (
ID
,Name
,Email
)
VALUES (
1
,'Bob'
,'bob#email.com'
)
,(
2
,'Frank'
,'frank#email.com'
)
,(
3
,'Jim'
,'jim#email.com'
)
SELECT
*
FROM
#TempTable
WHERE
ID = #ID
DROP TABLE #TempTable