Doing data checking of input data in staging tables prior to loading into the main system tables. Because I want to use the same field checking in Apex I have implemented the checks as regular expressions using 'not regexp_like' to return any field that does not match its pattern. However, for a mandatory 10 character field I expected a pattern such as:
'^.{1,10}$'
to return any field that was null with the fields that were too long, it does not, null fields are not returned.
Is there a regexp pattern that will do this?
Example:
select col
from (select null col from dual
union all
select 'A string' from dual)
where not regexp_like(col, '^.{1,10}$')
Removing the 'not' returns 1 row as expected...
The point here is to force the pattern to find the null, not to use regexp_like to find the good values and then exclude them, or to use an NVL on the field. This checking is already expensive enough!
I am assuming, that like most Oracle code involving null what is happening here is that regexp_like sees null and therefore returns null?
Does Java script and Java behave in the same way?
Thanks
Bob
Oracle has 3 states in the result of a boolean expression: TRUE, FALSE or NULL (unknown).
WHERE REGEXP_LIKE( NULL, '.' )
Will return NULL and not TRUE or FALSE.
Negating the expression to:
WHERE NOT REGEXP_LIKE( NULL, '.' )
Will also return NULL and not TRUE or FALSE and, since it is not TRUE then the row will never be returned.
Instead you need to explicitly check for a NULL value:
WHERE NOT REGEXP_LIKE( col, '^.{1,10}$' )
OR col IS NULL
or, more simply:
WHERE LENGTH( col ) > 10
OR col IS NULL
If you only need to get the strings with zero or more than 10 characters, you can simply use:
length(col) > 10 or col is null
About NULL, the only check you can rely on is is [not] null.
For example:
select length(null) from dual
gives NULL, not 0.
Related
I'm trying to make a select to a bigquery table, but I need to assign a default value to a column if it is null, because next in the process I need the default or the real item_id value, I was trying to use the CASE validation but I'm not pretty sure if I can use this clause for this purpose, and I'm getting the next error:
Expected end of input but got keyword case.
Select
p.item_id CASE WHEN item_id IS NULL THEN 'XXXXX' ELSE item_id END AS item_id,
from items
where -- rest of the query
Any ideas?
try this
Select
IFNULL(p.item_id,'XXXXX') AS item_id,
from items
where -- rest of the query
IFNULL() function in BigQuery
IFNULL(expr, null_result)
Description
If expr is NULL, return null_result. Otherwise, return expr. If expr is not NULL, null_result is not evaluated.
expr and null_result can be any type and must be implicitly coercible to a common supertype. Synonym for COALESCE(expr, null_result).
more details: https://cloud.google.com/bigquery/docs/reference/standard-sql/functions-and-operators#ifnull
Use
isnull(yourcolumn,'')
--or
isnull(yourcolumn,99999999)
and you can put anything between those quotes. If the value is an int that won't work though. You will need to make it some default number, preferably something that would never be used like 9999999999.
The conventional handling of null in SQL, and the language specification, is that if any part of an expression is null, the whole expression is null.
However in Oracle, text concatenation converts null to a <blank>, eg:
select concat(concat('foo', null), 'bar') from dual; --> returns "foobar"
select 'foo' || null || 'bar' from dual; --> returns "foobar"
I want the conventional behaviour, where the result would be null if any term is null.
Is there a method or function provided by Oracle that concatenates text using a single expression, without recoding any term, such that if any term is null, the result is null?
Notes:
I don't want to repeat any terms, which would be required by a case etc, because the terms are very long and complex, and besides it's bad practice to repeat code
I can’t define any functions. I must use just a single SQL query using nothing but standard syntax and plain Oracle provided functions/operators
Side-stepping the no-repeat requirement by using a subquery or a CTE isn’t answering the question, it’s avoiding it: I want to know if Oracle can concatenate Strings using a single expression in the same way every other database I know does
Yes, concat and || do not work in a standard (as in SQL92 spec) way. I guess, this is because there is no distinction between an empty string and null value in Oracle DB.
You can create a user defined function and use it in SQL.
CREATE OR REPLACE FUNCTION standard_concat (
a VARCHAR2,
b VARCHAR2
) RETURN VARCHAR2
AS
BEGIN
IF
a IS NULL OR b IS NULL
THEN
RETURN NULL;
ELSE
RETURN a || b;
END IF;
END;
/
Using this function gives you these results:
select standard_concat(standard_concat('foo', ''), 'bar') from dual; returns null
select standard_concat(standard_concat('foo', null), 'bar') from dual; returns null
select standard_concat(standard_concat('foo', 'foo'), 'bar') from dual; returns "foofoobar"
As you can see, empty string will be treated as null since this is the way Oracle DB treats Strings. There is no way to make a distinction here.
If this function is only needed for one query you can inline your function definition into the SQL itself as in:
WITH
FUNCTION standard_concat (
a VARCHAR2,
b VARCHAR2
) RETURN VARCHAR2
AS
BEGIN
IF
a IS NULL OR b IS NULL
THEN
RETURN NULL;
ELSE
RETURN a || b;
END IF;
END;
SELECT
standard_concat(standard_concat('foo',''),'bar')
FROM
dual;
I hope it helps.
I'm not avere of a SQL function and this NULL behavior on VARCHAR2 is posible conventional, but for sure not usual expected. The reason is that Oracle doesn't distinct betwen NULL and a string with length zero (''). For string concatenation the NULLs are considered as empty strings.
Anyway you may use subqueries to avoid repeating the expressions:
with t1 as (
select 'foo' col1, null col2, 'bar' col3 from dual union all
select null col1, null col2, null col3 from dual union all
select 'foo' col1, 'baz' col2, 'bar' col3 from dual
)
select col1,col2,col3,
case when col1 is not NULL and col2 is not NULL and col3 is not NULL then
col1||col2||col3 end as concat
from t1;
returns
COL COL COL CONCAT
--- --- --- ---------
foo bar
foo baz bar foobazbar
Alternatively you may write the predicate in teh CASE statement a bit more compact using the Group Comparison Conditions
select
case when 0 < ALL(length(col1),length(col2),length(col3)) then
col1||col2||col3 end as concat
from t1;
Unfortunately the Group Comparison Conditions doesn't allow a dierct IS NULL test, so a workaround with length must be used.
The third option is a bit ugly (as requires some special string that doesn't exists in regular strings, but probably meets best your requriements.
Simple NVL all strings before concatenation and than exclude those mappend by NVL
with t2 as
(select nvl(col1,'#§$%')||nvl(col2,'#§$%')||nvl(col3,'#§$%') as concat
from t1)
select
case when concat not like '%#§$\%%' escape'\' then concat end as concat
from t2;
There ins't a single expression that will do what you want unfortunately - there isn't a stanrd-compliant equivalent of concat() or the concatenation operator.
I'm sure this doesn't meet the criteria either, but as requested (if slightly mis-advertised), an 'XNL' (well, XMLDB anyway) workaround/hack/abomination:
set null "(null)"
select xmlquery(
'if (min((string-length($x), string-length($y), string-length($z))) = 0)
then "" else concat($x, $y, $z)'
passing 'foo' as "x", null as "y", 'bar' as "z"
returning content)
from dual;
XMLQUERY('IF(MIN((STRING-LENGTH($X),STRING-LENGTH($Y),STRING-LENGTH($Z)))=0)THEN
--------------------------------------------------------------------------------
(null)
With no null expressions:
select xmlquery(
'if (min((string-length($x), string-length($y), string-length($z))) = 0)
then "" else concat($x, $y, $z)'
passing 'foo' as "x", 'bar' as "y", 'baz' as "z"
returning content)
from dual;
XMLQUERY('IF(MIN((STRING-LENGTH($X),STRING-LENGTH($Y),STRING-LENGTH($Z)))=0)THEN
--------------------------------------------------------------------------------
foobarbaz
As well as being a hack, of course, it suffers from the problem of not being general as the number of terms is fixed. I haven't come up with a way to pass any number of terms in.
I have the following piece of code:
IF i.cd_ver IS NULL OR i.cd_ver = '0'
THEN
--Set value of v_cd_ver
v_cd_ver := CASE i.cd_ver
WHEN NULL
THEN '9'
WHEN '0'
THEN '10'
END;
END if;
"i" is the record found in the cursor referenced to an external table. The value is NULL or blank in the external table:
SELECT cd_ver FROM table_xtl WHERE cd_id = '123';
..This returned one row blank.
HOWEVER, v_cd_ver did not get set to '9' as expected. My work-around was to use IF THEN statements instead and it works. Why is the CASE statement not working as expected??
UPDATE:
When I tried instead the following it worked:
v_cd_ver := CASE
WHEN i.cd_ver IS NULL
THEN '9'
WHEN i.cd_ver = '0'
THEN '10'
END;
Is this a bug or there's some reason why the former did not work?
case i.cd_ver when NULL ... is logically equivalent to case when i.cd_ver = NULL ... You know full well that such a comparison results in UNKNOWN, not in TRUE, so the case expression falls through to the default value, which is NULL. (To test this for yourself, add else 'x' before end and run it again, you'll see.)
The right way to write it is case when i.cd_ver is null then '9' when i.cd_ver = '0' then '10' end.
That form of 'simple' case expression compares the value - i.cd_ver here - with each of the comparison expressions, looking for one that is equal.
In a simple CASE expression, Oracle Database searches for the first WHEN ... THEN pair for which expr is equal to comparison_expr and returns return_expr ...
The important bit there is "for which expr is equal to ...". Since nothing is ever equal to null, even itself, there is no match.
To look for nulls you would need a searched case exprssion instead:
v_cd_ver := CASE
WHEN i.cd_ver IS NULL THEN '9'
WHEN i.cd_ver = '0' THEN '10'
END;
Not related, but are you sure 0, 9 and 10 should really be treated as strings?
Try to check empty ('') value or is NULL
I'm trying to show a default value, "Others", when the query does not return any result for one of the selected columns. I'll show you the example.
This query returns an empty value for os(agent) SO (in the first row):
select country, os(agent) SO, count(*) from clicks_data
where country is not null and os(agent) is not null
group by country, os(agent);
Output:
ZA 4
ZA Android 4
ZA Mac 8
ZA Windows 5
Instead, I would like to get this result:
ZA Others 4
ZA Android 4
ZA Mac 8
ZA Windows 5
My next attempt was this query, but it's not really working, either:
select country, regexp_replace(os(agent),'','Others') SO, count(*) from clicks_data
where country is not null and os(agent) is not null
group by country, os(agent);
This is the result:
ZA Others 4
ZA OthersAOthersnOthersdOthersrOthersoOthersiOthersdOthers 4
ZA OthersMOthersaOtherscOthers 8
ZA OthersWOthersiOthersnOthersdOthersoOtherswOtherssOthers 5
Use LENGTH() to check the length of the column value. It returns > 0, if there is some value else return 0 for empty or NULL value.
Also frame the column value in CASE WHEN ... END block
The final query may look like:
SELECT country, CASE WHEN LENGTH(os(agent)) > 0 THEN os(agent) ELSE 'Others' END AS SO, COUNT(*)
FROM clicks_data
WHERE country IS NOT NULL AND os(agent) IS NOT NULL
GROUP BY country, os(agent);
Hope this help you!!!
COALESCE will be the best suitable and optimum solution for your case
Syntax:
COALESCE(VALUE,DEFAULT_VALUE): Function returns default value when values is null else VALUE;
Query
SELECT country, COALESCE(os(agent),'Others') AS SO, COUNT(*)
FROM clicks_data
WHERE country IS NOT NULL AND os(agent) IS NOT NULL
GROUP BY country, os(agent);
Hope this would be the efficient solution for your problem.
='' maybe the easiest way to go.
e.g.
CASE WHEN col='' THEN xxx ELSE yyy END
AS col_new;
Another possible solution. Incase you would like to simply replace all the NULL values by an empty string when you export the data, you could do as such by feeding the sed command the output of your sql
$ hive -e 'set hive.cli.print.header=true; select * from db_name.table_name;' | sed 's/[\t]/,/g; s/^NULL,/,/g; s/,NULL,/,,/g; s/,NULL$/,/g;' > test.csv
Credit
For anyone who has similar issues, I'd like to summarize it here.
Well, this is a relatively old question. The provided SQL filters out NULL, so all you need to handle is the empty string "". But this conflicts with the title, which specifically says both NULL and empty strings should be considered. So I'll stick with the title.
COALESCE and NVL works only for NULL, but it doesn’t work for the empty string "".
Both LENGTH and !="" (along with CASE WHEN) are feasible as they are compatible with NULL and "". Note, when one of the parameters of the = operation is NULL, it evaluates to NULL.
One more thing to note is we should make expressions in the GROUP BY clause conform to those of the SELECT clause. By this I mean, when you SELECT an expression like CASE WHEN..., you should GROUP BY the same expression CASE WHEN.
This leads to code repetition, which can be improved by position alias (possible since version 0.11.0). So the final query may be sth like this:
--Only needed for version 0.11 through 2.1.x.
SET hive.groupby.orderby.position.alias = true;
SELECT
country,
CASE
WHEN os(agent)!="" THEN os(agent) --This also implies that it's not NULL.
ELSE 'Others'
END AS SO,
COUNT(*)
FROM clicks_data
WHERE country IS NOT NULL
GROUP BY
1, 2
;
I have a stored procedure with an optional parameter like this
create or replace
PROCEDURE "my_stored_procedure" (param1 int, param2 int default null)
IS
BEGIN
[select some fields from some tables]
...
I need a where clause with a if on the default parameter (if it is set), but I don't even know if it's possible...
Something like
where param1=123 and (if param2 is not null then some_value != param2)
The select clause it's pretty long and complex, so I'll prefer to have a "flexible" WHERE rather than a structure like
if param2 is not null then
[long and complex select] where param1=123 and some_value != param2
else
[long and complex select] where param1=123
Is this possible?
In this case you can do:
where param1=123 and (param2 is null or param2 != some_value)
If param2 is not null - then it's true only if param2 != some_value - as expected
If param2 is null - then it returns true no matter what some_value is
Initially we used this syntax:
WHERE (pParameter = COLUMN OR pParameter IS NULL)
until a database tuning specialist found out that this causes Oracle to perform full table scans and ignore the indexes, because of using OR.
Now we use
WHERE decode(pParameter,
null, 1, --if parameter is null, then return 1
COLUMN, 1, -- if parameter matches the value in the column, then return 1
0) --else return 0
= 1
this construct allows an easy and readable way to add special handling for special values
WHERE decode(pParameter,
null, 1, --if parameter is null, then allow the row
'abc', 1, --if parameter is 'abc', then always allow the row
'xyz', 1, --if parameter is 'xyz', then always reject the row
COLUMN, 1, -- if parameter matches the value in the column, then allow
0) --else return 0 to reject the row
= 1
Alternatively, you can rewrite this with COALESCE or with CASE:
WHERE COALESCE(pParameter, COLUMN, 'ok') = COALESCE(COLUMN, 'ok')
or, sample using CASE:
THEN (
case
when pParameteris null then 1
when pParameter= COLUMN then 1
else 0
end
) = 1