error using regexp_extract to extract a particular part of the string from Hive - hadoop

I have a table with a column that has urls. I want to query out a particular url param value from each record. the url param can occur in any position in the url data and the url can contain hashbangs and this param can contain special chars like -, _ and |.
data table column:
url
http://www.url.com?like=hobby&name=tom-_green
http://www.url.com?name=bob|ghost&like=hobby
and I want the query results to be
name
srini
tom-_green
bob|ghost
I tried a query like
Select regexp_extract(url, '(?<=name=)[^&?]*(?:|$&)',2) as name
From table_name
I see java exceptions when I run this query. the exceptions are pretty vague and checking if someone can help.

I found another Hive implementation for handling URLs specifically..
Select parse_url(url, 'QUERY', 'name') as name From table_name and this worked :)
ref: parse_url(string urlString, string partToExtract [, string keyToExtract])
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF

Related

Oracle - JSON_VALUE returns NULL

I have a JSON string as below, and when I run the following query, it does not return anything. The JSON is valid, so am I missing something?
SELECT JSON_VALUE(json_content, '$.section_1') FROM messages;
{"section_1":{"section_publish":true,"section_body":"<p><b>Overview<\/b><\/p><p>Launched on 19\/06\/1992, the NUVEEN NEW YORK SELECT TAX-FREE INCOME (NXN) is a passively managed exchange traded fund designed to provide a broad exposure to<\/p>"}}
From the documentation:
The function uses the path expression to evaluate expr and find a scalar JSON value that matches, or satisfies, the path expression.
Your path expression would return a JSON object, not a scalar, I believe. That's an error and the default behavior is to return null. You could try adding an ERROR ON ERROR clause and seeing if that raises an error.
As eaolson stated, JSON_VALUE will only return a SCALAR value. If you want to return the JSON string contained in "section_1" you can use JSON_TABLE while using the keywords FORMAT JSON.
WITH
messages
AS
(SELECT '{"section_1":{"section_publish":true,"section_body":"<p><b>Overview<\/b><\/p><p>Launched on 19\/06\/1992, the NUVEEN NEW YORK SELECT TAX-FREE INCOME (NXN) is a passively managed exchange traded fund designed to provide a broad exposure to<\/p>"}}' AS json_content
FROM DUAL)
SELECT j.*
FROM messages
CROSS JOIN
JSON_TABLE (
json_content,
'$'
COLUMNS section1 FORMAT JSON PATH '$.section_1',
section1_publish PATH '$.section_1.section_publish',
section1_body PATH '$.section_1.section_body') j;

Laravel - WhereExists returning "Invalid parameter number: parameter was not defined"

I'm trying to use whereExists() on an existing Eloquent query builder (called $trips):
$trips = $trips->whereExists(function ($query) use ($filterValue) {
$query->from(DB::raw("jsonb_array_elements(passengers->'adults'->'persons') as p(person)"))
->whereRaw("p.person->>'name' LIKE '?%'", $filterValue);
});
The query I'm trying to create in raw postgres format is the following (this query works fine using pgAdmin):
SELECT *
from trips
WHERE exists (select *
from jsonb_array_elements(passengers -> 'adults' -> 'persons') as p(person)
where p.person ->> 'name' LIKE 'Prof%');
And I'm receiving this error:
Invalid parameter number: parameter was not defined
I think the problem is small, but I can't see it myself.
The parameter definition in your whereRaw() statement is not quite correct. Parameterized queries are not just string replacements. Your query as written doesn't have a parameter in it, it has a string literal of '?%'. You need to change this to a query parameter, and append the % wildcard to the string you pass in.
Try this:
->whereRaw("p.person->>'name' LIKE ?", $filterValue.'%')

Getting invalid identifier SQL query

I am trying to execute next query:
MERGE INTO NOTIFICATION_OBJS p
USING (SELECT
:fcsNotif_id as doc_id,
:OKPD2_code as OKPD2_code,
:OKPD2_name as OKPD2_name,
:quantity_value as quantity,
:purchaseObject_price as price
FROM DUAL
) v
ON (p.doc_id=v.doc_id)
WHEN MATCHED THEN
UPDATE SET
p.OKPD2_code = v.OKPD2_code,
p.OKPD2_name = v.OKPD2_name,
p.quantity_value = v.quantity_value,
p.price = v.price
WHEN NOT MATCHED THEN
INSERT (p.doc_id, p.OKPD2_code, p.OKPD2_name, p.quantity_value, p.price)
VALUES(v.doc_id, v.OKPD2_code, v.OKPD2_name, v.quantity_value, v.price)
I am sending to bind method next dictionary:
{'OKPD2_code': '62.02.30.000', 'OKPD2_name': 'some text', 'purchaseObject_price': '20466982.25', 'quantity_value': '1', 'fcsNotif_id': '18941152'}
But I am getting error:
ORA-00904: "P"."OKPD2_NAME": invalid identifier
All other query with binding are working. Please help me to find error.
There's no OKPD2_NAME column in NOTIFICATION_OBJS table.
If you used double quotes while creating that table (and its columns), you should
recreate the table without double quotes, or
reference the table (and its columns that use mixed case) using double quotes again, specifying case EXACTLY as it was while creating the table
[EDIT, after the screenshot has been uploaded]
Column name really is created using mixed case, so you'll have to reference it exactly like that: "OKPD2_name" paying attention to double quotes and mixed case.
If you use "okpd2_name" or "okPD2_NAME" or anything but "OKPD2_name", it won't work. Once again: get rid of double quotes.

How to match exact Id within Comma seperated database column using LINQ and Lambda

Hi I have SQL Table where I am storing values like this:
Column Name: Registration_ID
180,1801,1803,18011,220
180,1801,
180,1801,1803
No I want to match exact Registration_ID and get records based on the Registration_ID. I have tried Contains but is not matching exact values.
Here is my query:
var Result=db.Entity_StudentRepository.Get(x =>
x.Registration_ID.Contains(Used_For_Id.ToString())).Select(x => x.Registration_ID).ToArray();
Could you please try the following query and let know if it works-
db.Entity_StudentRepository.AsEnumerable().Where(t=> Registration_ID.Split(',').Select(int.Parse).Contains(Used_For_Id));

codenigiter: sql injection and xss_clean

I am using active record in codenigiter to do some query and I read some docs that say using AR will escape the parameter automaticly.I want to comfirm this and I read the source code of AR class,but I am confused!
I do some test,eg,I access the url as follow is:
http://wrecruit.tudouya.com/company/editProfile/4
then I enable the profiler in CI.
when I add a single quote at the end of url as follows:
http://wrecruit.tudouya.com/company/editProfile/4'
I see the real sql statement the query execute and I got this:
SELECT *
FROM (`wy_company`)
WHERE `id` = '4%27'
the single quote is escaped to '%27',I want to know how this escape happen?maybe it's escaped by the input class?
Url adress is always encoded with urlencode function. This function replace all symbols with codes.
If you want to prevent xss you should to check you parameter. May be like this (in controller Company):
public function editProfile($id)
{
$id = (int)$id;
if($id){
// model code execute - loading DB data
}
}
And in DB query you should use Query Bindings, or use Active record. Them both are prevent injection.

Resources