I made a simple query to a remote database. The table where I make query has all fields with VARCHAR2. However, some fields returns "?" in characters like º, £. I checked enconding and get:
NLS_CHARACTERSET: AL32UTF8
NLS_NCHAR_CHARACTERSET:AL16UTF16
Checking my /etc/default/locale file. These are the results:
LANG="en_US.UTF-8"
LC_NUMERIC="pt_BR.UTF-8"
LC_TIME="pt_BR.UTF-8"
LC_MONETARY="pt_BR.UTF-8"
LC_PAPER="pt_BR.UTF-8"
LC_NAME="pt_BR.UTF-8"
LC_ADDRESS="pt_BR.UTF-8"
LC_TELEPHONE="pt_BR.UTF-8"
LC_MEASUREMENT="pt_BR.UTF-8"
LC_IDENTIFICATION="pt_BR.UTF-8"
The enconding from both edges are UTF-8. Is there another configuration that I missing?
When you say 'some fields return "?" in characters like ...' I assume that is shown on your screen, right? You don't know what is REALLY returned, you only know what's on your screen.
To see what is REALLY returned, you could do something like
select dump('£500') from dual;
DUMP('£500')
--------------------------
Typ=96 Len=4: 163,53,48,48
EDIT: As discussed in the comments below, if you type EXACTLY that command at your terminal and you do, in fact, have a display problem, you will see garbage on the way in. Rather, to see what is stored in the database, you must refer to an actual table, and a column that has those string values in it. For example if the column name is COL1 in the table TBL, and there is also an ID column and for ID = 1000 you have a COL1 value with the pound sign in it, run
select dump(COL1) from TBL where ID = 1000;
Obviously, there are no issues with the INPUT since the input no longer has a pound sign in it (like my first example did). But on the way out, the DUMP may show the proper character is there - however your display is not able to show it correctly.
END EDIT
If you see the code 163 in the DUMP, that means the pound sign is stored correctly in the database, and the issue is just how it is displayed on your screen. In that case, you may have an issue with your NLS_LANG setting. There is excellent information here:
http://www.oracle.com/technetwork/products/globalization/nls-lang-099431.html
If you find that you have to work with different character sets often, you may benefit from reading this article carefully. It will show you how to find out what your current character set is, how to change it, and why the "obvious" things one would look at are in fact not very helpful. The issue is not too complicated, but not trivial either.
Related
In one scenario we are dynamically creating sql to create temp tables on-fly. There is no issue with table_name as it is decided by us however the column-names are provided by sources not in our control.
Usually we would check the column names using below query:
select ..
where NOT REGEXP_LIKE (Column_Name_String,'^([a-zA-Z])[a-zA-Z0-9_]*$')
OR Column_Name_String is NULL
OR Length(Column_Name_String) > 30
However is there any build in function which can do a more extensive check. Also any input on the above query is welcome as well.
Thanks in advance.
Final query based on below answers:
select ..
where NOT REGEXP_LIKE (Column_Name_String,'^([a-zA-Z])[a-zA-Z0-9_]{0,29}$')
OR Column_Name_String is NULL
OR Upper(Column_Name_String) in (select Upper(RESERVED_WORDS.Keyword) from V$RESERVED_WORDS RESERVED_WORDS)
Particularly not happy with character's like $ in column name either hence won't be using..
dbms_assert.simple_sql_name('VALID_NAME')
Instead with regexp I can decide my own set of character's to allow.
This answer does not necessarily offer either a performance or logical improvement, but you can actually validate the column names using a single regex:
SELECT ...
WHERE NOT
REGEXP_LIKE (COALESCE(Column_Name_String, ''), '^([a-zA-Z])[a-zA-Z0-9_]{0,29}$')
This works because:
It uses the same pattern to match columns, i.e. starting with a letter and afterwards using only alphanumeric characters and underscore
NULL column names are mapped to empty string, which fails the regex
We use a length quantifier {0,29} to check the column length directly in the regex
" is there any build in function which can do a more extensive check."
Oracle has the DBMS_ASSERT.SIMPLE_SQL_NAME() function. This returns the passed name if it meets the Oracle naming rules ...
select dbms_assert.simple_sql_name('VALID_NAME') from dual;
... and hurls ORA-44003 if the name is invalid.
Valid names permit any characters if the name is double-quoted (yuck, but then so is creating "temp tables on-fly"). Also the function doesn't check the length of the name, so you will still need to validate that yourself.
Find out more in the docs.
Also here is a SQL Fiddle.
"creating a table with comment column is not possible as its a invalid identifier"
Fair point. DBMS_ASSERT is primarily aimed at preventing SQL injection. So it verifies that a value conforms to Oracle's naming rules, not that the value is a valid Oracle name. To catch things like comment you will also need to check the value against V$RESERVED_WORDS, probably where reserved != 'Y'. As this is a V$ view select on it is not granted by default; if you don't have access you'll need to ask your friendly DBA to help out.
" For validating column names I believe I should check with the entire list"
Up to you. The distinction is that some keywords can legitimately be used as identifiers. For instance TYPE only became a reserved word in Oracle version 8 when they introduced the object-relational stuff. But there were a lot of tables and views in existing systems which used 'TYPE' as a column name (not least the Oracle data dictionary). If Oracle had made TYPE a properly reserved word it would have broken all those systems. So the list of reserved words which cannot be used as identifiers is a sub-set of all the Oracle keywords.
Opinions on the general task:
"we are getting data from external sources (files) and the job of the program/script is to push that data to oracle tables."
There are two parts to this task.
The first is that you should have agreed a standard format for these files with the third parties. There should be no need for discovery of the files' structure or content. (Or if there is such a need because the files are randomly sourced from a carousel of third parties probably you should not be using a relational database but something else: Endeca? Python Pandas library?)
The second is the creating tables on the fly. If you have an agreed file structure then you should be loading into standard tables, using either SQL*Loader or external tables according to your circumstances. If you're on 12c maybe SQL*Loader Express Mode could be of interest.
How can I select a table column by column header name with XPath?
My attempt is:
//table/tbody/tr/td[count(//table/thead/tr/th[.="$columnName"]/preceding-sibling::th)+1]
This is not working.
It always selects the first column no matter what value I provide for $columnName.
There is barely any information in this post, but my educated guess would be that
count(//table/thead/tr/th[.="$columnName"]/preceding-sibling::th)+1
is always equal to 1 because you use $columnName in quotes - which makes it a string, not a variable.
If this is indeed the problem, using
count(//table/thead/tr/th[.=$columnName]/preceding-sibling::th)+1
would solve it. If it doesn't, you really need to give more information - show the whole input document, indicate the programming language, show all of that code.
I am currently using Oracle sql developer v4.0.3.16. When I do auto-complete of table columns, they show up in ascending order. I've got table with over 200 columns. Auto-completion window shows only 40 positions. When i choose last position '...' - window is being refreshed with all column names, but still in ascending order.
How can i change the way of presenting result in prompt, to show all column names in order as is returned by 'DESCRIBE table' command?
You can't influence the order, we do alphabetical only. Main reason is because a general user isn't going to have your data model memorized.
To get the best bang for your buck with this feature, you're going to want to type the first 1 to however many characters of your column or table or procedure name.
We don't show 200 columns by default, because that's not very helpful at all.
But give us the first letter, and now things become much more interesting.
Starting in a version very close to what you have, the automatic insight feature doesn't even make a suggestion unless you have 10 or fewer 'hits' - although this is configurable.
I'm having trouble with the number formatting in Apex 3.2.1. NLS_TERRITORY is set to GERMANY, NLS_LANGUAGE to GERMAN. However, the default number formatting is wrong and I would like to fix/change it. I'm wondering why there is an application date format setting but no application number format setting. That would have been really helpful right now. So my question is: how can I set a new number format which will be applied everywhere in my application? Is it possible? If not, what would be the least painful way to to apply the new format mask. I'm mainly concerned with interactive reports. Additionally, if it were possible to change the default format mask for the territory in some configuration file, that would also be an option. Thanks for your input in advance.
Edit:
My problem with the default format is that numbers like 0.34 are being displayed as ",34", but the correct output would be "0,34".
Okay, purely the display then. I don't think there is an easy way to do this for all your fields, but specifying a format mask on those fields, by using for example FM999G999G990D000.
That would be annoying for a lot of fields across the application though, i agree. It is even more annoying when only by specifying a format mask, items are recognised as being number fields. Maybe making a plugin, which is identical to a number field, but with a default format mask may do the trick.
Or query the apex views to find items, then update them (though you will need access to the 'internal' tables of apex, the wwv_* things).
select application_name, page_id, page_name, item_name, display_as, display_as_code, format_mask from apex_application_page_items where page_id = 6
Which you could then transform into an update...
I've nothing else to offer, so far i've always specified the format mask during development...
How do you guys treat empty strings with Oracle?
Statement #1: Oracle treats empty string (e.g. '') as NULL in "varchar2" fields.
Statement #2: We have a model that defines abstract 'table structure', where for we have fields, that can't be NULL, but can be "empty". This model works with various DBMS; almost everywhere, all is just fine, but not with Oracle. You just can't insert empty string into a "not null" field.
Statement #3: non-empty default value is not allowed in our case.
So, would someone be so kind to tell me - how can we resolve it?
This is why I've never understood why Oracle is so popular. They don't actually follow the SQL standard, based on a silly decision they made many years ago.
The Oracle 9i SQL Reference states (this has been there for at least three major versions):
Oracle currently treats a character value with a length of zero as null. However, this may not continue to be true in future releases, and Oracle recommends that you do not treat empty strings the same as nulls.
But they don't say what you should do. The only ways I've ever found to get around this problem are either:
have a sentinel value that cannot occur in your real data to represent NULL (e.g, "deoxyribonucleic" for a surname field and hope that the movie stars don't start giving their kids weird surnames as well as weird first names :-).
have a separate field to indicate whether the first field is valid or not, basically what a real database does with NULLs.
Are we allowed to say "Don't support Oracle until it supports the standard SQL behaviour"? It seems the least pain-laden way in many respects.
If you can't force (use) a single blank, or maybe a Unicode Zero Width Non-Break Space (U+FEFF), then you probably have to go the whole hog and use something implausible such as 32 Z's to indicate that the data should be blank but isn't because the DBMS in use is Orrible.
Empty string and NULL in Oracle are the same thing. You want to allow empty strings but disallow NULLs.
You have put a NOT NULL constraint on your table, which is the same as a not-an-empty-string constraint. If you remove that constraint, what are you losing?