recently I started to work with db2, and created few databases.
To drop a single DB I should use db2 drop db demoDB, is there a way to drop all DBs at once?
Thanks
Taking into account the previous answer, this set of lines do the same without creating a script.
db2 list db directory | tail -n +6 | sed 'N;N;N;N;N;N;N;N;N;N;N;s/\n/ /g' | awk '$28 = /Indirect/ {print "db2 drop database "$7}' | source /dev/stdin
This filters the local databases, and executes the generated output.
(Only works in English environment)
first , i don't think there is any db2 nature way to do that. But I usually do the following thing. At start, the way to see all the databases on your instance is one of the following:
db2 list db directory
db2 list active active databases
Depends on your need ( all DBs or just the active DBs)
I'm sure there is more DBs lists you can get.(at DB2 user guide)
The way I usually drop all my DBs is by using shell script:
1. create new script by using 'vi db2_drop_all.sh' or some other way you want.
2. paste the code:
#!/bin/bash -x
for db_name in $(db2 list db directory | grep Database | \
grep name | cut -d= -f2); do
db2 drop db $db_name || true
done
exit 0
3. save changes
4. and just run the script (after you switched to your instance of course ) sh db2_drop_all.sh
Notice that in step 2 you can change the list of DBs as you wish. ( for example to db2 list active databases)
Hope it helped you. :)
Related
I am testing monetdb for a colunmnar storage.
I already installed and run the server
but, when I connect to the client and run a query, the response does not show the time to execute the query.
I am connecting as:
mclient -u monetdb -d voc
I already tried to connect with interactive like:
mclient -u monetdb -d voc -i
Output example:
sql>select count(*) from voc.regions;
+---------+
| L3 |
+=========+
| 5570699 |
+---------+
1 tuple
As mkersten mentioned, I would read through the options of the mclient utility first.
To get server and client timing measurements I used --timer=performance option when starting mclient.
Inside mclient I would then disable the result output by setting \f trash to ignore the results when measuring only.
Prepend trace to your query and you get your results like this:
sql>\f trash
sql>trace select count(*) from categories;
sql:0.000 opt:0.266 run:1.713 clk:5.244 ms
sql:0.000 opt:0.266 run:2.002 clk:5.309 ms
The first of the two lines shows you the server timings, the second one the overall timing including passing the results back to the client.
If you use the latest version MonetDB-Mar18 you have good control over the performance timers, which includes parsing, optimization, and runtime at server. See mclient --help.
I have a Local directory it is used to store the hive table data.
I need to list all tables which are using Local directory .
These tables (managed tables) are stored in hive Default DB , this DB allows to store Data in other Local directories .
My Local directory : /abc/efg/data/
Table data is Stored in sub folders like 123 , 456,789 etc
For table xyz location is /abc/efg/data/123 , PQR location is /abc/efg/data/456 like that.
I am trying to use
hive -e " show tables " > All_tables list all tables and redirect to a file
For each line(each table) in All_tables
hive -e " desc formatted $line " | grep '/abc/efg/data/' >> Tables_My_local_dir
but it will result some performance issue as i have 6000 tables in DB .
please help me to list all tables which are using Local directory with a best performance.
I assume that you wanted to list table and its corresponding location information by extracting it from the desc formatted command for managed tables in default database.
If my understanding is correct, I suggest you to go with querying the Hive Meta-store, provided its an externally configured one and you have necessary permissions to fetch the same information
Query on meta-store:
SELECT T.TBL_NAME AS TABLE_NAME,S.LOCATION AS LOCATION FROM TBLS T LEFT JOIN SDS S ON T.SD_ID=S.SD_ID WHERE T.TBL_TYPE='MANAGED_TABLE' AND T.DB_ID=1 ;
note: in the query, DB_ID for default database is 1
Output:
------------+------------------------------------------------------------+
| TABLE_NAME | LOCATION |
+------------+------------------------------------------------------------+
| sample | hdfs://********:8020/user/hive/warehouse/sample |
...
.
Based on the rule
HADOOP TABLES ARE DIRECTORIES
I have created a shell script to do the below steps.
Step 1. Find all the directories which are not being modified since last 14 days .
Step 2 . Separate real tables and real folders 2.1execute "desc $dir_name "
2.2 based on return status($？) redirect $dir_name to two files(one for real tables and other for directories )
Now I have the required tables in a file.
I am working on a test in which I must find out the number of partitions of a table and check if it is right. If I use show partitions TableName I get all the partitions by name, but I wish to get the number of partitions, like something along the lines show count(partitions) TableName (which retuns OK btw.. so it's not good) and get 12 (for ex.).
Is there any way to achieve this??
Using Hive CLI
$ hive --silent -e "show partitions <dbName>.<tableName>;" | wc -l
--silent is to enable silent mode
-e tells hive to execute quoted query string
You could use:
select count(distinct <partition key>) from <TableName>;
By using the below command, you will get the all partitions and also at the end it shows the number of fetched rows. That number of rows means number of partitions
SHOW PARTITIONS [db_name.]table_name [PARTITION(partition_spec)];
< failed pictoral example >
You can use the WebHCat interface to get information like this. This has the benefit that you can run the command from anywhere that the server is accessible. The result is JSON - use a JSON parser of your choice to process the results.
In this example of piping the WebHCat results to Python, only the number 24 is returned representing the number of partitions for this table. (Server name is the name node).
curl -s 'http://*myservername*:50111/templeton/v1/ddl/database/*mydatabasename*/table/*mytablename*/partition?user.name=*myusername*' | python -c 'import sys, json; print len(json.load(sys.stdin)["partitions"])'
24
In scala you can do following:
sql("show partitions <table_name>").count()
I used following.
beeline -silent --showHeader=false --outputformat=csv2 -e 'show partitions <dbname>.<tablename>' | wc -l
Use the following syntax:
show create table <table name>;
When i dump a table with uppercase letters using mysqldump it comes out as lower case in my > dump.sql file. I found a report here in 2006, almost 4 years old http://bugs.mysql.com/bug.php?id=19967
A solution here suggest making linux insensitive. I rather not if possible. Whats the easiest way to copy a win32 db into linux?
According to the MySQL manuals, you only have a limited number of options:
Use lower_case_table_names=1 on all systems. The main disadvantage
with this is that when you use SHOW
TABLES or SHOW DATABASES, you do not
see the names in their original
lettercase.
Use lower_case_table_names=0 on Unix and lower_case_table_names=2 on
Windows. This preserves the lettercase
of database and table names. The
disadvantage of this is that you must
ensure that your statements always
refer to your database and table names
with the correct lettercase on
Windows. If you transfer your
statements to Unix, where lettercase
is significant, they do not work if
the lettercase is incorrect.
Exception: If you are using InnoDB tables and you are trying to
avoid these data transfer problems,
you should set lower_case_table_names
to 1 on all platforms to force names
to be converted to lowercase.
See: http://dev.mysql.com/doc/refman/5.0/en/identifier-case-sensitivity.html for full details.
Today I've had to make it so. I already have windows db in lower case and need to import to linux db with case sensitive table names, so the play with lowecase_table_names option in not an option :)
It looks that 'show tables' displays appropriately sorted table names and the dump have escaped table names with ` character. I've succesfully imported the database with following algorithm:
I have mydb.sql with lowercase windows dump
I started application to create database schema in Linux, with case sensitive names.
Then I've had lower case names in dump, and case sensitive names in mysql database. I converted the dump using sed & awk with following script:
#!/bin/bash
MYSQL="mysql -u root -p mydb"
FILE=mydb.sql
TMP1=`mktemp`
TMP2=`mktemp`
cp $FILE $TMP1
for TABLE in `echo "show tables" | $MYSQL`; do
LCTABLE=`echo $TABLE| awk '{print tolower($0)}'`
echo "$LCTABLE --> $TABLE"
cat $TMP1 | sed "s/\`$LCTABLE\`/\`$TABLE\`/" > $TMP2
cp $TMP2 $TMP1
done
cp $TMP1 $FILE.conv
rm $TMP1
rm $TMP2
And the dump has been converted properly. Everything works after import in Linux.
I have an Oracle database backup file (.dmp) that was created with expdp.
The .dmp file was an export of an entire database.
I need to restore 1 of the schemas from within this dump file.
I don't know the names of the schemas inside this dump file.
To use impdp to import the data I need the name of the schema to load.
So, I need to inspect the .dmp file and list all of the schemas in it, how do I do that?
Update (2008-09-18 13:02) - More detailed information:
The impdp command i'm current using is:
impdp user/password#database directory=DPUMP_DIR
dumpfile=EXPORT.DMP logfile=IMPORT.LOG
And the DPUMP_DIR is correctly configured.
SQL> SELECT directory_path
2 FROM dba_directories
3 WHERE directory_name = 'DPUMP_DIR';
DIRECTORY_PATH
-------------------------
D:\directory_path\dpump_dir\
And yes, the EXPORT.DMP file is in fact in that folder.
The error message I get when I run the impdp command is:
Connected to: Oracle Database 10g Enterprise Edition ...
ORA-31655: no data or metadata objects selected for the job
ORA-39154: Objects from foreign schemas have been removed from import
This error message is mostly expected. I need the impdp command be:
impdp user/password#database directory=DPUMP_DIR dumpfile=EXPORT.DMP
SCHEMAS=SOURCE_SCHEMA REMAP_SCHEMA=SOURCE_SCHEMA:MY_SCHEMA
But to do that, I need the source schema.
impdp exports the DDL of a dmp backup to a file if you use the SQLFILE parameter. For example, put this into a text file
impdp '/ as sysdba' dumpfile=<your .dmp file> logfile=import_log.txt sqlfile=ddl_dump.txt
Then check ddl_dump.txt for the tablespaces, users, and schemas in the backup.
According to the documentation, this does not actually modify the database:
The SQL is not actually executed, and the target system remains unchanged.
If you open the DMP file with an editor that can handle big files, you might be able to locate the areas where the schema names are mentioned. Just be sure not to change anything. It would be better if you opened a copy of the original dump.
Update (2008-09-19 10:05) - Solution:
My Solution: Social engineering, I dug real hard and found someone who knew the schema name.
Technical Solution: Searching the .dmp file did yield the schema name.
Once I knew the schema name, I searched the dump file and learned where to find it.
Places the Schemas name were seen, in the .dmp file:
<OWNER_NAME>SOURCE_SCHEMA</OWNER_NAME>
This was seen before each table name/definition.
SCHEMA_LIST 'SOURCE_SCHEMA'
This was seen near the end of the .dmp.
Interestingly enough, around the SCHEMA_LIST 'SOURCE_SCHEMA' section, it also had the command line used to create the dump, directories used, par files used, windows version it was run on, and export session settings (language, date formats).
So, problem solved :)
Assuming that you do not have the log file from the expdp job that generated the file in the first place, the easiest option would probably be to use the SQLFILE parameter to have impdp generate a file of DDL (based on a full import). Then you can grab the schema names from that file. Not ideal, of course, since impdp has to read the entire dump file to extract the DDL and then again to get to the schema you're interested in, and you have to do a bit of text file searching for the various CREATE USER statements, but it should be doable.
The running the impdp command to produce an sqlfile, you will need to run it as a user which has the DATAPUMP_IMP_FULL_DATABASE role.
Or... run it as a low privileged user and use the MASTER_ONLY=YES option, then inspect the master table. e.g.
select value_t
from SYS_IMPORT_TABLE_01
where name = 'CLIENT_COMMAND'
and process_order = -59;
col object_name for a30
col processing_status head STATUS for a6
col processing_state head STATE for a5
select distinct
object_schema,
object_name,
object_type,
object_tablespace,
process_order,
duplicate,
processing_status,
processing_state
from sys_import_table_01
where process_order > 0
and object_name is not null
order by object_schema, object_name
/
http://download.oracle.com/otndocs/products/database/enterprise_edition/utilities/pdf/oow2011_dp_mastering.pdf
Step 1: Here is one simple example. You have to create a SQL file from the dump file using SQLFILE option.
Step 2: Grep for CREATE USER in the generated SQL file (here tables.sql)
Example here:
$ impdp directory=exp_dir dumpfile=exp_user1_all_tab.dmp logfile=imp_exp_user1_tab sqlfile=tables.sql
Import: Release 11.2.0.3.0 - Production on Fri Apr 26 08:29:06 2013
Copyright (c) 1982, 2011, Oracle and/or its affiliates. All rights reserved.
Username: / as sysdba
Processing object type SCHEMA_EXPORT/PRE_SCHEMA/PROCACT_SCHEMA Job "SYS"."SYS_SQL_FILE_FULL_01" successfully completed at 08:29:12
$ grep "CREATE USER" tables.sql
CREATE USER "USER1" IDENTIFIED BY VALUES 'S:270D559F9B97C05EA50F78507CD6EAC6AD63969E5E;BBE7786A5F9103'
Lot of datapump options explained here http://www.acehints.com/p/site-map.html
You need to search for OWNER_NAME.
cat -v dumpfile.dmp | grep -o '<OWNER_NAME>.*</OWNER_NAME>' | uniq -u
cat -v turn the dumpfile into visible text.
grep -o shows only the match so we don't see really long lines
uniq -u removes duplicate lines so you see less output.
This works pretty well, even on large dump files, and could be tweaked for usage in a script.
My solution (similar to KyleLanser's answer) (on a Unix box):
strings dumpfile.dmp | grep SCHEMA_LIST
In my case, based on Aldur's and slafs' answers I came up with this expression that should tell you just the name of the original schema:
cat -v file.dmp | grep 'SCHEMA_LIST' | uniq -u | grep -o -P '(?<=SCHEMAS\=).*(?=content)'
Tested for a DMP file from Oracle 19.8 version.