How to delete temp table created by VerticaPy - error the table "v_temp_schema"."train" already exists - vertica

I want to read a csv file using VerticaPy - https://www.vertica.com/python/documentation_last/utilities/read_csv/
I am running Vertica using a Docker image on my laptop.
The Jupyter code is
import sys
!{sys.executable} -m pip install vertica-python
!{sys.executable} -m pip install verticapy
from verticapy import *
conn_info = {'host': '127.0.0.1',
'port': 5433,
'user': 'dbadmin',
'password': '',
'database': 'kaggle_titanic'}
train = read_csv("train.csv")
train
The code runs fine the first time but when I run it again, I get error
NameError Traceback (most recent call last)
~\AppData\Local\Temp/ipykernel_8344/783506563.py in <module>
23 # cur.execute("drop table kaggle_titanic.v_temp_schema.train")
24
---> 25 train = read_csv("train.csv")
26 train
27 #iris2
~\Anaconda3\lib\site-packages\verticapy\utilities.py in read_csv(path, cursor, schema, table_name, sep, header, header_names, dtype, na_rep, quotechar, escape, genSQL, parse_n_lines, insert, temporary_table, temporary_local_table, ingest_local)
918 result = cursor.fetchall()
919 if (result != []) and not (insert):
--> 920 raise NameError(
921 'The table "{}"."{}" already exists !'.format(schema, table_name)
922 )
NameError: The table "v_temp_schema"."train" already exists !
Looks like VerticaPy creates a temp table in the first run. How do I delete it?
I tried adding this code
with vertica_python.connect(**conn_info) as conn:
cur = conn.cursor()
cur.execute("drop table kaggle_titanic.v_temp_schema.train")
But got error
MissingSchema: Severity: ROLLBACK, Message: Schema "v_temp_schema" does not exist, Sqlstate: 3F000, Routine: RangeVarGetObjid, File: /data/jenkins/workspace/RE-ReleaseBuilds/RE-Jackhammer_3/server/vertica/Catalog/Namespace.cpp, Line: 281, Error Code: 4650, SQL: 'drop table kaggle_titanic.v_temp_schema.train'
I had to stop/start the database (from docker image) to run the code but clearly that isn't the right way.
How do I delete v_temp_schema.train?

I answered to this request on Github: https://github.com/vertica/VerticaPy/issues/285

Related

How could I resolve xmlrpc.client.Fault while trying to search a package through pip3? [duplicate]

I am getting this error in pip search while studying python.
The picture is an error when I pip search. Can you tell me how to fix it?
$ pip search pdbx
ERROR: Exception:
Traceback (most recent call last):
File "*/lib/python3.7/site-packages/pip/_internal/cli/base_command.py", line 224, in _main
status = self.run(options, args)
File "*/lib/python3.7/site-packages/pip/_internal/commands/search.py", line 62, in run
pypi_hits = self.search(query, options)
File "*/lib/python3.7/site-packages/pip/_internal/commands/search.py", line 82, in search
hits = pypi.search({'name': query, 'summary': query}, 'or')
File "/usr/lib/python3.7/xmlrpc/client.py", line 1112, in __call__
return self.__send(self.__name, args)
File "/usr/lib/python3.7/xmlrpc/client.py", line 1452, in __request
verbose=self.__verbose
File "*/lib/python3.7/site-packages/pip/_internal/network/xmlrpc.py", line 46, in request
return self.parse_response(response.raw)
File "/usr/lib/python3.7/xmlrpc/client.py", line 1342, in parse_response
return u.close()
File "/usr/lib/python3.7/xmlrpc/client.py", line 656, in close
raise Fault(**self._stack[0])
xmlrpc.client.Fault: <Fault -32500: 'RuntimeError: This API has been temporarily disabled due to unmanageable load and will be deprecated in the near future. Please use the Simple or JSON API instead.'>
The pip search command queries PyPI's servers, and PyPI's maintainers have explained that the API endpoint that the pip search command queries is very resource intensive and too expensive for them to always keep open to the public. Consequently they sometimes throttle access and are actually planning to remove it completely soon.
See this GitHub issues thread ...
The solution I am using for now is to pip install pip-search (a utility created by GitHub user #victorgarric).
So, instead of 'pip search', I use pip_search. Definitely beats searching PyPI via a web browser
Follow the suggestion from JRK at the discussion at github (last comment) the search command is temporarily disabled, use your browser to search for packages meanwhile:
Check the thread on github and give him a thumb up ;)
search on website, https://pypi.org/,
then install the package you wanted
The error says
Please use the Simple or JSON API instead
You can try pypi-simple to query the pip repository
https://pypi.org/project/pypi-simple/
It gives an example too, I tried to use it here:
pypi-simple version 0.8.0 DistributionPackage' object has no attribute 'get_digest':
!/usr/bin/env python3
-*- coding: utf-8 -*-
"""
Created on Thu Nov 11 17:40:03 2020
#author: Pietro
"""
from pypi_simple import PyPISimple
def simple():
package=input('\npackage to be checked ')
try:
with PyPISimple() as client:
requests_page = client.get_project_page(package)
except:
print("\n SOMETHING WENT WRONG !!!!! \n\n",
"CHECK INTERNET CONNECTION OR DON'T KNOW WHAT HAPPENED !!!\n")
pkg = requests_page.packages[0]
print(pkg)
print(type(pkg))
print('\n',pkg,'\n')
print('\n'+pkg.filename+'\n')
print('\n'+pkg.url+'\n')
print('\n'+pkg.project+'\n')
print('\n'+pkg.version+'\n')
print('\n'+pkg.package_type+'\n')
#print('\n'+pkg.get_digest()+'\n','ENDs HERE !!!!') #wasnt working
if __name__ == '__main__':
simple()
got -4 so far for this answer don't know why , figureout I can try to check for a package with:
# package_name = input('insert package name : ')
package_name = 'numpy'
import requests
url = ('https://pypi.org/pypi/'+package_name+'/json')
r = requests.get(url)
try:
data = r.json()
for i in data:
if i == 'info':
print('ok')
for j in data[i]:
if j == 'name':
print((data[i])[j])
print([k for k in (data['releases'])])
except:
print('something went south !!!!!!!!!!')

"Commands out of sync" error when trying to execute a procedure in MySQL 8

When executing below code in phpMyAdmin:
use db;
DELIMITER $$
DROP PROCEDURE IF EXISTS McaTest3$$
CREATE PROCEDURE McaTest3()
BEGIN
SELECT
cl.*
FROM `condition_library` cl
LEFT JOIN condition_custom cc on cl.condition_library_id = cc.condition_library_id
and cc.active = 1
AND (cc.permit_application_id = 20231 OR cc.permit_id = NULL)
WHERE FIND_IN_SET(cl.`condition_library_id`, '13070')
AND cl.active = 1
and cc.condition_library_id IS NULL;
END$$
DELIMITER ;
call McaTest3();
Getting error:
Error
Static analysis:
1 errors were found during analysis.
Missing expression. (near "ON" at position 25)
SQL query: Edit Edit
SET FOREIGN_KEY_CHECKS = ON;
MySQL said: Documentation
#2014 - Commands out of sync; you can't run this command now
This happens when there is no record found in the table which is at LEFT JOIN.
When the same is ran in MySQL Workbench: NO ERROR and return empty dataset.
Same procedure when executed from Application (Appian) is failing as well… Any clues?
Another question on Stackoverflow answered my issue:
link: MySQL error #2014 - Commands out of sync; you can't run this command now

Pyspark - error when save data into Hive table "unresolved operator 'InsertIntoTable HiveTableRelation'"

I use the following:
pyspark library , version 2.3.1
python, version 2.7.1
hadoop, version 2.7.3
hive, version 1.2.1000.2.6.5.30-1
spark version 2
My hive table looks following:
CREATE TABLE IF NOT EXISTS my_database.my_table
(
division STRING COMMENT 'Sample column'
)
I want to save data into HIVE using pyspark. I use the following code:
spark_session = SparkSession.builder.getOrCreate()
hive_context = HiveContext(spark_session.sparkContext)
hive_table_schema = hive_context.table("my_database.my_table").schema
df_to_save = spark_session.createDataFrame([["a"],["b"],["c"]], schema=hive_table_schema)
df_to_save.write.mode("append").insertInto("my_database.my_table")
But the following error occur:
Traceback (most recent call last):
File "/home/my_user/mantis service_quality_check__global/scripts/row_counts_preprocess.py", line 147, in <module> df_to_save.write.mode("append").insertInto(hive_table_row_counts_str)
File "/usr/hdp/current/spark2-client/python/lib/pyspark.zip/pyspark/sql/readwriter.py", line 716, in insertInto
File "/usr/hdp/current/spark2-client/python/lib/py4j-0.10.6-src.zip/py4j/java_gateway.py", line 1160, in __call__
File "/usr/hdp/current/spark2-client/python/lib/pyspark.zip/pyspark/sql/utils.py", line 69, in deco
pyspark.sql.utils.AnalysisException: u"unresolved operator 'InsertIntoTable HiveTableRelation `my_database`.`my_table`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, [division#14], false, false;;\n'InsertIntoTable HiveTableRelation `my_database`.`my_table`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, [division#14], false, false\n+- LogicalRDD [division#2], false\n"
Please is there someone who cane help with this? I am stuck with this few days
I found the issue. SparkSession has to support hive. The method enableHiveSupport() has to be call when spark session is created.
Then creation of spark session will looks like following
spark_session = SparkSession.builder.enableHiveSupport().getOrCreate()

how to delete database of odoo in terminal from debian

How can i delete the database of odoo 9 using terminal , but i have no idea how to do this . please share with me the process .
File "/usr/lib/python2.7/dist-packages/openerp/tools/cache.py", line 85, in lookup
value = d[key] = self.method(*args, **kwargs)
File "/usr/lib/python2.7/dist-packages/openerp/addons/base/ir/ir_model.py", line 814, in check
, (model_name, uid,)
File "/usr/lib/python2.7/dist-packages/openerp/sql_db.py", line 141, in wrapper
return f(self, *args, **kwargs)
File "/usr/lib/python2.7/dist-packages/openerp/sql_db.py", line 220, in execute
res = self._obj.execute(query, params)
InternalError: current transaction is aborted, commands ignored until end of transaction block
This error will be found , when i open the odoo . so i think , if i delete the database there may be solve the problem
Thanks
you can manage databases (and delete them) via /web/database/manager.
On linux you can handle databases as described in official docs:
# dropdb mydb

"pymysql.err.InternalError: (1046, u'No database selected')" error message when running Python script from command line

I'm trying to run a Python script that connects to a MySQL database through PyMySQL. The script is effectively:
import pymysql
cnx = pymysql.connect(read_default_file = "/directory/my.cnf", cursorclass = pymysql.cursors.DictCursor)
cursor = cnx.cursor()
# Do stuff.
When I run the script in the interpreter, I don't get any errors, but when I try to run it from the command line, I get the following error:
Traceback (most recent call last):
File "s02_prepare_data_RNN.py", line 264, in <module>
(omniture, urls, years, global_regions) = get_omniture_data("omniture_results")
File "s02_prepare_data_RNN.py", line 76, in get_omniture_data
sso_to_accountid = get_sso_accountids()
File "s02_prepare_data_RNN.py", line 31, in get_sso_accountids
cursor.execute(query)
File "/home/rdu/malcorn/.local/lib/python2.6/site-packages/pymysql/cursors.py", line 134, in execute
result = self._query(query)
File "/home/rdu/malcorn/.local/lib/python2.6/site-packages/pymysql/cursors.py", line 282, in _query
conn.query(q)
File "/home/rdu/malcorn/.local/lib/python2.6/site-packages/pymysql/connections.py", line 768, in query
self._affected_rows = self._read_query_result(unbuffered=unbuffered)
File "/home/rdu/malcorn/.local/lib/python2.6/site-packages/pymysql/connections.py", line 929, in _read_query_result
result.read()
File "/home/rdu/malcorn/.local/lib/python2.6/site-packages/pymysql/connections.py", line 1125, in read
first_packet = self.connection._read_packet()
File "/home/rdu/malcorn/.local/lib/python2.6/site-packages/pymysql/connections.py", line 893, in _read_packet
packet.check_error()
File "/home/rdu/malcorn/.local/lib/python2.6/site-packages/pymysql/connections.py", line 369, in check_error
err.raise_mysql_exception(self._data)
File "/home/rdu/malcorn/.local/lib/python2.6/site-packages/pymysql/err.py", line 120, in raise_mysql_exception
_check_mysql_exception(errinfo)
File "/home/rdu/malcorn/.local/lib/python2.6/site-packages/pymysql/err.py", line 115, in _check_mysql_exception
raise InternalError(errno, errorvalue)
pymysql.err.InternalError: (1046, u'No database selected')
Modifying my code from:
import pymysql
cnx = pymysql.connect(read_default_file = "/directory/my.cnf", cursorclass = pymysql.cursors.DictCursor)
cursor = cnx.cursor()
# Do stuff.
to:
import pymysql
cnx = pymysql.connect(read_default_file = "/directory/my.cnf", cursorclass = pymysql.cursors.DictCursor)
cursor = cnx.cursor()
if __name__ == "__main__":
# Do stuff.
fixed the error. I had the thought that Python might be trying to execute the queries before the connection was established, so I tried putting the main part of my program under if __name__ == "__main__": and that fixed it. I'm still not 100% what is going on, though. I had assumed the code would wait for the connection to be established before proceeding to the following lines, but this fix suggests that's not the case.
It's also worth noting that I was only getting the error when running the original script from the command line on a server that has Python 2.6. When I ran the original script from the command line on my local machine that has Python 2.7, I did not get the error.
Anyway, if __name__ == "__main__": is good Python style, so I'll make sure to use it in the future.
Sometimes this may simply happen when the database URI is not built correctly. For me, the error occurred when my pymysql connection string (URL) looked like this:
mysql+pymysql://:#localhost/
To fix it, it needed to look like:
mysql+pymysql://<my-database-user-name>:#localhost/<my-database-name> # or replace localhost with your particular host name
If you don't select the database in connection, you must add this line.
connection = pymysql.connect(host=DB_host, user=DB_user, password=DB_password) # NO DB_NAME SPECIFED
cursor = connection.cursor()
cursor.execute(f'CREATE DATABASE IF NOT EXISTS {DB_NAME}') # ADD THIS LINE!
connection.select_db(DB_NAME)

Resources