I am trying to fetch historics data of the day and to print it.
the error i am getting is :
sqlite3 : Operational error near "(" : syntax error
import sqlite3 as sqlite
import sys
import time
conn = sqlite.connect('places.sqlite.db')
c = conn.cursor()
today = str(time.time())
here i am selecting the 10 first caracter because i want to search for a unix epoch match in seconds and not in milliseconds ( so only the first 10 are interesting to me)
c.execute("SELECT * FROM moz_places WHERE LEFT(last_visit_date, 10)='"+today+"'")
user1 = c.fetchone()
print(user1)
As mentionned earlier, i get "sqlite3 : Operational error near "(" : syntax error"
Do you what is wrong there ?
Here is how to convert moz_places.last_update_time into a string, 'YYYY-MM-DD HH:MM:SS':
UTC: datetime(last_visit_date/1000000, 'unixepoch')
Local time zone: datetime(last_visit_date/1000000, 'unixepoch','localtime')
Here is a link to SQLite doc on Date and Time Functions.
The today string created in python should match format exactly (because it will be doing a string comparison).
From the comments: the name of the places database in Firefox is places.sqlite (not places.sqlite.db). The database name should include full or relative path if it is not in your current working directory.
Related
I am trying to create a textfsm template with the Netmiko library. While it works for most of the commands, it does not work when I try performing "inc" operation in the network device. The textfsm index file seems like it is not recognizing the same command for 2 different templates; for instance:
If I am giving the command - show running | inc syscontact
And give another command - show running | inc syslocation
in textfsm index; the textfsm template seems like it is recognizing only the first command; and not the second command.
I understand that I can get the necessary data by the regex expression for syscontact and syslocation for the commands( via the template ), however I want to achieve this by the "inc" command from the device itself. Is there a way this can be done?
you need to escape the pipe in the index file. e.g. sh[[ow]] ru[[nning]] \| inc syslocation
There is a different way to parse that you want all datas which is called TTP module. You can take the code I wrote below as an example. You can create your own templates.
from pprint import pprint
from ttp import ttp
import json
import time
with open("showSystemInformation.txt") as f:
data_to_parse = f.read()
ttp_template = """
<group name="Show_System_Information">
System Name : {{System_Name}}
System Type : {{System_Type}} {{System_Type_2}}
System Version : {{Version}}
System Up Time : {{System_Uptime_Days}} days, {{System_Uptime_HR_MIN_SEC}} (hr:min:sec)
Last Saved Config : {{Last_Saved_Config}}
Time Last Saved : {{Last_Time_Saved_Date}} {{Last_Time_Saved_HR_MIN_SEC}}
Time Last Modified : {{Last_Time_Modified_Date}} {{Last_Time_Modifed_HR_MIN_SEC}}
</group>
"""
parser = ttp(data=data_to_parse, template=ttp_template)
parser.parse()
# print result in JSON format
results = parser.result(format='json')[0]
print(results)
Example run:
[appadmin#ryugbz01 Nokia]$ python3 showSystemInformation.py
[
{
"Show_System_Information": {
"Last_Saved_Config": "cf3:\\config.cfg",
"Last_Time_Modifed_HR_MIN_SEC": "11:46:57",
"Last_Time_Modified_Date": "2022/02/09",
"Last_Time_Saved_Date": "2022/02/07",
"Last_Time_Saved_HR_MIN_SEC": "15:55:39",
"System_Name": "SR7-2",
"System_Type": "7750",
"System_Type_2": "SR-7",
"System_Uptime_Days": "17",
"System_Uptime_HR_MIN_SEC": "05:24:44.72",
"Version": "C-16.0.R9"
}
}
]
I am trying to run the below code using H2DB (via junit test), while doing so i get error message as below. I understand that, there are no function available as "days" in H2. So i am trying to write a custom function, but it does not work out, can any one help on writing this function.
SQLBuilder class code:
public String dummy() {
return new StringBuilder(new SQL() {
{
SELECT("date(CREATE_TMS)");
SELECT("CASE WHEN date(CREATE_TMS) >= (CURRENT DATE - cast('1' AS integer) days) THEN 'Y' ELSE 'N' END NEW_B");
FROM("Q.DUMMY");
}
}.toString().concat(" FOR READ ONLY WITH UR")).toString();
}
Error message:
org.springframework.jdbc.BadSqlGrammarException:
### Error querying database. Cause: org.h2.jdbc.JdbcSQLSyntaxErrorException: Syntax error in SQL statement "SELECT DATE(CREATE_TMS), CASE WHEN DATE(CREATE_TMS) >= (CURRENT DATE - CAST('1' AS INTEGER) DAYS[*]) THEN 'Y' ELSE 'N' END NEW_BILLING
FROM Q.DUMMY FOR READ ONLY WITH UR "; expected "[, ::, *, /, %, +, -, ||, ~, !~, NOT, LIKE, ILIKE, REGEXP, IS, IN, BETWEEN, AND, OR, ,, )"; SQL statement:
SELECT date(CREATE_TMS), CASE WHEN date(CREATE_TMS) >= (CURRENT DATE - cast('1' AS integer) days) THEN 'Y' ELSE 'N' END NEW_BILLING
FROM Q.DUMMY FOR READ ONLY WITH UR [42001-199]
For some reason days are converted to DAYS[*], we can see that in error message.
Customer method i tried in schema-db2.sql:
drop ALIAS if exists days;
CREATE ALIAS days as '
import java.lang.String;
#CODE
java.lang.String days() throws Exception {
return "days";
}
';
applicaiton.properties:
spring.datasource.url=jdbc:h2:mem:testdb;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=false;Mode=DB2
DAYS is not a function and is not a something that other databases support. Db2 also uses non-standard interval literals.
If you can build H2 from its current sources, you can use cast('1' AS integer) day in it (not the days) and such construction is also supported by Db2. You can also simply use 1 DAY, it is supported by current H2 and Db2 too.
(CURRENT_DAY - 1 DAY)
Sources of H2 are available on GitHub:
https://github.com/h2database/h2database
Building instructions are here:
https://h2database.com/html/build.html#building
You need a jar target.
To compile H2 from the current sources you need JDK 8, 9, 10, 11, or 12. Compiled jar will be compatible with more recent versions.
I'm trying to change the CRS of a geopandas dataframe. The current CRS is:
Name: unknown
Axis Info [ellipsoidal]:
- lon[east]: Longitude (degree)
- lat[north]: Latitude (degree)
Area of Use:
- undefined
Datum: World Geodetic System 1984
- Ellipsoid: WGS 84
- Prime Meridian: Greenwich
When I try dfTrans.to_crs('epsg:4326') I get the following error:
pyproj.exceptions.CRSError: Invalid projection: epsg:4326: (Internal Proj Error: proj_create: cannot build geodeticCRS 4326: SQLite error on SELECT name, ellipsoid_auth_name, ellipsoid_code, prime_meridian_auth_name, prime_meridian_code, area_of_use_auth_name, area_of_use_code, publication_date, deprecated FROM geodetic_datum WHERE auth_name = ? AND code = ?: no such column: publication_date)
For a simple command in pyproj, pyproj.CRS.from_epsg(4326), I get the same error:
File "pyproj/_crs.pyx", line 1738, in pyproj._crs._CRS.__init__
pyproj.exceptions.CRSError: Invalid projection: epsg:4326: (Internal Proj Error: proj_create: cannot build geodeticCRS 4326: SQLite error on SELECT name, ellipsoid_auth_name, ellipsoid_code, prime_meridian_auth_name, prime_meridian_code, area_of_use_auth_name, area_of_use_code, publication_date, deprecated FROM geodetic_datum WHERE auth_name = ? AND code = ?: no such column: publication_date)
I don't know enough to know what's going on, but it seems like there's an underlying function that calls a column that doesn't exist. Any ideas how to fix this or work around it?
I got that same error when using Proj-5.x. It seems that the 'publication_date' column is a Proj-6 or Proj-7 item (which both require SQLite.)
I'm trying to run a test task on Airflow but I keep getting the following error:
FAILED: ParseException 2:0 cannot recognize input near 'create_import_table_fct_latest_values' '.' 'hql'
Here is my Airflow Dag file:
import airflow
from datetime import datetime, timedelta
from airflow.operators.hive_operator import HiveOperator
from airflow.models import DAG
args = {
'owner': 'raul',
'start_date': datetime(2018, 11, 12),
'provide_context': True,
'depends_on_past': False,
'retries': 2,
'retry_delay': timedelta(minutes=5),
'email': ['raul.gregglino#leroymerlin.ru'],
'email_on_failure': True,
'email_on_retry': False
}
dag = DAG('opus_data',
default_args=args,
max_active_runs=6,
schedule_interval="#daily"
)
import_lv_data = HiveOperator(
task_id='fct_latest_values',
hive_cli_conn_id='metastore_default',
hql='create_import_table_fct_latest_values.hql ',
hiveconf_jinja_translate=True,
dag=dag
)
deps = {}
# Explicity define the dependencies in the DAG
for downstream, upstream_list in deps.iteritems():
for upstream in upstream_list:
dag.set_dependency(upstream, downstream)
Here is the content of my HQL file, in case this may be the issue and I can't figure:
*I'm testing the connection to understand if the table is created or not, then I'll try to LOAD DATA, hence the LOAD DATA is commented out.
CREATE TABLE IF NOT EXISTS opus_data.fct_latest_values_new_data (
id_product STRING,
id_model STRING,
id_attribute STRING,
attribute_value STRING
) ROW FORMAT DELIMITED FIELDS TERMINATED ',';
#LOAD DATA LOCAL INPATH
#'/media/windows_share/schemas/opus/fct_latest_values_20181106.csv'
#OVERWRITE INTO TABLE opus_data.fct_latest_values_new_data;
In the HQL file it should be FIELDS TERMINATED BY ',':
CREATE TABLE IF NOT EXISTS opus_data.fct_latest_values_new_data (
id_product STRING,
id_model STRING,
id_attribute STRING,
attribute_value STRING
) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',';
And comments should start with -- in HQL file, not #
Also this seems incorrect and causing Exception hql='create_import_table_fct_latest_values.hql '
Have a look at this example:
#Create full path for the file
hql_file_path = os.path.join(os.path.dirname(__file__), source['hql'])
print hql_file_path
run_hive_query = HiveOperator(
task_id='run_hive_query',
dag = dag,
hql = """
{{ local_hive_settings }}
""" + "\n " + open(hql_file_path, 'r').read()
)
See here for more details.
Or put all HQL into hql parameter:
hql='CREATE TABLE IF NOT EXISTS opus_data.fct_latest_values_new_data ...'
I managed to find the answer for my issue.
It was related to the path my HiveOperator was calling the file from. As no Variable had been defined to tell Airflow where to look for, I was getting the error I mentioned in my post.
Once I have defined it using the webserver interface (See picture), my dag started to work propertly.
I made a change to my DAG code regarding the file location for organization only and this is how my HiveOperator looks like now:
import_lv_data = HiveOperator(
task_id='fct_latest_values',
hive_cli_conn_id='metastore_default',
hql='hql/create_import_table_fct_latest_values2.hql',
hiveconf_jinja_translate=True,
dag=dag
)
Thanks to (#panov.st) who helped me in person to identify my issue.
So after spending quite a lot of time trying to fix my iCalendar app
from the Spinning beach ball lag every time I make a new event or edit an event,
I found this error in the console application:
"error: sql cached statement NSSQLiteStatement <0x7f8eef4c0fd0> on entity 'Group' with sql text 'SELECT t0.Z_ENT, t0.Z_PK, t0.Z_OPT, t0.ZCOLORSTRING, t0.ZISENABLED, ....... ( t0.Z_PK IN (SELECT * FROM _Z_intarray0) AND t0.Z_ENT >= ? AND t0.Z_ENT <= ?) ' failed due to missing variable binding for (null) with expecting bindings (
"<NSSQLBindVariable: 0x7f8eef47bdd0>",
"<NSSQLBindVariable: 0x7f8eef47be70>"
) but actual substitution variables {
objects = "{<NSManagedObject: 0x7f8eef540140> (entity: ExchangePrincipal; id: 0x240092b <x-coredata://93547915-498F-4251-8E7E-23DD04782C04/ExchangePrincipal/p9> ; data: <fault>)}";
}
error: sql cached statement NSSQLiteStatement <0x7f8eef48daa0> on entity 'Attendee' with sql text 'SELECT 0, t0.Z_PK, t0.Z_OPT, t0.ZADDRESSSTRING, t0.ZCOMMONNAME, t0.ZEMAIL, t0.ZINCLUDEDINALLRESPONDED, t0.ZINVITERNAME, t0.ZISSELFINVITED, t0.ZLIKENESSDATASTRING, t0.ZOMITSYNCRECORD, t0.ZPROPOSALENDDATE, t0.ZPROPOSALSTARTDATE, t0.ZPROPOSALSTATUS, t0.ZROLE, t0.ZRSVP, t0.ZSCHEDULEAGENT, t0.ZSCHEDULEFORCESEND, t0.ZSCHEDULESTATUS, t0.ZSTATUS, t0.ZSTATUSMODIFIEDDATE, t0.ZTYPE, t0.ZEVENT, t0.ZMYATTENDEEFOREVENT FROM ZATTENDEE t0 WHERE t0.ZMYATTENDEEFOREVENT IN (SELECT * FROM _Z_intarray0) ' failed due to missing variable binding for (null) with expecting bindings (
) but actual substitution variables {
destinations = "{0x122c009eb <x-coredata://93547915-498F-4251-8E7E-23DD04782C04/Event/p1163>, 0x1230009eb <x-coredata://93547915-498F-4251-8E7E-23DD04782C04/Event/p1164>, 0x1234009eb <x-coredata://93547915-498F-4251-8E7E-23DD04782C04/Event/p1165>, 0x123c009eb <x-coredata://93547915-498F-4251-8E7E-23DD04782C04/Event/p1167>, 0x1258009eb <x-coredata://93547915-498F-4251-8E7E-23DD04782C04/Event/p1174>, 0x1264009eb <x-coredata://93547915-498F-4251-8E7E-23DD04782C04/Event/p1177>, 0x126c009eb <x-coredata://93547915-498F-4251-8E7E-23DD04782C04/Event/p1179>, 0x127c009eb <x-coredata://93547915-498F-4251-8E7E-23DD04782C04/Event/p1183>, 0x1280009eb <x-coredata://93547915-498F-4251-8E7E-23DD04782C04/Event/p1184>, 0x1284009eb <x-coredata://93547915-498F-4251-8E7E-23DD04782C04/Event/p1185>}";
}"
There are around 8-10 of such errors each time I make a new event.
Can you please help me with this issue?
I already reinstalled mac os sierra few times,
but it made no difference.
What fixed it for me was a new clean install of the Mac OS.
Now I don't get the same issue (and Calendar is a pleasure to use),
but if the issue re-appears I will make an update.