How to avoid this DEBUG information when running Functions and python connected to a NoSQL table in OCI
"01F4J52HHK1BT062RZJ00065AZ - urllib3.connectionpool - DEBUG XXXXXXXXXXX"
You could create a separate logger for urllib3 or alternatively you could set your main logger level to something lower than debug
urllib3_logger = logging.getLogger('urllib3')
urllib3_logger.setLevel(logging.WARNING)
or
logger = logging.getLogger()
logger.setLevel(logging.INFO)
Related
I am automating Dialogflow CX using Python client libraries. That includes agent/intent/entity etc. creation/updation/deletion.
But for the first time run, I am encountering the below error from python.
If I login to console and set the location from there and rerun the code, it is working fine. I am able to create agent.
Followed this URL of GCP -
https://cloud.google.com/dialogflow/cx/docs/concept/region
I am looking for code to automate the region & location setting before running the python code. Kindly provide me with the code.
Below is the code I am using to create agent.
Error -
google.api_core.exceptions.FailedPrecondition: 400 com.google.apps.framework.request.FailedPreconditionException: Location settings have to be initialized before creating the agent in location: us-east1. Code: FAILED_PRECONDITION
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.FAILED_PRECONDITION
details = "com.google.apps.framework.request.FailedPreconditionException: Location settings have to be initialized before creating the agent in location: us-east1. Code: FAILED_PRECONDITION"
debug_error_string = "{"created":"#1622183899.891000000","description":"Error received from peer ipv4:142.250.195.170:443","file":"src/core/lib/surface/call.cc","file_line":1068,"grpc_message":"com.google.apps.framework.request.FailedPreconditionException: Location settings have to be initialized before creating the agent in location: us-east1. Code: FAILED_PRECONDITION","grpc_status":9}"
main.py -
# Import Libraries
import google.auth
import google.auth.transport.requests
from google.cloud import dialogflowcx as df
from google.protobuf.field_mask_pb2 import FieldMask
import os, time
import pandas as pd
# Function - Authentication
def gcp_auth():
cred, project = google.auth.default(scopes=["https://www.googleapis.com/auth/cloud-platform"])
auth_req = google.auth.transport.requests.Request()
cred.refresh(auth_req)
# Function - Create Agent
def create_agent(agent_name, agent_description, language_code, location_id, location_path):
if location_id == "global":
agentsClient = df.AgentsClient()
else:
agentsClient = df.AgentsClient(client_options={"api_endpoint": f"{location_id}-dialogflow.googleapis.com:443"})
agent = df.Agent(display_name=agent_name, description=agent_description, default_language_code=language_code, time_zone=time_zone, enable_stackdriver_logging=True)
createAgentRequest = df.CreateAgentRequest(agent=agent, parent=location_path)
agent = agentsClient.create_agent(request=createAgentRequest)
return agent```
Currently, Dialogflow does not support configuring the location settings through the API, thus you can not initialise location settings through it. You can only set the location through the Console.
As an alternative, since the location setting has to be initialised only once for each region per project you could set the location and automate the agent creation process, some useful links: 1 and 2.
On the other hand, if you would find this feature useful, you can file a Feature Request, here. It will be evaluated by the Google's product team.
Many thanks Alexandre Moraes. I have raised a feature request for the same.
I'm trying to execute a SqlSensor task in Airflow using a connection to Teradata database. The connection is configured as follow:
I have provide in particular 2 driver paths separated by ", " but I am not sure if it's the proper way to do it?
/home/airflow/java_sample/tdgssconfig.jar
/home/airflow/java_sample/terajdbc4.jar
When the DAG executes, it triggers the error message
[2017-08-02 02:32:45,162] {models.py:1342} INFO - Executing <Task(SqlSensor): check_running_batch> on 2017-08-02 02:32:12
[2017-08-02 02:32:45,179] {base_hook.py:67} INFO - Using connection to: jdbc:teradata://myservername.mycompanyname.org/database=MYDBNAME,TMODE=ANSI,CHARSET=UTF8
[2017-08-02 02:32:45,313] {sensors.py:109} INFO - Poking: SELECT BATCH_KEY FROM MYDBNAME.AUDIT_BATCH WHERE BATCH_OWNER='ARO_TEST' AND AUDIT_STATUS_KEY=1;
[2017-08-02 02:32:45,316] {base_hook.py:67} INFO - Using connection to: jdbc:teradata://myservername.mycompanyname.org/database=MYDBNAME,TMODE=ANSI,CHARSET=UTF8
[2017-08-02 02:32:45,497] {models.py:1417} ERROR - java.lang.RuntimeException: Class com.teradata.jdbc.TeraDriver not found
What am I doing wrong?
The appropriate way to input multiple jars in the connections page is to separate both fully qualified paths with a comma which you did above.
I can confirm this is the approach I took and it worked (Airflow 10.1.1 and 10.1.2).
See: https://github.com/apache/airflow/blob/master/airflow/hooks/jdbc_hook.py#L51
Bonus: If you use Ad Hoc Query in Data Profiling to test it out, you'll notice that you'll get an error when you send a SELECT statement because Airflow wraps it in a LIMIT clause which TD doesn't support.
The solution provided by my team member was to merge the two jar into a single jar file. After doing it and indicating that new jar file in the driver path, it worked as expected.
Here is the link to the JAR file: https://github.com/alexisrolland/linux-setup/blob/master/teradataDriverJdbc.jar
Here is a code snippet example to use the connection in a SQLSensor Task:
CheckRunningBatch = SqlSensor(
task_id='check_running_batch',
conn_id='ed_data_quality_edw_dev',
sql="SELECT CASE WHEN MAX(BATCH_KEY) IS NOT NULL THEN 0 ELSE 1 END FROM DATABASE.AUDIT_BATCH WHERE STATUS_KEY=1;",
poke_interval=300,
dag=dag)
How to MonetDB log settings.
I want to change the log level of "merovingian.log".
I want to output the ERROR and WARN the merovingian.log.
But now merovingian.log is outputting only MSG log.
2016-07-22 18:12:03 MSG merovingian[7825]: proxying client x.x.x.x:51609 for database 'test' to mapi:monetdb:///var/MonetDB/dbfarm/test/.mapi.sock?database=test
2016-07-22 18:12:03 MSG merovingian[7825]: target connection is on local UNIX domain socket, passing on filedescriptor instead of proxying
OS is "CentOS 6.4",
MonetDB version is "MonetDB-11.19.7".
Any advice how to solve this problem?
No fine-tuning is provided in where log information ends up.
I'm connecting to MongoDB using the Ruby Driver and I don't see how to turn off the logging. Here's the current output from an IRB session:
D, [2015-06-06T11:01:05.622513 #30526] DEBUG -- : MONGODB | COMMAND | namespace=admin.$cmd selector={:ismaster=>1} flags=[] limit=-1 skip=0 project=nil | runtime: 0.7546ms
D, [2015-06-06T11:01:15.623716 #30526] DEBUG -- : MONGODB | COMMAND | namespace=admin.$cmd selector={:ismaster=>1} flags=[] limit=-1 skip=0 project=nil | runtime: 0.7083ms
....
It repeats the lines above over and over, which makes it very difficult to type in the IRB session. Here's the code used to create the client connection:
client = Mongo::Client.new('mongodb://127.0.0.1:27017/mydb')
Anyone know how to turn that off? FWIW, I tried setting verbose=false in /etc/mongod.conf and that did not help.
You can change Mongo's logger instance and/or verbosity.
From the docs:
You can either use the default driver logger or set your own. To set your own:
Mongo::Logger.logger = other_logger
Please see the Ruby Logger documentation for more information on the default logger API and available levels.
To change the logger level:
Mongo::Logger.logger.level = Logger::WARN
I'm in the process of executing Maven commands to run tests in the console (MacOSX). Recently, development efforts have produced extraneous messages in the console (info, debug, warning, etc.) I'd like to know how to remove messages like this:
INFO c.c.m.s.c.p.ApplicationProperties - Loading application properties from: app-config/shiro.properties
I've used this code to remove messages from the dbunit tests:
ch.qos.logback.classic.Logger Logger = (ch.qos.logback.classic.Logger)LoggerFactory.getLogger("org.dbunit");
Logger.setLevel(Level. ERROR);
However, I'm unsure how to disable these additional (often verbose and irritating) messages from showing up on the console so that I can see the output more easily. Additional messages appear as above and these:
DEBUG c.c.m.s.c.f.dao.AbstractDBDAO - Adding filters to the Main Search query.
WARN c.c.m.s.c.p.JNDIConfigurationProperties - Unable to find JNDI value for name: xxxxx
INFO c.c.m.a.t.d.DatabaseTestFixture - * executing sql: xxxxx
The successful answer was:
SOLUTION: Solution to issue IS adding a 'logback-test.xml' file to the root of my test folder. I used the default contents (as instructed by the documentation - thanks #Charlie). Once file exists there, FIXED!