I am trying to build on a test Twilio script in order that the POST request can just use values returned by method is a script.
The values are set as environmental variables and I have checked that they are correct in the Python shell using the same command as the script.
Python 2.7 being used.
The environmental variables are stored as strings and return as such.
from twilio.rest import TwilioRestClient
import os
ACCOUNT_SID = os.environ['ACCOUNT_SID']
AUTH_TOKEN = os.environ['AUTH_TOKEN']
client = TwilioRestClient(ACCOUNT_SID, AUTH_TOKEN)
def target_number():
return os.environ['TARGET_NUMBER']
def my_number():
return os.environ['MY_NUMBER']
message = client.messages.create(
body="foo bar", # Message body, if any
to=target_number(),
from_=my_number(),
)
print message.body
I am getting the following error:
python my_twilio_testing_script.py
Traceback (most recent call last):
File "my_twilio_testing_script.py", line 19, in <module>
to=target_number(),
File "my_twilio_testing_script.py", line 9, in target_number
return os.environ['TARGET_NUMBER']
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/UserDict.py", line 23, in __getitem__
raise KeyError(key)
KeyError: 'TARGET_NUMBER'
Twilio developer evangelist here.
It appears that some of your environment variables are working, otherwise the script would blow up when you set the account SID and auth token.
So it seems that maybe TARGET_NUMBER is perhaps not set. In the terminal window in which you run the script, are you able to perform the following bash command:
$ echo $TARGET_NUMBER
Does that show the number you expect? If not, you may need to export the environment variable again.
Related
I am learning python and PySimpleGUI seems like a good start for exercises. With this self-exercise I'm working on, I would like to view and edit a YAML file. So far, I am able to create a prompt to browse and select a yaml. I am able to print the data in the console. But my next step is to view the yaml view PySimpleGUI window. I will work on how to edit the yaml content once I can figure out how to display it.
Here is my code:
import PySimpleGUI as sg
import yaml
from yaml.loader import SafeLoader
import os
working_directory = os.getcwd()
layout = [
[sg.Text("Shoose your yaml file:")],
[sg.InputText(key="-FILE_PATH-"),
sg.FileBrowse(initial_folder=working_directory, file_types=[("YAML Files","*.yaml")])],
[sg.Button("Submit"), sg.Exit()],
[sg.Multiline(size=(30,5), key= data)]
]
window = sg.Window("File Loader", layout).Finalize()
while True:
event, values = window.read()
if event in (sg.WIN_CLOSED, 'Exit'):
break
elif event == "Submit":
file_path = values["-FILE_PATH-"];
with open(file_path) as f:
data = yaml.load(f, Loader=SafeLoader)
# print(data)
print(values[data])
window.close()
Running this code i get this error:
Traceback (most recent call last):
File "yaml_gui.py", line 13, in <module>
[sg.Multiline(size=(30,5), key= data)]
NameError: name 'data' is not defined
I'm stuck because I am not sure why it is returning this error. The code works if I decide to just print the results in my terminal by using print(data). But when I use print(values[data]), it doesn't work.
I am getting this error in pip search while studying python.
The picture is an error when I pip search. Can you tell me how to fix it?
$ pip search pdbx
ERROR: Exception:
Traceback (most recent call last):
File "*/lib/python3.7/site-packages/pip/_internal/cli/base_command.py", line 224, in _main
status = self.run(options, args)
File "*/lib/python3.7/site-packages/pip/_internal/commands/search.py", line 62, in run
pypi_hits = self.search(query, options)
File "*/lib/python3.7/site-packages/pip/_internal/commands/search.py", line 82, in search
hits = pypi.search({'name': query, 'summary': query}, 'or')
File "/usr/lib/python3.7/xmlrpc/client.py", line 1112, in __call__
return self.__send(self.__name, args)
File "/usr/lib/python3.7/xmlrpc/client.py", line 1452, in __request
verbose=self.__verbose
File "*/lib/python3.7/site-packages/pip/_internal/network/xmlrpc.py", line 46, in request
return self.parse_response(response.raw)
File "/usr/lib/python3.7/xmlrpc/client.py", line 1342, in parse_response
return u.close()
File "/usr/lib/python3.7/xmlrpc/client.py", line 656, in close
raise Fault(**self._stack[0])
xmlrpc.client.Fault: <Fault -32500: 'RuntimeError: This API has been temporarily disabled due to unmanageable load and will be deprecated in the near future. Please use the Simple or JSON API instead.'>
The pip search command queries PyPI's servers, and PyPI's maintainers have explained that the API endpoint that the pip search command queries is very resource intensive and too expensive for them to always keep open to the public. Consequently they sometimes throttle access and are actually planning to remove it completely soon.
See this GitHub issues thread ...
The solution I am using for now is to pip install pip-search (a utility created by GitHub user #victorgarric).
So, instead of 'pip search', I use pip_search. Definitely beats searching PyPI via a web browser
Follow the suggestion from JRK at the discussion at github (last comment) the search command is temporarily disabled, use your browser to search for packages meanwhile:
Check the thread on github and give him a thumb up ;)
search on website, https://pypi.org/,
then install the package you wanted
The error says
Please use the Simple or JSON API instead
You can try pypi-simple to query the pip repository
https://pypi.org/project/pypi-simple/
It gives an example too, I tried to use it here:
pypi-simple version 0.8.0 DistributionPackage' object has no attribute 'get_digest':
!/usr/bin/env python3
-*- coding: utf-8 -*-
"""
Created on Thu Nov 11 17:40:03 2020
#author: Pietro
"""
from pypi_simple import PyPISimple
def simple():
package=input('\npackage to be checked ')
try:
with PyPISimple() as client:
requests_page = client.get_project_page(package)
except:
print("\n SOMETHING WENT WRONG !!!!! \n\n",
"CHECK INTERNET CONNECTION OR DON'T KNOW WHAT HAPPENED !!!\n")
pkg = requests_page.packages[0]
print(pkg)
print(type(pkg))
print('\n',pkg,'\n')
print('\n'+pkg.filename+'\n')
print('\n'+pkg.url+'\n')
print('\n'+pkg.project+'\n')
print('\n'+pkg.version+'\n')
print('\n'+pkg.package_type+'\n')
#print('\n'+pkg.get_digest()+'\n','ENDs HERE !!!!') #wasnt working
if __name__ == '__main__':
simple()
got -4 so far for this answer don't know why , figureout I can try to check for a package with:
# package_name = input('insert package name : ')
package_name = 'numpy'
import requests
url = ('https://pypi.org/pypi/'+package_name+'/json')
r = requests.get(url)
try:
data = r.json()
for i in data:
if i == 'info':
print('ok')
for j in data[i]:
if j == 'name':
print((data[i])[j])
print([k for k in (data['releases'])])
except:
print('something went south !!!!!!!!!!')
I am trying to use gspread, but I need the library to mesh well with another async library I am using.
After digging through the docs for gspread, I found this function that I can use:
class gspread.Client(auth, session=None)
An instance of this class communicates with Google API.
Parameters:
auth – An OAuth2 credential object. Credential objects are those created by the oauth2client library. https://github.com/google/oauth2client
session – (optional) A session object capable of making HTTP requests while persisting some parameters across requests. Defaults to requests.Session.
Which gives me an optional session parameter. How would I specify the session to use aiohttp?
I wrote a bit of test code, which compiles fine, but running the code crashes.
import aiohttp
import gspread
import random
from oauth2client.service_account import ServiceAccountCredentials
scope = ['https://spreadsheets.google.com/feeds',
'https://www.googleapis.com/auth/drive']
creds = ServiceAccountCredentials.from_json_keyfile_name('client_secret.json', scope)
c = gspread.authorize(creds)
client = gspread.Client(auth=c, session=aiohttp.ClientSession)
sheet = client.open_by_key('1Hkwo9gSpk3NjgPLPkG8kh0zBNw2nxsYWRw0cVdn0JA0')
ws = sheet.get_worksheet(0)
rcount = ws.row_count
msg = ws.cell(random.randint(1,rcount),1).value
print(msg)
The error message I get is below:
Traceback (most recent call last):
File "c:\Users\xxxxx\.vscode\extensions\ms-python.python-2018.9.0\pythonFiles\experimental\ptvsd_launcher.py", line 118, in <module>
vspd.debug(filename, port_num, '', '', run_as)
File "c:\Users\xxxxx\.vscode\extensions\ms-python.python-2018.9.0\pythonFiles\experimental\ptvsd\ptvsd\debugger.py", line 37, in debug
run(address, filename, *args, **kwargs)
File "c:\Users\xxxxx\.vscode\extensions\ms-python.python-2018.9.0\pythonFiles\experimental\ptvsd\ptvsd\_local.py", line 79, in run_file
run(argv, addr, **kwargs)
File "c:\Users\xxxxx\.vscode\extensions\ms-python.python-2018.9.0\pythonFiles\experimental\ptvsd\ptvsd\_local.py", line 140, in _run
_pydevd.main()
File "c:\Users\xxxxx\.vscode\extensions\ms-python.python-2018.9.0\pythonFiles\experimental\ptvsd\ptvsd\_vendored\pydevd\pydevd.py", line 1751, in main
debugger.connect(host, port)
File "c:\Users\xxxxx\.vscode\extensions\ms-python.python-2018.9.0\pythonFiles\experimental\ptvsd\ptvsd\_vendored\pydevd\pydevd.py", line 1107, in run
return self._exec(is_module, entry_point_fn, module_name, file, globals, locals)
File "c:\Users\xxxxx\.vscode\extensions\ms-python.python-2018.9.0\pythonFiles\experimental\ptvsd\ptvsd\_vendored\pydevd\pydevd.py", line 1114, in _exec
pydev_imports.execfile(file, globals, locals) # execute the script
File "c:\Users\xxxxx\.vscode\extensions\ms-python.python-2018.9.0\pythonFiles\experimental\ptvsd\ptvsd\_vendored\pydevd\_pydev_imps\_pydev_execfile.py", line 25, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "c:\Users\xxxxx\Desktop\Coding\Discord Bot\Testing\test.py", line 14, in <module>
ws = sheet.get_worksheet(0)
File "C:\Users\xxxxx\AppData\Local\Programs\Python\Python36\lib\site-packages\gspread\models.py", line 141, in get_worksheet
sheet_data = self.fetch_sheet_metadata()
File "C:\Users\xxxxx\AppData\Local\Programs\Python\Python36\lib\site-packages\gspread\models.py", line 123, in fetch_sheet_metadata
r = self.client.request('get', url, params=params)
File "C:\Users\xxxxx\AppData\Local\Programs\Python\Python36\lib\site-packages\gspread\client.py", line 73, in request
headers=headers
TypeError: get() missing 1 required positional argument: 'url'
PS C:\Users\xxxxx\Desktop\Coding\Discord Bot>
Any ideas?
It's impossible to use aiohttp with current version of gspread (3.0.1). the gspread library uses synchronous calls and aiohttp uses asynchronous calls.
Please, reconsider to use compatible library like requests or httplib2.
For anyone wondering, once I posted this question, I found that someone made an async wrapper for gspread. Check out the library here and show this guy your appreciation. I sure am!
I'm trying to make an Airflow task intentionally fail and error out by passing in a Bash line (thisshouldnotrun) that doesn't work. Airflow is outputting the following:
[2017-06-15 17:44:17,869] {bash_operator.py:94} INFO - /tmp/airflowtmpLFTMX7/run_bashm2MEsS: line 7: thisshouldnotrun: command not found
[2017-06-15 17:44:17,869] {bash_operator.py:97} INFO - Command exited with return code 127
[2017-06-15 17:44:17,869] {models.py:1417} ERROR - Bash command failed
Traceback (most recent call last):
File "/home/ubuntu/.local/lib/python2.7/site-packages/airflow/models.py", line 1374, in run
result = task_copy.execute(context=context)
File "/home/ubuntu/.local/lib/python2.7/site-packages/airflow/operators/bash_operator.py", line 100, in execute
raise AirflowException("Bash command failed")
AirflowException: Bash command failed
[2017-06-15 17:44:17,871] {models.py:1433} INFO - Marking task as UP_FOR_RETRY
[2017-06-15 17:44:17,878] {models.py:1462} ERROR - Bash command failed
Traceback (most recent call last):
File "/home/ubuntu/.local/bin/airflow", line 28, in <module>
args.func(args)
File "/home/ubuntu/.local/lib/python2.7/site-packages/airflow/bin/cli.py", line 585, in test
ti.run(ignore_task_deps=True, ignore_ti_state=True, test_mode=True)
File "/home/ubuntu/.local/lib/python2.7/site-packages/airflow/utils/db.py", line 53, in wrapper
result = func(*args, **kwargs)
File "/home/ubuntu/.local/lib/python2.7/site-packages/airflow/models.py", line 1374, in run
result = task_copy.execute(context=context)
File "/home/ubuntu/.local/lib/python2.7/site-packages/airflow/operators/bash_operator.py", line 100, in execute
raise AirflowException("Bash command failed")
airflow.exceptions.AirflowException: Bash command failed
Will Airflow send an email for these kind of errors? If not, what would be the best way to send an email for these errors?
I'm not even sure if airflow.cfg is setup properly... Since the ultimate goal is to test the email alerting notification, I want to make sure airflow.cfg is setup properly. Here's the setup:
[email]
email_backend = airflow.utils.email.send_email_smtp
[smtp]
# If you want airflow to send emails on retries, failure, and you want to use
# the airflow.utils.email.send_email_smtp function, you have to configure an
# smtp server here
smtp_host = emailsmtpserver.region.amazonaws.com
smtp_starttls = True
smtp_ssl = False
# Uncomment and set the user/pass settings if you want to use SMTP AUTH
# smtp_user = airflow_data_user
# smtp_password = password
smtp_port = 587
smtp_mail_from = airflow_data_user#domain.com
What is smtp_starttls? I can't find any info for it in the documentation or online. If we have 2-factor authentication needed to view emails, will that be an issue here for Airflow?
Here's my Bash command:
task1_bash_command = """
export PATH=/home/ubuntu/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin
export rundate=`TZ='America/Los_Angeles' date +%F -d "yesterday"`
export AWS_CONFIG_FILE="/home/ubuntu/.aws/config"
/home/ubuntu/bin/snowsql -f //home/ubuntu/sql/script.sql 1> /home/ubuntu/logs/"$rundate"_dev.log 2> /home/ubuntu/logs/"$rundate"_error_dev.log
if [ -e /home/ubuntu/logs/"$rundate"_error_dev.log ]
then
exit 64
fi
And my task:
task1 = BashOperator(
task_id = 'run_bash',
bash_command = task1_bash_command,
dag = dag,
retries = 2,
email_on_failure = True,
email = 'username#domain.com')
smtp_starttls basically means Use TLS
Set this to False and set smtp_ssl to True if you want to use SSL instead. You probably need smtp_user and smtp_password for either.
Airflow will not handle 2 step authentication. However, is you are using AWS you likely don't need it as your SMTP (SES) credentials are different from your AWS credentials.
See here.
EDIT:
For airflow to send an email on failure, there are a couple things that need to be set on your task, email_on_failure and email.
See here for example:
def throw_error(**context):
raise ValueError('Intentionally throwing an error to send an email.')
t1 = PythonOperator(task_id='throw_error_and_email',
python_callable=throw_error,
provide_context=True,
email_on_failure=True,
email='your.email#whatever.com',
dag=dag)
Use below link for creating airflow dag.
How to trigger daily DAG run at midnight local time instead of midnight UTC time
Approach 1 :
You can setup SMTP locally and make it send email on jobs failure.
[email]
email_backend = airflow.utils.email.send_email_smtp
[smtp]
smtp_host = localhost
smtp_starttls = False
smtp_ssl = False
smtp_port = 25
smtp_mail_from = noreply#company.com
Approach 2 : You can use Gmail to send email.
I have written an article to do this.
https://helptechcommunity.wordpress.com/2020/04/04/airflow-email-configuration/
If we have 2-factor authentication needed to view emails, will that be an issue here for Airflow?
You can use google app password to get your way around 2 factor authentication
https://support.google.com/mail/answer/185833?hl=en-GB
Source - https://docs.aws.amazon.com/mwaa/latest/userguide/configuring-env-variables.html
I'm trying to run a Python script that connects to a MySQL database through PyMySQL. The script is effectively:
import pymysql
cnx = pymysql.connect(read_default_file = "/directory/my.cnf", cursorclass = pymysql.cursors.DictCursor)
cursor = cnx.cursor()
# Do stuff.
When I run the script in the interpreter, I don't get any errors, but when I try to run it from the command line, I get the following error:
Traceback (most recent call last):
File "s02_prepare_data_RNN.py", line 264, in <module>
(omniture, urls, years, global_regions) = get_omniture_data("omniture_results")
File "s02_prepare_data_RNN.py", line 76, in get_omniture_data
sso_to_accountid = get_sso_accountids()
File "s02_prepare_data_RNN.py", line 31, in get_sso_accountids
cursor.execute(query)
File "/home/rdu/malcorn/.local/lib/python2.6/site-packages/pymysql/cursors.py", line 134, in execute
result = self._query(query)
File "/home/rdu/malcorn/.local/lib/python2.6/site-packages/pymysql/cursors.py", line 282, in _query
conn.query(q)
File "/home/rdu/malcorn/.local/lib/python2.6/site-packages/pymysql/connections.py", line 768, in query
self._affected_rows = self._read_query_result(unbuffered=unbuffered)
File "/home/rdu/malcorn/.local/lib/python2.6/site-packages/pymysql/connections.py", line 929, in _read_query_result
result.read()
File "/home/rdu/malcorn/.local/lib/python2.6/site-packages/pymysql/connections.py", line 1125, in read
first_packet = self.connection._read_packet()
File "/home/rdu/malcorn/.local/lib/python2.6/site-packages/pymysql/connections.py", line 893, in _read_packet
packet.check_error()
File "/home/rdu/malcorn/.local/lib/python2.6/site-packages/pymysql/connections.py", line 369, in check_error
err.raise_mysql_exception(self._data)
File "/home/rdu/malcorn/.local/lib/python2.6/site-packages/pymysql/err.py", line 120, in raise_mysql_exception
_check_mysql_exception(errinfo)
File "/home/rdu/malcorn/.local/lib/python2.6/site-packages/pymysql/err.py", line 115, in _check_mysql_exception
raise InternalError(errno, errorvalue)
pymysql.err.InternalError: (1046, u'No database selected')
Modifying my code from:
import pymysql
cnx = pymysql.connect(read_default_file = "/directory/my.cnf", cursorclass = pymysql.cursors.DictCursor)
cursor = cnx.cursor()
# Do stuff.
to:
import pymysql
cnx = pymysql.connect(read_default_file = "/directory/my.cnf", cursorclass = pymysql.cursors.DictCursor)
cursor = cnx.cursor()
if __name__ == "__main__":
# Do stuff.
fixed the error. I had the thought that Python might be trying to execute the queries before the connection was established, so I tried putting the main part of my program under if __name__ == "__main__": and that fixed it. I'm still not 100% what is going on, though. I had assumed the code would wait for the connection to be established before proceeding to the following lines, but this fix suggests that's not the case.
It's also worth noting that I was only getting the error when running the original script from the command line on a server that has Python 2.6. When I ran the original script from the command line on my local machine that has Python 2.7, I did not get the error.
Anyway, if __name__ == "__main__": is good Python style, so I'll make sure to use it in the future.
Sometimes this may simply happen when the database URI is not built correctly. For me, the error occurred when my pymysql connection string (URL) looked like this:
mysql+pymysql://:#localhost/
To fix it, it needed to look like:
mysql+pymysql://<my-database-user-name>:#localhost/<my-database-name> # or replace localhost with your particular host name
If you don't select the database in connection, you must add this line.
connection = pymysql.connect(host=DB_host, user=DB_user, password=DB_password) # NO DB_NAME SPECIFED
cursor = connection.cursor()
cursor.execute(f'CREATE DATABASE IF NOT EXISTS {DB_NAME}') # ADD THIS LINE!
connection.select_db(DB_NAME)