I've been using the Python SDK for quite a while and haven't experienced any issues.
That changed on Saturday (Aug 11) when I started getting this exception from list_catalog
..."/3.6.5/lib/python3.6/site-packages/squareconnect/rest.py", line 66, in __init__
self.data += chunk.decode('utf8')
UnicodeDecodeError: 'utf-8' codec can't decode bytes in position 8190-8191: unexpected end of data
Here's a snippet that reproduces the error.
squareconnect.configuration.access_token = sqcredentials.access_token
api = squareconnect.apis.catalog_api.CatalogApi()
cursor = ''
page = 1
while cursor is not None:
print('Processing item page {:d}'.format(page))
resp = api.list_catalog(cursor=cursor)
cursor = resp.cursor
page += 1
Any ideas ?
Related
I get the following error while I'm playing YouTube audio with my bot
[tls # 0000024ef8c4d480] Error in the pull function.
[matroska,webm # 0000024ef8c4a400] Read error
[tls # 0000024ef8c4d480] The specified session has been invalidated for some reason.
Last message repeated 1 times
It seems like YouTube links expire? I don't really know but I need to fix this issue. This is my code:
class YTDLSource(discord.PCMVolumeTransformer):
def __init__(self, source, *, data, requester):
super().__init__(source)
self.requester = requester
self.title = data['title']
self.description = data['description']
self.uploader = data['uploader']
self.duration = data['duration']
self.web_url = data['webpage_url']
self.thumbnail = data['thumbnail']
def __getitem__(self, item: str):
return self.__getattribute__(item)
#classmethod
async def create_source(cls, ctx, player, search: str, *, loop, download=True):
async with ctx.typing():
loop = loop or asyncio.get_event_loop()
to_run = partial(ytdl.extract_info, url=search, download=download)
raw_data = await loop.run_in_executor(None, to_run)
if 'entries' in raw_data:
# take first item from a playlist
if len(raw_data['entries']) == 1:
data = raw_data['entries'][0]
else:
data = raw_data['entries']
#loops entries to grab each video_url
total_duration = 0
try:
for i in data:
webpage = i['webpage_url']
title = i['title']
description = i['description']
uploader = i['uploader']
duration = i['duration']
thumbnail = i['thumbnail']
total_duration += duration
if download:
source = ytdl.prepare_filename(i)
source = cls(discord.FFmpegPCMAudio(source), data=i, requester=ctx.author)
else:
source = {'webpage_url': webpage, 'requester': ctx.author, 'title': title, 'uploader': uploader, 'description': description, 'duration': duration, 'thumbnail': thumbnail}
player.queue.append(source)
except Exception as e:
print(e)
return
embed=discord.Embed(title="Playlist", description="Queued", color=0x30a4fb, timestamp=datetime.now(timezone.utc))
embed.set_author(name=ctx.author.display_name, icon_url=ctx.author.avatar_url)
embed.set_thumbnail(url=data[0]['thumbnail'])
embed.add_field(name=raw_data['title'], value=f"{len(data)} videos queued.", inline=True)
embed.set_footer(text=raw_data["uploader"] + ' - ' + '{0[0]}m {0[1]}s'.format(divmod(total_duration, 60)))
await ctx.send(embed=embed)
return
embed=discord.Embed(title="Playlist", description="Queued", color=0x30a4fb, timestamp=datetime.now(timezone.utc))
embed.set_author(name=ctx.author.display_name, icon_url=ctx.author.avatar_url)
embed.set_thumbnail(url=data['thumbnail'])
embed.add_field(name=data['title'], value=(data["description"][:72] + (data["description"][72:] and '...')), inline=True)
embed.set_footer(text=data["uploader"] + ' - ' + '{0[0]}m {0[1]}s'.format(divmod(data["duration"], 60)))
await ctx.send(embed=embed)
if download:
source = ytdl.prepare_filename(data)
else:
source = {'webpage_url': data['webpage_url'], 'requester': ctx.author, 'title': data['title'], 'uploader': data['uploader'], 'description': data['description'], 'duration': data['duration'], 'thumbnail': data['thumbnail']}
player.queue.append(source)
return
source = cls(discord.FFmpegPCMAudio(source), data=data, requester=ctx.author)
player.queue.append(source)
#classmethod
async def regather_stream(cls, data, *, loop):
loop = loop or asyncio.get_event_loop()
requester = data['requester']
to_run = partial(ytdl.extract_info, url=data['webpage_url'], download=True)
data = await loop.run_in_executor(None, to_run)
return(cls(discord.FFmpegPCMAudio(data['url']), data=data, requester=requester))
I'm using the rewrite branch of discord.py for the bot.
I'm not sure if I need to provide more details? Please let me know, I really need to get this fixed...
In fact it isn't really a problem with your code (and many people complain of this error).
This is just a possible issue when streaming a video. If you absolutely want to stream it, you have to accept this as a potential issue. Note how (almost) every music bots set limitations for the video/music you want to listen to.
If you need to ensure you do not get this issue, you have to fully download the music. (Which will also make the bot loading longer before playing).
Would you be able to post all your code? i may have a solution for you if i was able to see the whole code.
the solution i would recomend is to download the soong and then delete it after.
You could set your download to true and then add this in your player_loop
try:
# We are no longer playing this song...so, lets delete it!
with YoutubeDL(ytdlopts) as ydl:
info = ydl.extract_info(source.web_url, download=False)
filename = ydl.prepare_filename(info)
try:
if os.path.exists(filename):
os.remove(filename)
else:
pass
except Exception as E:
print(E)
await self.np.delete()
except discord.HTTPException:
pass
bit botched but could be cleaned up, this was the best solution i have found for me.
I am using tesseract to perform OCR on screengrabs. I have an app using a tkinter window leveraging self.after in the initialization of my class to perform constant image scrapes and update label, etc values in the tkinter window. I have searched for multiple days and can't find any specific examples how to leverage CREATE_NO_WINDOW with Python3.6 on a Windows platform calling tesseract with pytesseract.
This is related to this question:
How can I hide the console window when I run tesseract with pytesser
I have only been programming Python for 2 weeks and don't understand what/how to perform the steps in the above question. I opened up the pytesseract.py file and reviewed and found the proc = subprocess.Popen(command, stderr=subproces.PIPE) line but when I tried editing it I got a bunch of errors that I couldn't figure out.
#!/usr/bin/env python
'''
Python-tesseract. For more information: https://github.com/madmaze/pytesseract
'''
try:
import Image
except ImportError:
from PIL import Image
import os
import sys
import subprocess
import tempfile
import shlex
# CHANGE THIS IF TESSERACT IS NOT IN YOUR PATH, OR IS NAMED DIFFERENTLY
tesseract_cmd = 'tesseract'
__all__ = ['image_to_string']
def run_tesseract(input_filename, output_filename_base, lang=None, boxes=False,
config=None):
'''
runs the command:
`tesseract_cmd` `input_filename` `output_filename_base`
returns the exit status of tesseract, as well as tesseract's stderr output
'''
command = [tesseract_cmd, input_filename, output_filename_base]
if lang is not None:
command += ['-l', lang]
if boxes:
command += ['batch.nochop', 'makebox']
if config:
command += shlex.split(config)
proc = subprocess.Popen(command, stderr=subprocess.PIPE)
status = proc.wait()
error_string = proc.stderr.read()
proc.stderr.close()
return status, error_string
def cleanup(filename):
''' tries to remove the given filename. Ignores non-existent files '''
try:
os.remove(filename)
except OSError:
pass
def get_errors(error_string):
'''
returns all lines in the error_string that start with the string "error"
'''
error_string = error_string.decode('utf-8')
lines = error_string.splitlines()
error_lines = tuple(line for line in lines if line.find(u'Error') >= 0)
if len(error_lines) > 0:
return u'\n'.join(error_lines)
else:
return error_string.strip()
def tempnam():
''' returns a temporary file-name '''
tmpfile = tempfile.NamedTemporaryFile(prefix="tess_")
return tmpfile.name
class TesseractError(Exception):
def __init__(self, status, message):
self.status = status
self.message = message
self.args = (status, message)
def image_to_string(image, lang=None, boxes=False, config=None):
'''
Runs tesseract on the specified image. First, the image is written to disk,
and then the tesseract command is run on the image. Tesseract's result is
read, and the temporary files are erased.
Also supports boxes and config:
if boxes=True
"batch.nochop makebox" gets added to the tesseract call
if config is set, the config gets appended to the command.
ex: config="-psm 6"
'''
if len(image.split()) == 4:
# In case we have 4 channels, lets discard the Alpha.
# Kind of a hack, should fix in the future some time.
r, g, b, a = image.split()
image = Image.merge("RGB", (r, g, b))
input_file_name = '%s.bmp' % tempnam()
output_file_name_base = tempnam()
if not boxes:
output_file_name = '%s.txt' % output_file_name_base
else:
output_file_name = '%s.box' % output_file_name_base
try:
image.save(input_file_name)
status, error_string = run_tesseract(input_file_name,
output_file_name_base,
lang=lang,
boxes=boxes,
config=config)
if status:
errors = get_errors(error_string)
raise TesseractError(status, errors)
f = open(output_file_name, 'rb')
try:
return f.read().decode('utf-8').strip()
finally:
f.close()
finally:
cleanup(input_file_name)
cleanup(output_file_name)
def main():
if len(sys.argv) == 2:
filename = sys.argv[1]
try:
image = Image.open(filename)
if len(image.split()) == 4:
# In case we have 4 channels, lets discard the Alpha.
# Kind of a hack, should fix in the future some time.
r, g, b, a = image.split()
image = Image.merge("RGB", (r, g, b))
except IOError:
sys.stderr.write('ERROR: Could not open file "%s"\n' % filename)
exit(1)
print(image_to_string(image))
elif len(sys.argv) == 4 and sys.argv[1] == '-l':
lang = sys.argv[2]
filename = sys.argv[3]
try:
image = Image.open(filename)
except IOError:
sys.stderr.write('ERROR: Could not open file "%s"\n' % filename)
exit(1)
print(image_to_string(image, lang=lang))
else:
sys.stderr.write('Usage: python pytesseract.py [-l lang] input_file\n')
exit(2)
if __name__ == '__main__':
main()
The code I am leveraging is similar to the example in the similar question:
def get_string(img_path):
# Read image with opencv
img = cv2.imread(img_path)
# Convert to gray
img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
# Apply dilation and erosion to remove some noise
kernel = np.ones((1, 1), np.uint8)
img = cv2.dilate(img, kernel, iterations=1)
img = cv2.erode(img, kernel, iterations=1)
# Write image after removed noise
cv2.imwrite(src_path + "removed_noise.png", img)
# Apply threshold to get image with only black and white
# Write the image after apply opencv to do some ...
cv2.imwrite(src_path + "thres.png", img)
# Recognize text with tesseract for python
result = pytesseract.image_to_string(Image.open(src_path + "thres.png"))
return result
When it gets to the following line, there is a flash of a black console window for less than a second and then it closes when it runs the command.
result = pytesseract.image_to_string(Image.open(src_path + "thres.png"))
Here is the picture of the console window:
Program Files (x86)_Tesseract
Here is what is suggested from the other question:
You're currently working in IDLE, in which case I don't think it
really matters if a console window pops up. If you're planning to
develop a GUI app with this library, then you'll need to modify the
subprocess.Popen call in pytesser.py to hide the console. I'd first
try the CREATE_NO_WINDOW process creation flag. – eryksun
I would greatly appreciate any help for how to modify the subprocess.Popen call in the pytesseract.py library file using CREATE_NO_WINDOW. I am also not sure of the difference between pytesseract.py and pytesser.py library files. I would leave a comment on the other question to ask for clarification but I can't until I have more reputation on this site.
I did more research and decided to learn more about subprocess.Popen:
Documentation for subprocess
I also referenced the following articles:
using python subprocess.popen..can't prevent exe stopped working prompt
I changed the original line of code in pytesseract.py:
proc = subprocess.Popen(command, stderr=subprocess.PIPE)
to the following:
proc = subprocess.Popen(command, stderr=subprocess.PIPE, creationflags = CREATE_NO_WINDOW)
I ran the code and got the following error:
Exception in Tkinter callback Traceback (most recent call last):
File
"C:\Users\Steve\AppData\Local\Programs\Python\Python36-32\lib\tkinter__init__.py",
line 1699, in call
return self.func(*args) File "C:\Users\Steve\Documents\Stocks\QuickOrder\QuickOrderGUI.py", line
403, in gather_data
update_cash_button() File "C:\Users\Steve\Documents\Stocks\QuickOrder\QuickOrderGUI.py", line
208, in update_cash_button
currentCash = get_string(src_path + "cash.png") File "C:\Users\Steve\Documents\Stocks\QuickOrder\QuickOrderGUI.py", line
150, in get_string
result = pytesseract.image_to_string(Image.open(src_path + "thres.png")) File
"C:\Users\Steve\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pytesseract\pytesseract.py",
line 125, in image_to_string
config=config) File "C:\Users\Steve\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pytesseract\pytesseract.py",
line 49, in run_tesseract
proc = subprocess.Popen(command, stderr=subprocess.PIPE, creationflags = CREATE_NO_WINDOW) NameError: name 'CREATE_NO_WINDOW'
is not defined
I then defined the CREATE_NO_WINDOW variable:
#Assignment of the value of CREATE_NO_WINDOW
CREATE_NO_WINDOW = 0x08000000
I got the value of 0x08000000 from the above linked article. After adding the definition I ran the application and I didn't get any more console window popups.
I"m getting the following error message, and I can't figure out why renaming is trigger this error message:
Traceback (most recent call last):
File "C:\Apps\UtilitiesByMarc\make hypertext list from directory aaa.py", line 17, in
os.rename(fullname, nufullname)
WindowsError: [Error 32] The process cannot access the file because it is being used by another process
Any insights would be much appreciated.
Here's my code:
import os
dir_path = r'E:\ddelete'
dir_list = os.listdir(dir_path)
html_file = r"E:\ddelete\AAA__ReadMeFirst___dirlist.html"
text_file = open(html_file, "w")
i = 10000
for item in sorted(dir_list):
fullname = os.path.join(dir_path,item)
ext = ext = os.path.splitext(fullname)[1]
nufullname = os.path.join(dir_path, str(i) + ext)
nufilename = str(i) + ext
print fullname
print nufullname
os.rename(fullname, nufullname)
temp_item = '''''' + item + '<br />' + '\r\n'
text_file.write(temp_item)
i += 1
text_file.close()
I just realized that I was attempting to rename the html_file to which I was writing text.
I'm sorry for wasting your time.
Marc
I am trying to make a small API that logs into a internal monitoring tool via web and retrieves images on pages that I specify using the login credentials I specify. It's not passing any authentication to the last section after it has already built the URL. After the URL is built, it is put into variable y. I then attempt to open y and save it, that is where the authentication problem is happening. Please scroll down for examples.
import urllib
import urllib2
import lxml, lxml.html
ID = raw_input("Enter ID:")
PorS = raw_input("Enter 1 for Primary 2 for Secondary:")
base_link = 'http://statseeker/cgi/nim-report?rid=42237&command=Graph&mode=ping&list=jc-'
dash = '-'
end_link = '&tfc_fav=range+%3D+start_of_today+-+1d+to+now%3B&year=&month=&day=&hour=&minute=&duration=&wday_from=&wday_to=&time_from=&time_to=&tz=America%2FChicago&tfc=range+%3D+start_of_today+-+1d+to+now%3B&rtype=Delay&graph_type=Filled&db_type=Average&x_step=1&interval=60&y_height=100&y_gridlines=5&y_max=&y_max_power=1&x_gridlines=on&legend=on'
urlauth = base_link + jConnectID + dash + jConnectPorS + end_link
print urlauth
realm = 'statseeker'
username = 'admin'
password = '*****'
auth_handler = urllib2.HTTPBasicAuthHandler()
auth_handler.add_password(realm, urlauth, username, password)
opener = urllib2.build_opener(auth_handler)
urllib2.install_opener(opener)
data = opener.open(urlauth).read()
html = lxml.html.fromstring(data)
imgs = html.cssselect('img.graph')
for x in imgs:
y = 'http://statseeker%s' % (x.attrib['src'])
g = urllib2.urlopen(y).read()
urllib2.urlopen(test.jpg, 'wb').write(g)
print 'http://statseeker%s' % (x.attrib['src'])
with open('statseeker.html', 'a') as f:
f.write(y)
Result:
C:\Users\user\Documents\Scripting>python test.py
Enter ID:4050
Enter 1 for Primary 2 for Secondary:1
http://statseeker/cgi/nim-report?rid=42237&command=Graph&mode=ping&list=jc-4050-
1&tfc_fav=range+%3D+start_of_today+-+1d+to+now%3B&year=&month=&day=&hour=&minute
=&duration=&wday_from=&wday_to=&time_from=&time_to=&tz=America%2FChicago&tfc=ran
ge+%3D+start_of_today+-+1d+to+now%3B&rtype=Delay&graph_type=Filled&db_type=Avera
ge&x_step=1&interval=60&y_height=100&y_gridlines=5&y_max=&y_max_power=1&x_gridli
nes=on&legend=on
Traceback (most recent call last):
File "JacksonShowAndSave.py", line 35, in <module>
g = urllib2.urlopen(y).read()
File "C:\Python27\lib\urllib2.py", line 127, in urlopen
return _opener.open(url, data, timeout)
File "C:\Python27\lib\urllib2.py", line 410, in open
response = meth(req, response)
File "C:\Python27\lib\urllib2.py", line 523, in http_response
'http', request, response, code, msg, hdrs)
File "C:\Python27\lib\urllib2.py", line 448, in error
return self._call_chain(*args)
File "C:\Python27\lib\urllib2.py", line 382, in _call_chain
result = func(*args)
File "C:\Python27\lib\urllib2.py", line 531, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 401: Authorization Required
C:\Users\user\Documents\Scripting>
How do I fix these errors to pass the Authentication to any web page opened on that website? I thought that any urlopen request would use the same authentication as above.
The Section that is Failing:
y = 'http://statseeker%s' % (x.attrib['src'])
g = urllib2.urlopen(y).read()
urllib2.urlopen(test.jpg, 'wb').write(g)
print 'http://statseeker%s' % (x.attrib['src'])
with open('statseeker.html', 'a') as f:
I built another auth handler for the request:
auth_handler2 = urllib2.HTTPBasicAuthHandler()
auth_handler2.add_password(realm, y, username, password)
opener2 = urllib2.build_opener(auth_handler2)
urllib2.install_opener(opener2)
link = urllib2.Request(y)
response = urllib2.urlopen(link)
output = open('out2.jpg','wb')
output.write(response.read())
output.close()
I'm playing around with mongoengine(0.8.2) and trying to write some images using the GridFS functionality. This is how I have declared my model:
class Lady(DynamicDocument):
id = ObjectIdField()
name = StringField()
offers = ListField(EmbeddedDocumentField(Offer))
pictures = ListField(ImageField())
So what I want to do is to append the pictures to the pictures field. What I tried is the following:
fileA = open('/home/evermean/Pictures/a.jpg', 'r')
fileB = open('/home/evermean/Pictures/b.jpg', 'r')
upload = [fileA, fileB]
files = []
for f in upload:
mf = ImageGridFsProxy()
mf.put(f, content_type = 'image/jpeg')
files.append(mf)
lady.pictures = files
lady.save()
What I get is the following:
Error
Traceback (most recent call last):
File "/home/evermean/Code/django/pourlamour/core/tests.py", line 38, in test_basic_addition
mf.put(f, content_type = 'image/jpeg')
File "/home/evermean/Code/django/env/pourlamour/local/lib/python2.7/site-packages/mongoengine/fields.py", line 1257, in put
field = self.instance._fields[self.key]
AttributeError: 'NoneType' object has no attribute '_fields'
I tried to figure out what goes wrong and the following line from the fields.py seems to be responsible although I cannot quite see what to do about it.
field = self.instance._fields[self.key]
I also tried to use the GridFSProxy which works ... so why does it fail with ImageGridFsProxy?
Any ideas? Where is my mistake? Thanks for your help.