I'm using the Ruby official Ruby C interface and am not able to bzip working. It did build with bzip support, ./configure said:
checking bzlib.h usability... yes
checking bzlib.h presence... yes
checking for bzlib.h... yes
So I wrote this example program that just writes an entry to two files, one supposedly bzip'd and one not. Neither is compressed; aside from the simple file size test at the end I can edit the with_bzip.tcb
file and see the raw string text there.
require 'tokyocabinet'
include TokyoCabinet
def write filename, options
File.unlink filename if File.exists? filename
bdb = BDB::new
bdb.tune(0, 0, 0, -1 -1, options) or raise "Couldn't tune"
bdb.open(filename, BDB::OWRITER | BDB::OCREAT | BDB::OLCKNB) or raise "Couldn't open"
bdb["test"] = "This string should be compressed and not appear raw.\n" * 10000
bdb.close
end
write 'without_bzip.tcb', 0
write 'with_bzip.tcb', BDB::TBZIP
puts "Not actually compressed" unless File.size('with_bzip.tcb') < File.size('without_bzip.tcb')
What makes it worse is that if I try a preview release of Oklahoma Mixer (example following - though I don't have the reputation to add the new tag), it compresses fine. When I tucked some debugging into its try() call, it seems to be making the same call to tune(0, 0, 0, -1, -1, 4). I'm pretty completely stumped - can anyone tell me what my code above is doing wrong?
require 'oklahoma_mixer'
OklahomaMixer.open("minimal_om.tcb", :opts => 'lb') do |db|
db["test"] = "This string should be compressed and not appear raw.\n" * 10000
end
It is an evil, subtle bug. I left out a comma in the tune() call and wrote -1 -1 instead of -1, -1. All the arguments are optional, so it was quietly not bzipping. Argh.
Related
I'm trying to compare certain pixel values in my pyautogui script, but it crashes with following error message after either multiple successful runs, or sometimes just straight on the first call:
Traceback (most recent call last):
File "F:\Koodit\Python\HeroWars NNet\Assets\autodataGet.py", line 219, in <module>
battle = observeBattle()
File "F:\Koodit\Python\HeroWars NNet\Assets\autodataGet.py", line 180, in observeBattle
statii = getHeroBattlePixels()
File "F:\Koodit\Python\HeroWars NNet\Assets\autodataGet.py", line 32, in getHeroBattlePixels
colormatch = pyautogui.pixelMatchesColor(location[0], location[1], alive, tolerance=5)
File "E:\Program Files\Python\lib\site-packages\pyscreeze\__init__.py", line 557, in pixelMatchesColor
pix = pixel(x, y)
File "E:\Program Files\Python\lib\site-packages\pyscreeze\__init__.py", line 582, in pixel
return (r, g, b)
File "E:\Program Files\Python\lib\contextlib.py", line 120, in __exit__
next(self.gen)
File "E:\Program Files\Python\lib\site-packages\pyscreeze\__init__.py", line 111, in __win32_openDC
raise WindowsError("windll.user32.ReleaseDC failed : return 0")
OSError: windll.user32.ReleaseDC failed : return 0
My code (this is called multiple times, sometimes it crashes on first run, sometimes it runs nicely for around 100 calls before failing, also, my screen is 4K, so the resolutions get big):
def getSomePixelStatuses():
someLocations= [
[1200, 990],
[1300, 990],
[1400, 990],
[1500, 990],
[1602, 990],
[1768, 990],
[1868, 990],
[1968, 990],
[2068, 990],
[2169, 990]
]
status = []
someValue= (92, 13, 12)
for location in someLocations:
colormatch = pyautogui.pixelMatchesColor(location[0], location[1], someValue, tolerance=5)
status.append(colormatch)
return status
I have no idea how to mitigate this problem. It would seem that pyautogui uses pyscreeze to read pixel values on screen, and most probable candidate for the place where error occurs is the pyscreeze pixel function:
def pixel(x, y):
"""
TODO
"""
if sys.platform == 'win32':
# On Windows, calling GetDC() and GetPixel() is twice as fast as using our screenshot() function.
with __win32_openDC(0) as hdc: # handle will be released automatically
color = windll.gdi32.GetPixel(hdc, x, y)
if color < 0:
raise WindowsError("windll.gdi32.GetPixel failed : return {}".format(color))
# color is in the format 0xbbggrr https://msdn.microsoft.com/en-us/library/windows/desktop/dd183449(v=vs.85).aspx
bbggrr = "{:0>6x}".format(color) # bbggrr => 'bbggrr' (hex)
b, g, r = (int(bbggrr[i:i+2], 16) for i in range(0, 6, 2))
return (r, g, b)
else:
# Need to select only the first three values of the color in
# case the returned pixel has an alpha channel
return RGB(*(screenshot().getpixel((x, y))[:3]))
I installed these libraries just yesterday, and I'm running python 3.8 on windows 10, and pyscreeze is version 0.1.25 so in theory everything should be up to date, but somehow something ends up crashing. Is there a way to mitigate this, either modifying my code, or even the library itself, or is my environment not suitable for this operation?
Well I know it's not particularly helpful; but for me, this error was fixed simply by running my code on 3.7 instead of 3.8. There shouldn't be any changes you have to make to your code, however (unless you were using walrus!)
On Windows, this can be done with the -3.7 command line flag, as long as 3.7 is installed
PyScreeze and PyAutoGUI maintainer here. This is an issue that has been fixed in PyScreeze 0.1.28, so you just need to update it by running pip install -U pyscreeze.
For more context, here's the GitHub issue where it was reported: https://github.com/asweigart/pyscreeze/pull/73
It's a bug. You were on the right track, as the problem is indeed in this line of the pixel() function:
with __win32_openDC(0) as hdc
That function uses cyptes.windll which doesn't seem to do well with the negative values sometimes returned from windll.user32.GetDC(), which subsequently creates an exception when windll.user32.ReleaseDC() is called.
The folks at pillow helped track this down and propose a fix.
issue filed at pyautogui
issue filed at pillow which led to the solution
pending PR at pyscreeze to address
I can use pixel function on Python 3.8 like this:
try:
a = pixel(100,100)
> except:
> a = pixel(100,100)
I don't have any clue why this works, but it works.
I had this error too and i fixed it. Just use try and except.
While true:
try:
x,y = pyautogui.position()
print(pyautogui.pixel(x,y))
except:
print("Cannot get pixel for the moment")
Given that you might be taking pixels multiple times, or you can do so, try and except works wonders to solve any pyscreeze for pyautogui issue. Honestly i dont know whats up with pyscreeze, but this works for me. Cheers
I'm working with a Ruby script that reads a .json file.
Here is the JSON file:
{
"feed.xml": "93d5b140dd2b4779edef0347ac835fb1",
"index.html": "1cbe25936e392161bad6074d65acdd91",
"md5.json": "655d7c1dbf83a271f348a50a44ba4f6a",
"test.sh": "9be192b1b5a9978cb3623737156445fd",
"index.html": "c064e204040cde216d494776fdcfb68f",
"main.css": "21b13d87db2186d22720e8c881a78580",
"welcome-to-jekyll.html": "01d7c7d66bdeecd9cd69feb5b4b4184d"
}
It is completely valid, and is checked for its existence before trying to read from it. Example:
if File.file?("md5.json")
puts "MD5s exists"
mddigests = File.open("md5.json", "r")
puts "MD5s" + mddigests.read
items = JSON.parse(mddigests.read) <--- Where it all goes wrong.
puts items["feed.xml"]
Everything works up until that point:
MD5s exists
MD5s{
"feed.xml": "93d5b140dd2b4779edef0347ac835fb1",
"index.html": "1cbe25936e392161bad6074d65acdd91",
"md5.json": "655d7c1dbf83a271f348a50a44ba4f6a",
"test.sh": "9be192b1b5a9978cb3623737156445fd",
"index.html": "c064e204040cde216d494776fdcfb68f",
"main.css": "21b13d87db2186d22720e8c881a78580",
"welcome-to-jekyll.html": "01d7c7d66bdeecd9cd69feb5b4b4184d"
}
common.rb:156:in `initialize': A JSON text must at least contain two octets! (JSON::ParserError)
I've searched and tried a lot of different things, to no avail. I'm stumped. Thanks!
You have a duplicate call to read() at the point that it all goes wrong. Replace the second call to read() with the variable mddigests and all should be fine.
This code should work like you'd expect:
if File.file?("md5.json")
puts "MD5s exists"
mddigests = File.open("md5.json", "r")
digests = mddigests.read
puts "MD5s" + digests
items = JSON.parse(digests) #<--- This should work now!
puts items["feed.xml"]
end
The reason is that the file pointer is moved after the first read(), and by the second read(), it's at the end of file, hence the message requiring at least 2 octets.
I was trying to debug a Python 2 script running on Raspbian (Raspberry Pi flavoured Debian Linux) which had code like
euid = os.geteuid()
if euid != 0:
print("you must be root!")
exit(1)
It seemed like, in the user's environment, euid would sometimes be nonzero even if the script was called with sudo.
To investigate whether this was actually the case, I tried to figure out what os.geteuid() is actually doing.
Since the os module is pretty OS-specific by its nature, the source doesn't actually have a clear definition for os.geteuid().
I also tried hg cloneing the source and compiling it, then using inspect.findsource(os.geteuid), but:
TypeError: <built-in function geteuid> is not a module, class, method, function, traceback, frame, or code object
It's... a builtin? Then "geteuid" in dir(__import__("__builtin__")) should be True, but it isn't.
Is geteuid's definition hidden because it could be spoofed into returning the wrong thing (and that would be bad)? Where can I see these sorts of functions' actual source?
ASCII stupid question, get a stupid ANSI.
I did try full-text searching the source, but apparently I used the wrong command the first time and gave up.
$ grep -rnw '.' -e "geteuid"
./Misc/setuid-prog.c:129: uid_t euid = geteuid();
./Lib/site.py:209: if hasattr(os, "getuid") and hasattr(os, "geteuid"):
./Lib/site.py:211: if os.geteuid() != os.getuid():
./Lib/test/test_shutil.py:84: #unittest.skipIf(hasattr(os, 'geteuid') and os.geteuid() == 0,
./Lib/test/test_httpservers.py:339: if os.name == 'posix' and os.geteuid() != 0:
./Lib/test/test_httpservers.py:395:#unittest.skipIf(hasattr(os, 'geteuid') and os.geteuid() == 0,
./Lib/test/test_spwd.py:8:#unittest.skipUnless(hasattr(os, 'geteuid') and os.geteuid() == 0,
./Lib/test/test_posix.py:44: "getegid", "geteuid", "getgid", "getgroups",
./Lib/test/test_argparse.py:1532:#unittest.skipIf(hasattr(os, 'geteuid') and os.geteuid() == 0,
Binary file ./Lib/tarfile.py matches
./Lib/rexec.py:148: 'getcwd', 'getuid', 'getgid', 'geteuid', 'getegid')
Binary file ./Modules/posixmodule.o matches
./Modules/posixmodule.c:4047:"geteuid() -> euid\n\n\
./Modules/posixmodule.c:4053: return _PyInt_FromUid(geteuid());
./Modules/posixmodule.c:8944: {"geteuid", posix_geteuid, METH_NOARGS, posix_geteuid__doc__},
./Doc/library/rexec.rst:234: 'times', 'uname', 'getpid', 'getppid', 'getcwd', 'getuid', 'getgid', 'geteuid',
./Doc/library/os.rst:136:.. function:: geteuid()
Binary file ./python matches
Binary file ./libpython2.7.a matches
./Modules/posixmodule.c:4053, indeed:
#ifdef HAVE_GETEUID
PyDoc_STRVAR(posix_geteuid__doc__,
"geteuid() -> euid\n\n\
Return the current process's effective user id.");
static PyObject *
posix_geteuid(PyObject *self, PyObject *noargs)
{
return _PyInt_FromUid(geteuid());
}
#endif
I don't know what I expected, subprocess.check_output(["id"])?
It uses the C standard library, it's never wrong.
The Ruby driver tutorial states:
The second argument to the method is options defining whether the operations should be executed in order and what write concern should be used.
But it doesn't say how to set the write concern. I've tried every way I can possibly think of:
collection.bulk_write(operations, ordered: false, w: 0)
collection.bulk_write(operations, ordered: false, write: 0)
collection.bulk_write(operations, ordered: false, writeConcern: 0)
collection.bulk_write(operations, ordered: false, write: {w: 0})
None of them work, and the function still raises a BulkWriteError complaining about duplicate key errors.
So how do you set write concern to 0 when using bulk_write?
I'm using MongoDB Ruby driver version 2.0
I want to leave it to Blakes Seven to get credit for a proper answer, but I would just like to share my solution. I use bulk writes like this in several locations, so I wrote a function for specifically ignoring duplicate errors:
def ignore_bulk_write_duplicates
fail "Expected a block" unless block_given?
begin
yield
rescue Mongo::Error::BulkWriteError => e
e.result[:write_errors].each { |error|
if error["code"] == 11000
# Duplicate error is OK
else
raise
end
}
end
end
And I use it like this:
ignore_bulk_write_duplicates {
collection.bulk_write(operations, ordered: false, w: 0)
}
At this point I'm assuming that w: 0 sets write concern to 0, but I have yet to confirm that... Eagerly awaiting Blake's upcoming answer :-)
Is there a way in Ruby to find the version of a file, specifically a .dll file?
For Windows EXE's and DLL's:
require "Win32API"
FILENAME = "c:/ruby/bin/ruby.exe" #your filename here
s=""
vsize=Win32API.new('version.dll', 'GetFileVersionInfoSize',
['P', 'P'], 'L').call(FILENAME, s)
p vsize
if (vsize > 0)
result = ' '*vsize
Win32API.new('version.dll', 'GetFileVersionInfo',
['P', 'L', 'L', 'P'], 'L').call(FILENAME, 0, vsize, result)
rstring = result.unpack('v*').map{|s| s.chr if s<256}*''
r = /FileVersion..(.*?)\000/.match(rstring)
puts "FileVersion = #{r ? r[1] : '??' }"
else
puts "No Version Info"
end
The 'unpack'+regexp part is a hack, the "proper" way is the VerQueryValue API, but this should work for most files. (probably fails miserably on extended character sets.)
What if you want to get the version info with ruby, but the ruby code isn't running on Windows?
The following does just that (heeding the same extended charset warning):
#!/usr/bin/ruby
s = File.read(ARGV[0])
x = s.match(/F\0i\0l\0e\0V\0e\0r\0s\0i\0o\0n\0*(.*?)\0\0\0/)
if x.class == MatchData
ver=x[1].gsub(/\0/,"")
else
ver="No version"
end
puts ver
As of Ruby 2.0, the DL module is deprecated. Here is an updated version of AShelly's answer, using Fiddle:
version_dll = Fiddle.dlopen('version.dll')
s=''
vsize = Fiddle::Function.new(version_dll['GetFileVersionInfoSize'],
[Fiddle::TYPE_VOIDP, Fiddle::TYPE_VOIDP],
Fiddle::TYPE_LONG).call(filename, s)
raise 'Unable to determine the version number' unless vsize > 0
result = ' '*vsize
Fiddle::Function.new(version_dll['GetFileVersionInfo'],
[Fiddle::TYPE_VOIDP, Fiddle::TYPE_LONG,
Fiddle::TYPE_LONG, Fiddle::TYPE_VOIDP],
Fiddle::TYPE_VOIDP).call(filename, 0, vsize, result)
rstring = result.unpack('v*').map{|s| s.chr if s<256}*''
r = /FileVersion..(.*?)\000/.match(rstring)
puts r[1]
If you are working on the Microsoft platform, you should be able to use the Win32 API in Ruby to call GetFileVersionInfo(), which will return the information you're looking for.
http://msdn.microsoft.com/en-us/library/ms647003.aspx
For any file, you'd need to discover what format the file is in, and then open the file and read the necessary bytes to find out what version the file is. There is no API or common method to determine a file version in Ruby.
Note that it would be easier if the file version were in the file name.