I am using a Lua script to determine the file size:
local filesize=0
local filePath = "somepath.bin"
local file,msg = io.open(filePath, "r")
if file then
filesize=file:seek("end")
file:close()
filePresent = true
end
However, this only seem to work for files up to 2GB. For larger files filesize is always nil. Is there any limitation on io.open? And if so, how could I work around this?
Running Lua 5.1.4 on Windows Server 2008 R2 64bit
The problem is not in io.open, but file:seek. You can check the error like this:
filesize, err = file:seek("end")
if not filesize then
print(err)
end
The error message is probably Invalid argument. That's because for files larger than 2GB, its size is over what 32-bit long can hold, which causes the C function fseek fail to work.
In POSIX systems, Lua uses fseeko which takes the size of off_t instead of long in fseek. In Windows, there's a _fseeki64 which I guess does similar job. If these are not available, fseek is used, and it would cause the problem.
The relevant source is liolib.c(Lua 5.2). As #lhf points out, in Lua 5.1, fseek is always used (source). Upgrading to Lua 5.2 could possibly solve the problem.
Internally, Lua uses the ISO C function long int ftell(FILE *stream); to determine the return value for file:seek(). A long int is always 32 bits on Windows, so you are out of luck here. If you can, you should use some external library to detect the file size -- I recommend luafilesystem.
On old Lua versions (where file:seek() is limited to 2Gb) you can ask cmd.exe to get file size:
function filesize(filename)
-- returns file size (or nil if the file doesn't exist or unable to open)
local command = 'cmd /d/c for %f in ("'..filename..'") do #echo(%~zf'
return tonumber(io.popen(command):read'*a')
end
print(filesize[[C:\Program Files\Windows Media Player\wmplayer.exe]])
--> 73728
print(filesize[[E:\City.of.the.Living.Dead.1980.720p.BluRay.x264.Skazhutin.mkv]])
--> 8505168882
Related
I was trying to print a backtrace using dladdr(). info.dli_fname in the following code snippet is displaying the file name of an ELF file. Could you please tell me if it is possible to resolve and print the name of the source file and line number programmatically without the help of addr2line or gdb?
Code:
void print_caller(void)
{
int rc = -1;
Dl_info info = {0};
void *pc = __builtin_return_address(0);
rc = dladdr(pc, &info);
printf(" ==> %p: %s (in %s)\n", pc, info.dli_sname, info.dli_fname);
}
Output:
$ ./a.out
==> 0x55a6b04a1589: foo2 (in ./a.out)
tell me if it is possible to resolve and print the name of the source file and line number programmatically
It is definitely possible -- addr2line and gdb do that.
But it is very non-trivial -- it requires understanding and decoding (possibly multiple) complicated debugging info formats.
If you only care about a single platform (looks like Linux), things are a bit easier -- you only need to decode DWARF.
But that format is still pretty complicated. You should start with a helper library, such as libdwarf.
I'm trying to load a C shared library within Ruby using Fiddle.
Here is a minimal example:
require 'fiddle'
require 'fiddle/import'
module Era
extend Fiddle::Importer
dlload './ServerApi.so'
extern 'int era_init_lib()'
extern 'void era_deinit_lib()'
extern 'int era_process_request(const char* request, char** response)'
extern 'void era_free(char* response)'
end
Era.era_init_lib
begin
# ...
ensure
Era.era_deinit_lib
end
The shared library loads without issues. However when I call Era.era_init_lib it tries to load additional libraries (Network.so and Protobuf.so). I have these file located in the current working directory (in the same directory as ServerApi.so).
However when I try to execute the code above I receive the following error:
! Failed to load library: /home/username/.rvm/rubies/ruby-2.6.5/bin/Network.so, error: /home/username/.rvm/rubies/ruby-2.6.5/bin/Network.so: cannot open shared object file: No such file or directory
If I place the file at the location the error describes everything works fine.
My guess is that the C working directory of fiddle is different from the Ruby working directory. I would like to keep the project files within the project and not in the Ruby installation directory.
How can I use Network.so from my project folder?
All the *.so files are provided by a third-party. I do not have the source and as a result cannot change these files. The function signatures are provided by the documentation.
Searching for Network.so in the strace gives me these results:
readlink("/proc/self/exe", "/home/username/.rvm/rubies/ruby-2."..., 4096) = 44
openat(AT_FDCWD, "/home/username/.rvm/rubies/ruby-2.6.5/bin/Network.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
futex(0x7fcc16666d90, FUTEX_WAKE_PRIVATE, 2147483647) = 0
futex(0x7fcc16b44520, FUTEX_WAKE_PRIVATE, 2147483647) = 0
write(2, "! Failed to load library: ", 26! Failed to load library: ) = 26
write(2, "/home/username/.rvm/rubies/ruby-2."..., 50/home/username/.rvm/rubies/ruby-2.6.5/bin/Network.so) = 50
write(2, ", error: ", 9, error: ) = 9
write(2, "/home/username/.rvm/rubies/ruby-2."..., 109/home/username/.rvm/rubies/ruby-2.6.5/bin/Network.so: cannot open shared object file: No such file or directory) = 109
write(2, "\n", 1) = 1
I've also written a C script which does the same thing which works perfectly fine when the files are dropped into the same directory. So it might be the fault of the library, which I assume checks the location of the current running program, then tries to load the library from that folder. This would explain the behavior when ran as a Ruby script (since it runs as part of the Ruby program), whereas a C binary runs standalone.
For those that want to re-create the (Linux) issue. You can download the necessary files from here. Which gives you the server-linux-x86_64.sh file.
Supported distros are: Suse, Ubuntu, Debian, Red Hat and CentOS but others may also work fine.
You can either run the installer, which should place the files in /opt/eset/RemoteAdministrator/Server. Or, assuming most of you don't want to install the full application you can run the following command:
sed '1,/^# Start of TAR\.GZ file #$/d' server-linux-x86_64.sh | sed '1d' > server-linux-x86_64.tar.gz
Which removes all the installer instructions from the .sh file and only leaves the binary .tar.gz data, writing it to server-linux-x86_64.tar.gz.
Copy the files ServerApi.so, Protobuf.so and Network.so into a directory of your liking. Create a Ruby script (with the question code) in the same directory and run the script.
Because ServerApi.so checks /proc/self/exe for the location of all subsequent files to load, and it is very difficult to modify this target by normal means, it is easier to just modify ServerApi.so itself so that it uses something else besides proc for the source.
If we run strings ServerApi.so, we can verify that the location to check is stored inside a string in ServerApi.so:
strings ServerApi.so | grep 'proc/self/exe'
B/proc/self/exe
So now all we need to do is modify this string to something else that works for us.
The easiest way to modify the string is to replace it with something that is exactly the same length as the original. This way we do not have to worry about changing the end-of-string zero padding or accidentally changing the total size of ServerApi.so.
Here we can see a suitable candidate could be /tmp/scriptexe:
/proc/self/exe
/tmp/scriptexe <- same length
So let's do that:
sed -e 's/proc\/self\/exe/tmp\/scriptexe/' ServerApi.so > ServerApi_Mod.so
Now we can verify the change:
strings ServerApi_Mod.so | grep scriptexe
B/tmp/scriptexe
Next we need to create /tmp/scriptexe to actually point to our Ruby script:
ln -s /the/full/path/to/our/ruby/script.rb /tmp/scriptexe
Then we modify our script:
dlload './ServerApi_Mod.so
Now we can run it as normal:
ruby script.rb
And everything should work.
If we read the strace output we see that the library obtains the current executable location from /proc/self/exe, and then searches subsequent libraries from there.
/proc/self/exe is not easily modifiable, but by using a hard link to a Ruby executable in the current directory we can trick it to point to a new folder.
Problem is making a hard link requires root.
In any case, here is a self-contained solution (note that it will ask for root password the first time you run it, in order to create the hard link).
Put this at the top of your script:
# Obtain path to current executable
exe = File.readlink("/proc/self/exe")
# Check if we are running the hard-liked version
if !exe.match /localruby/
if !File.exist?('localruby')
# Create a hard link to the current Ruby exe using sudo
system("sudo ln #{exe} localruby")
end
puts "Restarting..."
# In order to prevent infinite busy loop in case of some mishap
sleep 1
# Rerun self using the hard-linked Ruby executable.
# This will make /proc/self/exe point to the hard-link, which then
# allows the ESET library to search for .so files in current folder.
exec('./localruby', File.expand_path(__FILE__))
end
require 'fiddle'
require 'fiddle/import'
# ...rest of your script goes here...
A simple solution without any extra Ruby code is to just create the hard link manually, and then always run the script with ./localruby myscript.rb, instead of using the normal ruby myscript.rb.
I am writing a code in Fortran to read a list of properties from a file, and am using the IOSTAT function to skip invalid data. The relevant section of code is as follows:
do j=1,1000
read(22,*,IOSTAT=ios) step,T,K,U,Tot,P
If(IS_IOSTAT_END(ios)) Exit !exits loop if value is not number or end of file
IF(ios.ne.0) cycle
sumT=sumT+T
sumU=sumU+U
sumK = sumK + K
sumKsq = sumKsq + (K**2.d0)
end if
end do
This code has previously worked fine when running on Linux, but when running on Mac OSX I get the error message 'IS_IOSTAT_END has no implicit type'. Could somebody please explain how to correct this?
Thanks
The intrinsic procedure IS_IOSTAT_END(i) is defined since Fortran 2003. Sufficiently recent compiler version must be used.
I'm still learning ruby, so I'm sure I'm doing something wrong here, but using ruby 1.9.3 on windows, I'm having a problem writing a file with random ascii garbage to be a specific size. I need to be able to write these files for a test on an application I'm QAing. On Mac and on *nix, the file size is written correctly every time. But on windows, it generates files of random size, generally between 1,024 bytes and 1,031 bytes.
I'm sure the problem is one of the characters that the rstr is generating is counting as two characters but... it seems like this shouldn't happen.
Here is my code:
num = 10
k = 1
for i in 1..num
fname = "f#{i}.txt"
f = File.new(fname, "w")
for k in 1..size
rstr = "#{(1..1024).map{rand(255).chr}.join}"
f.write rstr
print " #{rstr.size} " # this returns 1024 every time.
rstr = ""
end
f.close
end
Also tried:
opts = {}
opts[:encoding] = "UTF-8"
fname = "f#{i}.txt"
f = File.new(fname, "w", opts)
By default files open in Windows are open with text mode meaning that line endings and other details are adjusted.
If you want the files be written byte-to-byte exactly as you want, you need to open the files in binary mode:
File.new("foo", "wb") do |f|
# ...
end
The b is a ignored on POSIX operating systems, so your scripts are now cross-platform compatible.
Note: I used block syntax to manage the file so it properly closes and disposes the file handler once the block is executed. You no longer need to worry about closing the file ;-)
Hope this helps.
There is not any 255 ASCII. The values goes from 0~254.
If you try to printf 255.chr, you'll get a multibyte character.
As Windows does not standard utf-8, you'll get incorrect values. Hence the problem you're facing!
Try adding #coding: utf-8 at the top of your file. It should get things working.
I am running a Rails (3.2.3) application with Ruby 1.9.3p194 on the basic Ubuntu lucid32 image in a Vagrant virtual box. The virtual box is running on Leopard, for what it's worth. I'm trying to use rubyzip in the application to decompress a zip archive - 2009_da_lmp.zip. Using code directly from examples in the rubyzip repository, I can confirm that I can list the archive file contents:
#f is the absolute path to 2009_da_lmp.zip (string)
Zip::ZipFile.open(f) { |zf| zf.entries[0] }
=> 20090101_da_lmp.csv #that is indeed a file in the archive.
Using some more code from the examples in the repository, I try to get at an actual file in the archive:
Zip::ZipInputStream.open(f) { |zis|
entry = zis.get_next_entry
print "first line of '#{entry.name}' (#{entry.size} bytes: ) "
puts "'#{zis.gets.chomp}'" }
=> first line of '20090101_da_lmp.csv' (826610 bytes: ) Zlib::DataError:
invalid stored block lengths #and a long stack trace I can provide
#if that might help
The Mac OS decompression utility unzips the archive fine. I was wondering if it was some kind of encoding-related thing (my locale is set to en_US.UTF-8 because to make using PostgreSQL in dev less painful), but I don't know how to tell if that's the case. I can't find any information on what might cause this error.
This is a typical error found when feeding random data to an inflater. In fact you will get this error about 1/4 of the time from random data (when the low three bits of the first byte are 000 or 001). So I would guess that the inflation is simply starting at the wrong byte for some reason.