Numpy genfromtxt() error using converters: Converter function does not skip footer - converters

I am using numpy.genfromtxt() to load time data in string format HH:MM:SS.S, and wishing to modify it according to a converter function (translating string UTC-time into float decimal hours). The code generates errors at the end of the file, where the converter tries to read in footer lines even when set to skip.
Here is my code:
utc_converter= lambda hhmmss: int(hhmmss.split(':')[0])+ int(hhmmss.split(':')[1])/60 + float(hhmmss.split(':')[2])/3600
testtime= np.genfromtxt(path_to_gps, dtype=None, skip_header=headerlines, skip_footer=2, usecols=(9), converters={9: utc_converter}, encoding='utf-8')
Generates the error:
Converter #0 is locked and cannot be upgraded: (occurred line #18859 for value '(21910')
Line #18859 is indeed a footer message. When no converter is used, the data is read in correctly without including the footer. When the footer lines are manually deleted from the data file, and the code modified accordingly (skip_footer=0), the conversion works fine.
How can I solve this?

Related

How I can load a font file with PIL.ImageFont.truetype in replit

I'm trying to make this welcome card thingy in discord.py and I'm running my bot on replit.com as of now.
font = ImageFont.truetype("arial.ttf", 28)
I got some examples and it worked great as long as I'm running it on my PC but wen I get to replit.com it gives the error saying
Command raised an exception: OSError: cannot open resource
How should I go about correcting this?
I don't know discord or replit but presume the issue is that you can't upload your binary font file.
If so, you have a couple of options:
find your desired font online somewhere and use requests.get(URL) to grab it in your code on replit, or
make a base64 variable in your code and decode it
The first option is covered here.
Let's look at the second. Say your font is called /Fonts/funky.ttf on your PC. Now you want that in base64 which you can do with a commandline tool on your local PC:
base64 < /Fonts/funky.txt
That will make a long string of characters. Copy it, and in your Python code add a string called font64 and set it equal to the pasted string, i.e.
font64 = 'PASTED STRING'
Now in your code you can convert the string back from base64 to binary, then wrap it in a BytesIO to make it look like a file and load it:
import base64
import io
from PIL import ImageFont
font64 = 'PASTED STRING'
# decode from base64 to binary
binary = base64.b64decode(font64)
# wrap in BytesIO to make file-like object
FileLike = io.BytesIO(binary)
# load font
font = ImageFont.truetype(FileLike, 28)

Don't show data notices in worksheet

I'm aware of data validations in Axlsx, however I'm not sure that's what I'm looking for here. I don't want to validate any data per se, but want any cell notices (in Excel here) to to be ignored.
For example, a notice I'm seeing on a couple of cells is:
The number in this cell is formatted as text or preceded by an apostrophe.
I tried doing something like this to select the whole worksheet and avoid showing any validation erros, however it didn't work for me:
sheet.add_data_validation("A1:H100", {
showErrorMessage: false
})
Can I use a Data validation to avoid displaying these notices in Excel?
This is one of the error checking options found in Excel Options ► Formulas ► Error checking rules ► Numbers formatted as text or preceded by an apostrophe. (or Alt+F,T,F then Alt+H)
To halt the display of these error control messages through VBA, run the following line of code.
Application.ErrorCheckingOptions.NumberAsText = False

What kind of encoding does posFlag requires?

How can I encode the position of the form /pathto/file.go:40:32 which is returned by token.Position.String() to a posFlag param required by ParseQueryPos which looks like /pathto/file.go:#550.
Why?
I'm using the Oracle tool to do some static analysis. I need to run Oracle.Query which requires a param of type *QueryPos. The only way to get *QueryPos is using ParseQueryPos.
The source to tools/pos.go called by ParseQueryPos says
// parsePosFlag parses a string of the form "file:pos" or
// file:start,end" where pos, start, end match #%d and represent byte
// offsets, and returns its components.
If you really had to convert from line:column strings, you'd look at the file contents and count up bytes (including newlines) leading to that line:column. But since you're working with a token.Position, it looks like you can get what you need from token.Position.Offset.

iText - adding Image element generates a corrupt PDF file

I'm using iText® 5.2.1 ©2000-2012 1T3XT BVBA and Integration Designer 8.0 to create a PDF file that is exported in an byte array.
I am creating a document with a fair amount of text and want to add a logo at the beginning.
Part of the code that is adding the image is as follows:
BASE64Decoder decoder = new BASE64Decoder();
byte[] decodedBytes = decoder.decodeBuffer(Stringovi.SLIKA1);
Image image1 = Image.getInstance(decodedBytes);
image1.setAbsolutePosition(30f, 770f);
image1.scalePercent(60f);
document.add(image1);
The input image is in byte array format because of the system requirements.
The rest of the document consists of different tables with various content and it's all text.
When I add the image in the before mentioned way the program finishes and i get an byte output that i run trough a Base64 decoder. Resulting PDF can not be opend and the error shown is:
"Error [PDF Structure 40]:Invalid reference table (xref)"
I can't see where my mistake is so if anybody could be so kind and point me in the right direction I would very much appreciate it.
The document you presented as a "broken PDF file" is not a complete PDF file. It doesn't end with %%EOF, it doesn't have a cross-reference table,... It's a PDF document that isn't complete.
This means that you don't have the following line in your code:
document.close();
If you do have this line, it isn't reached. For instance: an exception is thrown causing the code to jump to a catch clause, skipping the close() operation.
The error message saying Invalid reference table (xref) is consistent with that diagnosis. This isn't a problem caused by iText. It's a problem caused by bad coding: not closing the document and/or not dealing with exceptions correctly.

Ruby 1.9.3 - How does CSV.table know if no headers are in a CSV file?

I have been doing some testing with CSV.table. I have two small and almost identical CSV files, however one is missing the header row.
When I run CSV.table against the CSV file with the header row, everything works as expected.
When I run it against the CSV file without the header row I get:
NoMethodError: undefined method `encode' for nil:NilClass
I tried this with different types of data, with different types of headers, and get the same results.
I am curious to the magic of CSV.table. If I use CSV.parse with headers set to true, then it always makes the first row be headers no matter what. So, I have been using CSV.table to check if a CSV file being imported has a header row but I am not too comfortable with this because I don't understand if or when it will or will not work the way I'm using it.
begin
CSV.table(csv_file_path)
rescue
# Add error to log or something.
end
Does anyone know?
P.S. I've already read through this and the source code it provides on each method - http://www.ruby-doc.org/stdlib-1.9.3/libdoc/csv/rdoc/CSV.html
There isn't any magic involved, and it won't work for you in general.
As you can see from the source, table literally just calls read with headers: true. But it also converts the header to symbols (header_converters: :symbol) and this is the key to why it appears to work.
You get an error without headers because you have a blank column in your first row of data (something like a,b,,d,e). The blank gets read in as nil, and since nil can't be converted to a symbol, it blows up.
Try it with some data that doesn't have a blank in the first row - you'll see that table will treat that row of data as headers just like any of the other methods.

Resources