I'm currently working on lattices. To solve some problems, I have to generate a big number of matrices of the same basis. This takes a lot of time. For example, to generate 10'000 bases, I have to launch the code when I go to bed and retrieve the list of basis in the morning. The problem is that I can't do it every day.
So I'd like to save my list of 1000 matrices once for all in a text file. The problem is that when I do it, I get strings.
The matrix list is named BB.
with open('yourfile.csv', 'w') as f1:
writefile = csv.writer(f1)
writefile.writerows(BB)
import csv
with open('yourfile.csv','rU') as f1:
data=list( csv.reader(f1) )
Do you know how I could find a way to save the matrix list and then, directly recover a list? I'm working on the Sage notebook.
The correct and easiest ways to save and load Sagemath objects via a file are
save(your_list_of_matrix, 'filename.sobj')
your_list_of_matrix = load('filename.sobj')
Saving Sagemath objects to CSV will need to convert the values into strings and will lose precision.
Refer to the official document for more detail.
Related
I am generating and storing PDFs in a database.
The pdf data is stored in a text field using Convert.ToBase64String(pdf.ByteArray)
If I generate the same exact PDF that already exists in the database, and compare the 2 base64strings, they are not the same. A big portion is the same, but it appears about 5-10% of the text is different each time.
What would make 2 pdfs different if both were generated using the same method?
This is a problem because I can't tell if the PDF was modified since it was last saved to the db.
Edit: The 2 pdfs visually appear exactly the same when viewing the actual pdf, but the base64string of the bytes are different
Two PDFs that look 100% the same visually can be completely different under the covers. PDF producing programs are free to write the word "hello" as a single word or as five individual letters written in any order. They are also free to draw the lines of a table first followed by the cell contents, or the cell contents first, or any combination of these such as one cell at a time.
If you are actually programmatically creating the PDFs and you create two PDFs using completely identical code you still won't get files that are 100% identical. There's a couple of reasons for this, the most obvious is that PDFs support creation and modification dates. These will obviously change depending on when they are created. You can override these (and confuse everyone else so I don't recommend this) using something like this:
var info = writer.Info;
info.Put(PdfName.CREATIONDATE, new PdfDate(new DateTime(2001,01,01)));
info.Put(PdfName.MODDATE, new PdfDate(new DateTime(2001,01,01)));
However, PDFs also support a unique identifier in the trailer's /ID entry. To the best of my knowledge iText has no support for overriding this parameter. You could duplicate your PDF, change this manually and then calculate your differences and you might get closer to a comparison.
Then there's fonts. When subsetting fonts, producers create a unique internal name based on the original name and an arbitrary selection of six uppercase ASCII letters. So for the font Calibri the font's name could be JLXWHD+Calibri one time and SDGDJT+Calibri another time. iText doesn't support overriding of this because you'd probably do more harm than good. These internal names are used to avoid font subset collisions.
So the short answer is that unless you are comparing two files that are physical duplicates of each other you can't perform a direct comparison on their binary contents. The long answer is that you can tweak some of the PDF entries to remove unique parts for comparison only but you'd probably be doing more work than it would take to just re-store the file in the database.
I trained my model in Nvidia Digits 5 and I would now like to extract the accuracy and loss plots that were generated during training for a report. Is this data saved somewhere so that it would possible to extract the data for these plots so that I could plot it in Python and perhaps ultimately modify the plots to compare different models etc?
The best solution I have found is to either look at the HTML file or to scan the text file caffe_output.log that is produced by Caffe. The text file is usually stored in /var/digits/jobs/insert_your_job_id/ but you can also just run on linux systems:
locate caffe_output.log
Go to your DIGITS job folder and locate your job's subfolder. Inside you'll find a file status.pickle, which is a pickled object containing all your job's information.
You can load it in python like so:
import digits
import pickle
data = pickle.load(open('status.pickle','rb'))
This object is somewhat generic and may contain multiple tasks. For a typical classification task it will likely be just one, but you will still need to access it via data.tasks[0]. From there you can grab the plots:
data.tasks[0].combined_graph_data()
which returns a somewhat convoluted dict (unfortunately - since your network can produce many accuracy/loss outputs, as well as even custom ones). It contains everything you need though - I managed to plot accuracy with:
plt.plot( data.tasks[0].combined_graph_data()['columns'][2][1:] )
but it's likely that you'll have to write a bit of custom code. As always, dir() is your friend.
I have a directory with a processing script and some .mp4 files, how do i choose a random one to display?
Break your problem down into smaller steps.
Can you write a program that simply lists all of the files in a directory? The File class might help, and the Java API is your best friend.
Can you write a program that takes that list of files and creates an array or ArrayList that contains all of them?
Can you write a program that takes an array or ArrayList and chooses a random element from it? Use hard-coded String values for testing.
When you get all of these individual steps working, you can combine them into a single program that chooses a random file from a directory. If you get stuck on a specific step, you can post a MCVE of just that step, and we'll go from there.
I am creating a survey in Qualtrics with many photos, say 1000. I want to have each survey participant answer, say 6, questions per photo. Each participant will see 5 photos that are randomly assigned.
Before looking into things, I assumed that there would be a way to upload the 1000 photos, create one block in Qualtrics (with the 6 questions) and then simply randomize the photo that occurs and have this be repeated this 5 times.
But it seems like this is either not possible or not obvious. I called Qualtrics and they said that I would manually need to create 1000 blocks (each block would be exactly the same with the exception of the title and the photo). I would then need to go into the Survey Flow and use the Randomizer there and manually add all 1000 blocks and have it randomly present 5 of the elements.
I really hope that there is a better way. This will take a ton of time if I have to do it this way.
If not, is there any way to automate anything?
Creating new blocks and automatically populating the photos. I know python and could possibly write a script to generate blocks, BUT the photo names are changed from their original names into some complicated code that Qualtrics generates.
Loading the photos into Qualtrics all at once (it currently requires one to load photos one at a time).
It turns out that there is a much better faster way to do this than the 1000 blocks fix.
There is a bunch of stuff going on to accomplish it, but it is possible.
First, one needs to put the photos into Qualtrics through the Graphics Library. The best way to do this is to simply drag and drop the photos into the desired location. Luckily one does not have to do this one-by-one. Make sure that they are in the order you want.
Second, create a block with a "question" where you want the random photo to appear. This block should also have all 6 questions.
Third, create a column in a spreadsheet (in, eg. Excel) of the URLs corresponding to the photos. This should be in order. One way to do this is mentioned at the bottom.
Fourth, go to the Loop and Merge option for this block. Copy and paste the column of URLs to, say, Field 1. Luckily this option exists and one does not have to do this one-by-one either. A sidenote is that if one changes the numbers in the gray boxes to the left of the rows, this changes what appears in the results. But there is no apparent way to change these more than one-by-one at a time.
Then you should be all set.
Finally, a little bit about how to get the URLs of the photos. Once again, make sure the photos in the library are in the order you want. Then you can use web scraping to scrape the image names, which can then be put into the proper URL. I used Python's Selenium and BeautifulSoup to accomplish this. Here is what I did, using a mac. The code at least gives you the idea:
from bs4 import BeautifulSoup
import codecs
import os
from selenium import webdriver
import re
chromedriver = "File path to /chromedriver"
os.environ["webdriver.chrome.driver"] = chromedriver
driver = webdriver.Chrome(chromedriver)
*In the Chrome browser that has appeared, manually navigate to the photos library page, then:
abc = driver.find_elements_by_css_selector(".thumbframe")
file = codecs.open('outputURLs.txt', 'w', encoding = 'utf-8')
urls = {}
for i in range(0,len(abc)):
h = abc[i].get_attribute("innerHTML")
soup = BeautifulSoup(h)
t = soup.find_all("img", attrs={"p4":re.compile('.*')})
urls[i] = t[0]['p1']
file.write("<img src=*Qualtrics Path/Graphic.php?IM=" + urls[i] + "/> + '\n')
One can find the proper first part to stick in "Qualtrics Path" by, eg. going to the Qualtrics Survey Editor, inserting a photo using Rich HTML Editing (or something similar), inserting the photo, clicking on View Source, and then looking at the pattern file path to use. It may begin with something like https://qualtrics.com/...
Then copy the results into a spreadsheet program and you should be ready to copy and paste.
I am using Mathematica to enhance and thin images. I used it for single image, now i want to use it for multiple images. so I have to import 6 images, do the thining and store them in a list for example. Can any one show me how to do that??
The images will be used for biometrics identification system.
Since you want a list as a result you might think of using either Table or Map. Either of those can do n things, one after another, and put the result into your final list.
Since you didn't show the steps you used for processing a single list it is a little difficult to tell you exactly how to wrap Table or Map around this.
If you have a list of image file names then you could use Map to process those names one at a time. The processing could either be a compound function to Import the image and then enhance and thin and the output of that function would be a single processed image. Map would then do the repetition over all the names.
Table might work in a similar way, but you use each iteration to get the file name, do the processing and store the result in your desired list.