I'm trying to use dft_r2c and dft in the same code block. The following code quits with fatal error: Attempted to read or write protected memory. If the 2nd Measure was changed to Estimate, it worked.
Is it possible to import 2 wisdom files for 2 fftw plans in the same code block?
fftw.import_wisdom_from_file("wisdom1");
fftplan1 = fftw_dft_r2c_1d(......fftw.Measure);
fftw.execute(fftplan1);
fftw.export_wisdom_to_file("wisdom1");
fftw.import_wisdom_from_file("wisdom2");
fftplan2 = fftw_dft_1d(......fftw.Measure);
fftw.execute(fftplan2);
fftw.export_wisdom_to_file("wisdom2");
Related
I am studying Hugging Face course and trying to compare the performance of different parallelization options.
I am using the code below. Actually, it is the sample code on the site here.
slow_tokenizer = AutoTokenizer.from_pretrained("bert-base-cased", use_fast=False)
def slow_tokenize_function(examples):
return slow_tokenizer(examples["review"], truncation=True)
tokenized_dataset = drug_dataset.map(slow_tokenize_function, batched=True, num_proc=8)
But I got NameError: name 'slow_tokenizer' is not defined error.
The error raises when num_proc parameter is greater than 1.
I am using jupyter notebook in VS Code.
How can I handle this error?
Thanks in advance.
I have executed the same code in the terminal and I got the error below.
An attempt has been made to start a new process before the
current process has finished its bootstrapping phase.
This probably means that you are not using fork to start your
child processes and you have forgotten to use the proper idiom
in the main module:
if __name__ == '__main__':
freeze_support()
...
The "freeze_support()" line can be omitted if the program
is not going to be frozen to produce an executable.
I am trying to use mutliprocessing Pool.map() to speed up my code. In the function where I have computation occurring for each process I reference an xarray.DataArray that was opened using xarray.open_rasterio(). However, I receive errors similar to this:
rasterio.errors.RasterioIOError: Read or write failed. /net/home_stu/cfite/data/CDL/2019/2019_30m_cdls.img, band 1: IReadBlock failed at X offset 190, Y offset 115: Unable to open external data file: /net/home_stu/cfite/data/CDL/2019/
I assume this is some issue related to the same file being referenced simultaneously while another worker is opening it too? I use DataArray.sel() to select small portions of the raster grid that I work with since the entire .img file is way to big to load all at once. I have tried opening the .img file in the main code and then just referencing to it in my function, and I've tried opening/closing it in the function that is being passed to Pool.map() - and receive errors like this regardless. Is my file corrupted, or will I just not be able to work with this file using multiprocessing Pool? I am very new to working with multiprocessing, so any advice is appreciated. Here is an example of my code:
import pandas as pd
import xarray as xr
import numpy as np
from multiprocessing import Pool
def select_grid(x,y):
ds = xr.open_rasterio('myrasterfile.img') #opening large file with xarray
grid = ds.sel(x=slice(x,x+50), y=slice(y,y+50))
ds.close()
return grid
def myfunction(row):
x = row.x
y = row.y
mygrid = select_grid(x,y)
my_calculation = mygrid.sum() #example calculation, but really I am doing multiple calculations
my_calculation.to_csv('filename.csv')
with Pool(30) as p:
p.map(myfunction, list_of_df_rows)
I'm currently developing a QGIS plug-in.
When i start editing a layer either with with edit(QgsVectorLayer) or with QgsVectorLayer.startediting() this RuneTimeError happens the majority of runs: RuntimeError: wrapped C/C++ object of type QgsVectorLayer has been deleted. I can run 10 times the script and have no error and then run it another 10 times and get 10 times in a row the error. It feels completely random.
As i understood by reading post such as Understanding the "underlying C/C++ object has been deleted" error it might be a garbage collector problem C++ side. But none of the post i saw was about QgsVectorLayer so i'm not really sure it applies.
It really annoys me to the point where i start creating empty layers to store modified features instead of editing.
I tried to move start editing before the loop as i was thinking to continually start editing and commit changes for each feature might cause the issue but the error still appears.
Then i thought it might be the use of break at the end but removing it doesn't resolve the error.
As it is the first time i really use PyQGIS i spent sometimes reading the developer cookbook or searching online (Anita Graser - creating and editing a new vector layer) but i could not find any solutions.
I tried with different version, LTR or not. With another computer by despair but the issue is still here.
I also read somewhere that the progress bar was the issue, so i removed the feedback in my script also without success.
Here are some code example :
nodesLayer = self.parameterAsVectorLayer(parameters, self.INPUT_NODE, context)
arcsLayer = self.parameterAsVectorLayer(parameters, self.INPUT_LINE, context)
# Fill node Id_line_x
# Create spatial index
index = QgsSpatialIndex(nodesLayer.getFeatures())
for line in arcsLayer.getFeatures():
# Construct a geometry engine to speed up spatial relationship
engine = QgsGeometry.createGeometryEngine(line.geometry().constGet())
engine.prepareGeometry()
# Get potential neighbour
candidateIds = index.intersects(line.geometry().boundingBox())
request = QgsFeatureRequest().setFilterFids(candidateIds)
for node in nodesLayer.getFeatures(request):
# Get real neighbour
if engine.intersects(node.geometry().constGet()):
# Fill the Id_line fields for the number of neighbour
for fld in range(1, node["Nb_seg"] + 1):
if node["fk_Id_line_%d" %fld] == NULL:
with edit(nodesLayer):
node["fk_Id_line_%d" %fld] = line["Id_line"]
nodesLayer.updateFeature(node)
break
And the exact error :
Traceback (most recent call last):
File "/some/path/to/a/file.py", line 331, in processAlgorithm
nodesLayer.updateFeature(node)
RuntimeError: wrapped C/C++ object of type QgsVectorLayer has been deleted
Hope the example is enough. The goal of the code is for the nodes to be aware of their surroundings without going through the lines. it's just for treatment and those fields would be removed in the final output.
I want to write data to zedboard's sdcard. I am able to write data to DRAM. Now I want to read DRAM's data and write it Sdcard. I have followed this (http://elm-chan.org/fsw/ff/00index_e.html) but it does not fulfill my requirement. I am not able to find any tutorial any example etc for this.
Please any tutorial link or any example. Thanks.
If you're using Vivado SDK, which I assume you are, it is really straightforward to use the SD Card.
To include the Fat File System, inside Xilinx SDK, open your Board Support Package (system.mss file) an select Modify this BSP's Settings. Under Overview, you can select xilffs.
Next, you must write the software to access the SD Card. This library offers a wide variety of functions. You can either look at here_1, in here_2 or in here_3. In this second reference is provided a wide variety of complex functions.
Aside from this, in order to use the SD Card, what you should basically do is written below. Note that this is just an overview, and you should refer to the references I gave you.
# Flush the cache
Xil_DCacheFlush();
Xil_DCacheDisable();
# Initialize the SD Card
f_mount(0, 0);
f_mount(0, <FATFS *>)
# Open a file using f_open
# Read and Write Operations
# Either use f_write or f_read
# Close your file with f_close
# Unmount the SD with f_mount(0,0)
Note that experience teaches me that you need to write to the file in blocks that are multiples of the block size of the file system, which for the FAT files syste, is typically 512 bytes. Writing less that 512 bytes and closing the file will make it zero bytes in length.
In new version of Xilffs (Fatfs) lib syntax is little changed.
New syntax is:
static FATFS FS_instance; // File System instance
const char *path = "0:/"; // string pointer to the logical drive number
static FIL file1; // File instance
FRESULT result; // FRESULT variable
static char fileName[24] = "FIL"; // name of the log
result = f_mount(&FS_instance, path, 0); //f_mount
result = f_open(&file1, (char *)fileName, FA_OPEN_ALWAYS | FA_WRITE); //f_open
May be this can help you.
Please have a look at the following OpenCV code
Mat *curent;
current = new Mat();
cv::Rect bRect = cv::boundingRect(Mat(*points).reshape(2));
Mat roi = *current(bRect);
Here, I am trying to get a ROI to the Mat called roi. But whenever I try to get execute the last line of the above code I get the error term does not evaluate to a function taking 1 arguments. I have followed the same technique of getting an ROI without pointers number of times before in C++ and they worked. I guess the issue is with pointer current ? current must be a pointer because local variable slowed the application in an unbelievable way.
So, how can I solve this issue and get the ROI ?
please, throw out those pointers!
you're going to wreck havoc on the internal Mat refcounts, produce undefined behaviour and memleaks
"local variable slowed the application in an unbelievable way."
really, you think, copying a 58 byte struct is the reason ? i just don't believe you.
well i'll give you a hint, anyway - the ( ) operator has a higher precedence than the * operator.