I have a pointer to some floating point data that I'd like to copy in to an Eigen::ArrayXf. Is there a simple way of doing this other than looping through the memory and assigning values to the Eigen::ArrayXf?
Something along the lines of:
float* someData = new float[64];
...
Eigen::ArrayXf newArary( 64 );
newArray.data() = someData;
This should do the job:
Eigen::ArrayXf newArary(Eigen::ArrayXf::Map(someData, 64));
You can also just use the mapped data directly.
Check the map tutorial for more details and alternative syntax:
http://eigen.tuxfamily.org/dox/group__TutorialMapClass.html
If you are looking to copy from and std::vector, then see the assignment operator in the Eigen Do Better library.
Related
I am currently trying to make some TensorFlow Inference (C backend) using Boost::GIL (challenging). I need a few thinks, I have been able to load my png image (rgb8_image_t)
and did a conversion to rgb32_f_image_t.
I still need 3 thinks, the raw pointer of the data, memory allocated, and dimensions.
for the memory allocated unfortunately the function total_allocated_size_in_bytes() is private, so I did this:
boost::gil::view(dest).size() * boost::gil::view(dest).num_channels() * sizeof(value_type);
Which is valid, if I do not have any extra padding for alignment story. But does it exist any nice alternative?
For the dimension, I should match with numpy (from PILLOW), I hope both libraries are using the same memory layout pattern. From my understanding, by default, datas are interleaved and contiguous so, it should be good.
Last the raw pointer _memory, it is a private data member of the Image class with no dedicated function. boost::gil::view(dest).row_begin(0) returns a iterator on the first pixel but I not sure how I could get the pointer of the data _memory. Any suggestions ?
Thank you very much,
++t
ps: TensorFlow proposes a C++ backend, however, it is not installed from any package managers, and manipulate Bazel is beyond my strength.
GIL documentation pretty accurately documents the various memory layouts.
The point of the library, though, is to abstract away the memory layouts. If you require some representation (planar/interleaved, packed or unpacked) you are doing things "the hard way" for the library interface.
So, I think you can read and convert in one go, e.g. for a jpeg:
gil::rgb32f_image_t img;
gil::image_read_settings<gil::jpeg_tag> settings;
read_and_convert_image("input.jpg", img, settings);
Now getting the raw data is possible:
auto* raw_data = gil::interleaved_view_get_raw_data(view(img));
It happens to be the case that the preferred implementation storage is interleaved, which is likely what you're expecting. If your particular image storage is planar, the call will not compile (and you'd probably want planar_view_get_raw_data(vw, plane_index) instead).
Note that you'll have to reinterpret_cast to float [const]* if you need that, because there is not public interface to get a reference to the scoped_channel_value<>::value_, but the BaseChannelValue type is indeed float and you can assert that the wrapper doesn't add additional weight:
static_assert(sizeof(float) == sizeof(raw_data[0]));
Alternative Approach:
Conversely, you can setup your own raw pixel buffer, mount a mutable view into it and use that to read/convert your initial load into:
// get dimension
gil::image_read_settings<gil::jpeg_tag> settings;
auto info = gil::read_image_info("input.jpg", settings).get_info();
// setup raw pixel buffer & view
using pixel = gil::rgb32f_pixel_t;
auto data = std::make_unique<pixel[]>(info._width * info._height);
auto vw = gil::interleaved_view(info._width, info._height, data.get(),
info._width * sizeof(pixel));
// load into buffer
read_and_convert_view("input.jpg", vw, settings);
I've actually checked that it works correctly by writing out the resulting view:
//// just for test - doesn't work for 32f, so choose another pixel format
//gil::write_view("output.png", vw, gil::png_tag());
I have an RDD like:
(A,AA,1)
(A,BB,0)
(A,CC,0)
(B,AA,2)
(B,BB,1)
(B,CC,4)
and I want to convert it into the following RRD:
([1,0,0],[2,1,4])
the order is important for me since the main propose is using RowMatrix to convert the second RDD to a matrix.
Your need to be careful with the wording, when you ask for a Matrix, do you mean something like the spark.mllib.matrix ? If so, you will need to follow very specific instructions to create one. However, it seems to me that your problem can be solved in a much easier way. Just using zipWithIndex with groupBy
//Here is how I see it
val test = sc.parallelize(Array(("A","AA",1),("A","BB",0),("A","CC",0),("B","AA",2),("B","BB",1),("B","CC",4))).zipWithIndex
val grouptest = test.groupBy(_._1._1).map(x=>(Vectors.dense(x._2.map(y=>(y._2,y._1._3)).toArray.sortBy(_._1).map(z=>z._2.toDouble))))
In your example, you seem to want the result as a vector? So I used spark's Vector (which by the way, only allows Doubles).
Result looks like:
[1.0,0.0,0.0]
[2.0,1.0,4.0]
I am trying to get a set of binary images' eccentricity and solidity values using the regionprops function. I obtain the label matrix using the vision.ConnectedComponentLabeler function.
This is the code I have so far:
files = getFiles('images');
ecc = zeros(length(files)); %eccentricity values
sol = zeros(length(files)); %solidity values
ccl = vision.ConnectedComponentLabeler;
for i=1:length(files)
I = imread(files{i});
[L NUM] = step(ccl, I);
for j=1:NUM
L = changem(L==j, 1, j); %*
end
stats = regionprops(L, 'all');
ecc(i) = stats.Eccentricity;
sol(i) = stats.Solidity;
end
However, when I run this, I get an error says indicating the line marked with *:
Error using ConnectedComponentLabeler/step
Variable-size input signals are not supported when the OutputDataType property is set to 'Automatic'.'
I do not understand what MATLAB is talking about and I do not have any idea about how to get rid of it.
Edit
I have returned back to bwlabel function and have no problems now.
The error is a bit hard to understand, but I can explain what exactly it means. When you use the CVST Connected Components Labeller, it assumes that all of your images that you're going to use with the function are all the same size. That error happens because it looks like the images aren't... hence the notion about "Variable-size input signals".
The "Automatic" property means that the output data type of the images are automatic, meaning that you don't have to worry about whether the data type of the output is uint8, uint16, etc. If you want to remove this error, you need to manually set the output data type of the images produced by this labeller, or the OutputDataType property to be static. Hopefully, the images in the directory you're reading are all the same data type, so override this field to be a data type that this function accepts. The available types are uint8, uint16 and uint32. Therefore, assuming your images were uint8 for example, do this before you run your loop:
ccl = vision.ConnectedComponentLabeler;
ccl.OutputDataType = 'uint8';
Now run your code, and it should work. Bear in mind that the input needs to be logical for this to have any meaningful output.
Minor comment
Why are you using the CVST Connected Component Labeller when the Image Processing Toolbox bwlabel function works exactly the same way? As you are using regionprops, you have access to the Image Processing Toolbox, so this should be available to you. It's much simpler to use and requires no setup: http://www.mathworks.com/help/images/ref/bwlabel.html
As you could probably tell from the below code I am working on a project which creates csv reports from data in mongoDB. After getting the data I need in, I need to structure the data into something more sensible then how it exists in the db, which is fairly horrendous (not my doing) and near impossible to print the way I need it. The structure that makes the most sense to me is a slice (for each document of data) of maps of the name of the data to a structure holding the data for that name. Then I would simply have to loop through the document and stuff values into the structs where they belong.
My implementation of this is
type mongo_essential_data_t struct {
caution string
citation string
caution_note string
}
mongo_rows_struct := make([]map[string]mongo_essential_data_t, len(mongodata_rows))
//setting the values goes like this
mongo_rows_struct[i][data_name].caution_note = fmt.Sprint(k)
//"i" being the document, "k" being the data I want to store
This doesn't work however. When doing "go run" it returns ./answerstest.go:140: cannot assign to mongo_rows_struct[i][data_name].caution_note. I am new to Go and not sure why I am not allowed to do this. I'm sure this is an invalid way to reference that particular data location, if it is even possible to reference it in Go. What is another way to accomplish this setting line? If it is too much work to accomplish this the way I want, I am willing to use a different type of data structure and am open to suggestions.
This is a known issue of Golang, known as issue 3117. You can use a temporary variable to get around it:
var tmp = mongo_rows_struct[i][data_name]
tmp.caution_note = fmt.Sprint(k)
mongo_rows_struct[i][data_name] = tmp
as per my understanding, when you write:
mongo_rows_struct[i][data_name]
compiler will generate code, which will return copy of mongo_essential_data_t struct(since struct in go is value type, not reference type), and
mongo_rows_struct[i][data_name].caution_note = fmt.Sprint(k)
will write new value to that copy. And after that copy will be discarded. Obviously, its not what you expect. So Go compiler generate error to prevent this misunderstanding.
In order to solve this problem you can:
1. Change definition of your data type to
[]map[string]*mongo_essential_data_t
2. Explicitly create copy of your struct, make changes in that copy and write it back to the map
data := mongo_rows_struct[i][data_name]
data.caution_note = fmt.Sprint(k)
mongo_rows_struct[i][data_name] = data
Of course, first solution is preferable because you will avoid unnecessary copying of data
Using Xcode/Cocoa and the ExtAudioFile API, I'm trying to store away AudioBufferList* objects away for later use, but I'm running into trouble with re-accessing the data. These objects are coming from repeated ExtAudioFileRead calls.
I realize that I can't just stuff these AudioBufferList* objects into an NSArray, but I was under the impression that an NSPointerArray would work for this purpose. However, when trying to access audioBufferList->mBuffers[0].mData after storing the audio buffer lists in the NSPointerArray, they just come back zeroed out.
I was memcpying the audioBufferLists to new audioBufferList objects since I'm reusing the same audio buffer list for each ExtAudioFileRead call. I'm not sure if that's sufficient, though, and memcpying the void* audioBufferList->mBuffers[0].mData objects isn't helping either.
What's the easiest way to store these AudioBufferList* objects? Am I on the right track?
AudioBufferLists hold their data in audioBufferList.mBuffers[i].mData.
mData is void* and the actual type of the values is determined by the output format you specified.
Example:
If you defined
kAudioFormatFlagsCanonical,
kAudioFormatLinearPCM,
mBitsPerChannel = 32
and
mFramesPerPacket = 1
as your output format, the mData-array contains values of type AudioSampleType (which is Float32 a typedef)
If you have chosen another format the array might contain SInt16 values or something else.
So you must be aware of your output type when you want to copy the contents of mData around.
If you know the format you could simply create a c-array
dataCopy = calloc(dataSize, sizeof(Float32));
and memcpy audioBufferList.mBuffers[i].mData into that.
If you want to use a cocoa NSMutableArray, you would have to wrap the floats into a NSNumber object.