Default graph.visualize() function does not display task labels. Is there any way they can be added manually using graph = dask.delayed()(tasks)?
I need the labels to show basic business logic (which tables are joined with which etc.).
You can pass a filename to the visualize method with a .dot extension. Dask will then write out the result as a text file in the DOT language. You could then modify this file and add labels manually.
https://en.wikipedia.org/wiki/DOT_(graph_description_language)
Related
I am writing a script, that extracts different types of data from different kind of custom log files.
But before I continue to write, I want to determine in what output format / platform I want it to be, so it is displayed properly or it can be read properly.
examples:
sometimes it is certain lines of text with an important word in it
sometimes it is a block of text between a start and end phrase
sometimes it are data points, which i then want to visualize better in a line chart
....
OR it is a combination of those
At first i thought i write it so that it is in a markdown file format, so i can for instance create fold able blocks, so that i just unfold the part that i want to read.
But markdown is not versatile. Meaning I cant create line charts or other kinds of stuff (thinking about the future)
So know I put the different types of data in different type of output formats and visualize them in an HTML file.
meaning, the blocks of text in a markdown file, which I then import though a java-script markdown viewer
the data points, I create a line chart through a java-script chart
.....
HOWEVER, I am not sure that this is the best/correct way to go .....
What is your advice ?
I am using flow_from_directory in ImageDataGenerator from keras to train my convolution neural network. I downloaded the ILSVRC2012 images from this link and I am trying to train and validate my network. The training folder has images categorized in corresponding folders, but the validation images are not categorized into folders, which is not letting me use flow_from_directory to test the validation accuracy. Is there any easy way of categorizing the validation data into corresponding folders?
I tried with the below link of git code.
- https://raw.githubusercontent.com/soumith/imagenetloader.torch/master/valprep.sh
- https://github.com/pytorch/examples/tree/master/imagenet
This shell script worked for me to convert validation directory into sub directories where images categorized into corresponding folders.
Try to convert your validation directory into respective images categorized directories and give that parent directory of subdirectories as input to your code.
Thanks
You can use the libraries mxnet and gluon to classify the imagenet validation data in respective folders.
See https://gluon-cv.mxnet.io/build/examples_datasets/imagenet.html.
Then yuo can use the keras validation data generator.
I'm doing a finetuning in two classes, first you need create something like that in your code
path_directory = "../images/"
path_classes_name = ['class_0','class_2',...,'class_999'] full list in order
image.ImageDataGenerator(rescale=0).flow_from_directory(path_directory, target_size=(244, 244), batch_size= 128, class_mode='categorical', shuffle=True,classes=path_classes_name)
after this the path_directory need by organized exactly the blog.keras say. Keras will make the magic for you!
Tip: Don't forget then create the folders in order and with synset name (something like n04111531)! Not the name literal of classes =]
I am putting together a project where I need to be able to source outside data as a means of inputting skeleton joint positions into Maya. Basically I have a spreadsheet of sequential joint positions for the skeleton which I would like to load into Maya and then link to the skin. Does anyone know a way to upload or reference these positions (as FK into Maya)?
Probably the easiest thing to do is to convert your spreadsheet data to atom format
Atom is a json based format for exchanging animation data and, since its JSON based you should be able to concoct a CSV to ATOM translator using Python's built in csv and json modules.
I am using DMExpress tasks to do taransformations on my business data. These business data come in multiple format/layout. I need to be able to use single task for transformation on multiple source layouts. Any DMExpress experts here??
One way that I found out for doing transformation on multiple source layouts with the help of single task was by using the Dmexpress SDK to write the script for the task rather than building the tasks using the GUI task editor. SDK gives lot more flexibility compared to the GUI editor.
But if you are bound to GUI then there is a way around for this specific purpose. You should define a common name for the source layout. Only the source layout name is binded to the task but not the actual layout definition. So you can alter the layout definition while keeping the layout name constant to get a generic task.
FYI- DMExpress is now called DMX (Syncsort changed the name about a year ago).
Do you have multiple different record types within a single file or is each type of record in a separate file? Your question is not clear on this.
If they are in separate files, this is very easy, but you will need to create a separate DMX task for each file. In each of these tasks, define one of the files as the source and create a record layout that matches the format of that file.
If they are in the SAME file, it is only a little more difficult. You can split them into separate files by creating multiple targets and defining a named condition for each target using the SourceName() function (this function returns the name of the file that the current record came from). Then you can process them as separate files (see above). This works UNLESS you have a parent-child relationship going on between the different types of records in that single file. If that is the case, please post some sample data and I can advise on how to handle it.
i'm using xmgrace in the terminal, and want the data to be displayed directly as dots instead of lines. Achieving this in the GUI is simple, but I have to read in multiple files, and do not want to change it every time i start xmgrace. Can I add a command to the files that are read in? Or can I use an option in the terminal when I start xmgrace?
The correct way to set the appearance of a plot from the commandline is to use an existing parameter file, specified using the flag
-param settings.par
The parameter file can be stored beforehand, using the GUI to modify the appearance of an existing, similar plot. Modify the plot as you like, then save the appearance settings in a parameter file (convention is to use the extension .par) using Plot > Save Parameters.
A typical example command would then be
xmgrace -block data2.dat -bxy 1:4 -block data2.dat -bxy 1:6 -param settings.par
In my experience, calling the
-param
flag last thing in your command works best.
There really is no need to be manually text-editing your grace plot files (.agr) to achieve this.
xmgrace has a full and complex language for expressing the configuration of the look and feel for the graph. There are two ways to go about what you described. The simple way is to load the dataset into xmgrace, change everything to make it look the way you want, then save the dataset. You will see the dataset now has tons of lines describing the configuration "#g0 on" "# s0 linestyle 1" etc with your dataset at the end, terminated by a &.
To replicate that graph, spit out the saved header, insert your data, and the insert the trailing &. Feed the result into xmgrace and everything will be all set up. Once you get comfortable you can start doing dynamic substitutions to rename the graph or change the symbol or whatever. See /usr/share/grace/examples for examples of what grace can do (and the config files which generate that).
The more complex method is to load the dataset, save it immediately, change it to look the way you want, and then save it again under a different name. Run diff on the two files and you will get a set of changes. You might need at most a handful of other lines from the non-changing portion, but that is somewhat rare. This produces the minimal set of fixed headers you need to prepend to the dataset. It usually isn't worth the effort to reduce the prefix size.