I have a folder of 10000 images of 120 different breeds of dogs with each individual image having a unique id example: 000bec180eb18c7604dcecc8fe0dba07 and each such id has a corresponding label name in another CSV file. What should I do to pass these images in mini-batches to a CNN?
You can use tf.keras.preprocessing.image.DirectoryIterator. But before using it you'll need to preprocess your images so that each category will have its own directory.
Preprocess step will something be like follows
1. Get name for each file in the folder.
2. For each file, look up its category in the csv
add it to a maintained list of category files
3. Create sub-directory for each category
4. Move the files in their respective directory
5. Now use the `DirectoryIterator`
Related
I have to list and collect several specific files on a S3 bucket from different dates.
The path looks like the following:
/path/to/20221201/files/specificfile/
/path/to/20221202/files/specificfile/
The "files" and "specificfile" folders contain several different files, where I am only interest in a specific one from each.
I tried changing the date with * thinking it would list any date, but I get no result.
Any suggestions?
Thanks
I am trying to extract tables from pdf files. I am using UiPath's Document Understanding for this purpose. I have to create a template for this purpose and then use that template for other similar invoices. The issue I am facing is that the number of items in the table is varying. For example some pdf's have table which contain 4 items whereas other pdf files have table which contain only 1 item. So if I create a template using the pdf which has a table containing 4 items then it works. But then the same template when used for files which have table containing 1 item then it does not properly extract the data in the table. Is there any solution for this?
The solution should be able to extract tables from similar invoices containing varying number of items in the table. The format and layout of the invoice and the table is similar. The only thing varying is the number of items in each table.
Thanks for your time and help!
I have a folder which contains many files and I got a configuration table in sql database which contains the list of file names which I need to load to Azure Blob Storage.
I tried getting the file names from the source folder using 'Get Metadata' activity and then used Filter activity to filter the file name but this way I have to hard code the filename inside the filter.
Can someone please let me know a way to do this?
here is an example:
I have below files in a folder.
And the below in sql Config table
This is how the sample pipeline looks like.
1. Lookup list of files from sql config table and using foreach actvity append to an array variable. In my example it is in config_files.
2. Using GetMetadata, list the childItems in the folder, and append the file names into another variable. In my example it is files
3. Use SetVariable activity to store the result i.e. the files that match from the entries in config table.
Expression: #intersection(variables('files'),variables('config_files'))
I need to import multiple files from a folder and I need each file's contents to be new columns in the resultant table.
There are multiple examples all over the web of how to include multiple files from a folder as an append (e.g., PowerQuery multiple files and add column) but I need the contents of each file to be merged as new columns in the original table.
Any help will be greatly appreciated.
I came up with my own answer. Once you append the files you can pivot on the file name to turn them into columns.
I have the following info in a text file.
Item Rate
pencil 2
eraser 1
laser 3
pencil 1
torch 4
eraser 1
Specifically, I want to know if any item in the above list has a different price.
For eg: In the above one, you can see that pencil has 2 rates ie 2 and 1.
The price of the eraser is same in both entries, so no problem.
Further complexities - The text file is very huge.
Since dicts don't allow us to store duplicate keys, please suggest ways to solve this problem along with appropriate data structure.
You Can use Hash Table with Separate Chaining Method.Hope it will works
Does the file have to be plain text ? I recommend tackling this problem by using XML format and parsing it with SAX (not DOM !). SAX will not load the entire file in the memory, so it works well with huge file sizes.
As for the data structure, you could always define your own or you could just use something like this Map<KeyType, List<ValueType>>. I feel it's counter-intuitive to have different prices mapped for the same product name. You could create a unique ID for every type of product and have a new field: quantity.