Quite new here, hope this isn't to much of simple question.
I am trying to get into deep learning, starting with pytorch.
Thing is, all tutorials I see use downloading in order to build a data set for training and testing. Unfortunately, I am restricted with my internet connection, meaning I can't download directly from the web.
What I can do is download files and transfer them into my computer.
So my question is - in order to use a previously downloaded dataset in pytorch:
Where should I store it?
How do I create a dataset once I have the files on my computer?
If any other information seems important to you I'll be glad to hear, I'm a serious newbie..
Thank you very much!
Not sure which dataset you are asking for. Since you mentioned the "tutorials", I am guessing that you just want to work with a dataset that comes with some library in PyTorch ecosystem (e.g. torchvision).
Dataset classess in PyTorch ecosystem have a "root" argument in their constrcutors.
mnist = MNIST(root='/some/path', download=True)
You can simply download it on some machine with internet, and transfer the contents of the folder /some/path to your machine at /my/machine/path. And simply point to it and turn off the download
# on your machine without internet
mnist = MNIST(root='/my/machine/path', download=False)
This should work.
Related
I have a web application where user uploads the images of their locations. I want to write a program to detect the type of location and list of objects from the image. I write a program in C# using alturos YOLO to detect objects in the image. The result is fine for me but the problem is i want to detect the place type from the image. Like, if you upload some image that has snow then it should detect the "Snow" keyword. If you upload the "Lake" image then it should show keywords like "Lake, water, river etc". I am a web developer and never done any Machine Learning or image processing thing. But i am keen to learn this. Is there any way to do this or anyone can tell me the right path to do this.
I found this "https://www.clarifai.com/" but i want to write my own code because i have large number of images.
All in all, I'm pretty sure that there's no single correct answer to this. You could implement image recognition in a hundred different equally correct ways using different tools. So here's my opinionated perspective. Anyone and everyone is free to agree/disagree with what I'm saying.
I've worked a bit with Open CV (Python) in the past. There are a great number of libraries available based on it, so you can probably find a working base to build off of. I think that it should be capable of doing the task you specify, although I'm not quite sure how it would be done.
The other framework for machine learning and object recognition that I have seen is Apple's Create ML/ Core ML system (Swift or Objective-C). My experience with that one is as limited as cloning a git repo and poking around inside, but it looks pretty powerful.
someone asked a similar question a few years ago, but I need some additional detail.
I'm looking to build out a searchable image library for my company. We have thousands of images and I'm trying to find the best way to "tag them" so-to-speak. The images are stored on a network drive and accessed by both Mac and Windows users. I work on a Mac and have Adobe Bridge, but not everyone accessing these images will. I've seen that in Windows, you can add tags to files for searching, and in OS X, you can add Spotlight comments (delimited by commas, it can be used as search tags), however, the 2 don't carry over across platforms.
Is there anything out there that may help in accomplishing this goal?
I'm not sure how XML works. If I apply tags in Bridge or another software, will these tags be available to everyone, on whatever platform, if they are using a software that can read them? How does this work? Do they simply open the folder in one of those programs and have the ability to search and filter?
Thanks in advance!
I am afraid I don't have a panacea for all file formats (JPG/PNG/TIFF etc) across all OSes (macOS, Linux, Windows) across disparate application software (Photoshop/Lightroom/GIMP and MS-Paint or whatever Microsoft's offering is). However, if I put what I know, maybe someone else will know better and tell us so! Or maybe someone will know some better solutions for some pieces of the puzzle. So, in that vein...
If you have, know and love Adobe Bridge and you are "The Keeper" of the data, I would input, manage and control all metadata through that.
Then I believe you will need to find a way to do a nightly/weekly automated distribution of that data in a format intelligible to the "users" of the data. So, you will need to achieve the following:
Schedule an export from Bridge - my idea would be OSX's launchd
Export metadata from Adobe Bridge - my idea would be Adobe Extendscript
At this point I am thinking I would want to go with the easiest to parse, probably CSV (Comma Separated Values)
Now you have the info and will have to look at what your client users are using for application software and OS, and generate something intelligible to them from the CSV. I am thinking of tools like jhead, exiftool, ImageMagick (which can insert IPTC data).
So, there's a marker in the sand... if anyone knows better - please share your knowledge!
Other possibilities that spring to mind, are Lightroom which is reputedly excellent with metadata handling, or a dedicated picture-searching web-based application on your Intranet that runs on a small database updated nightly from exported Bridge metadata.
I guess the nature of the users' enquiries will be pretty important in deciding a way to go....
I have an engineer who is currently using the DOS based CAD program called Anvil. We are unable to update his PC. We are looking to convert these to AutoCAD files or is there anyone has successfully updated to Anvil Express? We have one other user using Anvil Express with limited success. We have to run old drivers and non-standard resolutions.
Thank-you
If you can run Anvil, export the files to IGES or STEP, then import these exchange files into Autocad, or probably more appropriately Autodesk Inventor (since Anvil is most likely being used for mechanical design).
The data won't likely come through perfectly, but there's probably nothing better to do.
How do I convert a pdf to an image (say a .png or .tiff file)?
Is there a way to do this without purchasing a third-party component?
Take a look at Imagemagic you can call these programs from roughly any programing language. And it is available on Unix, Windows, Mac, ...
I assume that you want to manually convert a PDF. Yo have two ways of doing it:
Online:
PRO: very very easy.
CONTRA: if your document is somewhat private your are giving it away.
I used last week Neevia and worked perfectly: http://convert.neevia.com/pdfconvert/
In your machine
PRO: More Secure.
CONTRA: You need to download a program and have it setup.
Here you can find 3 ways of converting it online and another three from your pc.
http://www.makeuseof.com/tag/6-ways-to-convert-a-pdf-file-to-a-jpg-image/
You can use any softwares available in market to convert a pdf to an image, like adobe, ClassicPDF or may be nitro. There are trials available you can download any one of them easily.
For example, try the trial of ClassicPDF and convert your file in seconds, as you asked, its a free one and it does not require purchasing any software, its completely free:
http://www.classicpdf.com/
EDIT: Removed direct download to .exe file and made link to main website. This is a serious no no and should be the users choice to download a file.
Two years ago, we shipped a multi-gigabyte Windows application, with lots of video files. Now we're looking to release a significant update, with approximately 1 gigabyte of new and changed data.
We're currently looking at DVD fulfillment houses (like these folks, for example), which claim to be able to ship DVDs to our customers for $5 and up. Does anyone have any experience with these companies?
We've also looked at a bunch of network-based "updater" software. Unfortunately, most of these tools are intended for much smaller programs. Are there any libraries or products which handle gigabyte-sized updates well?
Thank you for your advice!
BITS is a library from Microsoft for downloading files piece by piece using unused bandwidth. You can basically have your clients trickle-download the new video files. The problem, however, is that you'll have to update your program to utilize BITS first.
Depending on who the end user is you have a few options:
Shipping DVD's
This option tends to be rather expensive, and may not be the best way, what if you are shipping it to someone that no longer has the software installed.
HTTP hosting (using Akamai, or any other CDN)
This works rather well for other companies, for example Apple and I believe Microsoft as well.
Bittorrent
It is not just used for illegal content, it will allow you to offload some of the work load of sending the file, and at the same time it is a fast protocol, if you make sure the that the machine seeding has the correct file, the bittorrent protocol will make sure the end user gets the same file with the exact same hash.
You can use the rsync algorithm: http://samba.anu.edu.au/rsync/