Every week at work, I am responsible for manually moving 100-200 files from one folder into a corresponding subfolder. After doing this for a couple of weeks, I thought to myself: This can be done faster!
I have used Python 2.7 and 3.X a bit at school, but mostly with (very) basic search engines and text search.
I found another thread, where a guy was told to use either os.rename or shutil.move. I made a simple test with os.rename:
os.rename("path/to/current/file.foo", "path/to/new/desination/for/file.foo")
And it works, so far so good.
Is there any way to make python run through every file from a folder and move it into a corresponding subdirectory in another folder? The original directory contains all the files, while the target directory contains all the folders.
Every file (A_file, B_file, etc.) has the same name as the folder(A_folder, B_folder, etc), which means they are in the correct order.
This makes me think a simple iteration could work, as in(More of an algorithm than code):
for file in original_dir
move file to folder_x in tar_dir
x += 1
Obviously this is not complete, but maybe someone can point me in the right direction.
This makes directories recursively.
os.makedirs(path)
So you pass the path to the directory you want. eg /path/to/
Which you would follow up with the copy.
def move_file(new_path_to_file, file_to_move):
file_name = file_to_move.split(os.path.sep)[-1]
os.makedirs(new_path_to_file)
os.rename(file_to_move, os.path.join(new_path_to_file, file_name))
You could also make it easier by passing in the filename as well.
Related
I have a batch file that I use (and have used all day successfully) to copy some badly named files (all are 1.csv) in a horrendously long dir tree to a different location with a more appropriate name (and shorter route to the file). It suddenly quit working; specifically, it IS making the new directory and it IS making the file with the correct name; however, the 'new' file is basically void. The source file may be 26mb but the destination file created is 287k. I'm using
xcopy "Z:\jobs\!q-z\clientname\tars\2016_06_06\home\abcd\xyz\qwert\more\moarr\deeper\deepest\x\y\z\1.csv" "Z:\jobs\!q-z\clientname\daily\160606\biz.csv*"
As I said earlier, this was working just fine all day long and suddenly began creating all dest files exactly the same size, which is an empty shell of a csv file. I have a feeling this might have to do with some sort of "cache" issue, but if so, I don't know how to clear it. Or is there some error in my syntax that allowed it to work almost accidentally before today? I've tried using pretty much every switch for xcopy, with no better results.
Also, the only thing that ever changes in the batch file is the date.
So, is there a possible way to move Test.txt to C:\ProgramData\CsD2\Tools\("Unknown Folder Name")\data\per Using command prompt?
using foxidrives solution for your previous question for detecting the correct directory, then just
move test.txt "%folder%\"
Short answer: yes. Not quite sure what the situation is that has left only the middle part of your path unknown, and the need to use the comnand line, but I have encountered similar cases on Linux and expect the algoirthm can be adapted to Windows commands. It's possible to do this by hand rather than writing a shell script, but it's up to you and your skills.
Permissions matter. Make sure you elevate yours enough to read and write in Tools before continuing.
First, change directory to C:\ProgramData\CsD2\Tools\
Presumably there are many items here. Some may be "hidden," so list the contents of this directory and be sure to include an option to show hidden files and folders. If you can, restrict the search to directories only.
It's tempting to display contents recursively in the above step. It's up to you, but I find it makes the output cluttered without a script to do the rest of the work.
Now it's time to search for the subfolder set that theoretically only exists in your target folder. Suppose Tools contains the directories fldr1, fldr2, and fldr3. Use your command to list a directory's contents with the path "fldr1\data\per", then use "fldr2\data\per", and so on until it doesn't return an error. Per may be empty, but that should look different from the path not found error.
Now you've found the name of your mystery folder. Write it down for future reference.
At thus point, you know the path to Test.txt, and the full path to the destination directory. Do a move command to relocate Test.txt, and you're done. I like to relist the contents of the target directory after to be comfortable that it arrived.
On TextMate 2 and opening two files in two different locations such as /path/1/file.txt and /path/2/file.txt, I am no longer seeing a way to perform diffs as before since one cannot select files in the project "drawer." We now have a file browser that seems to have taken its place and thus no way to pick the two opposing files. This also precludes any other command that requires multi file selection that are not within the file structure.
Am I missing something that would allow this to work properly when dealing with files in two different paths?
This isn't a new trick. It's one we learned when grep in project would go insane when you had a project with files whose common ancestor was root or some directory far above the files. Instead of opening your files like:
mate /foo/bar/baz /quix/quacks/quux
You do the following, assuming you're in an empty directory or don't care that its files will be included in the project as well
ln /foo/bar/baz /quix/quacks/quux . && mate .
That can obviously be wrapped up into a function to reduce the syntactical difference. In fact, at one point, I actually wrote a wrapper script around mate to do that transparently when needed AND clean up the hard linked files after I closed the project or quit TextMate. That went away with some bad hard drive though.
Anyhow I HTH
I'm using the split linux command to split huge xml files into node-sized ones. The problem is now I have directory with hundreds of thousands of files.
I want a way to get a file from the directory (to pass to another process for import into our database) without needing to list everything in it. Is this how Dir.foreach already works? Any other ideas?
You can use Dir.glob to find the files you need. More details here, but basically, you pass it a pattern like Dir.glob 'dir/*.rb' and get back filenames matching that pattern. I assume it's done in a reasonably good way, but it will depend on your platform and implementation.
As to Dir.foreach, this should be efficient too - the concern would be if it has to process the entire directory for every pass around the loop. But that would be awful implementation, and is not the case.
I'm writing some code that at run time may create or delete directories within the project path. I haven't really used ruby for file processing so i'm really uneasy about having code that, with a few mistypes weeks down the line, could result in wiping other directories outside of my project path.
Is there anyway to make it impossible for the program to delete files outside of its own path regardless of whats typed in destructive calls?
Pathname is a wrapper class for almost any file operations.
require "pathname"
path= Pathname.new("/home/johannes")
path.directory? # => true
path.children # => [#<Pathname:.bash_history>, #<Pathname:Documents>, #<Pathname:Desktop>]
path.children.each do |p|
p.delete if p.file?
end
Pathname#children does not contain . or .. so you don't accidently walk up the tree instead of down. If you still don't trust in the code, you can even check if on path is contained in another
Pathname.new("test") <=> Pathname.new("test/123") # => -1
You might want to create a wrapper method around your favourite delete method (or, perhaps, around whole class, because not only deleting files is potentially destructive file operation), which would expand all the submitted paths and check whether they begin with your "sandbox" path). You can also try to redefine delete method, if you are willing to cripple it through whole application.
And maybe the cleanest solution of them all would be to create a new user on your system and run your program as him.
On a POSIX system, you can use Dir.chroot to change the root that your application sees. Then ALL actions, not just delete ones, will be limited to the project directory. This does mean that external commands will be unavailable unless you make them part of your project directory as well.
This is the standard 'sandboxing' method used in Unix based systems. It can be difficult to setup (eliminating all external dependancies is sometimes hard), but affords significant protection when configured properly.
You could generate an Array of filenames in your project directory using
my_files = Dir["/bla/bla/your/directory/**/*"]
and then simply check if the filename passed to your "delete" function exist in your my_files array.
I'm sure there is a more elegant solution, but this could work ^_^
You could use File.expand_path and File.dirname on the input, and check that against __FILE__. So something like this might work:
File.delete(path) if File.dirname(File.expand_path(path)).include? File.dirname(File.expand_path(__FILE__))
I've got automated tests that routinely create and wipe out directories. I've taken two approaches:
Use /tmp as much as possible. The 'tmpdir' standard library module will create temporary directories which will be destroyed when your program exits. Or,
When the code creates a directory that it will later be deleting, it drops a marker file into the directory. When it comes time to delete the directory, if the marker file is not found, the code refuses to delete the directory. A marker file might be called ".ok_to_delete", for example.