Batch file for copying every nth file from folder1 to folder2? - windows

I am trying to copy every 30th file from one folder to another and automate the process for other folders. I have already tried the batch script in this thread: windows batch file script to copy every tenth file from a folder to another folder and just get "The syntax in that command is incorrect" when I run the file (and yes, I've tried both versions).
My folders do have spaces in the names (not my choice and cannot be changed). The files are named image00000X.jpg and yes, there are over 100k of them (which is why I really want the script to work).
Ideally, I'd like a way to set the script up so that I could just change the input and output paths and not have to move the script between the different folders when running it but I'll settle for whatever I can get at this point because I have tried just about everything else (including robocopy, Xcopy, five Powershell scripts, and a few BASH scripts).
Thanks!

Here is a simple batch file:
:: copyNth.bat interval sourcePath destinationPath
#echo off
setlocal
set /a n=0
for %%F in ("%~f2.\*") do 2>nul set /a "1/(n=(n+1)%%%1)" || copy "%%F" %3
sampleUsage:
copyNth 30 "c:\someSourcePath" "d:\someDestinationPath"
The "%~f2. is syntax that allows you to safely append a file (or file mask) to any provided path.
The trick to getting every Nth value is to let SET /A intentionally raise a division by 0 error. I redirect the error message to nul and conditionally copy the file only when there was an error.

You can also use just a standard for loop. I added some params as well, so you can change the source, destination, and skip count on the fly:
param(
[string]$Source = $( throw "You Must Specify Source Directory" ),
[string]$Destination = $( throw "You Must Specify Destination Directory" ),
[int]$Skip = 30
)
$Files = Get-ChildItem -Path $Source -File
for( $idx = 0; $idx -lt $Files.count; $idx += $Skip ) {
$Files[$idx] | Move-Item -Destination $Destination
}
Source and Destination are required params, but Skip defaults to 30 if you don't specify a value. To use, name it something like move30th.ps1, and run it like:
.\move30th.ps1 -Source "C:\Path\To\Files" -Destination "C:\New\Path" -Skip 30

You could do a simple Do/While loop like:
$Files = Get-ChildItem C:\Path\To\Files
$i = 0
Do{
$files[$i]|Move-Item -Dest C:\New\Path
$i=$i+30
}While($i -le $files.count)

If you want to use Python you could do something like this.
import glob
import shutil
files = glob.glob("data/set1/*.png")
n = 30
for file in files[0::n]:
shutil.move(file, "data/set3")

Related

Windows Compare filenames and delete 1 version of the filename

I have several hundred folders where I will have multiple files called filename.ext but also another file called filename.ext.url
I need a way of checking if filename.pdf.url exists does filename.ext exist. If they both exist delete filename.ext.url
I can't just do a search and delete all *.url files as they will be needed if the normal file does not exist
I then need to repeat that in all subdirectories of a specific directory.
I don't mind its its a batch script, powershell script that does it or any other way really. I'm just stumped on how to do what I want.
Currently I'm doing it folder by folder, manually comparing file names, file size and file icon.
foreach ($file in ls -Recurse c:\files\*.url) {
if (ls -ErrorAction Ignore "$($file.PSParentPath)\$($file.basename)") {
remove-item $file.fullname -whatif
}
}
remove whatif when ready to delete.
the basename removes the extension, so if they are all .ext.url then it will check if that file exists. It also removes the path, so we pull that as well.
an alternative way (that more matches what you're explaining) is something like
foreach ($file in ls -Recurse "c:\files\*.url") {
### replacing '.url$' means .url at the end of the line in Regex
if (ls -ErrorAction Ignore ($file.FullName -replace '\.url$')) {
remove-item $file.fullname -whatif
}
}
for /r "startingdirectoryname" %b in (*.url) do if exist "%~dpnb" ECHO del "%b"
This is expected to be executed directly from the prompt. If the requirement is as a batch line, each % needs to be doubled (ie. %%).
This also assumes that the whatever.ext file is to be in the same directory as the whatever.ext.url file.
Note that the filenames that are to be deleted will merely be echoed to the console. To actually delete the files, remove the echo keyword.
Test against a test directory first!
[untested]
To check for "filename.ext", check for "filename.ext.", they are the same file.
In CMD.EXE, you can do "IF EXIST "filename.ext." CALL :DoIt

Automatic copying and renaming of a file as soon as it is in the directory

I'm relatively new to programming so I need some help.
I have 2 directories
C:\test1
C:\test2
So in test1 will get constantly get files.
Which look like this:
testA000_00001.txt0..txt
test00A0_00102.txt1..txt
test00A0_00102_00123.txt45..txt
...
testG000_00999.txt999..txt
testH000_00013.txt0..txt
Since its essential that the files in test1 stay the way that they are I'm gonna need them in test2.
And since test2 needs to be the current version it is needed to be done the moment the files are in test1.
But without the .txt0. - .txt999. part.
testA000_00001.txt
test00A0_00102.txt
test00A0_00102_00123.txt
...
testG000_00999.txt
testH000_00013.txt
Its also essential that these files are only copied once since they aren't gonna stay in test2 for long.
I tried it with xcopy and some other versions of copy but each time it copies the files back into test2 and after I move the files from test2 the files are copied into it again.
(sry cant comment yet)
I dont have exact code (working on it)
What i think would work is to:
Check a text file for things it alr coppied
schedual a task to run every minute or so
move to the folder you want
Use the dir command to take all files in the current directory
Copy them
Write these to a text file
Change The names
loop
When i get the code done i will edit.
Also moving them assoon as they enter the directory is very resource intensive but might be possible if the code is very efficient.
#make a folder
md "C:\test\test3"
#move to test1
cd "C:\test\test1"
# Presets
$txt = ".txt"
$files = dir
#clean up the files list
$files = $files -split "`r`n"
#for each file it found
foreach ($line in $files) {
#move the file from test1 to test 3
Copy-Item "C:\test\test1\$line" -destination "C:\Test\test3"
#take the first 14 chars from the file name
$line_name = $line.Substring(0, 14)
#add a .txt extension
$line_name = $line_name + $txt
#rename the files in test3
Rename-Item -path "C:\test\test3\$line" -NewName "$line_name"
#move the files to test 2 and if they alr exist replace(update) them
Move-Item -Path "C:\test\test3\$line_name" -Destination "C:\Test\test2" -Force
}
Remove-Item –path "C:\test\test3" –recurse -force
This Code is super close to working but i might not be able to finish today.
It only failes to move the file: test00A0_00102_00123.txt45..txt
due to the name not working. I will update the script when it works fully.

Adding part of source path to destination path - and more

I feel like this one should be easy, but it's giving me a bit of trouble. The goal is to get downloaded wsus updates migrated to a separate drive that can be burned to a disk and moved to a secure environment.
The current process was set up by another guy and I am taking over. He has 2 batch files that run we will call them first.bat and second.bat
first.bat is run that spits out a log of how many new files there are. We'll call it new.txt this simply contains the hash file paths for the changes i.e. C:\folder\sub\1A\HASHFILE.cab
Then, we copy the file paths in new.txt by hand and manually paste them into second.bat ... add in an xcopy function and a destination folder in a new drive. i.e. xcopy C:\folder\sub\1A\HASHFILE.cab E:\eport\**1A** However, we have to manually add in the hash folder identifier (1A for example) I would like the script to pick it up from the source folder path and add it into the destination to eliminate the potential for human error.
What I am trying to write a shell script to accomplish is this.
run first.bat
export info from new.txt and modify to
add xcopy parameter to all new files in new.txt as well as the destination folder path
automate the addition of the hash folder callout (i.e. 1A to the end of the destination folder path. i.e. E:\export\**1A**)
run second.bat
Anyone have any ideas? I would like to wrap all of this up (or a similar function if that's easier, i imagine it might be) into a handy script that will automate this tedious process.
I have tinkered with a few ideas, I get it to spit out the destination path, but it doesn't actually move anything. Here is what I have so far that successfully tags the hash marker at the end of the destination folder, but does nothing else:
$source = "C:\temp\test folder\"
$dest = "C:\Temp2\export\"
$folders = Get-ChildItem $source | Where-Object {$_.PSIsContainer -eq $true} | Sort-Object
foreach ($folder in $folders){
$fname = $folder.FullName
$sourcefolder = $fname.Replace($source,"")
$newdest = $dest + $sourcefolder
Write-Output $newdest
}

Issue with Robocopy on Windows Server 2019

I've had a robocopy script running on Windows Server 2008 (Powershell version 4) with no issues for about 6 months now.
We've recently had to migrate off of this machine to initiate the same script off of a Windows Server 2019 (Powershell 5) machine. The exact same script no longer works, and i'm not quite sure what the issue is. The script is:
# Run BCP Bat file to get list of folders that have changed in the past 12 days and save them to
\\xxxx\xxxx\xxx\xxx\xxx.bat
# Select the starting directory to pull the CSV from. The assumption is that the DB file can #be sent to a dedicated directory that just has only the CSV files
$Filerecentdir = "\\ZZZ\Z\ZZZ\ZZZZZ\ZZZZ\ZZZZZZ"
#
# Filtering for only CSV files
$Filter = "*.csv"
#
# This PS command will search for the file with the latest timestamp on it. Coupling this with #the filter above, we turn $Filerecent into a variable consisting of the most recent CSV
$Filerecent = Get-ChildItem -Path $Filerecentdir -Filter $Filter | Sort-Object LastAccessTime -Descending | Select-Object -First 1
#
# I concatenate $Filerecentdir with the $Filerecent variable to form the full path to the CSV
$FullPath = Join-Path -path $Filerecentdir -ChildPath $Filerecent
#
# $roboSource variable uses import-csv cmdlet to parameterize the one (headerless) column that #the DB file creates
$roboSource = Import-Csv -Header #("a") -Path $FullPath
#
# Arbitrary directory that i'm using to store the logs. We'll change this to something on the #file server so it can be viewable
$logPath = "\\AAAA\A\AAAA\AAAA\AAAA\AAAA\"
#creates a folder to seperate weekly logs based off the output of the bat script
$weeklylogfolder = $Filerecent -replace '_.csv'
New-Item -ItemType Directory -Force -Path "$($logPath)$($weeklylogfolder)"
#
# For each loop to iterate over every single row in the CSV
Foreach($script in $roboSource)
{
# I used the two below variables to replace the two last trailing '\' in each entry
$prefix = $script.a -replace '\{.+'
$suffix = $script.a.Substring($prefix.Length) -replace '\\', '_' #keeping the {} in the file #name.
#$suffix = $script.a.Substring($prefix.Length) -replace '[{}]' -replace '\\', '_'
#
#$logFileName = $prefix + $suffix
$logFileName = $suffix
#
# Same switches that we used
#$StandardSwitches = "/copy:DAT /s /dcopy:DAT /V /L" #no copy
$StandardSwitches = "/copy:DAT /s /dcopy:DAT /V" #Copy
#
# Creates the log file in the same format that we used, so one log file per entry
$log = "/log:`"$($logPath)$($weeklylogfolder)\$($logFileName).log`""
#
# Iterates through each row to create the source and destination
$FileSource = "$($script.a)"
$FileDestination = "$($script.a)"
#
# used this to surround the certain variables with double quotes, otherwise Robo fails
$RoboArgs = '"I:\{0}" "Z:\{1}" {2} {3}' -f
$FileSource, $FileDestination, $StandardSwitches, $log
#
Robocopy $RoboArgs
}
I can't seem to pinpoint what would be causing the issue's I'm seeing. I've tried to run each of the commands within the script alone, and when I do, I am noticing that the $RoboArgs variable seems to cut off which then generates an incomplete $FileDestination
$RoboArgs with the above generates the exact same output as it does on server 2008/Powershell 4. However, it seems that powershell 5 processes this differently. Is there something I'm doing wrong or need to add in order to get this to process correctly?
EDIT:
Here's an example of what $RoboArgs is defined as in Win2019/Powershell5:
PS C:\> echo $RoboArgs
"I:\Fake-directory\Fake-directory\D\{DAFD6721-E854-46F3-B5CF-7EE1861348A6}\Fake\FakeDate" "Z:\Fake-Directory\Fake-Directory\D\{DAFD6721-E854-46F3-B
5CF-7EE1861348A6}\Responses\01182020" /copy:DAT /s /dcopy:DAT /V /log:"I:\FakeDirectory\FakeDirectory\Fake\Logs\Folders_Changed_20200118_21.1
\{DAFD6721-E854-46F3-B5CF-7EE1861348A6}_Responses_01182020.log"
It basically cuts off where the first line ends, and generates the destination in the same manner:
PS C:\> Robocopy "I:\Fake-directory\Fake-directory\D\{DAFD6721-E854-46F3-B5CF-7EE1861348A6}\Fake\FakeDate" "Z:\Fake-Directory\Fake-Directory\D\{DAFD6721-E854-46F3-B
5CF-7EE1861348A6}\Responses\01182020" /copy:DAT /s /dcopy:DAT /V /log:"I:\FakeDirectory\FakeDirectory\Fake\Logs\Folders_Changed_20200118_21.1
\{DAFD6721-E854-46F3-B5CF-7EE1861348A6}_Responses_01182020.log"
2020/01/18 23:17:21 ERROR 123 (0x0000007B) Opening Log File I:\FakeDirectory\FakeDirectory\Fake\Logs\Folders_Changed_20200118_21.1
\{DAFD6721-E854-46F3-B5CF-7EE1861348A6}_Responses_01182020.log
The filename, directory name, or volume label syntax is incorrect.
-------------------------------------------------------------------------------
ROBOCOPY :: Robust File Copy for Windows
-------------------------------------------------------------------------------
Started : Saturday, January 18, 2020 11:17:21 PM
Source - I:\FakeDirectory\FakeDirectory\Fake\D\{DAFD6721-E854-46F3-B5CF-7EE1861348A6}\Responses\01182020\
Dest - Z:\FakeDirectory\FakeDirectory\Fake\D\{DAFD6721-E854-46F3-B
Edit 2:
Comparing this to Win2008/Powershell4, it actually does the same thing in regards to failing if there's a line break. I think the real problem is how the script is trying to define the source and destination. On 2008/Powershell4, running the script generates everything correctly, with the correct source, destination and log file. Running the exact same script on 2019/Powershell5 seems to generate a problem with how Robocopy is defining these paths, placing both Source, Destination, and Log path all into the source variable, even though that's not how it's defined:
PS C:\> echo $FileSource
FakeDirectory\FakeDirectory\D\{DAFD6721-E854-46F3-B5CF-7EE1861348A6}\Responses\01182020
PS C:\> echo $FileDestination
FakeDirectory\FakeDirectory\D\{DAFD6721-E854-46F3-B5CF-7EE1861348A6}\Responses\01182020
PS C:\> echo $log
/log:"I:\FakeDirectory\FakeDirectory\FakeDirectory\Logs\Folders_Changed_20200118_21.1\{DAFD6721-E854-46F3-B5CF-7EE1861348A6}_Responses_01182020.log"
PS C:\> echo $StandardSwitches
/copy:DAT /s /dcopy:DAT /V

split an argument and add include the items in url

I'm trying to solve a problem at work, where it's required to access project folders on a shared drive.
However the naming convention is a bit tricky. The URLs on the server start with the static FS\XXX\00 followed by the project number (6 digits long) which is split into pieces of two and between slashes. For example the project folder for project 123456 would look like FS\XXX\00\12\34\56.
What I'm trying to sort out is how to create a .bat file, put it in the environment path and call it with the Run command, so for example I would call the file ex.bat by entering the following sequence in the Run console:
ex 123456
Then the program should split the number, build up and open the following URL:
FS\XXX\00\12\34\56
Any ideas?
%1 is the first parameter. Save it to a variable (%p%) do be able to do substring substitution (see set /?) and build and output the desired string:
#echo off
set p=%1
echo FS\XXX\00\%p:~0,2%\%p:~2,2%\%p:~4,2%
pause
Because powershell was tagged, here's a powershell implementation. Save this as a script in the environment path somewhere:
[CmdLetBinding()]
param
(
[string] $ProjectNumber = '123457'
)
$root = 'FS\XXX\00\'
# Create a string with backslashes every 2 chars
$out = (&{for ($i = 0;$i -lt $ProjectNumber.length;$i += 2)
{
$ProjectNumber.substring($i,2)
}}) -join '\'
# Create final path
$path = Join-Path -Path $root -ChildPath "$out\"
# Run explorer with the path
explorer $path
and call from run/cmd like this:
powershell "& "Script.ps1 -ProjectNumber 123456""

Resources