I have a windows for /F command that works perfectly well on files up to 2gb but simply does nothing and proceeds to the next command when they are over 2gb
I have searched for documentation regarding a limit but I can not find any - can anyone confirm that this is a real limit?
Assuming that it is I need to copy a file deleting the top three lines - any ideas (the for command is pretty slow anyway).
Yeah, I believe for /F has a 2GB limit. I think your best bet is to use sed from gnuwin32. Google for gnuwin32 sed and click the first result. Download the binaries zip file and extract the contents of the bin folder somewhere in your path.
The appropriate command is as follows:
sed "1,3d" oldfile > newfile
This should be much faster than a batch for loop anyway.
Related
I have a project folder in integrity client from which I want to get the entire folder structure. This folder structure should have all the folders and subfolders but no files inside. I don't even know this is possible or not.
I want this to run from command line. Any suggestion from UI perspective is also fine. Thank you.
I don't think this can't be achieved with only one command;
my suggestion is to createa a small script or batch, where you will use the SI VIEWPROJECT --RECURSE command and filter out the lines not ending with ".pj" (subprojects)
Not sure if I'm much too late, but I just stumbled across your question.
You could use the following command (simply type into Windows command line) to inventarize a downloaded sandbox:
$ TREE /F <path to Sandbox>
So in my case it'd be:
$ TREE /F D:\Sandbox\Name
You could even guide the output into a .txt file and postprocess it from there...
$ TREE /F D:\Sandbox\Name > D:\Output.txt
Edit: You would still have to take out all the filenames afterwards in your favourite programming language of choice (just check for the dash characters in front of folders and delete everything else...)
You can use the si viewproject command with filter by name activated to get only subprojects.
si viewproject --hostname={SI_HOST} --port={SI_PORT} --project={Project_Path} --filter=attribute:name=project.pj -R
Depends on your server configuration, the project.pj can be different.
I am trying to get a directory listing of only files with a given extension. At first blush this seems to be a simple thing to do, however check out this simple example:
C:\CODE\metcal>dir /b *.exe
metcal.exe
metcal.exe1
Notice that this returns metcal.**exe** and metcal.**exe1** as matches.
With python files a similar thing happens:
C:\CODE\metcal>dir /b *.py
metcal.py
metcal.pyc
Notice again Windows has determined that *.py takes anything that starts with *.py so it captures the .pyc files as well.
Is there a way to get only the extensions that match exactly? In the above python files example I would like the following to occur (obviously with the correct syntax substituted for *.py)
C:\CODE\metcal>dir /b *.py
metcal.py
As a note the matching under Windows not as simple as it seems.
*.exe matches foo.exe, foo.exe1, foo.exeabcde but not foo.exe.bak
There are other questions on SO that are similar that are related to long/short file names. The *.py and *.pyc example here should not introduce name mangling machinery.
**I have experimented on XP and Win7 machines and this behavior is not consistent at the cmd Prompt and file open dialogs. This inconsistant behavior makes me suspect this problem is related to settings of somekind. **
It's because windows wildcards on extensions check both long and short names as explained in this answer:
https://superuser.com/questions/238900/winxp-dir-command-3-and-4-char-extensions-are-the-same#238930
Solution there is to disable 8.3 names creation and then striping them on ntfs volumes which will also improve performance.
Microsoft Docs: Fsutil 8dot3name
Remarks:
Permanently removing 8dot3 file names and not modifying registry keys that point to the 8dot3 file names may lead to unexpected application failures, including the inability to uninstall an application. It is recommended you first back up your directory or volume before you attempt to remove 8dot3 file names.
So if you want to get only those extensions (.py and .pyc), you should try like this way :
#echo off
dir /b *.py*
pause
You can use the Unix ls command from the Windows Subsystem for Linux to do this, assuming you have WSL installed. It's freely available from Microsoft. From your Windows command prompt, type wsl followed by the ls command to list the files you want.
So, for your example, wsl ls metcal.py returns only metcal.py. To get the same results as you're seeing with dir, use wsl ls metcal.py*.
I'm using command line 7zip to zip up the contents of a folder (in Windows) thus:
7za a myzip.zip * -tzip -r
I've discovered that running exactly the same command line twice will produce two different ZIP files - they've got the same size but if you run a binary compare (ie fc /b file1.zip file2.zip) they are different. To complicate matters it seems that if you make the two zips in rapid succession then they are the same. But if you do them on different days or separated by a few hours they are not.
I presume that there's a date/time stamp in the ZIP file somewhere but I can't find anything on the 7zip site to confirm that.
Assuming I'm right does anyone know how to suppress the date/time? Or is something else causing the binaries to be different?
7-zip has the switch -m with parameter tc which has value on by default if not specified on command line.
With -mtc=on all 3 dates of a file stored on an NTFS partition are stored in the archive:
the last modification time,
the creation time, and also
the last access time.
See in help of 7-zip the page with title -m (Set compression Method) switch.
The last access time of the files is most likely responsible for the differences between the archive files.
You have to append -mtc=off to avoid storage of the NTFS timestamps in archive file.
I have a very large CSV file, over 2.5GB, that, when importing into SQL Server 2005, gives an error message "Column delimiter not found" on a specific line (82,449).
The issue is with double quotes within the text for that column, in this instance, it's a note field that someone wrote "Transferred money to ""MIKE"", Thnks".
Because the file is so large, I can't open it up in Notepad++ and make the change, which brought me to find VIM.
I am very new to VIM and I reviewed the tutorial document which taught me how to change the file using 82,449 G to find the line, l over to the spot, x the double quotes.
When I save the file using :saveas c:\Test VIM\Test.csv, it seems to be a portion of the file. The original file is 2.6GB and the new saved one is 1.1GB. The original file has 9,389,222 rows and the new saved one has 3,751,878. I tried using the G command to get to the bottom of the file before saving, which increased the size quite a bit, but still didn't save the whole file; Before using G, the file was only 230 MB.
Any ideas as to why I'm not saving the entire file?
You really need to use a "stream editor", something similar to sed on Linux, that lets you pipe your text through it, without trying to keep the entire file in memory. In sed I'd do something like:
sed 's/""MIKE""/"MIKE"/' < source_file_to_read > cleaned_file_to_write
There is a sed for Windows.
As a second choice, you could use a programming language like Perl, Python or Ruby, to process the text line by line from a file, writing as it searches for the doubled-quotes, then changing the line in question, and continuing to write until the file has been completely processed.
VIM might be able to load the file, if your machine has enough free RAM, but it'll be a slow process. If it does, you can search from direct mode using:
:/""MIKE""/
and manually remove a doubled-quote, or have VIM make the change automatically using:
:%s/""MIKE""/"MIKE"/g
In either case, write, then close, the file using:
:wq
In VIM, direct mode is the normal state of the editor, and you can get to it using your ESC key.
You can also split the file into smaller more manageable chunks, and then combine it back. Here's a script in bash that can split the file into equal parts:
#!/bin/bash
fspec=the_big_file.csv
num_files=10 # how many mini-files you want
total_lines=$(cat ${fspec} | wc -l)
((lines_per_file = (total_lines+num_files-1) / num_files))
split --lines=${lines_per_file} ${fspec} part.
echo "Total Lines = ${total_lines}"
echo "Lines per file = ${lines_per_file}"
wc -l part.*
I just tested it on a 1GB file with 61151570 lines, and each resulting file was almost 100 MB
Edit:
I just realized you are on Windows, so the above may not apply. You can use a utility like simple text splitter a Windows program which does the same thing.
When you're able to open the file without errors like E342: Out of memory!, you should be able to save the complete file, too. There should at least be an error on :w, a partial save without error is a severe loss of data, and should be reported as a bug, either on the vim_dev mailing list or at http://code.google.com/p/vim/issues/list
Which exact version of Vim are you using? Using GVIM 7.3.600 (32-bit) on Windows 7/x64, I wasn't able to open a 1.9 GB file without out of memory. I was able to successfully open, edit, and save (fully!) a 3.9 GB file with the 64-bit version 7.3.000 from here. If you're not using that native 64-bit version yet, give it a try.
I have a couple of flat files(.txt) in a directory.all those files are in the format *.txt.txt so i want to rename it to *.txt ?Is there any simple way to rename all together?
when I tried ren *.txt.txt *.txt is is not working
Any experts please suggest?It is amazing I have not got any answer yet
Please be noted that I need an out of the format filename.txt.
This should work
ren *.txt.txt *.
The reason your command didn't work is because, to windows, the file file.txt.txt is called file.txt with a .txt extension.
Only the last extension is the real extension, the first then becomes part of the filename, hence why your command changes it to what it already is.
If you did ren *.txt.txt *.pdf you would get file.txt.pdf.
My command will just remove the last one, thereby leaving the first, which then becomes the only and real extension.