I’m using Toad for Oracle to access 11G environment and my dataset has 1,000,000 records. While downloading I want to split the files in 100,000 each 10 files. I do not have the authority to create tables instead I can run and download only. Is their a way I can split the files while downloading?
You can split a text file using PowerShell or linux shell.
How to Split Large Text File into Smaller Files in Linux
example for linux
#split -l 100000 test.txt new
example for Windows
How can I split a text file using PowerShell?
split_log.ps1
param(
[string]$input_file = "",
[string]$count_line = ""
)
$lineCount = 0
$fileCount = 1
foreach ($line_file in get-content $input_file)
{
write $line_file | out-file -encoding ASCII -Append $input_file"_"$fileCount".out"
$lineCount++
if ($lineCount -eq $count_line)
{
$fileCount++
$lineCount = 0
}
}
PS C:\АСУ\Stackoverflow\split_log> ls
Каталог: C:\АСУ\Stackoverflow\split_log
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a--- 17.12.2018 6:03 10959355 1124.sql
-a--- 17.12.2018 7:27 357 split_log.ps1
PS C:\АСУ\Stackoverflow\split_log> .\split_log.ps1 .\1124.sql 10000
PS C:\АСУ\Stackoverflow\split_log> ls
Каталог: C:\АСУ\Stackoverflow\split_log
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a--- 17.12.2018 6:03 10959355 1124.sql
-a--- 17.12.2018 7:50 2461667 1124.sql_1.out
-a--- 17.12.2018 7:50 2461458 1124.sql_2.out
-a--- 17.12.2018 7:50 2461340 1124.sql_3.out
-a--- 17.12.2018 7:50 2461352 1124.sql_4.out
-a--- 17.12.2018 7:50 1113540 1124.sql_5.out
-a--- 17.12.2018 7:27 357 split_log.ps1
Related
So I don't really mind and it's not super important but I wrote a powershell script to set the time of a codec converted family film. The original files ext been named to MP44 and the resulting files ext are MP4. I just want to understand why the times are not exactly the same.
Directory of C:\Users\zzz\Desktop\family videos and images\hero 2021
2022-03-18 05:24 PM <DIR> .
2022-03-16 07:18 AM <DIR> ..
2020-09-02 07:15 PM 79,353,358 ffmpeg.exe
2020-09-02 07:15 PM 79,214,606 ffplay.exe
2020-09-02 07:15 PM 79,249,422 ffprobe.exe
2022-03-16 09:45 PM 482 go.bat
2021-12-24 09:14 PM 4,000,895,516 GX010066.MP44
2021-12-24 09:14 PM 741,345,212 GX010066.MP44.ffmpeg.mp4
2021-12-24 10:25 PM 4,003,210,355 GX010067.MP44
2021-12-24 10:25 PM 687,471,776 GX010067.MP44.ffmpeg.mp4
2021-12-24 09:15 PM 4,001,034,065 GX020066.MP44
2021-12-24 09:15 PM 719,404,024 GX020066.MP44.ffmpeg.mp4
2021-12-24 10:27 PM 3,629,297,689 GX020067.MP44
2021-12-24 10:27 PM 635,027,797 GX020067.MP44.ffmpeg.mp4
2021-12-24 09:17 PM 4,000,513,626 GX030066.MP44
2021-12-24 09:17 PM 690,608,291 GX030066.MP44.ffmpeg.mp4
2021-12-24 09:17 PM 714,946,960 GX040066.MP44
2021-12-24 09:17 PM 125,647,486 GX040066.MP44.ffmpeg.mp4
2022-03-18 05:36 PM 537 timechanger.ps1
17 File(s) 24,187,221,202 bytes
2 Dir(s)
Directory shows same times/dates.
I guess it's more of a windows question than a powershell question. Why would explorer show different modified dates?
Here is my powershell for posterity:
$files = Get-ChildItem *.mp44 | select Name, LastWritetime
foreach ($file in $files) {
$currentname = $file.name
#$currentname
$currentsubstring = $currentname.substring(0,8)
#$currentsubstring
$WriteTime = $file.LastWritetime
#$WriteTime
write-host "Going to get-item $currentsubstring.MP44.ffmpeg.mp4 and set $Writetime"
$targetfile = "$currentsubstring.MP44.ffmpeg.mp4"
$Targettime = $Writetime
$bacon = Get-Item $targetfile
$bacon.LastWritetime = $targettime
}
Thanks in advance for your time and assistance.
Because the 'Date' column in Windows Explorer that you have showing there is the Creation Date property, not the Last Modified Date.
Notice how the "Date" column matches the "Date created" column, not the "Date modified" column in my screenshot.
I have been a lurker at stackoverflow.com for many years (great site and users here), but never had the need to ask a question. Now the time has come :-) Let me begin:
OS: x64 Windows 8.0 to Windows 10 (15063.14) (the issue exists since years, but I have never pursued it fully yet, so we can exclude that it is specific to a specific Windows version)
FS: NTFS
Issue: 2 files with the same (long) name in the same directory and I cannot figure out how this is even possible. This happens to me since years whenever I manually upgrade my Email client. The main .EXE file of it (MailClient.exe) is never asking for replacement if copying the new one over to the same directory. Instead they are both placed there, with the exact same long name.
The issue has nothing to do with a specific directory, I can copy around both .EXE files to freshly created directories on the NTFS drive without issues (also getting no "overwrite" question there).
Let me show you:
C:\temp\2>dir
Volume in drive C is SSD 840 Pro
Volume Serial Number is 0C6D-D489
Directory of C:\temp\2
13.04.2017 02:29 <DIR> .
13.04.2017 02:29 <DIR> ..
21.10.2016 17:10 24.742.760 MailClient.exe
27.12.2016 03:26 24.911.872 MailCliеnt.exe
2 File(s) 49.654.632 bytes
2 Dir(s) 78.503.038.976 bytes free
However, if doing a dir /x, this comes up:
C:\temp\2>dir /x
Volume in drive C is SSD 840 Pro
Volume Serial Number is 0C6D-D489
Directory of C:\temp\2
13.04.2017 02:29 <DIR> .
13.04.2017 02:29 <DIR> ..
21.10.2016 17:10 24.742.760 MAILCL~2.EXE MailClient.exe
27.12.2016 03:26 24.911.872 MAILCL~1.EXE MailCliеnt.exe
2 File(s) 49.654.632 bytes
2 Dir(s) 78.503.038.976 bytes free
So they obviously have a different 8.3 name, OK, but the exact same long name. Here is another screenshot of the situation. Both files show the same location within the Windows "properties" dialog (right click) too. Unfortunately I am not allowed to post images just yet (it seems) - just tried. So you will have to take my word.
I cannot figure out how this is possible and this is bugging me ;) As soon as I rename both files for example to 1.exe, Windows starts telling me that there is already a file with that name in the same directory. So it obviously has something to do with the filename, but they are both exactly identical, no extra spaces, nothing, as you can see from the DIR command.
I´ve also tried to rename them and re-wrote the exact wording "MailCient.exe" manually for both, to make sure the characters are EXCACTLY the same, Windows still won´t complain, they both go there once again under the same name. However, renaming them to "Mail.exe" and "Mail.exe" will NOT work, then Windows is saying that another file with that name already exists. However, naming them both back to "MailClient.exe" is just absolutely fine, no complains by Windows with that.
Another fun fact about this, if I dir for mailclient.exe directly, this happens:
C:\temp\2>dir mailclient.exe
Volume in drive C is SSD 840 Pro
Volume Serial Number is 0C6D-D489
Directory of C:\temp\2
21.10.2016 17:10 24.742.760 MailClient.exe
1 File(s) 24.742.760 bytes
0 Dir(s) 78.501.998.592 bytes free
However, if looking for *.exe, this happens:
C:\temp\2>dir *.exe
Volume in drive C is SSD 840 Pro
Volume Serial Number is 0C6D-D489
Directory of C:\temp\2
21.10.2016 17:10 24.742.760 MailClient.exe
27.12.2016 03:26 24.911.872 MailCliеnt.exe
2 File(s) 49.654.632 bytes
0 Dir(s) 78.501.990.400 bytes free
This yields also interesting results:
C:\temp\2>ren mailclient.exe *.bak
C:\temp\2>dir
Volume in drive C is SSD 840 Pro
Volume Serial Number is 0C6D-D489
Directory of C:\temp\2
13.04.2017 02:50 <DIR> .
13.04.2017 02:50 <DIR> ..
21.10.2016 17:10 24.742.760 MailClient.bak
27.12.2016 03:26 24.911.872 MailCliеnt.exe
2 File(s) 49.654.632 bytes
2 Dir(s) 78.501.990.400 bytes free
And back:
C:\temp\2>ren mailclient.bak MailClient.exe
C:\temp\2>dir
Volume in drive C is SSD 840 Pro
Volume Serial Number is 0C6D-D489
Directory of C:\temp\2
13.04.2017 02:51 <DIR> .
13.04.2017 02:51 <DIR> ..
21.10.2016 17:10 24.742.760 MailClient.exe
27.12.2016 03:26 24.911.872 MailCliеnt.exe
2 File(s) 49.654.632 bytes
2 Dir(s) 78.501.982.208 bytes free
I´ve also checked permissions on the files and took ownership, it changes nothing. Additionally I´ve cleared the NTFS Journal and even the transaction log + run chkdsk, which reveals no errors either.
Any ideas on this mysterious situation? What am I missing?
Thanks so much:)
UPDATE #1:
I´ve just tried this: going to Windows explorer and renaming both files after each other by truncating their names. So I first renamed the first "MailClient.exe" to "MailClien.exe", then the seconds "MailClient.exe" to "MailClien.exe". Again, no message by Windows that they have the same name, it just renamed both fine. I then continued to "MailClie.exe". Worked.
However, as soon as I tried to renamed both to "MailCli.exe", Windows complained and told me that there is already another file with that name. Trying to rename both back from there to "MailClient.exe" also does not work, just for one of them, because then Windows says (and right so too) that a file with that name already exists. So it seems to come down to the "e" possibly having another ANSI-character in both filenames? I, however, wouldn´t know of another one for "e", or am I missing something?
Harry Johnston is right: one of the filenames contains a Unicode character that just looks the same as an ANSI character.
Read Naming Files, Paths, and Namespaces:
On newer file systems, such as NTFS, exFAT, UDFS, and FAT32, Windows
stores the long file names on disk in Unicode, which means that the
original long file name is always preserved. This is true even if a
long file name contains extended characters, regardless of the code
page that is active during a disk read or write operation.
Use the following PowerShell script 43381802b.ps1 to detect and show non-ANSI file names (see different calls below):
param( [string[]]$Path = '.',
[switch]$Cpp, ### list any non-ANSI character in file names like a C++ literal
### i.e. a prefix \u followed by a four digit Unicode code point
[switch]$All ### list all files including pure ANSI-encoded file names
)
Set-StrictMode -Version latest
$strArr = Get-ChildItem -path $Path
$arrDiff = #()
for ($i=0; $i -lt $strArr.Count; $i++) {
$strDiff = 'ANSI'
$strName = ''
$auxName = $strArr[$i].Name
for ( $k=0; $k -lt $auxName.Length; $k++ ) {
if ( [int][char]$auxName[$k] -gt 255 ) {
$strDiff = 'UCS2'
$strName += '\u{0:X4}' -f [int][char]$auxName[$k]
} else {
$strName += $auxName[$k]
}
}
if ( $All.IsPresent -or $strDiff -eq 'UCS2' ) {
$strArr[$i] | Add-Member NoteProperty Code $strDiff
$strArr[$i] | Add-Member NoteProperty CppName $strName
$arrDiff += $strArr[$i]
}
}
if ( $Cpp.IsPresent ) {
$arrDiff | Select-Object -Property Code, Mode, LastWriteTime, Length, CppName | ft
} else {
$arrDiff | Select-Object -Property Code, Mode, LastWriteTime, Length, Name | ft
}
Output:
PS D:\PShell> .\SO\43381802b.ps1 'C:\testC\43381802'
Code Mode LastWriteTime Length Name
---- ---- ------------- ------ ----
UCS2 -a---- 02/05/2017 11:47:53 317 MailCliеnt.txt
UCS2 -a---- 02/05/2017 11:49:04 317 МailClient.txt
UCS2 -a---- 02/05/2017 11:50:16 399 МailCliеnt.txt
PS D:\PShell> .\SO\43381802b.ps1 'C:\testC\43381802' -Cpp
Code Mode LastWriteTime Length CppName
---- ---- ------------- ------ -------
UCS2 -a---- 02/05/2017 11:47:53 317 MailCli\u0435nt.txt
UCS2 -a---- 02/05/2017 11:49:04 317 \u041CailClient.txt
UCS2 -a---- 02/05/2017 11:50:16 399 \u041CailCli\u0435nt.txt
PS D:\PShell> .\SO\43381802b.ps1 'C:\testC\43381802' -Cpp -All
Code Mode LastWriteTime Length CppName
---- ---- ------------- ------ -------
ANSI -a---- 02/05/2017 11:44:05 235 MailClient.txt
UCS2 -a---- 02/05/2017 11:47:53 317 MailCli\u0435nt.txt
UCS2 -a---- 02/05/2017 11:49:04 317 \u041CailClient.txt
UCS2 -a---- 02/05/2017 11:50:16 399 \u041CailCli\u0435nt.txt
Use the following 43381802a.ps1 script to get more info about non-ANSI characters (see the first call bellow) and their position in file names (see the latter call bellow with -Detail switch):
param( [string[]] $strArr = #('ΗGreek', 'НCyril', 'HLatin'),
[switch]$Detail )
Set-StrictMode -Version latest
$auxArr = #()
if ( ( Get-Command -Name Get-CharInfo -ErrorAction SilentlyContinue ) -and
( -not $Detail.IsPresent ) ) {
$auxArr = $strArr | Get-CharInfo |
Where-Object { [int]$_.Codepoint.Replace('U+', '0x') -ge 128 }
} else {
foreach ($strStr in $strArr) {
for ($i = 0; $i -lt $strStr.Length; $i++ ) {
if ( [int][char]$strStr[$i] -ge 128 ) {
$auxArr += [PSCustomObject] #{
Char = $strStr[$i]
CodePoint = 'U+{0:x4}' -f [int][char]$strStr[$i]
Category = $i + 1 ### 1-based index
Description = $strStr ### string itself
}
}
}
}
}
$auxArr
Output:
PS D:\PShell> .\SO\43381802a.ps1 ( Get-childitem -path 'C:\testC\43381802' ).Name
Char CodePoint Category Description
---- --------- -------- -----------
е U+0435 LowercaseLetter Cyrillic Small Letter Ie
М U+041C UppercaseLetter Cyrillic Capital Letter Em
М U+041C UppercaseLetter Cyrillic Capital Letter Em
е U+0435 LowercaseLetter Cyrillic Small Letter Ie
PS D:\PShell> .\SO\43381802a.ps1 ( Get-childitem -path 'C:\testC\43381802' ).Name -detail
Char CodePoint Category Description
---- --------- -------- -----------
е U+0435 8 MailCliеnt.txt
М U+041c 1 МailClient.txt
М U+041c 1 МailCliеnt.txt
е U+0435 8 МailCliеnt.txt
Tested on files:
==> dir /-C /X /A-D C:\testC\43381802\
Volume in drive C has no label.
Volume Serial Number is …
Directory of C:\testC\43381802
02/05/2017 11:44 235 MAILCL~1.TXT MailClient.txt
02/05/2017 11:47 317 MAILCL~2.TXT MailCliеnt.txt
02/05/2017 11:49 317 AILCLI~1.TXT МailClient.txt
02/05/2017 11:50 399 AILCLI~2.TXT МailCliеnt.txt
4 File(s) 1268 bytes
0 Dir(s) 69914857472 bytes free
==>
I am attempting to use 7 Zip through the command line. As you can see below, using the command 7z l lists the 3 files in the target zip file.
C:\Users\User1\Downloads>7z l recording_20130731180507.zip
--
Path = recording_20130731180507.zip
Type = zip
Physical Size = 311686
Date Time Attr Size Compressed Name
------------------- ----- ------------ ------------ ------------------------
2013-07-31 18:05:06 ..... 655 655 SD_DISK\20130731\18\2013073
1_180505_A4BC_00408CC2B40B\recording.xml
2013-07-31 18:05:06 ..... 309752 309752 SD_DISK\20130731\18\2013073
1_180505_A4BC_00408CC2B40B\20130731_18\20130731_180505_59EB_00408CC2B40B.mkv
2013-07-31 18:05:06 ..... 279 279 SD_DISK\20130731\18\2013073
1_180505_A4BC_00408CC2B40B\20130731_18\20130731_180505_59EB_00408CC2B40B.xml
------------------- ----- ------------ ------------ ------------------------
310686 310686 3 files, 0 folders
However, when I attempt to actually unzip the file, I get a "no files to process error". I've never tried unzipping from cmd before. Do I have to try to dig into the zip file to extract those 3 files?
C:\Users\User1\Downloads>7z e recording_20130731180507.zip o-C:\users\User1\do
cuments\folder1\test
No files to process
Files: 0
Size: 0
Compressed: 311686
The option is -o, not o-. Run the command like this:
7z e recording_20130731180507.zip -o"C:\users\User1\documents\folder1\test"
The command Get-Process gives output like below:
Handles NPM(K) PM(K) WS(K) VM(M) CPU(s) Id ProcessName
------- ------ ----- ----- ----- ------ -- -----------
65 6 1152 840 59 77.50 6048 Appx
78 8 2233 444 61 10.11 7878 Application
but I need a solution like below:
PM(K)=1152, ProcessName=Appx ; PM(K)=2233, ProcessName=Application
How do I parse output like above mentioned?
you can use -f to format your string :
PS>$resu=""
PS>gps | foreach {$resu+=("PM(K)= {0},appName={1};" -f ($_.pm/1KB),$_.name) }
PS>$resu
Try something like this:
$p = Get-Process | select #{n='PM(K)';e={$_.PM/1KB}}, ProcessName
($p | fl | Out-String) -replace "`n`n", ' ; ' -replace "`n", ', '
In Bash, we can combine two shell commands cd and ls like this:
function cd {
builtin cd "$#" && ls
}
#this will get a list of file after changing into a directory
Also this
mkcd () { mkdir -p "$#" && cd "$#"; }
#this will create and directory and change into it at once
Can we do similar stuff in Powershell? If so, I would like to make similar functions and put it in my $profile
Thanks for any help.
Steeluser
EDIT:
I realized this could be done from shell like this:
$> pwd|ls
Directory: D:\ps
Mode LastWriteTime Length Name
---- ------------- ------ ----
d---- 5/7/2011 9:40 PM config
d---- 5/7/2011 9:40 PM output
d---- 5/8/2011 3:37 AM static
-a--- 5/8/2011 3:36 AM 485 create-static-files.ps1
This could be put in a profile like this:
function pl
{
pwd|ls
}
and can be invoked from the shell
ps$ pl
Directory: D:\ps
Mode LastWriteTime Length Name
---- ------------- ------ ----
d---- 5/7/2011 9:40 PM config
d---- 5/7/2011 9:40 PM output
d---- 5/8/2011 3:37 AM static
-a--- 5/8/2011 3:36 AM 485 create-static-files.ps1
But I could not figure out how to do the mkcd function.
Something like this should work.
Function mkcd {
mkdir $args[0]
cd $args[0]
}
This is just an ordinary function in powershell. See http://technet.microsoft.com/en-us/library/dd347712.aspx for more.
You may want also manage exception directory already exists and also return the directory object to your caller:
Function mkcd {
if(!(Test-Path -path $args[0])) {
mkdir $args[0]
}
cd $args[0] -passthru
}