I'm trying to work around some logs to extract the data I want and to push it into another simplified .txt file (before going for the next step).
Here's the bit of code I've been trying to use to reach my goal:
setlocal ENABLEDELAYEDEXPANSION
for %%i in (C:\Test_Analyse\*files*.txt) do (
SET va=%%i
SET va=!va:~16,-31!
find /v /c "" %%i | FINDSTR /V /R /C:"^$">>C:\test_results\!va!log3.txt
set /p var=<C:\test_results\!va!log3.txt
set var=!var:~68,10!
echo !var!>>C:\test_results\!va!log2.txt
)
endlocal
The C:\test_results\!va!log3.txt file content is : ---------- C:\TEST_ANALYSE\1K43782_TEST_RENAMED_FILES_20210915.TXT: 223856.
As far as I know, it does its job except for the echo !var!>>C:\test_results\!va!log2.txt part. It prints Chinese characters in my output file instead of 223856. On a side note, when I discard the #echo OFF, I notice the ECHO line working properly in CMD - so I guess it's maybe about encryption? But I tried a few things around that, without success sadly.
Related
Just to be thorough, I'll state here my whole project and what I'm aiming at.
I intend to adapt a shell script to work in Windows cmd, as this is intended for people who are not going to have some sophisticate language available.
for g in $(curl -Ls https://api.chess.com/pub/player/hikaru/games/archives | jq -rc ".archives[]") ; do curl -Ls "$g" | jq -rc ".games[].pgn" ; done >> games.pgn
For some reason, Chess.com's API doesn't have a very important feature that Lichess' does, to export all games of a single player, so what I can do manually is to use https://api.chess.com/pub/player/hikaru/games/archives to export all available monthly archives and then hit the API for each one of them. (hikaru inside this will be a set variable, it's the nickname of the desired player to export).
The result for this command is something like
{"archives":["https://api.chess.com/pub/player/hikaru/games/2015/11","https://api.chess.com/pub/player/hikaru/games/2015/12","https://api.chess.com/pub/player/hikaru/games/2016/02","https://api.chess.com/pub/player/hikaru/games/2016/03","https://api.chess.com/pub/player/hikaru/games/2016/04","https://api.chess.com/pub/player/hikaru/games/2016/05"]}
to which I only have to append /pgn to get the desired result.
Obviously, cmd doesn't have jq available, so this involves "parsing" the string inside a batch file.
I figured if I just could replace every occurrence of " with a linebreak and echo the results, I could then use find (or findstr) to easily get a list of lines that only would need to be prefaced with curl and appended with /pgn to get my final result.
The big question is: how do I replace " with a linebreak in cmd? I found a few answers, but none of them seems to work with a special character, part of the problem is that I also didn't understand these answers enough to try and adapt them.
A second way of perhaps achieving the same result would be replacing [, ] and , with line breaks, but then I would also have to worry with deleting the final " to append /pgn, so if I'm able to do the former, it would be cleaner.
in batch/cmd, a for loop is used to process a list (separated by default delimiters like space, tab, comma). So just replace [ and ] with a space or comma, and you have a nice list to split. Finally, use find to filter the output to the relevant parts and you're done:
#Echo off
setlocal
set "string={"archives":["https://api.chess.com/pub/player/hikaru/games/2015/11","https://api.chess.com/pub/player/hikaru/games/2015/12","https://api.chess.com/pub/player/hikaru/games/2016/02","https://api.chess.com/pub/player/hikaru/games/2016/03","https://api.chess.com/pub/player/hikaru/games/2016/04","https://api.chess.com/pub/player/hikaru/games/2016/05"]}"
set "string=%string:[= %"
set "string=%string:]= %"
for %%a in (%string%) do echo %%~a|find "/"
Output:
https://api.chess.com/pub/player/hikaru/games/2015/11
https://api.chess.com/pub/player/hikaru/games/2015/12
https://api.chess.com/pub/player/hikaru/games/2016/02
https://api.chess.com/pub/player/hikaru/games/2016/03
https://api.chess.com/pub/player/hikaru/games/2016/04
https://api.chess.com/pub/player/hikaru/games/2016/05
(in case you wonder: the tilde in echo %%~a removes surrounding quotes)
Stephan's answer gave me the directions I needed to research more and build my own solution. This is not the final script to my project, but it does solve every problem presented in my original question:
#echo off
setLocal enabledelayedexpansion
for /f "delims=" %%a in (input.txt) do (
for %%b in (%%a) do (
set string=%%b
set "string=!string:[=,!"
set "string=!string:]=,!"
echo !string!>>replaced.txt
)
)
for /f "delims=" %%c in (replaced.txt) do (
for %%d in (%%c) do (
echo %%~d>>echo.txt
)
)
for /f %%e in (echo.txt) do echo curl %%~e/pgn|find ".">>list.txt
I basically run 3 sets of loops, the first one loads my input (this could not be done via set because there's a size limit, using a nested loop works around that) and replaces [ and ] for commas.
The second loop sorts again the output. This is done basically to trim unwanted characters from the first and last line.
The last loop generates a list of curl commands that will later be executed into a PGN file (which is a chess file).
This ends the scope of the question, but since my project wasn't that complex, I'll present it's final version, which improves on Compo's answer, in case someone else stumbles upon this question:
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: Chess.com and Lichess API Scraper ::
:: Author: fabiorzfreitas ::
:: Extract all games from a player from Chess.com and Lichess ::
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: This tool uses Chess.com and Lichess APIs to extract all games from a given player. ::
#echo off
setLocal enabledelayedexpansion
echo.
echo.
echo.
echo All input must be lowcase!
echo.
echo You can skip the input bellow by pressing Enter
echo.
echo.
echo.
set /p lichess="Input Lichess nickname and press Enter: "
set /p chess="Input Chess.com nickname and press Enter: "
echo.
:Lichess
if not defined lichess goto :Chess
curl https://lichess.org/api/games/user/%lichess% >> Games.pgn
:Chess
if not defined chess goto :End
(for /f "usebackq tokens=2 delims=[]" %%g in (`curl https://api.chess.com/pub/player/%chess%/games/archives`) do (
for %%h In (%%g) do curl "%%~h/pgn" >> Games.pgn
)
)
:End
exit
Based upon your own answer, it seems as if you could remove at least one of those steps by using the brackets [ and ], as delimiters.
You could also nest a for loop within another instead of having individual ones and writing to files.
Here it is as a single line batch-file:
#(For /F "UseBackQ Tokens=2 Delims=[]" %%G In ("input.txt") Do #For %%H In (%%G) Do #Echo curl.exe "%%~H/pgn") 1>"list.txt"
To do it directly in cmd:
(For /F "UseBackQ Tokens=2 Delims=[]" %G In ("input.txt") Do #For %H In (%G) Do #Echo curl.exe "%~H/pgn") 1>"list.txt"
I have a script that needs to extract a YouTube URL from a text file.
Here's what I have in the text file (output.txt):
---------- NUMBER11.TXT
<link itemprop="url" href="http://www.youtube.com/channel/UCnxGkOGNMqQEUMvroOWps6Q">
Note the text file has a line of empty space to start, which is annoying, and the URL is on line 3. Something that doesn't show up in the formatting for this site is the 11 spaces before the actual href starting as well. I'd like to separate it from the mass of other junk.
I've tried something like this:
set /p long= < output.txt
echo %long%
set short1=%long:^<link itemprop^="url" href^="=%
echo %short1% > o1.txt
I thought this would remove the selected text from the file, but I think this is a little over my head.
I'm getting the output.txt from firstly a curl of a youtube video page, and secondly from a find command here:
find "href=""http://www.youtube.com/channel/" %vd% > output.txt
Maybe I'm making this more complicated than it is?
Using batch-files to access files with special characters, like redirect, it can cause some problems, so it is not recommended, but I felt like posting an answer anyway, so given you exact example, here is one way. If your example is not as per your post, which I highly expect it to be, then this probably would not work.
#echo off
setlocal enabledelayedexpansion
for /f "usebackq delims=" %%i in ("output.txt") do for %%a in (%%i) do (
set "var=%%~a"
set "var=!var:>=!"
set "var=!var:"=!"
if "!var:~0,4!" == "http" echo !var!
)
#ECHO OFF
SETLOCAL
SET "sourcedir=U:\sourcedir"
SET "filename1=%sourcedir%\q64572433.txt"
set "url="
FOR /f "tokens=4,5delims=>= " %%a IN (%filename1%) DO if "%%~a"=="href" set "url=%%~b"
echo URL=%url%
GOTO :EOF
You would need to change the setting of sourcedir to suit your circumstances. The listing uses a setting that suits my system.
I used a file named q64572433.txt containing your data for my testing.
The for command tokenises each line of the file, using =, > and space as delimiters (the 3 characters between delims= and ")
On the line of interest, token 4 would be href and token 5 the url - and this is the only line where href is the fourth token. When that is detected, assign the 5th token (in %%b) to the variable, removing the quotes with ~ for good measure.
I would suggest you parse the results directly from your curl command instead of outputting them to a text file, and then using find against that output.
However, instead of using find.exe, I would suggest you use the following method using findstr.exe instead, to get the URL assigned to any line containing href= followed by "http: or "https and subsequently followed by youtube.com.
#Echo Off
SetLocal EnableExtensions DisableDelayedExpansion
For /F Tokens^=*EOL^= %%G In (
'%__APPDIR__%findstr.exe /IR "href=\"http[s:].*youtube\.com" "output.txt"'
) Do (Set "Line=%%G" & SetLocal EnableDelayedExpansion
For /F Tokens^=2Delims^=^" %%H In ("!Line:*href=!") Do EndLocal & Echo %%H)
Pause
If you want the output stored as a variable, instead of Echoing it, change Echo %%H to Set "URL=%%H". You could then use %URL%, (or "%URL%" if you need it doublequoted), elsewhere in your script.
I am working on a program using batch where the program read root directories from a text file and count total number of Folders and Files in all th give root directories. The program is working as it should be but I want to display the output in a certain way.
This is how I want to display output
0 : OF : 6
The first value should change each time program finish counting in one root directory. I have written code for it but the output I am getting is this.
Here is code I have written to change it.
:textUpdate
echo !counter! : OF : %number%
GOTO :EOF
where counter is the current number of root directory and number is total number of directories found in the text file. Is there any way to display the output like the first one.
you can abuse set /p to write to screen. It doens't append a line feed. You also need a Carriage Return to go back to the beginning of the line to overwrite the old output:
#echo off
setlocal EnableDelayedExpansion
for /f %%a in ('copy /Z "%~dpf0" nul') do set "CR=%%a"
for /l %%n in (1,1,6) do (
set /P "=Count %%n of 6!CR!" <nul
timeout 1 >nul
)
The first for /f loop is just to get a CR (Carriage Return). You have to use delayed expansion to be able to use it (%CR% does not work).
If I get my parameter with %1 and it is "Server" how can I add a + sign after every letter?
So my result would be "S+e+r+v+e+r"?
I think Batch file to add characters to beginning and end of each line in txt file this is a similar question but I don't know how to change the code for this purpose.
Any help would be great!
I'm pretty sure this has been asked and answered before, but I couldn't find it.
There is a really cool (and fast) solution that I saw posted somewhere. It uses a new cmd.exe process with the /U option so output is in unicode. The interesting thing about the unicode is that each ASCII character is represented as itself followed by a nul byte (0x00). When this is piped to MORE, it converts the nul bytes into newlines!. Then a FOR /F is used to iterate each of the characters and build the desired string. A final substring operation is used to remove the extra + from the front.
I tweaked my memory of the code a bit, playing games with escape sequences in order to get the delayed expansion to occur at the correct time, and to protect the character when it is appended - all to get the technique to preserve ^ and ! characters. This may be a new twist to existing posted codes using this general technique.
#echo off
setlocal enableDelayedExpansion
set "str=Server bang^! caret^^"
set "out="
for /f delims^=^ eol^= %%A in ('cmd /u /v:on /c echo(^^!str^^!^|more') do set "out=!out!+^%%A"
set "out=!out:~1!"
echo Before: !str!
echo After: !out!
--OUTPUT---
Before: Server bang! caret^
After: S+e+r+v+e+r+ +b+a+n+g+!+ +c+a+r+e+t+^
This batch file should do it:
#ECHO OFF
SETLOCAL EnableDelayedExpansion
SET Text=%~1
SET Return=
REM Batch files don't have a LEN function.
REM So this loop will process up to 100 chars by doing a substring on each.
FOR /L %%I IN (0,1,100) DO (
CALL SET Letter=!Text:~%%I,1!
REM Only process when a letter is returned.
IF NOT "!Letter!" == "" (
SET Return=!Return!+!Letter!
) ELSE (
REM Otherwise, we have reached the end.
GOTO DoneProcessing
)
)
:DoneProcessing
REM Remove leading char.
SET Return=%Return:~1,999%
ECHO %Return%
ENDLOCAL
Calling with Test.bat Server prints S+e+r+v+e+r to the console.
I have a file that contains the output of a file compare thats written to a text file:
Comparing files C:\LOGS\old.txt and C:\LOGS\NEW.TXT
***** C:\LOGS\old.txt
***** C:\LOGS\NEW.TXT
folder_thats_different
*****
I need to pull out the next to last line "folder_thats_different" and put in a new string:
folder contains a file that is different: folder_thats_different
Yes, I know I can use another language, but I'm stuck with batch files for now.
You can try to read it with a for-loop and take the current line, and always save the previous line
#echo off
setlocal EnableDelayedExpansion
for /f "delims=" %%x in (myFile.txt) do (
set "previous=!last!"
set "last=%%x"
)
echo !previous!
Here's an example you can use as a starting point. Just change the filename in the set command= line to the appropriate name (or replace the command with whatever will gerneate the log listing).
#echo off
#setlocal
(set command=type test.txt)
for /f "usebackq tokens=*" %%i in (`%command%`) do call :process_line %%i
echo next to last line: %old_line%
goto :eof
:process_line
(set old_line=%new_line%)
(set new_line=%*)
goto :eof
Of course, you'll probably want to do something other than simply echoing the found line.
The first answer works for me. I also added 2 lines after the end to allow it to repeat so I could watch an active log file without having to close and reopen it. I do a lot of debugging for the mods that are used in the game Space Engineers.
My version looks like this:
#echo off
setlocal EnableDelayedExpansion
for /f "delims=" %%x in (SpaceEngineers.log) do (
set "previous=!last!"
set "last=%%x"
)
echo !previous!
timeout 15 /nobreak
se_log
The line below stops the batch file from looping too fast and stop the key bypass. To change the amount of time in seconds just change the number "15" to anything you want. To stop the batch file just press ctrl+c.
timeout 15 /nobreak
The line below is the name of the batch file I made so it will tell CMD to run this again.
se_log