for /F not working as expected, not giving every element - for-loop

Question

I have a simple .bat with content:
#echo off
set "var=VAR1?VAR2?VAR3"
echo.%var%
echo.
for /F "delims=?" %%H in ('echo.%var%') do echo.%%~H
I would expect the following output:
VAR1?VAR2?VAR3
VAR1
VAR2
VAR3
Instead I get:
VAR1?VAR2?VAR3
VAR1
^ notice a trailing empty new line
Tried using different delimiter, usebackq, giving eol character, running through another for loop with different parameters, many other things; different variable name, different filename, setlocal w/o and w/ delayed expansion; no success.
If I include the tokens=1,2 parameter, it prints VAR1 into %%H and VAR2 into %%I as it should, but I need it to be "dynamic", give every element my %var% has.
EDIT: What is the most confusing to me is that if instead of writing my elements into a variable separated by a ?, I write it to a file, each element in its own line, then read the file line-by-line with "for /F "delims=*" %%H in ('type "VAR.txt"') do..."
it works perfeclty. Is it because the "delims=*", or is there a higher power messing with me? I don't see any difference from the FOR's perspective. If, by default, it should only give me the first token, why does it give every token from the file? Isn't token as new line the same as token as any other character?

Answer 1

You can also use this very simple trick that converts tokens into lines:
#echo off
setlocal EnableDelayedExpansion
set "var=VAR1?VAR2?VAR3"
echo %var%
echo/
for %%n in (^"^
%Do not remove this line%
^") do for /F "delims=" %%H in ("!var:?=%%~n!") do echo/%%~H
In this way, each token in the string: VAR1?VAR2?VAR3 is converted into separate lines like these ones:
VAR1
VAR2
VAR3
This is done replacing each ? character in the string by a NewLine (ASCII 10) character.
After that, you can process each line in any way you wish...

Answer 2

Each line read by a for/f is split into "tokens", substrings separated by any of the delimiters.
By default, tokens=1, so just the first token on the line is assigned to the metavariable %%H.
You could list each required token in a tokens= clause.
To assign each to %%H in turn, try
for %%H in (%var:?= %) do echo %%H
or if your elements contain spaces,
for %%H in ("%var:?=" "%") do echo.%%~H
The 'extra' newline is a red herring. You are misreading the result.

Answer 3

Here's a possible alternative methodology, which should be okay as long as each question mark delimited token does not contain a doublequote or exclamation mark character:
#Echo Off
SetLocal EnableExtensions DisableDelayedExpansion
Rem Dehine a variable named var.
Set "var=VAR 1?VAR,2?VAR|3?VAR;4?VAR<5?VAR>6?VAR&7?VAR=8"
Rem view the content of the variable named var.
Set /P "=%var%" 0<NUL & Echo(
Rem Display an optional empty line
Echo(
Rem Undefine and variables beginning with the string var[.
For /F "Delims==" %%G In ('"(Set var[) 2>NUL"') Do Set "%%G="
Rem define a increment counter
Set "i=1"
Rem Enable delayed variable expansion
SetLocal EnableDelayedExpansion
Rem Define variables named var[increment] splitting var at each question mark.
Set "var[%i%]=%var:?=" & Set /A i +=1 & Set "var[!i!]=%"
Rem Disable delayed variable expansion
Rem and display the individual incremented variable values.
For /F "Tokens=1-2,* Delims=[]=" %%G In ('"(Set var[) 2>NUL"') Do (EndLocal
Set /P "=%%I" 0<NUL & Echo()
Rem Display an optional empty line
Echo(
Pause
Please be aware that if there are more than nine tokens, the display order will not currently match the order in the initial variable. adjustments would have to be made in order to do that.
Expected output:
VAR 1?VAR,2?VAR|3?VAR;4?VAR<5?VAR>6?VAR&7?VAR=8
VAR 1
VAR,2
VAR|3
VAR;4
VAR<5
VAR>6
VAR&7
VAR=8
Press any key to continue . . .

Answer 4

Here's another possibility:
#Echo Off
SetLocal EnableExtensions DisableDelayedExpansion
Rem Define a variable named var.
Set "var=VAR 1?VAR,2?VAR;3?VAR=4?VAR!5?VAR&6?VAR)7?VAR^8?VAR#9"
Rem Display the content of the variable.
Set /P "=%var%" 0<NUL & Echo(
Rem Display an optional empty line.
Echo(
Rem Display the individual tokens.
For %%G In ("%var:?=","%") Do Set /P "=%%~G" 0<NUL & Echo(
Rem Display an optional empty line.
Echo(
Pause
This allows for an exclamation mark to exist in your file path tokens. It also has the additional benefit of being able to display the tokens in the same order, regardless of the number of them.
I should add that if you expect there to be percent characters in your tokens, you will have to perform an additional step, (due to those having a special meaning in cmd and batch files).
Expected output:
VAR 1?VAR,2?VAR;3?VAR=4?VAR!5?VAR&6?VAR)7?VAR^8?VAR#9
VAR 1
VAR,2
VAR;3
VAR=4
VAR!5
VAR&6
VAR)7
VAR^8
VAR#9
Press any key to continue . . .