Output the content of the specified line with special characters preserved to the greatest extent possible - China DOS Union Forum

In the CMD, the output processing of text content containing special characters has always been quite a headache: If you want to be compatible with special characters, you usually enclose the content in quotes and then output it. But in this way, quotes will be added at the beginning and end of all output lines. If you are very concerned about the quotes after output, this solution cannot be implemented. However, except for this solution, it seems there are no other solutions that can perfectly solve this difficult problem. Note: For the perfect solution, please refer to the code of bjsh on floor 23

Recently, I have had a little free time and thought about this problem again for a while. After several modifications, there is Code 1, which is sent out for everyone to test:

Code 1:


@echo off
:: Idea: Escape all special symbols and then output
:: Restriction: The file to be processed cannot be enclosed in quotes;
cd.>output.txt
for /f "delims=" %%i in ('findstr /n .* test.txt') do (
    set "str=%%i"
    call set "str=%%str:*:=%%"
    if defined str (call :output) else echo.>>output.txt
)
start output.txt
exit

:output
set "str=%str:^=^^%"
set "str=%str:>=^>%"
set "str=%str:<=^<%"
set "str=%str:|=^|%"
set "str=%str:&=^&%"
set "str=%str:"=^"%"
call echo.%%str%%>>output.txt
goto :eof

The modification from the code on floor 21 by bjsh is as follows. I personally think it is a relatively perfect solution:

Code 2:


@echo off
cd.>output.txt
for /f "delims=" %%i in ('findstr /n .* test.txt') do (
    set "str=%%i"
    call set "str=%%str:*:=%%"
    if defined str (call :output) else echo.>>output.txt
)
start output.txt
exit

:output
set "str=%str:^=^^%"
set "str=%str:"=%"
set "str=%str:>=^>%"
set "str=%str:<=^<%"
set "str=%str:&=^&%"
set "str=%str:|=^|%"
set "str=%str:="%"
(call echo.%%str%%)>>output.txt
goto :eof

The most perfect code is as follows (from the code of bjsh on floor 23, I only made a small modification):

Code 3:


@echo off
cd.>output.txt
for /f "delims=" %%i in ('findstr /n .* test.txt') do (
        set "var=%%i"
        setlocal enabledelayedexpansion
        set var=!var:*:=!
        (echo.!var!)>>output.txt
        endlocal
)
start output.txt

The content of the test file test.txt (please note: one of the blank lines is composed of spaces):


"aou"eo

;euou%^>
::::aeui
   
:::E2uo alejou 3<o2io|
^aue||%ou

!aue!
aoue eou 2
!str!auoeu!ueo &&
euo 8
ueyi^^^^aueuo2
~ ! @ # $ % ^ & * ( () " ok " No " <>nul
set ok=^

Analysis of Code 1 and Code 2 is as follows:

① According to the general idea, when referencing variables in the for statement, the setlocal enabledelayedexpansion statement is used to enable variable delay function. However, this function has a fatal flaw: when the string to be processed contains exclamation marks, the exclamation mark pairs and all the strings between them will be replaced with empty. So, Code 1 and Code 2 abandon the setlocal solution and use the solution of calling a sub-process;
② If the text to be processed contains an odd number of quotes, the echo.%str%>>output.txt statement will be wrong. So directly replace the quotes with special invisible characters and then output (that is, the black box displayed in the code); Thanks to the test of lxmxn and the analysis of bjsh;
③ In the :output sub-process, the sentence set "str=%str:^=^^%" must be placed before all replacement sentences, otherwise, ^ will be repeatedly replaced, resulting in inaccurate results; Thanks to the test of lxmxn;
④ The ordinary for statement will ignore the line content starting with a semicolon and also ignore blank lines. So, the findstr .* test.txt statement is used to display all lines (including blank lines); delims=: will discard all colons at the beginning of the line. So, the statement call set "str=%%str:*:=%%" is used to avoid this situation;
⑤ In the statement (call echo.%%str%%)>>output.txt, the dot after echo cannot be omitted. Otherwise, when the line content is a space, the output content will display the current state of echo; Using the call statement is to be compatible with lines with quotes; Using parentheses is to correctly handle the single numbers 1-9 separated by spaces at the end of the line.
⑥ The reason why the :output tag segment does not use the statement for %%i in (^^ ^> ^< ^| ^&) do call set "str=%%str:%%i=^%%i%%" is because this replacement statement cannot replace correctly. It seems that the call delay mechanism in the for statement is indeed a bit confusing.

Regarding Code 3, due to limited level, I can only make a superficial and vague analysis: In this code, the variable delay function is used to completely obtain special characters and terminate the variable delay at an appropriate time to avoid the problem that the string is recognized as a variable reference due to excessive variable delay. In fact, this is still the CMD preprocessing mechanism at work. It is worth noting that the position of the setlocal statement cannot be swapped with the set statement, otherwise, the exclamation mark will still be recognized as a variable reference symbol and thus be discarded.

[ Last edited by namejm on 2007-8-14 at 09:27 PM ]

Thanks to brother namejm for the original work. It's really gratifying that you can still care about the forum even with such a busy work schedule.

After a short test, it was found that there are no output errors.

[ Last edited by lxmxn on 2007-5-17 at 11:16 PM ]

Good stuff, but I can't understand it!

Except for quotes, the rest is actually easy

I used to write something similar too;
It's similar to Brother namejm's;
But I didn't consider the situation where a whole line is all spaces;
The last sentence is copied from namejm;
The usage of echo.%var% is really wonderful.


@echo off
for /f "tokens=1 delims=" %%a in ('findstr /n .* test.txt') do set "var=%%a" & call :change
goto :eof 
:change
set "var=%var:^=^^%"
set "var=%var:>=^>%"
set "var=%var:<=^<%"
set "var=%var:&=^&%"
set "var=%var:|=^|%"
set "var=%var:*:=%"
echo.%var%

This has solved the "problem;
Everyone also help test it;

To bjsh:

There is a small problem found in the test. Please see my test text below.

test.txt

~ ! @ # $ % ^ & * ( () " ok " No " <>nul
set ok=^

Screen display

There should not be > here.

Hey; the problem with three quotes;
It's like
echo fsdg"dsgsa"gds" >55.txt
won't be written into 55.txt;
I'll think about it again;
Is it really necessary to write it as an invisible character.

The result is incorrect.

aoueo

;euou%^>
::::aeui

:::E2uo alejou 3<o2io|
^aue||%ou

!aue!
!str!auoeu!ueo &&
ueyi^^^^aueuo2
=^

The first line "becomes garbled, and the last line is added"

To ttyp:

The first line's " is deliberately turned into that special character by the original poster.

Is your test text copied from the original poster's test text?

It works perfectly fine on my end.

It's copied, didn't see clearly from the first line. But there's an extra line at the end which is strange. My system is W2K + SP4

Originally posted by bjsh at 2007-5-18 12:53:
Oh, the problem with three quotation marks;
Just like
echo fsdg"dsgsa"gds" >55.txt
It won't be written into 55.txt;
I'll think about it again;
Do I really have to write it as an invisible character?

　　After testing, it should be that when there are an odd number of quotation marks, this situation occurs. This is a major defect of the echo statement when the call statement calls a sub - procedure. Maybe only replacing it with other characters will work.

　　The code in the top floor has simplified some statements. The set "str=%~1" is completely unnecessary. Hehe, my thinking was too complicated.

[ Last edited by namejm on 2007-5-18 at 01:47 PM ]

Oh, I see. It's because I've been testing the P processing and commented out EXIT.

Originally posted by namejm at 2007-5-18 01:26 PM:

After testing, it should be when the number of quotation marks is odd that this situation occurs. This is a major defect of the echo statement when the call statement calls a sub - procedure. Maybe only replacing it with other characters will work. ...

Yes; it is when it is odd;
echo dsg^"sdgsad^"gdgasdg^" >5.txt
It can be done;

In fact, "is also a special character;

Revise again


@echo off
set var=
for /f "delims=" %%a in ('findstr /n .* test.txt') do (
set "var=%%a"
call set "var=%%var:"=%%"
call :change
)
goto :eof
:change
set "var=%var:^=^^%"
set "var=%var:>=^>%"
set "var=%var:<=^<%"
set "var=%var:&=^&%"
set "var=%var:|=^|%"
set "var=%var:*:=%"
if not defined var echo. & goto :eof
call set "var=%%var:="%%"
echo.%var%

Test again to see what other problems there are

c:\>set "var=~ ! @ # $ % ^ & * ( () " ok " No " <>nul"
此时不应有 >.

c:\>for /f "delims=" %a in (test.txt) do set "var=%a"

c:\>set "var=~ ! @ # $ % ^ & * ( () " ok " No " <>nul"
c:\>set var
var=~ ! @ # $ % ^ & * ( () " ok " No " <>nul

Content of test.txt

~ ! @ # $ % ^ & * ( () " ok " No " <>nul

Did you find the difference between the two?
The former shows "此时不应有 >" (There should not be > here), while the latter just uses for to convert that string to %a and then assign it to var; no error occurs; Interesting!

When the number of quotation marks is odd, there will be no errors in the result of echo, but when the number of quotation marks is even, an error will occur. The solution is to change echo.%var% to call echo.%%var%%, but in this case, if the test content is ~!@#$%^&*(()" ok " No " a>a, all the content after the last quotation mark will be discarded. The reason is currently unknown.

————————————————————————————————
The above remarks belong to the test code used being incorrect, resulting in incorrect conclusions.

[ Last edited by namejm on 2007-5-18 at 02:53 PM ]

Floor1　namejm	Posted 2007-05-17 21:14
荣誉版主　Posts 1,737　Credits 5,226　From 成都
In the CMD, the output processing of text content containing special characters has always been quite a headache: If you want to be compatible with special characters, you usually enclose the content in quotes and then output it. But in this way, quotes will be added at the beginning and end of all output lines. If you are very concerned about the quotes after output, this solution cannot be implemented. However, except for this solution, it seems there are no other solutions that can perfectly solve this difficult problem. Note: For the perfect solution, please refer to the code of bjsh on floor 23 Recently, I have had a little free time and thought about this problem again for a while. After several modifications, there is Code 1, which is sent out for everyone to test: Code 1: @echo off :: Idea: Escape all special symbols and then output :: Restriction: The file to be processed cannot be enclosed in quotes; cd.>output.txt for /f "delims=" %%i in ('findstr /n .* test.txt') do ( set "str=%%i" call set "str=%%str::=%%" if defined str (call :output) else echo.>>output.txt ) start output.txt exit :output set "str=%str:^=^^%" set "str=%str:>=^>%" set "str=%str:<=^<%" set "str=%str:\|=^\|%" set "str=%str:&=^&%" set "str=%str:"=^"%" call echo.%%str%%>>output.txt goto :eof The modification from the code on floor 21 by bjsh is as follows. I personally think it is a relatively perfect solution: Code 2: @echo off cd.>output.txt for /f "delims=" %%i in ('findstr /n . test.txt') do ( set "str=%%i" call set "str=%%str::=%%" if defined str (call :output) else echo.>>output.txt ) start output.txt exit :output set "str=%str:^=^^%" set "str=%str:"=%" set "str=%str:>=^>%" set "str=%str:<=^<%" set "str=%str:&=^&%" set "str=%str:\|=^\|%" set "str=%str:="%" (call echo.%%str%%)>>output.txt goto :eof The most perfect code is as follows (from the code of bjsh on floor 23, I only made a small modification): Code 3: @echo off cd.>output.txt for /f "delims=" %%i in ('findstr /n . test.txt') do ( set "var=%%i" setlocal enabledelayedexpansion set var=!var::=! (echo.!var!)>>output.txt endlocal ) start output.txt The content of the test file test.txt (please note: one of the blank lines is composed of spaces): "aou"eo ;euou%^> ::::aeui :::E2uo alejou 3<o2io\| ^aue\|\|%ou !aue! aoue eou 2 !str!auoeu!ueo && euo 8 ueyi^^^^aueuo2 ~ ! @ # $ % ^ & ( () " ok " No " <>nul set ok=^ Analysis of Code 1 and Code 2 is as follows: ① According to the general idea, when referencing variables in the for statement, the setlocal enabledelayedexpansion statement is used to enable variable delay function. However, this function has a fatal flaw: when the string to be processed contains exclamation marks, the exclamation mark pairs and all the strings between them will be replaced with empty. So, Code 1 and Code 2 abandon the setlocal solution and use the solution of calling a sub-process; ② If the text to be processed contains an odd number of quotes, the echo.%str%>>output.txt statement will be wrong. So directly replace the quotes with special invisible characters and then output (that is, the black box displayed in the code); Thanks to the test of lxmxn and the analysis of bjsh; ③ In the :output sub-process, the sentence set "str=%str:^=^^%" must be placed before all replacement sentences, otherwise, ^ will be repeatedly replaced, resulting in inaccurate results; Thanks to the test of lxmxn; ④ The ordinary for statement will ignore the line content starting with a semicolon and also ignore blank lines. So, the findstr .* test.txt statement is used to display all lines (including blank lines); delims=: will discard all colons at the beginning of the line. So, the statement call set "str=%%str::=%%" is used to avoid this situation; ⑤ In the statement (call echo.%%str%%)>>output.txt, the dot after echo cannot be omitted. Otherwise, when the line content is a space, the output content will display the current state of echo; Using the call statement is to be compatible with lines with quotes; Using parentheses is to correctly handle the single numbers 1-9 separated by spaces at the end of the line. ⑥ The reason why the :output tag segment does not use the statement for %%i in (^^ ^> ^< ^\| ^&) do call set "str=%%str:%%i=^%%i%%" is because this replacement statement cannot replace correctly. It seems that the call delay mechanism in the for statement is indeed a bit confusing. Regarding Code 3, due to limited level, I can only make a superficial and vague analysis: In this code, the variable delay function is used to completely obtain special characters and terminate the variable delay at an appropriate time to avoid the problem that the string is recognized as a variable reference due to excessive variable delay. In fact, this is still the CMD preprocessing mechanism at work. It is worth noting that the position of the setlocal statement cannot be swapped with the set statement, otherwise, the exclamation mark will still be recognized as a variable reference symbol and thus be discarded. [ Last edited by namejm on 2007-8-14 at 09:27 PM* ]

Floor2　lxmxn	Posted 2007-05-17 22:44
版主　Posts 4,938　Credits 11,386
Thanks to brother namejm for the original work. It's really gratifying that you can still care about the forum even with such a busy work schedule. After a short test, it was found that there are no output errors. [ Last edited by lxmxn on 2007-5-17 at 11:16 PM ]

Floor3　ieutk	Posted 2007-05-17 23:25
初级用户　Posts 48　Credits 107
Good stuff, but I can't understand it!

Floor4　zh159	Posted 2007-05-18 00:11
金牌会员　Posts 1,467　Credits 3,687
Except for quotes, the rest is actually easy

Floor5　bjsh	Posted 2007-05-18 08:54
银牌会员　Posts 621　Credits 2,000
I used to write something similar too; It's similar to Brother namejm's; But I didn't consider the situation where a whole line is all spaces; The last sentence is copied from namejm; The usage of echo.%var% is really wonderful. @echo off for /f "tokens=1 delims=" %%a in ('findstr /n .* test.txt') do set "var=%%a" & call :change goto :eof :change set "var=%var:^=^^%" set "var=%var:>=^>%" set "var=%var:<=^<%" set "var=%var:&=^&%" set "var=%var:\|=^\|%" set "var=%var:*:=%" echo.%var% This has solved the "problem; Everyone also help test it;

Floor6　lxmxn	Posted 2007-05-18 12:36
版主　Posts 4,938　Credits 11,386
To bjsh: There is a small problem found in the test. Please see my test text below. test.txt ~ ! @ # $ % ^ & * ( () " ok " No " <>nul set ok=^ Screen display There should not be > here.

Floor7　bjsh	Posted 2007-05-18 12:53
银牌会员　Posts 621　Credits 2,000
Hey; the problem with three quotes; It's like echo fsdg"dsgsa"gds" >55.txt won't be written into 55.txt; I'll think about it again; Is it really necessary to write it as an invisible character.

Floor8　ttyp	Posted 2007-05-18 13:07
初级用户　Posts 84　Credits 180
The result is incorrect. aoueo ;euou%^> ::::aeui :::E2uo alejou 3<o2io\| ^aue\|\|%ou !aue! !str!auoeu!ueo && ueyi^^^^aueuo2 =^ The first line "becomes garbled, and the last line is added"

Floor9　lxmxn	Posted 2007-05-18 13:15
版主　Posts 4,938　Credits 11,386
To ttyp: The first line's " is deliberately turned into that special character by the original poster. Is your test text copied from the original poster's test text? It works perfectly fine on my end.

Floor10　ttyp	Posted 2007-05-18 13:21
初级用户　Posts 84　Credits 180
It's copied, didn't see clearly from the first line. But there's an extra line at the end which is strange. My system is W2K + SP4

Floor11　namejm	Posted 2007-05-18 13:26
荣誉版主　Posts 1,737　Credits 5,226　From 成都
Originally posted by bjsh at 2007-5-18 12:53: Oh, the problem with three quotation marks; Just like echo fsdg"dsgsa"gds" >55.txt It won't be written into 55.txt; I'll think about it again; Do I really have to write it as an invisible character? 　　After testing, it should be that when there are an odd number of quotation marks, this situation occurs. This is a major defect of the echo statement when the call statement calls a sub - procedure. Maybe only replacing it with other characters will work. 　　The code in the top floor has simplified some statements. The set "str=%~1" is completely unnecessary. Hehe, my thinking was too complicated. [ Last edited by namejm on 2007-5-18 at 01:47 PM ]

Floor12　ttyp	Posted 2007-05-18 13:27
初级用户　Posts 84　Credits 180
Oh, I see. It's because I've been testing the P processing and commented out EXIT.

Floor13　bjsh	Posted 2007-05-18 13:49
银牌会员　Posts 621　Credits 2,000
Originally posted by namejm at 2007-5-18 01:26 PM: After testing, it should be when the number of quotation marks is odd that this situation occurs. This is a major defect of the echo statement when the call statement calls a sub - procedure. Maybe only replacing it with other characters will work. ... Yes; it is when it is odd; echo dsg^"sdgsad^"gdgasdg^" >5.txt It can be done; In fact, "is also a special character; Revise again @echo off set var= for /f "delims=" %%a in ('findstr /n .* test.txt') do ( set "var=%%a" call set "var=%%var:"=%%" call :change ) goto :eof :change set "var=%var:^=^^%" set "var=%var:>=^>%" set "var=%var:<=^<%" set "var=%var:&=^&%" set "var=%var:\|=^\|%" set "var=%var:*:=%" if not defined var echo. & goto :eof call set "var=%%var:="%%" echo.%var% Test again to see what other problems there are

Floor14　bjsh	Posted 2007-05-18 14:08
银牌会员　Posts 621　Credits 2,000
c:\>set "var=~ ! @ # $ % ^ & * ( () " ok " No " <>nul" 此时不应有 >. c:\>for /f "delims=" %a in (test.txt) do set "var=%a" c:\>set "var=~ ! @ # $ % ^ & * ( () " ok " No " <>nul" c:\>set var var=~ ! @ # $ % ^ & * ( () " ok " No " <>nul Content of test.txt ~ ! @ # $ % ^ & * ( () " ok " No " <>nul Did you find the difference between the two? The former shows "此时不应有 >" (There should not be > here), while the latter just uses for to convert that string to %a and then assign it to var; no error occurs; Interesting!

Floor15　namejm	Posted 2007-05-18 14:09
荣誉版主　Posts 1,737　Credits 5,226　From 成都
When the number of quotation marks is odd, there will be no errors in the result of echo, but when the number of quotation marks is even, an error will occur. The solution is to change echo.%var% to call echo.%%var%%, but in this case, if the test content is ~!@#$%^&(()" ok " No " a>a, all the content after the last quotation mark will be discarded. The reason is currently unknown. ———————————————————————————————— The above remarks belong to the test code used being incorrect, resulting in incorrect conclusions. [ Last edited by namejm on 2007-5-18 at 02:53 PM* ]