How to extract the IPs I need from a txt file? - China DOS Union Forum

To extract the content from document1.txt and save it to a new document2.txt with the IP:port and the class attribute, you can use a programming language like Python. Here's a simple Python example using regular expressions:

```python
import re

# Read the content from document1.txt
with open('document1.txt', 'r', encoding='utf-8') as file:
content = file.read()

# Use regular expression to find the relevant parts
pattern = r'<a title=".*?" onMouseOver="s$'.*?'$" onMouseOut="d\()" class="(\w+)">(\d+\.\d+\.\d+\.\d+:\d+)</a>'
matches = re.findall(pattern, content)

# Write the results to document2.txt
with open('document2.txt', 'w', encoding='utf-8') as file:
for match in matches:
file.write(f"{match}----{match}\n")
```

This code uses regular expressions to find the IP:port pairs and their corresponding class attributes, then writes them to document2.txt. Make sure the Python script is in the same directory as document1.txt.

It is better to use sed for handling this kind of content, or ask classmates who are familiar with sed commands to do it.

sed -r "s/>[^<>]*<\/a>/&\n/g" 1.txt|sed -r "s/.*class=\x22([A-Z]{1})\x22>([^<>]*)<\/a>$/\2--\1/;/--([A-Z]{1})$/!d"|more>2.txt

Just a matter of tailoring, not considering too much matching

『Poster』: How to extract the content I want from a txt file?

</center><il><li><a title="AU" onMouseOver="s('AU')" onMouseOut="d()" class="D">144.140.22.190:80</a></li><li><a title="KR" onMouseOver="s('KR')" onMouseOut="d()" class="D">125.248.244.131:8080</a></li><li><a title="IN" onMouseOver="s('IN')" onMouseOut="d()" class="D">202.53.13.10:8080</a></li><li><a title="PH" onMouseOver="s('PH')" onMouseOut="d()" class="B">125.212.37.150:8080</a></li><li><a title="CN" onMouseOver="s('CN')" onMouseOut="d()" class="B">220.181.31.44:3128</a></li><li><a title="CO" onMouseOver="s('CO')" onMouseOut="d()" class="D">200-91-243-90-host.ifx.net.co:3128</a></li><li><a title="US" onMouseOver="s('US')" onMouseOut="d()" class="D">216.133.248.229:80</a></li><li><a title="US" onMouseOver="s('US')" onMouseOut="d()" class="D">216.133.248.228:80</a></li><li><a title="US" onMouseOver="s('US')" onMouseOut="d()" class="D">216.133.248.226:80</a></li><li><a title="JP" onMouseOver="s('JP')" onMouseOut="d()" class="D">neptun.ium.ne.jp:8094</a></li><li><a title="TR" onMouseOver="s('TR')" onMouseOut="d()" class="D">195.175.37.71:8080</a></li><li><a title="NL" onMouseOver="s('NL')" onMouseOut="d()" class="D">213.227.149.165:3128</a></li><li><a title="US" onMouseOver="s('US')" onMouseOut="d()" class="D">216.133.248.227:80</a></li></il><center>

The above content (all in the same line, not the multiple lines you see now) is document 1.txt. Please teach me how I can extract from 1.txt
144.140.22.190:80
xxx.xxx.xxx.xxx:xx
xxx.xxx.xxx.xxx:xx
...
...
...
xxx.xxx.xxx.xxx:xx
to a newly created another 2.txt?

It's best to also extract the type attribute after class, for example like this
144.140.22.190:80----D
xxx.xxx.xxx.xxx:xx
xxx.xxx.xxx.xxx:xx
220.181.31.44:3128----B
...
...
xxx.xxx.xxx.xxx:xx

Thanks in advance!

Just saw it, March 8th! I don't know where it went^


Set objFSO = CreateObject("Scripting.FileSystemObject") 

Set objText = objFSO.OpenTextFile("D:\Desktop\1.txt", 1)
inputstr = objText.ReadAll
objText.Close
outstr=replace(inputstr,"</a>",vbcrlf)
dim tep
tep=split(outstr,vbcrlf)
outstr=empty
dim i
for i=0 to ubound(tep)-2
outstr=outstr & right(tep(i),len(tep(i))-instrRev(tep(i),">")+3) & vbcrlf
next
outstr=replace(outstr,""">","--")


Set objText = objFSO.OpenTextFile("D:\Desktop\2.txt", 2,True)
objText.Write outstr
objText.Close

Result:

D--144.140.22.190:80
D--125.248.244.131:8080
D--202.53.13.10:8080
B--125.212.37.150:8080
B--220.181.31.44:3128
D--200-91-243-90-host.ifx.net.co:3128
D--216.133.248.229:80
D--216.133.248.228:80
D--216.133.248.226:80
D--neptun.ium.ne.jp:8094
D--195.175.37.71:8080
D--213.227.149.165:3128
D--216.133.248.227:80

In the recycle bin. I don't know how it happened.

Because the original title of the thread starter was very vague, and it was impossible to quickly know the general content of the post from the title, so it was moved to the recycle bin. After the thread starter modified the title, it has met the specifications, so it is moved back and related threads are merged.

Floor 4 is VBS? Floor 3's sed doesn't seem to work yet~~~~bat doesn't work? Went busy with business in the afternoon, didn't have time to come to the forum! Thanks to you several for your help here!

The following code can extract IP - formatted records, but it cannot extract the type attribute:


@echo off
setlocal enabledelayedexpansion

cls
for /f "delims=" %%i in (1.txt) do (
    set "str=%%i"
    set "str=!str:"=!"
    call :pickup "!str!"
)
pause
exit

:pickup
for /f "tokens=1* delims=<>" %%i in (%1) do (
    echo %%i|findstr "^\.">nul&&echo %%i
    set "str=%%j"
    if defined str call :pickup "!str!"
)
goto :eof

It's really useful for the format of XXX.XXX.XXX.XXX:XX.
Received it.
Thank you namejm!

Borrowing the code from namejm, you can extract records in IP format and type attributes:


@echo off
setlocal enabledelayedexpansion

for /f "delims=" %%i in (test.txt) do (
    set "str=%%i"
    set "str=!str:"=!"
    call :pickup "!str!"
)
pause>nul

:pickup
for /f "tokens=1* delims=<" %%i in (%1) do (
    echo "%%i"|findstr "class">nul && (
        for /f "tokens=1,2 delims=>" %%a in ("%%i") do (
           set class=%%a
           set class=!class:~-1!
           set IP=%%b
           echo !class!--!IP!
          )
        )
    set "str=%%j"
    if defined str call :pickup "!str!"
)

goto :eof

Result:


D--144.140.22.190:80
D--125.248.244.131:8080
D--202.53.13.10:8080
B--125.212.37.150:8080
B--220.181.31.44:3128
D--200-91-243-90-host.ifx.net.co:3128
D--216.133.248.229:80
D--216.133.248.228:80
D--216.133.248.226:80
D--neptun.ium.ne.jp:8094
D--195.175.37.71:8080
D--213.227.149.165:3128
D--216.133.248.227:80

This question was asked before by someone else, but that was in multiple lines.

[ Last edited by clonecd on 2007-3-9 at 04:32 PM ]

@sed "s/class/\n&/g;s/<\/a>/&\n/g" 1.txt|sed "/:/!d;s/.*\x22$.*$\x22>$\+$<.*/\2----\1/">2.txt
The above does not include 200-91-243-90-host.ifx.net.co:3128----D
If you need this line, use the following code
sed "s/class/\n&/g;s/<\/a>/&\n/g" 1.txt|sed "/:/!d;s/.*\x22$.*$\x22>$\+$<.*/\2----\1/">2.txt

[ Last edited by clonecd on 2007-3-9 at 04:40 PM ]

The one above doesn't seem to work well. Is there a general method that can extract all IP addresses in the following formats from any document:
XXX.XXX.XXX.XXX:XX
XXX.XXX.XXX.XX:XXXX
XX.XXX.XX.XXX:XXX
XX.XXX.XXX.XX:XXXX
Or the format:
XXX.XX.XXX.XXX--XX
XXX.XXX.XXX.XXX--XXXX
XX.XXX.XX.XXX--XXX
XX.XXX.XXX.XX--XX
? In the same line/not in the same line?

[ Last edited by 3391617 on 2007-3-9 at 10:23 AM ]

if defined str call :pickup "!str!"

Can anyone help explain this?

I saw in if /?:

If command extensions are enabled, IF will change as follows:

IF string1 compare-op string2 command
IF CMDEXTVERSION number command
IF DEFINED variable command

If an environment variable is defined, the effect of the DEFINED condition is the same as EXISTS, except that it takes an environment variable and returns true.

If the environment variable is not defined, then the subsequent command is not executed???

[ Last edited by xycoordinate on 2007-3-9 at 10:51 AM ]

Originally posted by 3391617 at 2007-3-9 23:18:
The above one doesn't seem to work well~
Is there any way to extract all content in the following formats from any document:
XXX.XXX.XXX.XXX:XX
XXX.XXX.XXX.XX:XXXX
XX.XXX.XX.XXX:XXX
XX.XXX.XXX.XX:XXXX
or format: ...

The code on floor 12 is written for the conditions you provided on floor 1. It's completely possible to use sed to handle the additional conditions you provided on floor 13. You can study sed by yourself and then research it.

Floor1　3391617	Posted 2007-03-09 02:41
初级用户　Posts 56　Credits 116
To extract the content from document1.txt and save it to a new document2.txt with the IP:port and the class attribute, you can use a programming language like Python. Here's a simple Python example using regular expressions: ```python import re # Read the content from document1.txt with open('document1.txt', 'r', encoding='utf-8') as file: content = file.read() # Use regular expression to find the relevant parts pattern = r'<a title=".?" onMouseOver="s\('.?'\)" onMouseOut="d\()" class="(\w+)">(\d+\.\d+\.\d+\.\d+:\d+)</a>' matches = re.findall(pattern, content) # Write the results to document2.txt with open('document2.txt', 'w', encoding='utf-8') as file: for match in matches: file.write(f"{match}----{match}\n") ``` This code uses regular expressions to find the IP:port pairs and their corresponding class attributes, then writes them to document2.txt. Make sure the Python script is in the same directory as document1.txt.

Floor2　ccwan	Posted 2007-03-09 02:50
金牌会员　Posts 1,160　Credits 2,725　From 河北廊坊
It is better to use sed for handling this kind of content, or ask classmates who are familiar with sed commands to do it.

Floor3　vkill	Posted 2007-03-09 03:22
金牌会员　Posts 1,744　Credits 4,103　From 甘肃.临泽
sed -r "s/>[^<>]<\/a>/&\n/g" 1.txt\|sed -r "s/.class=\x22([A-Z]{1})\x22>([^<>]*)<\/a>$/\2--\1/;/--([A-Z]{1})$/!d"\|more>2.txt Just a matter of tailoring, not considering too much matching

Floor4　slore	Posted 2007-03-09 03:23
铂金会员　Posts 2,478　Credits 5,212
『Poster』: How to extract the content I want from a txt file? </center><il><li><a title="AU" onMouseOver="s('AU')" onMouseOut="d()" class="D">144.140.22.190:80</a></li><li><a title="KR" onMouseOver="s('KR')" onMouseOut="d()" class="D">125.248.244.131:8080</a></li><li><a title="IN" onMouseOver="s('IN')" onMouseOut="d()" class="D">202.53.13.10:8080</a></li><li><a title="PH" onMouseOver="s('PH')" onMouseOut="d()" class="B">125.212.37.150:8080</a></li><li><a title="CN" onMouseOver="s('CN')" onMouseOut="d()" class="B">220.181.31.44:3128</a></li><li><a title="CO" onMouseOver="s('CO')" onMouseOut="d()" class="D">200-91-243-90-host.ifx.net.co:3128</a></li><li><a title="US" onMouseOver="s('US')" onMouseOut="d()" class="D">216.133.248.229:80</a></li><li><a title="US" onMouseOver="s('US')" onMouseOut="d()" class="D">216.133.248.228:80</a></li><li><a title="US" onMouseOver="s('US')" onMouseOut="d()" class="D">216.133.248.226:80</a></li><li><a title="JP" onMouseOver="s('JP')" onMouseOut="d()" class="D">neptun.ium.ne.jp:8094</a></li><li><a title="TR" onMouseOver="s('TR')" onMouseOut="d()" class="D">195.175.37.71:8080</a></li><li><a title="NL" onMouseOver="s('NL')" onMouseOut="d()" class="D">213.227.149.165:3128</a></li><li><a title="US" onMouseOver="s('US')" onMouseOut="d()" class="D">216.133.248.227:80</a></li></il><center> The above content (all in the same line, not the multiple lines you see now) is document 1.txt. Please teach me how I can extract from 1.txt 144.140.22.190:80 xxx.xxx.xxx.xxx:xx xxx.xxx.xxx.xxx:xx ... ... ... xxx.xxx.xxx.xxx:xx to a newly created another 2.txt? It's best to also extract the type attribute after class, for example like this 144.140.22.190:80----D xxx.xxx.xxx.xxx:xx xxx.xxx.xxx.xxx:xx 220.181.31.44:3128----B ... ... xxx.xxx.xxx.xxx:xx Thanks in advance! Just saw it, March 8th! I don't know where it went^ Set objFSO = CreateObject("Scripting.FileSystemObject") Set objText = objFSO.OpenTextFile("D:\Desktop\1.txt", 1) inputstr = objText.ReadAll objText.Close outstr=replace(inputstr,"</a>",vbcrlf) dim tep tep=split(outstr,vbcrlf) outstr=empty dim i for i=0 to ubound(tep)-2 outstr=outstr & right(tep(i),len(tep(i))-instrRev(tep(i),">")+3) & vbcrlf next outstr=replace(outstr,""">","--") Set objText = objFSO.OpenTextFile("D:\Desktop\2.txt", 2,True) objText.Write outstr objText.Close Result: D--144.140.22.190:80 D--125.248.244.131:8080 D--202.53.13.10:8080 B--125.212.37.150:8080 B--220.181.31.44:3128 D--200-91-243-90-host.ifx.net.co:3128 D--216.133.248.229:80 D--216.133.248.228:80 D--216.133.248.226:80 D--neptun.ium.ne.jp:8094 D--195.175.37.71:8080 D--213.227.149.165:3128 D--216.133.248.227:80

Floor5　ccwan	Posted 2007-03-09 03:41
金牌会员　Posts 1,160　Credits 2,725　From 河北廊坊
In the recycle bin. I don't know how it happened.

Floor6　namejm	Posted 2007-03-09 03:51
荣誉版主　Posts 1,737　Credits 5,226　From 成都
Because the original title of the thread starter was very vague, and it was impossible to quickly know the general content of the post from the title, so it was moved to the recycle bin. After the thread starter modified the title, it has met the specifications, so it is moved back and related threads are merged.

Floor7　3391617	Posted 2007-03-09 06:11
初级用户　Posts 56　Credits 116
Floor 4 is VBS? Floor 3's sed doesn't seem to work yet~~~~bat doesn't work? Went busy with business in the afternoon, didn't have time to come to the forum! Thanks to you several for your help here!

Floor8　namejm	Posted 2007-03-09 06:59
荣誉版主　Posts 1,737　Credits 5,226　From 成都
The following code can extract IP - formatted records, but it cannot extract the type attribute: @echo off setlocal enabledelayedexpansion cls for /f "delims=" %%i in (1.txt) do ( set "str=%%i" set "str=!str:"=!" call :pickup "!str!" ) pause exit :pickup for /f "tokens=1* delims=<>" %%i in (%1) do ( echo %%i\|findstr "^\.">nul&&echo %%i set "str=%%j" if defined str call :pickup "!str!" ) goto :eof

Floor9　3391617	Posted 2007-03-09 08:09
初级用户　Posts 56　Credits 116
It's really useful for the format of XXX.XXX.XXX.XXX:XX. Received it. Thank you namejm!

Floor10　youxi01	Posted 2007-03-09 12:00
高级用户　Posts 247　Credits 846　From 湖南==》广东
Borrowing the code from namejm, you can extract records in IP format and type attributes: @echo off setlocal enabledelayedexpansion for /f "delims=" %%i in (test.txt) do ( set "str=%%i" set "str=!str:"=!" call :pickup "!str!" ) pause>nul :pickup for /f "tokens=1* delims=<" %%i in (%1) do ( echo "%%i"\|findstr "class">nul && ( for /f "tokens=1,2 delims=>" %%a in ("%%i") do ( set class=%%a set class=!class:~-1! set IP=%%b echo !class!--!IP! ) ) set "str=%%j" if defined str call :pickup "!str!" ) goto :eof Result: D--144.140.22.190:80 D--125.248.244.131:8080 D--202.53.13.10:8080 B--125.212.37.150:8080 B--220.181.31.44:3128 D--200-91-243-90-host.ifx.net.co:3128 D--216.133.248.229:80 D--216.133.248.228:80 D--216.133.248.226:80 D--neptun.ium.ne.jp:8094 D--195.175.37.71:8080 D--213.227.149.165:3128 D--216.133.248.227:80

Floor11　clonecd	Posted 2007-03-09 15:14
初级用户　Posts 46　Credits 94
This question was asked before by someone else, but that was in multiple lines. [ Last edited by clonecd on 2007-3-9 at 04:32 PM ]

Floor12　clonecd	Posted 2007-03-09 16:30
初级用户　Posts 46　Credits 94
@sed "s/class/\n&/g;s/<\/a>/&\n/g" 1.txt\|sed "/:/!d;s/.\x22\(.\)\x22>\(\+\)<./\2----\1/">2.txt The above does not include 200-91-243-90-host.ifx.net.co:3128----D If you need this line, use the following code sed "s/class/\n&/g;s/<\/a>/&\n/g" 1.txt\|sed "/:/!d;s/.\x22\(.\)\x22>\(\+\)<./\2----\1/">2.txt [ Last edited by clonecd on 2007-3-9 at 04:40 PM ]

Floor13　3391617	Posted 2007-03-09 23:18
初级用户　Posts 56　Credits 116
The one above doesn't seem to work well. Is there a general method that can extract all IP addresses in the following formats from any document: XXX.XXX.XXX.XXX:XX XXX.XXX.XXX.XX:XXXX XX.XXX.XX.XXX:XXX XX.XXX.XXX.XX:XXXX Or the format: XXX.XX.XXX.XXX--XX XXX.XXX.XXX.XXX--XXXX XX.XXX.XX.XXX--XXX XX.XXX.XXX.XX--XX ? In the same line/not in the same line? [ Last edited by 3391617 on 2007-3-9 at 10:23 AM ]

Floor14　xycoordinate	Posted 2007-03-09 23:43
中级用户　Posts 228　Credits 493　From 安徽
if defined str call :pickup "!str!" Can anyone help explain this? I saw in if /?: If command extensions are enabled, IF will change as follows: IF string1 compare-op string2 command IF CMDEXTVERSION number command IF DEFINED variable command If an environment variable is defined, the effect of the DEFINED condition is the same as EXISTS, except that it takes an environment variable and returns true. If the environment variable is not defined, then the subsequent command is not executed??? [ Last edited by xycoordinate on 2007-3-9 at 10:51 AM ]

Floor15　clonecd	Posted 2007-03-09 23:56
初级用户　Posts 46　Credits 94
Originally posted by 3391617 at 2007-3-9 23:18: The above one doesn't seem to work well~ Is there any way to extract all content in the following formats from any document: XXX.XXX.XXX.XXX:XX XXX.XXX.XXX.XX:XXXX XX.XXX.XX.XXX:XXX XX.XXX.XXX.XX:XXXX or format: ... The code on floor 12 is written for the conditions you provided on floor 1. It's completely possible to use sed to handle the additional conditions you provided on floor 13. You can study sed by yourself and then research it.