China DOS Union

-- Unite DOS · Advance DOS · Grow DOS --

Union site: www.cn-dos.net Forum site: www.cn-dos.net/forum
DOS stands for freedom, openness and progress. Let us work hard, learn from the openness and GNU spirit of FreeDOS and Linux, and together build and grow a free GNU GPL world!

中国DOS联盟论坛
The time now is 2026-06-24 06:50
中国DOS联盟论坛 » DOS批处理 & 脚本技术(批处理室) » [Original] Batch Intelligent Download of Web Page Images [For Discussion] View 3,983 Replies 23
Floor 16 Posted 2008-03-17 00:48 ·  中国 安徽 马鞍山 电信
中级用户
★★
拟谷盗
Credits 312
Posts 108
Joined 2007-01-21 11:36
19-year member
UID 77238
Gender Male
Status Offline
### Part 1
The first part of the code is:
```
curl "http://220.181.38.82/m?ct=134217728&tn=baidusg,不得不爱  &word=mp3,http://www.yantaiblog.net/UploadFiles/2007-7/aGhmbGtmbGsx.mp3,,&si=%B2%BB%B5%C3%B2%BB%B0%AE;;%C5%CB%E7%E2%B0%D8;;51913;;51913&lm=16777216"

curl: (3) illegal character in range specification at pos 132
总感觉curl在某些方面有很多缺陷,为何出现这样的情况?
有时 url中 有 & 字符时必需使用在 & 前加转义字符,而有些时候却不需要,为何?

The translated text for the first part is:
The first code block is:
```
curl "http://220.181.38.82/m?ct=134217728&tn=baidusg,不得不爱  &word=mp3,http://www.yantaiblog.net/UploadFiles/2007-7/aGhmbGtmbGsx.mp3,,&si=%B2%BB%B5%C3%B2%BB%B0%AE;;%C5%CB%E7%E2%B0%D8;;51913;;51913&lm=16777216"

curl: (3) illegal character in range specification at pos 132
I always feel that curl has many defects in some aspects. Why does such a situation occur?
Sometimes when there are & characters in the URL, it is necessary to add an escape character before &, but sometimes it is not necessary. Why?

### Part 2
The second part of the code is:
curl http://list.mp3.baidu.com/topso/mp3topsong.html | sed "s/\x22/\xa/g;/baidump3/!d"

curl http://list.mp3.baidu.com/topso/mp3topsong.html | sed "s/\x22/\xa/g" | sed "/baidump3/!d"

这2个命令执行的结果为何不一样?

The translated text for the second part is:
The second code block is:
curl http://list.mp3.baidu.com/topso/mp3topsong.html | sed "s/\x22/\xa/g;/baidump3/!d"

curl http://list.mp3.baidu.com/topso/mp3topsong.html | sed "s/\x22/\xa/g" | sed "/baidump3/!d"

Why do the results of these two commands differ?
FLOSS
Floor 17 Posted 2008-03-17 13:53 ·  中国 浙江 杭州 电信
高级用户
★★
Credits 537
Posts 219
Joined 2007-08-04 09:43
18-year member
UID 94620
Gender Male
From 杭州--半山
Status Offline
I always feel that curl has many drawbacks in some aspects. Why does this happen?
Sometimes when there is an & character in the url, an escape character must be added before &, but sometimes it is not necessary. Why?
--------------------------
1 I think it should be related to the CMD.EXE syntax analysis. When cmd encounters &, it considers it as the end of a statement.


Why are the results of these two commands different?
--------------------------------------------
2. sed reads data line by line. The general process is to extract a line of data from curl, read the content and store it in the pattern space. After processing with sed "s/\x22/\xa/g", it seems to generate multiple lines. In fact, it is still stored in the pattern space as an unprocessed semi-finished product, and then processed by /baidump3/!d.
Anyway, remember
sed processes its input data line by line. The input may be in the form of pipes, files, communication in processes, etc.
The content of the pattern space is not treated in the same way as the input.

[ Last edited by junchen2 on 2008-3-17 at 01:54 PM ]
Floor 18 Posted 2008-03-18 16:24 ·  中国 甘肃 兰州 电信
金牌会员
★★★★
Credits 4,103
Posts 1,744
Joined 2006-01-20 13:00
20-year member
UID 49241
Gender Male
From 甘肃.临泽
Status Offline
Like the web page submitted by from can be solved with both wget and curl, just try both.
Floor 19 Posted 2008-03-18 19:02 ·  中国 北京 海淀区 联通
银牌会员
★★★
Credits 1,287
Posts 634
Joined 2007-05-02 15:06
19-year member
UID 87277
Gender Male
From cmd.exe
Status Offline
D:\>wget "http://mp3.baidu.com/u?u=http://www.yantaiblog.net/UploadFiles/2007-7/aGhmbGtmbGsx.mp3"
--17:52:54-- http://mp3.baidu.com/u?u=http://www.yantaiblog.net/UploadFiles/200
7-7/aGhmbGtmbGsx.mp3
=> `u@u=http:%2F%2Fwww.yantaiblog.net%2FUploadFiles%2F2007-7%2FaGhmbG
tmbGsx.mp3'
Resolving mp3.baidu.com... done.
Connecting to mp3.baidu.com:80... connected.
HTTP request sent, awaiting response... 302 Object moved
Location: http://www.yantaiblog.net/UploadFiles/2007-7/76386053.mp3
--17:52:55-- http://www.yantaiblog.net/UploadFiles/2007-7/76386053.mp3
=> `76386053.mp3'
Resolving www.yantaiblog.net... done.
Connecting to www.yantaiblog.net:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 4,505,113

0% 19,433 9.56K/s ETA 07:38
Floor 20 Posted 2008-03-18 19:12 ·  中国 浙江 温州 电信
中级用户
★★
Credits 458
Posts 196
Joined 2006-10-05 12:04
19-year member
UID 64614
Status Offline
Can the LZ explain under what circumstances the keepalive option of curl should be used and how to use it? Thanks
Floor 21 Posted 2008-03-18 21:07 ·  中国 安徽 马鞍山 电信
中级用户
★★
拟谷盗
Credits 312
Posts 108
Joined 2007-01-21 11:36
19-year member
UID 77238
Gender Male
Status Offline
Originally posted by zerocq at 2008-3-18 19:12:
Can the landlord explain when and how to use the keepalive option of curl?

Thank you

I haven't used the keepalive method
Reference

curl --manual

http://curl.haxx.se/docs/manpage.html

--keepalive-time <seconds>

This option sets the time a connection needs to remain idle before sending keepalive probes and the time between individual keepalive probes. It is currently effective on operating systems offering the TCP_KEEPIDLE and TCP_KEEPINTVL socket options (meaning Linux, recent AIX, HP-UX and more). This option has no effect if --no-keepalive is used. (Added in 7.18.0)

If this option is used multiple times, the last occurrence sets the amount.

[ Last edited by ngd on 2008-4-19 at 05:15 PM ]
FLOSS
Floor 22 Posted 2008-03-18 21:46 ·  中国 安徽 马鞍山 电信
中级用户
★★
拟谷盗
Credits 312
Posts 108
Joined 2007-01-21 11:36
19-year member
UID 77238
Gender Male
Status Offline
Originally posted by knoppix7 at 2008-3-18 19:02:
D:\>wget "http://mp3.baidu.com/u?u=http://www.yantaiblog.net/UploadFiles/2007-7/
aGhmbGtmbGsx.mp3"
--17:52:54-- http://mp3.baidu.com/u?u=http://www.yantaiblog.net/UploadFiles/2 ... Strange! wget 1.10.2 D:\>wget "http://mp3.baidu.com/u?u=http://www.yantaiblog.net/UploadFiles/2007-7/ aGhmbGtmbGsx.mp3" --20:32:27-- http://mp3.baidu.com/u?u=http://www.yantaiblog.net/UploadFiles/200 7-7/aGhmbGtmbGsx.mp3 => `u@u=http%3A%2F%2Fwww.yantaiblog.net%2FUploadFiles%2F2007-7%2FaGhm bGtmbGsx.mp3' Resolving mp3.baidu.com... 220.181.38.70 Connecting to mp3.baidu.com|220.181.38.70|:80... connected. HTTP request sent, awaiting response... 302 Found Location: http://www.baidu.com/search/error.html --20:32:28-- http://www.baidu.com/search/error.html => `error.html' Resolving www.baidu.com... 220.181.6.6, 220.181.37.55 Connecting to www.baidu.com|220.181.6.6|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 2,699 (2.6K) 100% 2,699 --.--K/s 20:32:38 (52.95 MB/s) - `error.html' saved
FLOSS
Floor 23 Posted 2008-03-19 19:41 ·  中国 北京 海淀区 联通
银牌会员
★★★
Credits 1,287
Posts 634
Joined 2007-05-02 15:06
19-year member
UID 87277
Gender Male
From cmd.exe
Status Offline
D:\>wget --help
GNU Wget 1.8.2, a non-interactive network retriever.
Floor 24 Posted 2008-08-08 11:38 ·  中国 广东 深圳 电信
初级用户
★★
Credits 153
Posts 103
Joined 2008-03-27 19:38
18-year member
UID 114210
Gender Male
Status Offline
curl -d "UserName=***&Password=******" http://www.netyi.net/jsLoginStatus.aspx Why can't I log in?
Forum Jump: