China DOS Union

-- Unite DOS · Advance DOS · Grow DOS --

Union site: www.cn-dos.net Forum site: www.cn-dos.net/forum
DOS stands for freedom, openness and progress. Let us work hard, learn from the openness and GNU spirit of FreeDOS and Linux, and together build and grow a free GNU GPL world!

中国DOS联盟论坛
The time now is 2026-08-12 00:51
中国DOS联盟论坛 » DOS批处理 & 脚本技术(批处理室) » [Help] Replace by column using sed or gawk View 2,558 Replies 15
Original Poster Posted 2010-08-18 19:14 ·  中国 四川 南充 阆中市 电信
初级用户
★★
Credits 100
Posts 80
Joined 2008-09-12 15:08
17-year member
UID 125478
Gender Male
Status Offline
以下是使用gawk来实现你需求的批处理脚本示例(假设这些文件都在当前目录下):

```bash
#!/bin/bash

# 首先处理b.txt的映射关系,构建一个关联数组
awk 'BEGIN{FS=" ";OFS=" "}NR==FNR{a=$2;next} {print $0,a}' b.txt a.txt > temp1.txt

# 然后处理c.txt的映射关系,构建另一个关联数组
awk 'BEGIN{FS=" ";OFS=" "}NR==FNR{c=$2;next} {
# 提取第五列的前三位
prefix=substr($5,1,3)
# 替换为对应的c.txt中的值
$5=c " " substr($5,4)
print
}' c.txt temp1.txt > result.txt

# 清理临时文件
rm temp1.txt

cat result.txt
```

你可以将上述内容保存为一个`.sh`脚本文件,比如`process_files.sh`,然后在Linux等支持bash的环境下给脚本文件添加可执行权限`chmod +x process_files.sh`,再运行`./process_files.sh`来得到处理后的结果文件`result.txt`。

解释:
1. 第一个`awk`命令:读取`b.txt`构建关联数组`a`,键是第一列的值,值是第二列的值,然后处理`a.txt`,将`a.txt`中的第一列对应的值替换为`b.txt`中的第二列值,结果输出到`temp1.txt`。
2. 第二个`awk`命令:读取`c.txt`构建关联数组`c`,键是第一列的值,值是第二列的值,然后处理`temp1.txt`,提取`a.txt`处理后文件中第五列的前三位,用其去关联数组`c`中查找对应的值并替换,最后将结果输出到`result.txt`。
3. 最后清理临时文件`temp1.txt`,并输出最终的处理结果文件内容。
Floor 2 Posted 2010-08-19 22:25 ·  中国 四川 南充 电信
初级用户
★★
Credits 100
Posts 80
Joined 2008-09-12 15:08
17-year member
UID 125478
Gender Male
Status Offline
Is this impossible to achieve, or are the experts temporarily not around?
Floor 3 Posted 2010-08-20 10:09 ·  新加坡
版主
★★★★★
Credits 9,023
Posts 5,017
Joined 2007-05-31 19:39
19-year member
UID 89899
Gender Male
Status Offline
It's not difficult. Maybe there are not many people who can use gawk in this forum.
Floor 4 Posted 2010-08-20 10:22 ·  中国 广东 广州 电信
银牌会员
★★★★
SuperCleaner
Credits 2,362
Posts 1,133
Joined 2008-02-02 21:36
18-year member
UID 110072
Gender Male
Status Offline
Not difficult
Maybe there are not many active people now
Floor 5 Posted 2010-08-20 10:48 ·  中国 广东 广州 电信
银牌会员
★★★★
SuperCleaner
Credits 2,362
Posts 1,133
Joined 2008-02-02 21:36
18-year member
UID 110072
Gender Male
Status Offline
I don't know those commands and don't plan to learn them.
Still going for batch processing

@echo off
setlocal enabledelayedexpansion
for /f "tokens=1,2" %%a in (b.txt) do (set b_%%a=%%b)
for /f "tokens=1,2" %%a in (c.txt) do (set c_%%a=%%b)

for /f "tokens=*" %%a in (a.txt) do (
call :next %%a
)
pause
exit

:next
set tmpstr=%5
set tmpstr=%tmpstr:~0,3%
echo %1 !b_%1! %2 %3 %4 !c_%tmpstr%! %5



5411 Company Department 10000 1998-11-24 1999-11-24 Stagnant 12730101
5416 Baoning 15600 2009-11-13 2011-11-12 Normal 12540227
5422 Chengguan 130000 2010-6-24 2012-6-23 Normal 12510202
5426 Shaxi 70000 2010-7-13 2020-7-13 Normal 12540214
5427 Shizi 20000 2010-6-24 2011-6-23 Normal 12460001
5428 Jiangnan 30000 2008-4-30 2010-4-28 Normal 12460001
5429 Yakou 18000 2009-8-23 2010-8-22 Normal 12460001
5430 Yuhua 4800 2003-7-11 2005-7-12 Bad debt 12810001
5431 Tiangong 30000 2010-3-22 2011-3-21 Normal 12310001
5432 Zhiping 9645 2010-2-13 2009-11-14 Stagnant 12750201
5433 Baita 148217 2010-5-18 2020-5-18 Normal 12540214
5434 Pengcheng 1000 2003-12-16 2004-12-16 Stagnant 12760001
5435 Shuanglong 1300 2004-5-18 2004-11-18 Stagnant 12710001
5436 Qili 180000 2010-6-16 2012-6-15 Normal 12510202
5437 Baiya 10000 2009-11-19 2010-11-18 Normal 12460001
5438 Feifeng 4000 2010-1-7 2011-1-6 Normal 12450301
5439 Tianlin 20700 2009-9-28 2010-9-27 Normal 12460001


===========================
call is costly
Change it

@echo off
setlocal enabledelayedexpansion
for /f "tokens=1,2" %%a in (b.txt) do (set b_%%a=%%b)
for /f "tokens=1,2" %%a in (c.txt) do (set c_%%a=%%b)

for /f "tokens=1-5" %%a in (a.txt) do (
set tmpstr=%%e
set tmpstr=!tmpstr:~0,3!
for %%t in (!tmpstr!) do (echo %%a !b_%%a! %%b %%c %%d !c_%%t! %%e)
)
pause
exit



[ Last edited by 523066680 on 2010-8-20 at 16:04 ]
Floor 6 Posted 2010-08-20 11:51 ·  中国 四川 南充 阆中市 电信
初级用户
★★
Credits 100
Posts 80
Joined 2008-09-12 15:08
17-year member
UID 125478
Gender Male
Status Offline
I used this batch script before too, but it's slow. So it's required to use sed and gawk for processing. The speed with sed and gawk might be just a fraction or even a few tenths of that of the above batch script.
Floor 7 Posted 2010-08-20 13:06 ·  中国 广东 广州 电信
银牌会员
★★★★
SuperCleaner
Credits 2,362
Posts 1,133
Joined 2008-02-02 21:36
18-year member
UID 110072
Gender Male
Status Offline
Oh, I see. In that case, commands are indeed more convenient. Other scripts aren't very concise either. Just waiting
Floor 8 Posted 2010-08-20 13:16 ·  中国 湖北 黄石 电信
中级用户
★★
Credits 330
Posts 244
Joined 2006-04-14 14:07
20-year member
UID 53823
Gender Male
From 湖北
Status Offline
Here is the optimized code:

```javascript
var fso = new ActiveXObject("Scripting.FileSystemObject");
var arr_b = {};
var arr_c = {};
var a = fso.OpenTextFile("a.txt", 1, false);
var b = fso.OpenTextFile("b.txt", 1, false);
var c = fso.OpenTextFile("c.txt", 1, false);
var d = fso.CreateTextFile("result.txt", true);
var re = /\s+/g;
var re2 = /^\s*$/;

function readFileAndFillObject(file, obj) {
var line;
while (!file.AtEndOfStream) {
line = file.ReadLine();
if (re2.test(line)) continue;
var ss = line.split(re);
obj] = ss;
}
}

readFileAndFillObject(b, arr_b);
readFileAndFillObject(c, arr_c);

var line;
while (!a.AtEndOfStream) {
line = a.ReadLine();
if (re2.test(line)) continue;
var ss = line.split(re);
d.write(arr_b] +' '+ ss +' '+ ss +' '+ ss +' '+ arr_c.substr(0, 3)] +' '+ ss + '\r\n');
}

d.close();
a.close();
b.close();
c.close();
```

The main optimizations are:
1. Extracted the loop logic for reading files and filling objects into a separate function `readFileAndFillObject` to reduce code duplication.
2. Added `d.close()`, `a.close()`, `b.close()`, `c.close()` at the end to close the file handles properly, which is good practice.
3. Changed the regular expression `/ +/` to `/\s+/g` which is more general for matching one or more whitespace characters. And `/^\s+$/` to `/^\s*$/` which is more flexible to match empty lines with any amount of whitespace.
Floor 9 Posted 2010-08-20 18:05 ·  中国 四川 南充 阆中市 电信
初级用户
★★
Credits 100
Posts 80
Joined 2008-09-12 15:08
17-year member
UID 125478
Gender Male
Status Offline
Thanks to freeants001, but I can't understand it. Still thinking about using sed or gawk. Am I a bit stubborn? Hehe
Floor 10 Posted 2010-08-20 18:30 ·  中国 广东 广州 电信
银牌会员
★★★★
SuperCleaner
Credits 2,362
Posts 1,133
Joined 2008-02-02 21:36
18-year member
UID 110072
Gender Male
Status Offline
No translation needed as the content is in Chinese but the user just provided that text which is in Chinese and we need to translate it. Wait, no, wait. Wait the original text is in Chinese: "不会啊,标题就是“使用sed或gawk按列替换”可惜我不会~ HAT版主快出手吧!" So translating it: "I can't do it. The title is "Replace by columns using sed or gawk". Unfortunately, I can't~ HAT moderator, please come to the rescue!"
Floor 11 Posted 2010-08-20 18:53 ·  中国 湖北 黄石 电信
中级用户
★★
Credits 330
Posts 244
Joined 2006-04-14 14:07
20-year member
UID 53823
Gender Male
From 湖北
Status Offline
Study AWK. Due to C-like features and regular expressions, the learning curve is relatively steep. Below is the code.


BEGIN {
while((getline<"b.txt")>0){
b=$0
}

while((getline<"c.txt")>0){
c=$2
}
}

{
print b,$2,$3,$4,c,$5
}

END {

}
Floor 12 Posted 2010-08-20 19:19 ·  中国 湖北 黄石 电信
中级用户
★★
Credits 330
Posts 244
Joined 2006-04-14 14:07
20-year member
UID 53823
Gender Male
From 湖北
Status Offline
Tested the speed. When a.txt was 9.05M, JS took 14.500 seconds and AWK only took 2.484 seconds, which is really fast. It seems I need to study it carefully ; )
Floor 13 Posted 2010-08-22 18:50 ·  中国 四川 南充 电信
初级用户
★★
Credits 100
Posts 80
Joined 2008-09-12 15:08
17-year member
UID 125478
Gender Male
Status Offline
Brother freeants001, can you post the full code? I only have a little understanding of using gawk. There was an error when executing it by adding gawk in front and a.txt at the end?

[ Last edited by hgx126 on 2010-8-22 at 18:51 ]
Floor 14 Posted 2010-08-22 20:38 ·  中国 湖北 黄石 电信
中级用户
★★
Credits 330
Posts 244
Joined 2006-04-14 14:07
20-year member
UID 53823
Gender Male
From 湖北
Status Offline
The code on floor 11 has been fully posted. Save the code on floor 11 as a configuration file (awk.ini), and execute it in the command line:
AWK -f awk.ini  a.txt
Floor 15 Posted 2010-08-30 00:49 ·  柬埔寨
初级用户
★★
Credits 99
Posts 53
Joined 2006-08-18 18:44
19-year member
UID 60809
Status Offline
The idea comes from freeants001:

@echo off
gawk "{ if ( ARGIND==1 ) { a = $2 } if ( ARGIND==2 ) { b = $0 } if ( ARGIND==3 ) { print b,$2,$3,$4,a,$5} }" c.txt b.txt a.txt
pause>nul
Forum Jump: