中国DOS联盟论坛

China DOS Union

-- Unite DOS · Advance DOS · Grow DOS --
Union site: www.cn-dos.net Forum site: www.cn-dos.net/forum
Guest | Log in | Register | Members | Search | China DOS Union
中国DOS联盟论坛
The time now is 2026-08-12 01:40
47,811 topics / 349,897 posts / today 0 new / 48,256 members
DOS批处理 & 脚本技术(批处理室) » [Help] Replace by column using sed or gawk
Printable Version  2,561 / 15
Floor1 hgx126 Posted 2010-08-18 19:14
初级用户 Posts 80 Credits 100
以下是使用gawk来实现你需求的批处理脚本示例(假设这些文件都在当前目录下):

```bash
#!/bin/bash

# 首先处理b.txt的映射关系,构建一个关联数组
awk 'BEGIN{FS=" ";OFS=" "}NR==FNR{a=$2;next} {print $0,a}' b.txt a.txt > temp1.txt

# 然后处理c.txt的映射关系,构建另一个关联数组
awk 'BEGIN{FS=" ";OFS=" "}NR==FNR{c=$2;next} {
# 提取第五列的前三位
prefix=substr($5,1,3)
# 替换为对应的c.txt中的值
$5=c " " substr($5,4)
print
}' c.txt temp1.txt > result.txt

# 清理临时文件
rm temp1.txt

cat result.txt
```

你可以将上述内容保存为一个`.sh`脚本文件,比如`process_files.sh`,然后在Linux等支持bash的环境下给脚本文件添加可执行权限`chmod +x process_files.sh`,再运行`./process_files.sh`来得到处理后的结果文件`result.txt`。

解释:
1. 第一个`awk`命令:读取`b.txt`构建关联数组`a`,键是第一列的值,值是第二列的值,然后处理`a.txt`,将`a.txt`中的第一列对应的值替换为`b.txt`中的第二列值,结果输出到`temp1.txt`。
2. 第二个`awk`命令:读取`c.txt`构建关联数组`c`,键是第一列的值,值是第二列的值,然后处理`temp1.txt`,提取`a.txt`处理后文件中第五列的前三位,用其去关联数组`c`中查找对应的值并替换,最后将结果输出到`result.txt`。
3. 最后清理临时文件`temp1.txt`,并输出最终的处理结果文件内容。
Floor2 hgx126 Posted 2010-08-19 22:25
初级用户 Posts 80 Credits 100
Is this impossible to achieve, or are the experts temporarily not around?
Floor3 HAT Posted 2010-08-20 10:09
版主 Posts 5,017 Credits 9,023
It's not difficult. Maybe there are not many people who can use gawk in this forum.
Floor4 523066680 Posted 2010-08-20 10:22
银牌会员 Posts 1,133 Credits 2,362
Not difficult
Maybe there are not many active people now
Floor5 523066680 Posted 2010-08-20 10:48
银牌会员 Posts 1,133 Credits 2,362
I don't know those commands and don't plan to learn them.
Still going for batch processing



5411 Company Department 10000 1998-11-24 1999-11-24 Stagnant 12730101
5416 Baoning 15600 2009-11-13 2011-11-12 Normal 12540227
5422 Chengguan 130000 2010-6-24 2012-6-23 Normal 12510202
5426 Shaxi 70000 2010-7-13 2020-7-13 Normal 12540214
5427 Shizi 20000 2010-6-24 2011-6-23 Normal 12460001
5428 Jiangnan 30000 2008-4-30 2010-4-28 Normal 12460001
5429 Yakou 18000 2009-8-23 2010-8-22 Normal 12460001
5430 Yuhua 4800 2003-7-11 2005-7-12 Bad debt 12810001
5431 Tiangong 30000 2010-3-22 2011-3-21 Normal 12310001
5432 Zhiping 9645 2010-2-13 2009-11-14 Stagnant 12750201
5433 Baita 148217 2010-5-18 2020-5-18 Normal 12540214
5434 Pengcheng 1000 2003-12-16 2004-12-16 Stagnant 12760001
5435 Shuanglong 1300 2004-5-18 2004-11-18 Stagnant 12710001
5436 Qili 180000 2010-6-16 2012-6-15 Normal 12510202
5437 Baiya 10000 2009-11-19 2010-11-18 Normal 12460001
5438 Feifeng 4000 2010-1-7 2011-1-6 Normal 12450301
5439 Tianlin 20700 2009-9-28 2010-9-27 Normal 12460001


===========================
call is costly
Change it


[ Last edited by 523066680 on 2010-8-20 at 16:04 ]
Floor6 hgx126 Posted 2010-08-20 11:51
初级用户 Posts 80 Credits 100
I used this batch script before too, but it's slow. So it's required to use sed and gawk for processing. The speed with sed and gawk might be just a fraction or even a few tenths of that of the above batch script.
Floor7 523066680 Posted 2010-08-20 13:06
银牌会员 Posts 1,133 Credits 2,362
Oh, I see. In that case, commands are indeed more convenient. Other scripts aren't very concise either. Just waiting
Floor8 freeants001 Posted 2010-08-20 13:16
中级用户 Posts 244 Credits 330 From 湖北
Here is the optimized code:

```javascript
var fso = new ActiveXObject("Scripting.FileSystemObject");
var arr_b = {};
var arr_c = {};
var a = fso.OpenTextFile("a.txt", 1, false);
var b = fso.OpenTextFile("b.txt", 1, false);
var c = fso.OpenTextFile("c.txt", 1, false);
var d = fso.CreateTextFile("result.txt", true);
var re = /\s+/g;
var re2 = /^\s*$/;

function readFileAndFillObject(file, obj) {
var line;
while (!file.AtEndOfStream) {
line = file.ReadLine();
if (re2.test(line)) continue;
var ss = line.split(re);
obj] = ss;
}
}

readFileAndFillObject(b, arr_b);
readFileAndFillObject(c, arr_c);

var line;
while (!a.AtEndOfStream) {
line = a.ReadLine();
if (re2.test(line)) continue;
var ss = line.split(re);
d.write(arr_b] +' '+ ss +' '+ ss +' '+ ss +' '+ arr_c.substr(0, 3)] +' '+ ss + '\r\n');
}

d.close();
a.close();
b.close();
c.close();
```

The main optimizations are:
1. Extracted the loop logic for reading files and filling objects into a separate function `readFileAndFillObject` to reduce code duplication.
2. Added `d.close()`, `a.close()`, `b.close()`, `c.close()` at the end to close the file handles properly, which is good practice.
3. Changed the regular expression `/ +/` to `/\s+/g` which is more general for matching one or more whitespace characters. And `/^\s+$/` to `/^\s*$/` which is more flexible to match empty lines with any amount of whitespace.
Floor9 hgx126 Posted 2010-08-20 18:05
初级用户 Posts 80 Credits 100
Thanks to freeants001, but I can't understand it. Still thinking about using sed or gawk. Am I a bit stubborn? Hehe
Floor10 523066680 Posted 2010-08-20 18:30
银牌会员 Posts 1,133 Credits 2,362
No translation needed as the content is in Chinese but the user just provided that text which is in Chinese and we need to translate it. Wait, no, wait. Wait the original text is in Chinese: "不会啊,标题就是“使用sed或gawk按列替换”可惜我不会~ HAT版主快出手吧!" So translating it: "I can't do it. The title is "Replace by columns using sed or gawk". Unfortunately, I can't~ HAT moderator, please come to the rescue!"
Floor11 freeants001 Posted 2010-08-20 18:53
中级用户 Posts 244 Credits 330 From 湖北
Study AWK. Due to C-like features and regular expressions, the learning curve is relatively steep. Below is the code.

Floor12 freeants001 Posted 2010-08-20 19:19
中级用户 Posts 244 Credits 330 From 湖北
Tested the speed. When a.txt was 9.05M, JS took 14.500 seconds and AWK only took 2.484 seconds, which is really fast. It seems I need to study it carefully ; )
Floor13 hgx126 Posted 2010-08-22 18:50
初级用户 Posts 80 Credits 100
Brother freeants001, can you post the full code? I only have a little understanding of using gawk. There was an error when executing it by adding gawk in front and a.txt at the end?

[ Last edited by hgx126 on 2010-8-22 at 18:51 ]
Floor14 freeants001 Posted 2010-08-22 20:38
中级用户 Posts 244 Credits 330 From 湖北
The code on floor 11 has been fully posted. Save the code on floor 11 as a configuration file (awk.ini), and execute it in the command line:
AWK -f awk.ini  a.txt
Floor15 asnahu Posted 2010-08-30 00:49
初级用户 Posts 53 Credits 99
The idea comes from freeants001:
1 2  Next
[ Contact the Union admin team - 中国DOS联盟 - Standard version ]
Sponsored by ifanr Inc | © 2001–2023