Super User Stack Exchange
Super User Stack Exchange

Questions tagged [awk]

AWK is a text-processing language. It is mainly used to manipulate and process structured data, and to generate reports.

AWK is a programming language created by Alfred V. Aho, Peter J. Weinberger and Brian W. Kernighan for processing text-based data, whether from files or data streams.

AWK is an example of a programming language that extensively uses the string datatype, associative arrays (that is, arrays indexed by key words) and regular expressions. The power, terseness, and limitations of programs made ​​in AWK and sed scripts inspired Larry Wall to create the Perl language.

620 questions
177
votes
18 answers

Removing ANSI color codes from text stream

Examining the output from perl -e 'use Term::ANSIColor; print color "white"; print "ABC\n"; print color "reset";' in a text editor (e.g., vi) shows the following: ^[[37mABC ^[[0m How would one remove the ANSI color codes from the output file? I…
user001
  • 3,474
  • 7
  • 24
  • 32
141
votes
11 answers

How to get video duration in seconds?

How can I get video duration in seconds? What I've tried: ffmpeg -i file.flv 2>&1 | grep "Duration" Duration: 00:39:43.08, start: 0.040000, bitrate: 386 kb/s mediainfo file.flv | grep Duration Duration : 39mn 43s This what close, but it's not…
user2783132
  • 1,797
  • 3
  • 16
  • 22
55
votes
4 answers

awk, mawk, nawk, gawk... WHAT?

I've just started learning awk and I'm a little confused about all those versions around. Is there any "version" which is found on all Unix-like systems? Like, you know, plain vi? Does the standard awk support the -F option?
Trollhorn
51
votes
11 answers

sed: how to replace line if found or append to end of file if not found?

With a single input file that only contains comments (starting with #) and VARIABLE=value lines, is it possible to replace a value for a single variable if found and, otherwise, append the pair to the end of file if not found? My current method…
BlakBat
  • 1,258
  • 2
  • 12
  • 19
22
votes
6 answers

How to get the pid of a running process using a single command that parse the output of ps?

I am looking for a single line that does return the pid of a running process. Currently I have: ps -A -o pid,cmd|grep xxx|head -n 1 And this returns the fist pid, command. I need only the first number from the output and ignore the rest. I suppose…
sorin
  • 11,660
  • 20
  • 63
  • 73
19
votes
2 answers

replace nth occurence of string in each line of a text file

I have large text files with space delimited strings (2-5). The strings can contain "'" or "-". I'd like to replace say the second space with a pipe. What's the best way to go? Using sed I was thinking of this: sed -r 's/(^[a-z'-]+ [a-z'-]+\b)…
dnkb
  • 387
  • 1
  • 4
  • 11
17
votes
2 answers

How can I find my awk version?

If I want to know the version of awk I get the following: $ awk --version awk: not an option: --version Checking in man awk I see that my awk is mawk - pattern scanning and text processing language
fedorqui
  • 1,877
  • 1
  • 16
  • 36
15
votes
7 answers

How can I replace a newline with its escape sequence?

Using sed to make text that contains certain characters suitable for use in string literals is straightforward: sed "s/\\\\/\\\\\\\\/g" sed "s/\\\"/\\\\\\\"/g" But how can I do something similar with a text file containing newline characters?
Melab
  • 983
  • 6
  • 18
  • 34
15
votes
3 answers

How can I make a table of contents for a markdown document with Python/AWK/SED?

I have the following markdown document: Heading-a ========== ---text--- Heading-b ------------ --- text --- Heading-c ---------- --- text--- Heading-d ======= --- text---- Heading-e --- ... I would like to make a clickable table of…
Léo Léopold Hertz 준영
  • 5,686
  • 12
  • 68
  • 115
14
votes
7 answers

Piping tail -f into awk

I am trying to pipe tail -f into awk to monitor a logfile in realtime, but according to examples, there should be not problem but I can't get it to work. here is the command I'm running tail -f logfile.log | awk -F" " '{print $1, $2, $7, $8}' But…
gimpycpu
  • 241
  • 1
  • 3
  • 4
14
votes
2 answers

How can I identify non-ASCII characters from the shell?

Is there a simple way to print all non-ASCII characters and the line numbers on which they occur in a file using a command line utility such as grep, awk, perl, etc? I want to change the encoding of a text file from UTF-8 to ASCII, but before doing…
user001
  • 3,474
  • 7
  • 24
  • 32
14
votes
4 answers

How to pipe awk output (with periodic, continuous input) to output file?

I am trying to write a command that pipes the continuous output of a free command (run every second) to an awk command that parses a specific value (available free memory) and outputs this to a file with a timestamp. Here are my current attempts at…
Mark
  • 370
  • 1
  • 2
  • 9
14
votes
5 answers

Add to every end of line in Notepad++

I have a long text file gallery-dl -g -i w4b027.txt > gallery-dl -g -i a4b028.txt > gallery-dl -g -i b4b029.txt > gallery-dl -g -i c4b030.txt > gallery-dl -g -i d4b031.txt > gallery-dl -g -i w4b032.txt > gallery-dl -g -i w4b033.txt > gallery-dl -g…
Roxion
  • 344
  • 1
  • 2
  • 11
13
votes
6 answers

Randomly shuffle rows in a large text file

I have a text file of ~1GB with about 6k rows (each row is very long) and I need to randomly shuffle its rows. Is it possible? Possibly with awk?
ddmichael
  • 357
  • 1
  • 4
  • 14
13
votes
1 answer

How to use awk separator of single quote?

I'd like to use awk to pull out information from SQL output like the following: (count(distinct serverclass)='2') And need to extract the number, which is 2 in this example. I'd like to do something like the following, but cannot figure out how to…
WilliamKF
  • 7,778
  • 34
  • 103
  • 145
1
2 3
…
41 42