When was the last time you used sed and/or awk?

Sadly, I've never come across either!

Neil

And rarely do they have non-cryptic names.

There's something for everyone in Linux, worth playing around with (say) a Pi's command line to explore, but finding all the interesting commands can take a lifetime. Just discovered the other day for the first time a command to search for human-readable text among the noise.. called strings

There's the obscure-sounding troff and groff, useful for converting text into printer format. And myriad other weird commands. nm is very cool, I used to use it a lot.. it searches binary files and tells you the names of all the functions inside.

One command I wanted, but never found (I'm sure it could be fashioned from sed/awk very easily from those in the know, but I wouldn't know how!) was a command to hunt rows of data that definitely contained text from a 'good list' but to definitely not include the lines from a 'bad list'. The reason being, it would be useful for quickly going through large log files, by first setting up the good and bad lists. The good list could contain (say) "ERROR" and "DEBUG" and the bad list could contain "LEVEL1" and then you'd see only errors or debug lines that were level 2 or higher (for instance). Since it was too hard to figure out with regexp and Linux tools, I did it in C++ using the normal string library.

Interesting fact from the 'AWK' Wiki site, AWK was created at Bell Labs in the 1970s,^{[6][better source needed]} and its name is derived from the surnames of its authors: Alfred Aho, Peter Weinberger, and Brian Kernighan.

I believe there were a couple more commands which were derived from initials of the developers.

Hi shabaz

I was going to write you something as way of 'pay back' for all the great blogs you've done, but I thought it seemed too simple a task and the existing tools must be able to do this already....

My solution so far is:

grep -f goodlist.txt dataset.txt | grep -vf badlist.txt dataset.txt

where -f is for fixed-string matching and -v is to get the inverse result, and the vertical line denotes the piping of data from one grep command into the other

This works better:

grep -f goodlist.txt dataset.txt | grep -vf badlist.txt

and you can add the -i option so the search isn't case sensitive:

grep -if goodlist.txt dataset.txt | grep -ivf badlist.txt dataset.txt

This works better:

grep -f goodlist.txt dataset.txt | grep -vf badlist.txt

and you can add the -i option so the search isn't case sensitive:

grep -if goodlist.txt dataset.txt | grep -ivf badlist.txt dataset.txt

Hi Rod,

No way! Thanks so much! I'm looking forward to trying this, it will really help to create favourite settings for repeated debug while troubleshooting all sorts of code. Although, as bad luck would have it, currently I'm having the opposite problem of too little debug! - working with an API that mostly only gives an ok/error indication and not much else : ( Refuse to give up until halfway through tomorrow's holiday at least..

That type of library would need a real debugger instead of log files.

If it's C or C++ on Linux, I may be ble to help you set it up.

Unfortunately it's just serial comms to a black box essentially, but not giving up until tomorrow. Although, might try anything unrelated tomorrow just to end the long weekend on something that is finally more productive!