When was the last time you used sed and/or awk?

Sadly, I've never come across either!

Neil

And rarely do they have non-cryptic names.

There's something for everyone in Linux, worth playing around with (say) a Pi's command line to explore, but finding all the interesting commands can take a lifetime. Just discovered the other day for the first time a command to search for human-readable text among the noise.. called strings

There's the obscure-sounding troff and groff, useful for converting text into printer format. And myriad other weird commands. nm is very cool, I used to use it a lot.. it searches binary files and tells you the names of all the functions inside.

One command I wanted, but never found (I'm sure it could be fashioned from sed/awk very easily from those in the know, but I wouldn't know how!) was a command to hunt rows of data that definitely contained text from a 'good list' but to definitely not include the lines from a 'bad list'. The reason being, it would be useful for quickly going through large log files, by first setting up the good and bad lists. The good list could contain (say) "ERROR" and "DEBUG" and the bad list could contain "LEVEL1" and then you'd see only errors or debug lines that were level 2 or higher (for instance). Since it was too hard to figure out with regexp and Linux tools, I did it in C++ using the normal string library.

Interesting fact from the 'AWK' Wiki site, AWK was created at Bell Labs in the 1970s,^{[6][better source needed]} and its name is derived from the surnames of its authors: Alfred Aho, Peter Weinberger, and Brian Kernighan.

I believe there were a couple more commands which were derived from initials of the developers.

Hi shabaz

I was going to write you something as way of 'pay back' for all the great blogs you've done, but I thought it seemed too simple a task and the existing tools must be able to do this already....

My solution so far is:

grep -f goodlist.txt dataset.txt | grep -vf badlist.txt dataset.txt

where -f is for fixed-string matching and -v is to get the inverse result, and the vertical line denotes the piping of data from one grep command into the other

This works better:

grep -f goodlist.txt dataset.txt | grep -vf badlist.txt

and you can add the -i option so the search isn't case sensitive:

grep -if goodlist.txt dataset.txt | grep -ivf badlist.txt dataset.txt

Hi Rod,

No way! Thanks so much! I'm looking forward to trying this, it will really help to create favourite settings for repeated debug while troubleshooting all sorts of code. Although, as bad luck would have it, currently I'm having the opposite problem of too little debug! - working with an API that mostly only gives an ok/error indication and not much else : ( Refuse to give up until halfway through tomorrow's holiday at least..

That type of library would need a real debugger instead of log files.

If it's C or C++ on Linux, I may be ble to help you set it up.

Unfortunately it's just serial comms to a black box essentially, but not giving up until tomorrow. Although, might try anything unrelated tomorrow just to end the long weekend on something that is finally more productive!

The 'strings' command is a handy tool for reading files that are not readable from a text editor.  You can view some of the contents of files such as a core file or a dump from a system.  quite handy.

That is a neat command. I did a fair bit of coding for a Solaris-based product in one of my first jobs, and although I didn't know about that particular command at the time, I grew to love and hate core-dumps.. one never really wants them, but if things do go wrong then I really did hope the customer was kind enough to save it : ) It would throw out development work by a week each time though.. unfortunately that's how long it would typically take me to get to the bottom of it (if possible) : (

Interesting.  My first gig with HP back in '99, although not me first job in Tech, was as a Back-Line Support Engineer for HP-UX.  Spent many a days having to troubleshoot folks systems where someone did something wrong and companies not backing up their systems even though they were a critical part of the company.  My favorite was getting calls from someone from a company who had no Unix experience, but since they were the only who knew how to type, they were tasked to make the call when something went south. Having to dictate commands to someone over the phone was a joy.

Later I went into a Storage group in HP and ended up spending like 6 years doing Test and Validation of Solaris against HP's high end storage arrays. HP at one point was even selling their servers with Solaris installed thinking they could take business away from Sun (at the time it was still Sun) where later HP would replace Solaris on the HP server with something like Windows or Linux not fully understanding the Solaris folks are like Apple folks and were not looking to give up Solaris for Windows or Linux.They called this "Sun Attack". HP got caught trying to sell Solaris Clusters without paying for Licensing and Safra Catz from Sun/Oracle shut the fun down.  I think it all got washed away at one point. Fun times.

Oddly, nearly 10 years after HP, I ended up working with Solaris again thinking it had died many moons ago.  It is still used in some dark corners of the globe, but slowing being decommissioned.

Interesting.  My first gig with HP back in '99, although not me first job in Tech, was as a Back-Line Support Engineer for HP-UX.  Spent many a days having to troubleshoot folks systems where someone did something wrong and companies not backing up their systems even though they were a critical part of the company.  My favorite was getting calls from someone from a company who had no Unix experience, but since they were the only who knew how to type, they were tasked to make the call when something went south. Having to dictate commands to someone over the phone was a joy.

Later I went into a Storage group in HP and ended up spending like 6 years doing Test and Validation of Solaris against HP's high end storage arrays. HP at one point was even selling their servers with Solaris installed thinking they could take business away from Sun (at the time it was still Sun) where later HP would replace Solaris on the HP server with something like Windows or Linux not fully understanding the Solaris folks are like Apple folks and were not looking to give up Solaris for Windows or Linux.They called this "Sun Attack". HP got caught trying to sell Solaris Clusters without paying for Licensing and Safra Catz from Sun/Oracle shut the fun down.  I think it all got washed away at one point. Fun times.

Oddly, nearly 10 years after HP, I ended up working with Solaris again thinking it had died many moons ago.  It is still used in some dark corners of the globe, but slowing being decommissioned.