Cfrtjryk

Suppose I have the following data file:

111 222 333

444 555 666

777 888 999

I'm able to calculate the sum per column with GNU Datamash like this:

cat foo | datamash -t  sum 1 sum 2 sum 3

1332 1665 1998

How would I do this with datamash if I didn't know the number of columns in my data file?

I'm asking because for example cut supports end of range symbols like - for its field selector.

cols=$( awk '{print NF; exit}' foo); cat foo | datamash -t  sum 1-$cols

or

cat foo | datamash -t  sum 1-$( awk '{print NF; exit}' foo)

datamash has a feature to specify column ranges, so calculate the number of columns and use that result as part of the range spec. In my example solution, I used awk to check only the first line of the file and exit, but you could use anything else that suits your fancy. datamash itself has a -check function whose output includes the number of columns, but in a format that would still need to be parsed for the specific number that's of interest to you.

I don't see an option to specify unknown range in datamash manual

Try this perl one-liner

$ perl -lane '$s[$_]+=$F[$_] for 0..$#F; END{print join " ", @s}' ip.txt

1332 1665 1998

• 
  -a option will auto split input line on whitespaces, results are saved in @F array


• 
  for 0..$#F to loop over the array, $#F gives index of last element


• 
  $s[$_]+=$F[$_] save the sum in @s array, by default initial value will be 0 in numeric context. $_ will have the index value for each iteration


• 
  END{print join " ", @s} after processing all input lines, print contents of @s array with space as separator

I don't know about datamash, but here is an awk solution:

$ awk '{ for( col=1; col<=NF; col++ ) { totals[col]+=$col } } END { for( col=0; col<length(totals); col++ ) {printf "%s ", totals[col]}; printf "n" } ' input

1332 1665 1998

To make that awk script more readable:

{      // execute on all records

  for( col=1; col<=NF; col++ ) { 

    totals[col]+=$col 

  }; 

} 

END {  // execute after all records processed

  for( col=0; col<length(totals); col++ ) {

    printf "%s ", totals[col]

  }; 

  printf "n";

} 

Using datamash and bash:

n=($(datamash -W check < foo)); datamash -W sum 1-${n[2]} < foo

Output:

1332    1665    1998

How it works:

1. datamash -W check < foo outputs the string "3 lines, 3 fields".




2. n=($(datamash -W check < foo)) loads that string into an array $n. We want the number of fields, which would be ${n[2]}.




3. datamash -W sum 1-${n[2]} < foo does the rest.

This can also be done with a POSIX shell, using a complex printf formatting string instead of an array, but it's gnarlier:

datamash -W sum 1-$(printf '%0.0s%0.0s%s%0.0s' $(datamash -W check < foo)) < foo

It can also be done with shell tools:

datamash -W sum 1-$(head -1 foo | wc -w) < foo