Cfrtjryk

No. There is nothing special about a binary or non-binary file. You can use heuristics like 'contains only characters in 0x01–0x7F', but that'll call text files with non-ASCII characters binary files, and unlucky binary files text files.

Now, once you've ignored that...

zip files

If its coming from your Windows user as a zip file, the zip format supports marking files as either binary or text in the archive itself. You can use unzip's -a option to pay attention to this and convert. Of course, see the first paragraph for why this may not be a good idea (the zip program may have guessed wrong when it made the archive).

zipinfo will tell you which files are binary (b) or text (t) in its zipfile listing.

other files

The file command will look at a file and try to identify it. In particular, you'll probably find its -i (output MIME type) option useful; only convert files with type text/*

A general solution to only process non-binary files in bash using file -b --mime-encoding:

while IFS= read -d '' -r file; do

  [[ "$(file -b --mime-encoding "$file")" = binary ]] &&

    { echo "Skipping   $file."; continue; }



  echo "Processing $file."



  # ...



done < <(find . -type f -print0)

I contacted the author of the file utility and he added a nifty -00 paramter in version 5.26 (released 2016-04-16, is e.g. in current Arch and Ubuntu 16.10) which prints fileresult for multiple files fed to it at once, this way you can do e.g.:

find . -type f -exec file -00 --mime-encoding {} + |

  awk 'BEGIN{ORS=RS=""}{if(NR%2)f=$0;else if(!/binary/)print f}' | …

(The awk part is to filter out every file that isn't non-binary. ORS is the output separator.)

Can be also used in a loop of course:

while IFS= read -d '' -r file; do



  echo "Processing $file."



  # ...



done < <(find . -type f -exec file -00 --mime-encoding {} + |

  awk 'BEGIN{ORS=RS=""}{if(NR%2)f=$0;else if(!/binary/)print f}')

Based of this and the previous I created a little bash script for filtering out binary files which utilizes the new method using the -00 parameter of file in newer versions of it and falls back to the previous method on older versions:

#!/bin/bash



# Expects files as arguments and returns the ones that do

# not appear to be binary files as a zero-separated list.

#

# USAGE:

#   filter_binary_files.sh [FILES...]

#

# EXAMPLE:

#   find . -type f -mtime +5 -exec ./filter_binary_files.sh {} + | xargs -0 ...

# 



[[ $# -eq 0 ]] && exit



if [[ "$(file -v)" =~ file-([1-9][0-9]|[6-9]|5.([3-9][0-9]|2[6-9])) ]]; then

  file -00 --mime-encoding -- "$@" |

    awk 'BEGIN{ORS=RS=""}{if(NR%2)f=$0;else if(!/binary/)print f}'

else

  for f do

    [[ "$(file -b --mime-encoding -- "$f")" != binary ]] &&

      printf '%s' "$f"

  done

fi

Or here a more POSIX-y one, but it requires support for sort -V:

#!/bin/sh



# Expects files as arguments and returns the ones that do

# not appear to be binary files as a zero-separated list.

#

# USAGE:

#   filter_binary_files.sh [FILES...]

#

# EXAMPLE:

#   find . -type f -mtime +5 -exec ./filter_binary_files.sh {} + | xargs -0 ...

# 



[ $# -eq 0 ] && exit



if [ "$(printf '%sn' 'file-5.26' "$(file -v | head -1)" | sort -V)" = 

    'file-5.26' ]; then

  file -00 --mime-encoding -- "$@" |

    awk 'BEGIN{ORS=RS=""}{if(NR%2)f=$0;else if(!/binary/)print f}'

else

  for f do

    [ "$(file -b --mime-encoding -- "$f")" != binary ] &&

      printf '%s' "$f"

  done

fi

Cas's answer is good, but it assumes sane filenames; in particular it is assumed that filenames will not contain newlines.

There's no good reason to make this assumption here, since it is quite simple (and actually cleaner in my opinion) to handle that case correctly as well:

find . -type f -exec sh -c 'file "$1" | grep -q "ASCII text"' sh {} ; -exec flip -u {} ;

The find command only makes use of POSIX-specified features. Using -exec to run arbitrary commands as boolean tests is simple, robust (handles odd filenames correctly), and more portable than -print0.

In fact, all parts of the command are specified by POSIX except for flip.

Note that file doesn't guarantee accuracy of the results it returns. However, in practice grepping for "ASCII text" in its output is quite reliable.

(It might miss some text files perhaps, but is very very unlikely to incorrectly identify a binary file as "ASCII text" and mangle it—so we are erring on the side of caution.)

The accepted answer didn't find all of them for me. Here is an example using grep's -I to ignore binaries, and ignoring all hidden files...

find . -type f -not -path '*/.*' -exec grep -Il '.' {} ; | xargs -L 1 echo 

Here it is in use in a practical application: dos2unix

https://unix.stackexchange.com/a/365679/112190

Hope that helps.

find . -type f -exec grep -I -q . {} ; -print

This will find all regular files (-type f) in the current directory (or below) that grep thinks are non-empty and non-binary.

It uses grep -I to distinguish between binary and non-binary files. The -I flag and will cause grep to exit with a non-zero exit status when it detects that a file is binary. A "binary" file is, according to grep, a file that contains character outside the printable ASCII range.

The -q option to grep will cause it to quit with a zero exit status if the given pattern is found, without emitting any data. The pattern that we use is a single dot, which will match any character.

If the file is found to be non-binary and if it contains at least one character, the name of the file is printed.

If you feel brave, you can plug your flip -u into it as well:

find . -type f -exec grep -I -q . {} ; -print -exec flip -u {} ;

Try this :

find . -type f -print0 | xargs -0 -r grep -Z -L -U '[^         -~]' | xargs -0 -r flip -u

Where the argument of grep '[^ -~]' is '[^<tab><space>-~]'.

If you type it on a shell command line, type Ctrl+V before Tab.
In an editor, there should be no problem.

• 
  '[^<tab><space>-~]' will match any character which is not ASCII text (carriage returns are ignore by grep).


• 
  -L will print only the filename of files who does not match


• 
  -Z will output filenames separated with a null character (for xargs -0)

Alternate solution:

The dos2unix command will convert line endings from Windows CRLF to Unix LF, and automatically skip binary files. I apply it recursively using:

find . -type f -exec dos2unix {} ;

sudo find / ( -type f -and -path '*/git/*' -iname ‘README’ ) -exec grep -liI '100644|100755' {} ; -exec flip -u {} ;

i.( -type f -and -path '*/git/*' -iname ‘README’ ): searches for files within a path containing the name git and file with name README. If you know any specific folder and filename to search for it will be useful.

ii.-exec command runs a command on the file name generated by find

iii.; indicates end of command

iv.{} is the output of the file/foldername found from the previous find search

v.Multiple commands can be run on subsequently. By appending -exec "command" ; such as with -exec flip -u ;

vii.grep

1.-l lists the name of the file

2.-I searches only non-binary files

3.-q quiet output

4.'100644|100755' searches for either 100644 or 100755 within the file found. if found it then runs flip -u. | is the or operator for grep. 

you can clone this test directory and try it out: https://github.com/alphaCTzo7G/stackexchange/tree/master/linux/findSolution204092017

more detailed answer here: https://github.com/alphaCTzo7G/stackexchange/blob/master/linux/findSolution204092017/README.md