Cfrtjryk

I have a system running. It creates 1 folder for each day, naming it by date. Inside each folder, it stores videos from security cameras, naming the files by time. You'd have something like this:

The folders are uploaded to the cloud, file by file, as it is created. However, my local storage limit is 16GB.

This way, I've made a script (in bash) to keep deleting old files as storage reaches a certain percentage. This is what the script should do:

• Detects the storage %.
  
  
  
  
  • If the storage percentage is at 75%:
    
    
    
    
    • Count the number of folders in the output's root folder.
      
      
      
      
      • If more than one folder:
        
        
        
        
        • Go to the oldest folder and count files.
          
          
          
          
          • If it has files, delete the 10 oldest files.
          
          
          • If it is empty, go back to the output's root folder and delete the oldest empty folder.
          
          
          
          
        
        
        
        
      
      
      • If only one folder:
        
        
        
        
        • Go to the only folder and delete the 10 oldest files.
        
        
        
        
      
      
      
      
    
    
    
    
  
  
  
  


• The cron job will run the script every minute, but the script will only delete stuff if the main condition is met.

This is the script itself (https://github.com/andreluisos/motioneyeos/blob/master/storage_cleaner_script/storage_cleaner_script.sh):

#! /bin/sh



##### VARIABLES #####



RootFolder="/data/output/Camera1" #Must change to your default output folder. Originally, it's "/data/output/Camera1".

cd $RootFolder

FileCounter=$(find . -name "*.mkv" -o -name "*.avi" -o -name "*.swf" -o -name "*.flv" -o -name "*.mov" -o -name "*.mp4" | wc -l | xargs) #Must change, if the output format if different than the basic ones available originally on MotionEyeOS.

FolderCounter=$(find . -mindepth 1 -maxdepth 1 -type d | wc -l | xargs)

CurrentStorage=`df -h | grep -vE "^Filesystem|tmpfs|/tmp|/boot|/home|/dev/root" | awk '{print $5}' | cut -d'%' -f1`

StorageThreshold=75 #Define the storage % to trigger the script.



##### FUNCTIONS #####



function multiplefolders () {

  echo "More than one folder detected."

  cd `ls -Ft | grep '/$' | tail -1`

  echo "Entering the oldest folder..."

    if [ "$FileCounter" -eq 0 ];

    then

      echo "No Files detected."

      cd $RootFolder

      echo "Going back to the output's root folder..."

      rm -rf `ls -Ft | grep '/$' | tail -1`

      echo "Detecting and removing the oldest empty folder...";

    else

      ls -t | tail -10 | xargs rm

      echo "Deleting the oldest files...";

    fi

}



function singlefolder () {

  echo "Only one folder detected."

  echo "Entering the only folder..."

  cd `ls -Ft | grep '/$' | head -1`

  ls -t | tail -10 | xargs rm

  echo "Deleting the oldest files."

}



function checkfolders () {

  if [ "$FolderCounter" -gt 1 ];

  then

    multiplefolders

  fi



  if [ "$FolderCounter" -eq 1 ];

  then

    singlefolder

  fi



  if [ "$FolderCounter" -eq 0 ];

  then

    echo "No folders detected. Please check if the output folder's path is correct."

  fi

}



##### SCRIPT STARTER #####



if [ $CurrentStorage -ge $StorageThreshold ];

then

  checkfolders

else

  echo "Storage threshold not yet reached."

fi

Thing is that the script isn't (apparently) counting the number of files inside the oldest folder (when more then one folder is detected) correctly.

Instead of going back to the root folder and delete the oldest empty folder, it keeps deleting files from the newest folder (apparently).

In other words, when you have 2 folders (the oldest one empty and the newest one with videos in it), it should delete the oldest and empty folder, but I keep getting:

More than one folder detected.

Entering the oldest folder...

Deleting the oldest files...

I hope we don't need to bother about whether we have a single folder or multiple folders. if we remove the files w.r.t modified time.

Just check the Storage and remove the oldest 10 files in the folder

if [ $CurrentStorage -ge $StorageThreshold ];

then

  find $RootFolder -type f -printf '%T+ %pn' | sort  | head -n 10 | awk '{print $NF}' | xargs rm -f

else

  echo "Storage threshold not yet reached."

fi

• 
  -type f -printf '%T+ %pn' print the files with last modified timestamp.


• 
  sort to get the oldest file on top.


• 
  head -n 10 to get 10 oldest files.


• 
  awk '{print $NF}' to extract the file path.


• 
  xargs rm -f remove the files which are extracted.

For MAC:

 find $RootFolder -type f -print0 | xargs -0 ls -ltr | head -n 10 | awk '{print $NF}' | xargs rm -f

And an empty folder would hardly occupy space of 4Kb. If you would like to remove all empty folder except the latest, include below code.

  find $RootFolder -type d -empty -printf '%T+ %pn' | sort | head -n -1 | xargs rm -rf

Or

 ls -lt $RootFolder/* | awk -F ":" '/total 0/{print last}{last=$1}' | tail -n +2 | xargs rm -rf

• It will remove all empty folder except the latest.