How to escape spaces etc in passed variable, for system call to cp in awk












4














This problem has enough layers of complexity to make simple escaping characters difficult for me. I have a bash script which, for a large part, has an embedded awk script that reads a file delimited by semicolons. In this text file, one of the fields is a directory path to some JPEGs. I'd like to copy the JPEGs somewhere else, but the original directory path has spaces in it (and could potentially have other strings that could do damage). I can't refer to the directory in question with single quotes because this stops the awk interpreter, and double quotes turns it into a literal string in awk.



I'm using gawk 4.1.1. Here's some awk code of what I'm trying to accomplish:



imageDirectory = $1;

cpJpegVariable="cp -r " imageDirectory " assets";

#screws up if imageDirectory has spaces and other naughty characters

system(cpJpegVariable);





shell-script awk quoting







share|improve this question





























    4














    This problem has enough layers of complexity to make simple escaping characters difficult for me. I have a bash script which, for a large part, has an embedded awk script that reads a file delimited by semicolons. In this text file, one of the fields is a directory path to some JPEGs. I'd like to copy the JPEGs somewhere else, but the original directory path has spaces in it (and could potentially have other strings that could do damage). I can't refer to the directory in question with single quotes because this stops the awk interpreter, and double quotes turns it into a literal string in awk.



    I'm using gawk 4.1.1. Here's some awk code of what I'm trying to accomplish:



    imageDirectory = $1;
    
cpJpegVariable="cp -r " imageDirectory " assets";
    
#screws up if imageDirectory has spaces and other naughty characters
    
system(cpJpegVariable);





    shell-script awk quoting







    share|improve this question



























      4












      4








      4


      1





      This problem has enough layers of complexity to make simple escaping characters difficult for me. I have a bash script which, for a large part, has an embedded awk script that reads a file delimited by semicolons. In this text file, one of the fields is a directory path to some JPEGs. I'd like to copy the JPEGs somewhere else, but the original directory path has spaces in it (and could potentially have other strings that could do damage). I can't refer to the directory in question with single quotes because this stops the awk interpreter, and double quotes turns it into a literal string in awk.



      I'm using gawk 4.1.1. Here's some awk code of what I'm trying to accomplish:



      imageDirectory = $1;
      
cpJpegVariable="cp -r " imageDirectory " assets";
      
#screws up if imageDirectory has spaces and other naughty characters
      
system(cpJpegVariable);





      shell-script awk quoting







      share|improve this question















      This problem has enough layers of complexity to make simple escaping characters difficult for me. I have a bash script which, for a large part, has an embedded awk script that reads a file delimited by semicolons. In this text file, one of the fields is a directory path to some JPEGs. I'd like to copy the JPEGs somewhere else, but the original directory path has spaces in it (and could potentially have other strings that could do damage). I can't refer to the directory in question with single quotes because this stops the awk interpreter, and double quotes turns it into a literal string in awk.



      I'm using gawk 4.1.1. Here's some awk code of what I'm trying to accomplish:



      imageDirectory = $1;
      
cpJpegVariable="cp -r " imageDirectory " assets";
      
#screws up if imageDirectory has spaces and other naughty characters
      
system(cpJpegVariable);





      shell-script awk quoting




      shell-script awk quoting






      share|improve this question















      share|improve this question













      share|improve this question




      share|improve this question








      edited Dec 20 '18 at 0:15









      Rui F Ribeiro

      39k1479130




      39k1479130










      asked Aug 5 '14 at 8:11









      Escher

      283615




      283615






















          3 Answers
          3






          active

          oldest

          votes


















          5














          You can use:



          awk '...
          
    cpJpegVariable="cp -r '''" imageDirectory "''' assets";
          
    ...'


          (note that ' doesn't need escaping for awk, but you need ''' for the shell). So when awk expands variable cpJpegVariable, it looks like:



          cp -r 'file_contain space' assets


          With that, you can avoid problem with all special characters, except ' itself. If imageDirectory may contain single quote characters, you can escape them with gsub(). Example:



          awk '{gsub("'''","'''"'''"",imageDirectory)}1'





          share|improve this answer























          • Worked perfectly, and embarassingly obivous in hindsight. Thanks very much.
            – Escher
            Aug 5 '14 at 8:34






          • 1




            Note that while it fixes the problem with space, tab, ;, |, & and many other characters, it does not fix it for filenames that contain backtick, backslash, $ or " characters (or whose name starts with -).
            – Stéphane Chazelas
            Aug 5 '14 at 8:46










          • @StéphaneChazelas: If change " to ', problem maybe fixed?
            – cuonglm
            Aug 5 '14 at 8:48










          • To ' you mean? Then, the only problematic character becomes ' itself (which you can escape with a gsub command).
            – Stéphane Chazelas
            Aug 5 '14 at 9:08










          • @StéphaneChazelas: Thanks for suggestion, fixed!
            – cuonglm
            Aug 5 '14 at 9:21



















          1














          If you are passing an Awk variable to system, you need to shell quote it:



          function quote(str,   d, m, x, y, z) {
          
  d = "47"; m = split(str, x, d)
          
  for (y in x) z = z d x[y] d (y < m ? "\" d : "")
          
  return z
          
}


          Example:



          system(sprintf("cp -r %s assets", quote($1)))


          Source






          share|improve this answer





























            0














            Executing external commands is meant for the shell, not for awk:



            while read image_dir __; do
            
    cp -r "$image_dir" assets
            
done < file





            share|improve this answer





















            • It is sometimes necessary or useful to execute a system command based on information that was read by awk. This is why system() exists.
              – Escher
              Aug 5 '14 at 10:30










            • system() is not consistent with all types of input. It's awfully difficult to sanitize it.
              – konsolebox
              Aug 5 '14 at 10:39











            Your Answer








            StackExchange.ready(function() {
            var channelOptions = {
            tags: "".split(" "),
            id: "106"
            };
            initTagRenderer("".split(" "), "".split(" "), channelOptions);

            StackExchange.using("externalEditor", function() {
            // Have to fire editor after snippets, if snippets enabled
            if (StackExchange.settings.snippets.snippetsEnabled) {
            StackExchange.using("snippets", function() {
            createEditor();
            });
            }
            else {
            createEditor();
            }
            });

            function createEditor() {
            StackExchange.prepareEditor({
            heartbeatType: 'answer',
            autoActivateHeartbeat: false,
            convertImagesToLinks: false,
            noModals: true,
            showLowRepImageUploadWarning: true,
            reputationToPostImages: null,
            bindNavPrevention: true,
            postfix: "",
            imageUploader: {
            brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
            contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
            allowUrls: true
            },
            onDemand: true,
            discardSelector: ".discard-answer"
            ,immediatelyShowMarkdownHelp:true
            });


            }
            });














            draft saved

            draft discarded


















            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f148454%2fhow-to-escape-spaces-etc-in-passed-variable-for-system-call-to-cp-in-awk%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown

























            3 Answers
            3






            active

            oldest

            votes








            3 Answers
            3






            active

            oldest

            votes









            active

            oldest

            votes






            active

            oldest

            votes









            5














            You can use:



            awk '...
            
    cpJpegVariable="cp -r '''" imageDirectory "''' assets";
            
    ...'


            (note that ' doesn't need escaping for awk, but you need ''' for the shell). So when awk expands variable cpJpegVariable, it looks like:



            cp -r 'file_contain space' assets


            With that, you can avoid problem with all special characters, except ' itself. If imageDirectory may contain single quote characters, you can escape them with gsub(). Example:



            awk '{gsub("'''","'''"'''"",imageDirectory)}1'





            share|improve this answer























            • Worked perfectly, and embarassingly obivous in hindsight. Thanks very much.
              – Escher
              Aug 5 '14 at 8:34






            • 1




              Note that while it fixes the problem with space, tab, ;, |, & and many other characters, it does not fix it for filenames that contain backtick, backslash, $ or " characters (or whose name starts with -).
              – Stéphane Chazelas
              Aug 5 '14 at 8:46










            • @StéphaneChazelas: If change " to ', problem maybe fixed?
              – cuonglm
              Aug 5 '14 at 8:48










            • To ' you mean? Then, the only problematic character becomes ' itself (which you can escape with a gsub command).
              – Stéphane Chazelas
              Aug 5 '14 at 9:08










            • @StéphaneChazelas: Thanks for suggestion, fixed!
              – cuonglm
              Aug 5 '14 at 9:21
















            5














            You can use:



            awk '...
            
    cpJpegVariable="cp -r '''" imageDirectory "''' assets";
            
    ...'


            (note that ' doesn't need escaping for awk, but you need ''' for the shell). So when awk expands variable cpJpegVariable, it looks like:



            cp -r 'file_contain space' assets


            With that, you can avoid problem with all special characters, except ' itself. If imageDirectory may contain single quote characters, you can escape them with gsub(). Example:



            awk '{gsub("'''","'''"'''"",imageDirectory)}1'





            share|improve this answer























            • Worked perfectly, and embarassingly obivous in hindsight. Thanks very much.
              – Escher
              Aug 5 '14 at 8:34






            • 1




              Note that while it fixes the problem with space, tab, ;, |, & and many other characters, it does not fix it for filenames that contain backtick, backslash, $ or " characters (or whose name starts with -).
              – Stéphane Chazelas
              Aug 5 '14 at 8:46










            • @StéphaneChazelas: If change " to ', problem maybe fixed?
              – cuonglm
              Aug 5 '14 at 8:48










            • To ' you mean? Then, the only problematic character becomes ' itself (which you can escape with a gsub command).
              – Stéphane Chazelas
              Aug 5 '14 at 9:08










            • @StéphaneChazelas: Thanks for suggestion, fixed!
              – cuonglm
              Aug 5 '14 at 9:21














            5












            5








            5






            You can use:



            awk '...
            
    cpJpegVariable="cp -r '''" imageDirectory "''' assets";
            
    ...'


            (note that ' doesn't need escaping for awk, but you need ''' for the shell). So when awk expands variable cpJpegVariable, it looks like:



            cp -r 'file_contain space' assets


            With that, you can avoid problem with all special characters, except ' itself. If imageDirectory may contain single quote characters, you can escape them with gsub(). Example:



            awk '{gsub("'''","'''"'''"",imageDirectory)}1'





            share|improve this answer














            You can use:



            awk '...
            
    cpJpegVariable="cp -r '''" imageDirectory "''' assets";
            
    ...'


            (note that ' doesn't need escaping for awk, but you need ''' for the shell). So when awk expands variable cpJpegVariable, it looks like:



            cp -r 'file_contain space' assets


            With that, you can avoid problem with all special characters, except ' itself. If imageDirectory may contain single quote characters, you can escape them with gsub(). Example:



            awk '{gsub("'''","'''"'''"",imageDirectory)}1'






            share|improve this answer














            share|improve this answer



            share|improve this answer








            edited Aug 5 '14 at 10:44

























            answered Aug 5 '14 at 8:20









            cuonglm

            102k23201301




            102k23201301












            • Worked perfectly, and embarassingly obivous in hindsight. Thanks very much.
              – Escher
              Aug 5 '14 at 8:34






            • 1




              Note that while it fixes the problem with space, tab, ;, |, & and many other characters, it does not fix it for filenames that contain backtick, backslash, $ or " characters (or whose name starts with -).
              – Stéphane Chazelas
              Aug 5 '14 at 8:46










            • @StéphaneChazelas: If change " to ', problem maybe fixed?
              – cuonglm
              Aug 5 '14 at 8:48










            • To ' you mean? Then, the only problematic character becomes ' itself (which you can escape with a gsub command).
              – Stéphane Chazelas
              Aug 5 '14 at 9:08










            • @StéphaneChazelas: Thanks for suggestion, fixed!
              – cuonglm
              Aug 5 '14 at 9:21


















            • Worked perfectly, and embarassingly obivous in hindsight. Thanks very much.
              – Escher
              Aug 5 '14 at 8:34






            • 1




              Note that while it fixes the problem with space, tab, ;, |, & and many other characters, it does not fix it for filenames that contain backtick, backslash, $ or " characters (or whose name starts with -).
              – Stéphane Chazelas
              Aug 5 '14 at 8:46










            • @StéphaneChazelas: If change " to ', problem maybe fixed?
              – cuonglm
              Aug 5 '14 at 8:48










            • To ' you mean? Then, the only problematic character becomes ' itself (which you can escape with a gsub command).
              – Stéphane Chazelas
              Aug 5 '14 at 9:08










            • @StéphaneChazelas: Thanks for suggestion, fixed!
              – cuonglm
              Aug 5 '14 at 9:21
















            Worked perfectly, and embarassingly obivous in hindsight. Thanks very much.
            – Escher
            Aug 5 '14 at 8:34




            Worked perfectly, and embarassingly obivous in hindsight. Thanks very much.
            – Escher
            Aug 5 '14 at 8:34




            1




            1




            Note that while it fixes the problem with space, tab, ;, |, & and many other characters, it does not fix it for filenames that contain backtick, backslash, $ or " characters (or whose name starts with -).
            – Stéphane Chazelas
            Aug 5 '14 at 8:46




            Note that while it fixes the problem with space, tab, ;, |, & and many other characters, it does not fix it for filenames that contain backtick, backslash, $ or " characters (or whose name starts with -).
            – Stéphane Chazelas
            Aug 5 '14 at 8:46












            @StéphaneChazelas: If change " to ', problem maybe fixed?
            – cuonglm
            Aug 5 '14 at 8:48




            @StéphaneChazelas: If change " to ', problem maybe fixed?
            – cuonglm
            Aug 5 '14 at 8:48












            To ' you mean? Then, the only problematic character becomes ' itself (which you can escape with a gsub command).
            – Stéphane Chazelas
            Aug 5 '14 at 9:08




            To ' you mean? Then, the only problematic character becomes ' itself (which you can escape with a gsub command).
            – Stéphane Chazelas
            Aug 5 '14 at 9:08












            @StéphaneChazelas: Thanks for suggestion, fixed!
            – cuonglm
            Aug 5 '14 at 9:21




            @StéphaneChazelas: Thanks for suggestion, fixed!
            – cuonglm
            Aug 5 '14 at 9:21













            1














            If you are passing an Awk variable to system, you need to shell quote it:



            function quote(str,   d, m, x, y, z) {
            
  d = "47"; m = split(str, x, d)
            
  for (y in x) z = z d x[y] d (y < m ? "\" d : "")
            
  return z
            
}


            Example:



            system(sprintf("cp -r %s assets", quote($1)))


            Source






            share|improve this answer


























              1














              If you are passing an Awk variable to system, you need to shell quote it:



              function quote(str,   d, m, x, y, z) {
              
  d = "47"; m = split(str, x, d)
              
  for (y in x) z = z d x[y] d (y < m ? "\" d : "")
              
  return z
              
}


              Example:



              system(sprintf("cp -r %s assets", quote($1)))


              Source






              share|improve this answer
























                1












                1








                1






                If you are passing an Awk variable to system, you need to shell quote it:



                function quote(str,   d, m, x, y, z) {
                
  d = "47"; m = split(str, x, d)
                
  for (y in x) z = z d x[y] d (y < m ? "\" d : "")
                
  return z
                
}


                Example:



                system(sprintf("cp -r %s assets", quote($1)))


                Source






                share|improve this answer












                If you are passing an Awk variable to system, you need to shell quote it:



                function quote(str,   d, m, x, y, z) {
                
  d = "47"; m = split(str, x, d)
                
  for (y in x) z = z d x[y] d (y < m ? "\" d : "")
                
  return z
                
}


                Example:



                system(sprintf("cp -r %s assets", quote($1)))


                Source







                share|improve this answer












                share|improve this answer



                share|improve this answer










                answered Jan 15 '17 at 6:57









                Steven Penny

                1




                1























                    0














                    Executing external commands is meant for the shell, not for awk:



                    while read image_dir __; do
                    
    cp -r "$image_dir" assets
                    
done < file





                    share|improve this answer





















                    • It is sometimes necessary or useful to execute a system command based on information that was read by awk. This is why system() exists.
                      – Escher
                      Aug 5 '14 at 10:30










                    • system() is not consistent with all types of input. It's awfully difficult to sanitize it.
                      – konsolebox
                      Aug 5 '14 at 10:39
















                    0














                    Executing external commands is meant for the shell, not for awk:



                    while read image_dir __; do
                    
    cp -r "$image_dir" assets
                    
done < file





                    share|improve this answer





















                    • It is sometimes necessary or useful to execute a system command based on information that was read by awk. This is why system() exists.
                      – Escher
                      Aug 5 '14 at 10:30










                    • system() is not consistent with all types of input. It's awfully difficult to sanitize it.
                      – konsolebox
                      Aug 5 '14 at 10:39














                    0












                    0








                    0






                    Executing external commands is meant for the shell, not for awk:



                    while read image_dir __; do
                    
    cp -r "$image_dir" assets
                    
done < file





                    share|improve this answer












                    Executing external commands is meant for the shell, not for awk:



                    while read image_dir __; do
                    
    cp -r "$image_dir" assets
                    
done < file






                    share|improve this answer












                    share|improve this answer



                    share|improve this answer










                    answered Aug 5 '14 at 8:23









                    konsolebox

                    1,22799




                    1,22799












                    • It is sometimes necessary or useful to execute a system command based on information that was read by awk. This is why system() exists.
                      – Escher
                      Aug 5 '14 at 10:30










                    • system() is not consistent with all types of input. It's awfully difficult to sanitize it.
                      – konsolebox
                      Aug 5 '14 at 10:39


















                    • It is sometimes necessary or useful to execute a system command based on information that was read by awk. This is why system() exists.
                      – Escher
                      Aug 5 '14 at 10:30










                    • system() is not consistent with all types of input. It's awfully difficult to sanitize it.
                      – konsolebox
                      Aug 5 '14 at 10:39
















                    It is sometimes necessary or useful to execute a system command based on information that was read by awk. This is why system() exists.
                    – Escher
                    Aug 5 '14 at 10:30




                    It is sometimes necessary or useful to execute a system command based on information that was read by awk. This is why system() exists.
                    – Escher
                    Aug 5 '14 at 10:30












                    system() is not consistent with all types of input. It's awfully difficult to sanitize it.
                    – konsolebox
                    Aug 5 '14 at 10:39




                    system() is not consistent with all types of input. It's awfully difficult to sanitize it.
                    – konsolebox
                    Aug 5 '14 at 10:39


















                    draft saved

                    draft discarded




















































                    Thanks for contributing an answer to Unix & Linux Stack Exchange!


                    • Please be sure to answer the question. Provide details and share your research!

                    But avoid …



                    • Asking for help, clarification, or responding to other answers.

                    • Making statements based on opinion; back them up with references or personal experience.


                    To learn more, see our tips on writing great answers.





                    Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


                    Please pay close attention to the following guidance:


                    • Please be sure to answer the question. Provide details and share your research!

                    But avoid …



                    • Asking for help, clarification, or responding to other answers.

                    • Making statements based on opinion; back them up with references or personal experience.


                    To learn more, see our tips on writing great answers.




                    draft saved


                    draft discarded














                    StackExchange.ready(
                    function () {
                    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f148454%2fhow-to-escape-spaces-etc-in-passed-variable-for-system-call-to-cp-in-awk%23new-answer', 'question_page');
                    }
                    );

                    Post as a guest















                    Required, but never shown





















































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown

































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown







                    -

                    Popular posts from this blog

                    Morgemoulin

                    Image
                    Morgemoulin — miejscowość i gmina — Państwo   Francja Region Grand Est Departament Moza Okręg Verdun Kod INSEE 55357 Powierzchnia 6,82 km² Populacja  (1990) • liczba ludności 75 • gęstość 11 os./km² Kod pocztowy 55400 Położenie na mapie Francji Morgemoulin 49°14′N   5°35′E / 49,233333   5,583333 Portal Francja Morgemoulin – miejscowość i gmina we Francji, w regionie Grand Est, w departamencie Moza. Według danych na rok 1990 gminę zamieszkiwało 75 osób, a gęstość zaludnienia wynosiła 11 osób/km² (wśród 2335 gmin Lotaryngii Morgemoulin plasuje się na 970. miejscu pod względem liczby ludności, natomiast pod względem powierzchni na miejscu 848.). Bibliografia | Francuski urząd statystyczny ( fr. ) . p   •   d   •   e Gminy okręgu Verdun Abaucourt-Hautecourt • Aincreville • Ambly-sur-Meuse • Amel-sur-l'Étang • Ancemont • Arrancy-sur-Crusne • Aubréville • Autré...
                    Read more

                    Scott Moir

                    Image
                    Scott Moir Tessa Virtue i Scott Moir w 2016 r. Reprezentacja   Kanada Data i miejsce urodzenia 2 września 1987 London Wzrost 180 cm Konkurencja Pary taneczne Partner sportowy Tessa Virtue Trener Marie-France Dubreuil, Patrice Lauzon, Romain Haguenauer Poprzednio: Marina Zujewa, Oleg Epstein, Igor Szpilband, Johnny Johns, Carol Moir, Paul MacIntosh, Suzanne Killing Klub Montreal International School of Skating Poprzednio: Arctic Edge FSC Ilderton Skating Club Lokalizacja treningowa Montreal Poprzednio: Canton Kitchener Rekordy życiowe ISU przed sezonem 2018/2019 Nota łączna 206,07 Igrzyska Olimpijskie 2018 Taniec krótki 83,67 Igrzyska Olimpijskie 2018 Taniec dowolny 122,40 Igrzyska Olimpijskie 2018 Dorobek medalowy Reprezentacja   Kanada Igrzyska olimpijskie Złoto Vancouver 2010 pary taneczne Złoto Pjongczang 2018 pary taneczne Złoto Pjongczang 2018 drużynowo Srebro S...
                    Read more

                    Souastre

                    Image
                    Souastre — miejscowość i gmina — Herb Państwo   Francja Region Hauts-de-France Departament Pas-de-Calais Okręg Arras Kod INSEE 62800 Powierzchnia 7,18 km² Populacja  (1990) • liczba ludności 352 • gęstość 49 os./km² Kod pocztowy 62111 Położenie na mapie Francji Souastre 50°09′N   2°34′E / 50,150000   2,566667 Portal Francja Souastre – miejscowość i gmina we Francji, w regionie Hauts-de-France, w departamencie Pas-de-Calais. Według danych na rok 1990 gminę zamieszkiwały 352 osoby, a gęstość zaludnienia wynosiła 49 osób/km² (wśród 1549 gmin regionu Nord-Pas-de-Calais Souastre plasuje się na 887. miejscu pod względem liczby ludności, natomiast pod względem powierzchni na miejscu 507.). Bibliografia | Francuski urząd statystyczny ( fr. ) . p   •   d   •   e Gminy okręgu Arras Ablain-Saint-Nazaire Ablainzevelle Acheville Achicourt Achiet-le-Grand Achiet-le-Pe...
                    Read more