headers
Martijn Rijkeboer | 2 Aug 2012 23:21
Favicon

Shellscript escaping problem

Hi,

I'm using the script below and it keeps giving me the following error:

  Unexpected remote arg: backup <at> fqdn:/data/dir/
  rsync error: syntax or usage error (code 1) at main.c(1214) [sender=3.0.9]

It looks like something is going wrong with my escaping, because when I
remove the --rsync-path line everything works fine. It also works fine
if I echo the last line of the script and copy and paste it in my shell
(KSH). I've tried all kinds of different escaping and nothing seems to
work. Any ideas on what I'm doing wrong?

Kind regards,

Martijn Rijkeboer

------------------------
#!/bin/sh

DATADIR="/data"
TODAY=`/bin/date +%Y%m%d`

RSYNC_CMD="/usr/local/bin/rsync -v -n \
    --rsync-path='rsync sudo' \
    --rsh=ssh \
    --archive --one-file-system --compress --hard-links --numeric-ids \
    --human-readable --delete-after --backup --backup-dir=deleted/$TODAY \
    --exclude=deleted"
(Continue reading)

Permalink | Reply |
headers
Matthew Dempsky | 2 Aug 2012 23:34
Favicon

Re: Shellscript escaping problem

On Thu, Aug 2, 2012 at 2:21 PM, Martijn Rijkeboer <martijn <at> bunix.org> wrote:
> RSYNC_CMD="/usr/local/bin/rsync -v -n \
>     --rsync-path='rsync sudo' \

This doesn't do what you think it does.  The single quotes are getting
literally passed to rsync, they're not reinterpreted after $RSYNC_CMD
is interpolated.

This is similar to running something like:

  rsync --rsync-path=\'rsync sudo\' backup <at> fqdn...
Permalink | Reply |
headers
Alexander Hall | 3 Aug 2012 00:03
Picon
Favicon

Re: Shellscript escaping problem

On 08/02/12 23:34, Matthew Dempsky wrote:
> On Thu, Aug 2, 2012 at 2:21 PM, Martijn Rijkeboer <martijn <at> bunix.org> wrote:
>> RSYNC_CMD="/usr/local/bin/rsync -v -n \
>>      --rsync-path='rsync sudo' \
>
> This doesn't do what you think it does.  The single quotes are getting
> literally passed to rsync, they're not reinterpreted after $RSYNC_CMD
> is interpolated.
>
> This is similar to running something like:
>
>    rsync --rsync-path=\'rsync sudo\' backup <at> fqdn...
>

Yep. $RSYNC_CMD will be splitted by space, tab or newline by default, or 
by the contents of $IFS, if set.

And no, that is _not_ a suggestion to fiddle with $IFS. :-)

if you need it to be reusable, I'd suggest making it a function or so:

...

synchronize() {
	/usr/local/bin/rsync -v -n \
	    --rsync-path='/usr/bin/sudo /usr/local/bin/rsync' \
	    --archive --one-file-system --compress --hard-links \
	    --numeric-ids --human-readable --delete-after \
	    --backup --backup-dir=deleted/$TODAY \
	    --exclude=deleted \
(Continue reading)

Permalink | Reply |
headers
Martijn Rijkeboer | 3 Aug 2012 10:10
Favicon

Re: Shellscript escaping problem

>>> RSYNC_CMD="/usr/local/bin/rsync -v -n \
>>>      --rsync-path='rsync sudo' \
>>
>> This doesn't do what you think it does.  The single quotes are getting
>> literally passed to rsync, they're not reinterpreted after $RSYNC_CMD
>> is interpolated.
>
> Yep. $RSYNC_CMD will be splitted by space, tab or newline by default, or
> by the contents of $IFS, if set.
>
> if you need it to be reusable, I'd suggest making it a function or so:
>
> synchronize() {
> 	/usr/local/bin/rsync -v -n \
> 	    --rsync-path='/usr/bin/sudo /usr/local/bin/rsync' \
> 	    --archive --one-file-system --compress --hard-links \
> 	    --numeric-ids --human-readable --delete-after \
> 	    --backup --backup-dir=deleted/$TODAY \
> 	    --exclude=deleted \
> 	    "$ <at> "
> }
>
> synchronize backup <at> fqdn:$DATADIR/dir/ $DATADIR/dir

Using a function works without problems. Thanks to all who gave input.

Kind regards,

Martijn Rijkeboer
(Continue reading)

Permalink | Reply |
headers
Nico Kadel-Garcia | 3 Aug 2012 22:24
Picon

Re: Shellscript escaping problem

On Fri, Aug 3, 2012 at 4:10 AM, Martijn Rijkeboer <martijn <at> bunix.org> wrote:
>>>> RSYNC_CMD="/usr/local/bin/rsync -v -n \
>>>>      --rsync-path='rsync sudo' \
>>>
>>> This doesn't do what you think it does.  The single quotes are getting
>>> literally passed to rsync, they're not reinterpreted after $RSYNC_CMD
>>> is interpolated.
>>
>> Yep. $RSYNC_CMD will be splitted by space, tab or newline by default, or
>> by the contents of $IFS, if set.
>>
>> if you need it to be reusable, I'd suggest making it a function or so:
>>
>> synchronize() {
>>       /usr/local/bin/rsync -v -n \
>>           --rsync-path='/usr/bin/sudo /usr/local/bin/rsync' \
>>           --archive --one-file-system --compress --hard-links \
>>           --numeric-ids --human-readable --delete-after \
>>           --backup --backup-dir=deleted/$TODAY \
>>           --exclude=deleted \
>>           "$ <at> "
>> }
>>
>> synchronize backup <at> fqdn:$DATADIR/dir/ $DATADIR/dir
>
> Using a function works without problems. Thanks to all who gave input.
>
> Kind regards,
>
>
(Continue reading)

Permalink | Reply |
headers
Martijn Rijkeboer | 4 Aug 2012 09:36
Favicon

Re: Shellscript escaping problem

> If you're doing complex rsync setups, you might also consider using
> "rsnapshot" as a wrapper. I just joined the maintainer list on that,
> it's a very useful old perl tool, well organized for frequent and well
> managed backups.

Rsnapshot is a nice tool, but for my current usage rsync suffices.

Kind regards,

Martijn Rijkeboer
Permalink | Reply |
headers
David Diggles | 4 Aug 2012 06:35
Picon

Re: Shellscript escaping problem

On Thu, Aug 02, 2012 at 11:21:01PM +0200, Martijn Rijkeboer wrote:
> Hi,

here's an example of how not to script rsync, when just starting
to learn how to script

it got over complicated over time.  i should rewrite it sometime :)

#!/bin/bash

SCRIPT=${0##*/}
BASE=/archive0/_backup
SSHOPTS="-q -o Ciphers=arcfour256 -o MACs=umac-64 <at> openssh.com"
RSYNCOPTS="--delete -avxlr"
CONF=/etc/$SCRIPT

f_f() {
  [ -d $1 ] && \
  for SRC in $(ls $1|grep -v "^\."); do
    echo \# $SRC \#
    if [ $# -ge 3 ]; then
      eval $(echo eval "$"3)
      DST=$2/$REV/${HOST%%.*}
      local i=4;while [ $i -le $# ]; do
        eval $(echo eval "$"$i)
      ((i++));done
    fi
  done
}
(Continue reading)

Permalink | Reply |
headers
Philip Guenther | 4 Aug 2012 07:41
Picon

Re: Shellscript escaping problem

On Fri, Aug 3, 2012 at 9:35 PM, David Diggles <david <at> elven.com.au> wrote:
...
> here's an example of how not to script rsync, when just starting
> to learn how to script
>
> it got over complicated over time.  i should rewrite it sometime :)
...

I guess I don't understand the point of sending that out.  It's like a
generic ghost story: "...and the code walks the corridors of the
office building to this day!  There it is!  Ahhhhhhhhh!"

If the goal is to help the inexperienced shell script writer avoid
that fate, you must provide instruction and suggestions, not just set
up your prior works up as warning.  The beginner will be dazzled by
the mess-o'-punctuation, but that doesn't help them see what they
should do instead when they, in turn, find their own scripts crawling
into the morass.

Philip Guenther
Permalink | Reply |
headers
Alexander Hall | 4 Aug 2012 22:37
Picon
Favicon

Re: Shellscript escaping problem

Philip Guenther <guenther <at> gmail.com> wrote:

>On Fri, Aug 3, 2012 at 9:35 PM, David Diggles <david <at> elven.com.au>
>wrote:
>...
>> here's an example of how not to script rsync, when just starting
>> to learn how to script
>>
>> it got over complicated over time.  i should rewrite it sometime :)
>...
>
>I guess I don't understand the point of sending that out.

+1
Permalink | Reply |
headers
David Diggles | 5 Aug 2012 08:00
Picon

Re: Shellscript escaping problem

On Fri, Aug 03, 2012 at 10:41:09PM -0700, Philip Guenther wrote:
> On Fri, Aug 3, 2012 at 9:35 PM, David Diggles <david <at> elven.com.au> wrote:
> ...
> > here's an example of how not to script rsync, when just starting
> > to learn how to script
> >
> > it got over complicated over time.  i should rewrite it sometime :)
> ...
> 
> I guess I don't understand the point of sending that out.  It's like a
> generic ghost story: "...and the code walks the corridors of the
> office building to this day!  There it is!  Ahhhhhhhhh!"
>
> If the goal is to help the inexperienced shell script writer avoid
> that fate, you must provide instruction and suggestions, not just set
> up your prior works up as warning.  The beginner will be dazzled by
> the mess-o'-punctuation, but that doesn't help them see what they
> should do instead when they, in turn, find their own scripts crawling
> into the morass.
> 
> 
> Philip Guenther

There were already excellent examples of what to do provided by others.

What is wrong with an example of how bad spaghetti scripting looks like?
I think there can be value in seeing an exagerrated example of what not to do.

Furthermore, I provided my own script, not someone elses, because I am
happy to own my own mistakes.  Another good thing to teach.
(Continue reading)

Permalink | Reply |
headers
Philip Guenther | 5 Aug 2012 08:47
Picon

Re: Shellscript escaping problem

On Sat, Aug 4, 2012 at 11:00 PM, David Diggles <david <at> elven.com.au> wrote:
> What is wrong with an example of how bad spaghetti scripting looks like?
> I think there can be value in seeing an exagerrated example of what not to do.

I don't think it's obvious to everyone that that was bad code.
Indeed, there was that person in the past that wrote that code and
decided it was the best they could do at that moment.  What's to say
some other person won't look at your code and say "huh, looks better
than some of my other scripts; I don't get what the problem is".

> Furthermore, I provided my own script, not someone elses, because I am
> happy to own my own mistakes.  Another good thing to teach.

True.

> IMO, a teaching method that only ever teaches what to do, and never what not to
> do, and only ever provides instructions and guidance, risks creating the kind of
> box that breeds idiots with no ability to think for themselves.

In my experience as a student and teacher, the discussions of about
what made something (a program, a work of art, a mathematical proof, a
piece of music) better or worse were the most valuable part of
reviewing the existing works.

If just seeing the bad stuff without thinking about and recognizing
what made it bad was enough to let you create good stuff, then
Sturgeon's law would be false.

Philip Guenther
(Continue reading)

Permalink | Reply |

Gmane