Small and fast POSIX compliant shell. ()

Mike Frysinger | 25 Oct 2011 23:58

Picon

[PATCH] echo: fix octal escaping with \1...\7

POSIX states that octal escape sequences should take the form \0num
when using echo.  dash however additionally treats \num as an octal
sequence.  This breaks some packages (like libtool) who attempt to
use strings with these escape sequences via variables to execute sed
(since sed ends up getting passed a byte instead of a literal \1).

The code that consumes this sequence includes a comment that indicates
it doesn't actually mean to do this.  So simplify the code a bit by
ignoring these sequences that lack a leading 0 and falling through to
the existing escape parsing logic.

before:
	$ echo '\1' | hexdump -C
	00000000  01 0a                 |..|
after:
	$ echo '\1' | hexdump -C
	00000000  5c 31 0a              |\1.|
(existing \01 sequence still works the same)

This also slightly shrinks the resulting compiled code :).

Signed-off-by: Mike Frysinger <vapier <at> gentoo.org>
---
Note: I assume this still applies to the latest git.  The last
	checkout I have is from Sep 2010 though, and kernel.org does
	not yet have the dash tree back on it.

 src/bltin/printf.c |   16 ++++------------
 1 files changed, 4 insertions(+), 12 deletions(-)

(Continue reading)

Stephane CHAZELAS | 26 Oct 2011 09:59

Picon

Re: [PATCH] echo: fix octal escaping with \1...\7

2011-10-25, 17:58(-04), Mike Frysinger:
> POSIX states that octal escape sequences should take the form \0num
> when using echo.

Only with the XSI option (Unix), for POSIX, echo "\whatever" is
unspecified. But as far as I can tell even with XSI, "echo
'\123'" is unspecified as well, so dash is free to do what it
likes here.

> dash however additionally treats \num as an octal
> sequence.  This breaks some packages (like libtool) who attempt to
> use strings with these escape sequences via variables to execute sed
> (since sed ends up getting passed a byte instead of a literal \1).

Given that the result of  

echo "\123" is unspecified, those scripts would not be POSIX and
they are those to be fixed.

--

-- 
Stephane

--
To unsubscribe from this list: send the line "unsubscribe dash" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Herbert Xu | 31 Oct 2011 04:41

Picon

Picon

Re: [PATCH] echo: fix octal escaping with \1...\7

Mike Frysinger <vapier <at> gentoo.org> wrote:
> POSIX states that octal escape sequences should take the form \0num
> when using echo.  dash however additionally treats \num as an octal
> sequence.  This breaks some packages (like libtool) who attempt to
> use strings with these escape sequences via variables to execute sed
> (since sed ends up getting passed a byte instead of a literal \1).

OK this is a bit of problem.  From our conversation I had the
impression that you were referring to the lack of support of
escape codes, rather than unwanted support.

If it was the former I could easily add it if POSIX said so,
however, as this is an existing feature there may well be scripts
out there that depend on it.  So removing it is not an option
unless it is explicitly forbidden by POSIX.

In any case, scripts that rely on escape codes like this are
simply broken and should either be fixed to use printf or just
run with #!/bin/bash.

Cheers,
--

-- 
Email: Herbert Xu <herbert <at> gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
--
To unsubscribe from this list: send the line "unsubscribe dash" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

(Continue reading)

Mike Frysinger | 31 Oct 2011 05:23

Picon

Re: [PATCH] echo: fix octal escaping with \1...\7

On Sunday 30 October 2011 23:41:58 Herbert Xu wrote:
> Mike Frysinger wrote:
> > POSIX states that octal escape sequences should take the form \0num
> > when using echo.  dash however additionally treats \num as an octal
> > sequence.  This breaks some packages (like libtool) who attempt to
> > use strings with these escape sequences via variables to execute sed
> > (since sed ends up getting passed a byte instead of a literal \1).
> 
> OK this is a bit of problem.  From our conversation I had the
> impression that you were referring to the lack of support of
> escape codes, rather than unwanted support.
> 
> If it was the former I could easily add it if POSIX said so,
> however, as this is an existing feature there may well be scripts
> out there that depend on it.  So removing it is not an option
> unless it is explicitly forbidden by POSIX.

i'm not seeing how this jives with dash's goal.  if it intends to be a 
fast/small POSIX compliant shell while punting (almost) all the rest, then why 
carry additional functionality that POSIX doesn't even mention in passing ?  
this isn't "documented but optional extended functionality", but rather the 
realm of "anything goes".  otherwise we approach the same realm that dash was 
created to avoid -- carrying lots of cruft that slow things down because 
scripts use it rather than POSIX mandating it.

as a comparison, bash/ksh/tcsh/zsh/busybox[ash] all behave the way my patch 
updates dash to operate ... i would test more shells, but these tend to be the 
standards that everyone compares against.  i can't see people writing scripts 
that only work under dash either.

(Continue reading)

Eric Blake | 31 Oct 2011 14:12

Picon

Re: [PATCH] echo: fix octal escaping with \1...\7

[adding bug-libtool]

On 10/30/2011 10:23 PM, Mike Frysinger wrote:
> On Sunday 30 October 2011 23:41:58 Herbert Xu wrote:
>> Mike Frysinger wrote:
>>> POSIX states that octal escape sequences should take the form \0num
>>> when using echo.  dash however additionally treats \num as an octal
>>> sequence.  This breaks some packages (like libtool) who attempt to
>>> use strings with these escape sequences via variables to execute sed
>>> (since sed ends up getting passed a byte instead of a literal \1).

That's a bug in libtool for using "echo '\1'" and expecting sane 
behavior.  Can you provide more details on this libtool bug, so we can 
get it fixed in libtool?  Or perhaps it has already been fixed in modern 
libtool, and you are just encountering it in an older version?

>>
>> OK this is a bit of problem.  From our conversation I had the
>> impression that you were referring to the lack of support of
>> escape codes, rather than unwanted support.
>>
>> If it was the former I could easily add it if POSIX said so,
>> however, as this is an existing feature there may well be scripts
>> out there that depend on it.  So removing it is not an option
>> unless it is explicitly forbidden by POSIX.
>
> i'm not seeing how this jives with dash's goal.  if it intends to be a
> fast/small POSIX compliant shell while punting (almost) all the rest, then why
> carry additional functionality that POSIX doesn't even mention in passing ?
> this isn't "documented but optional extended functionality", but rather the

(Continue reading)

Paul Gilmartin | 31 Oct 2011 14:35

Re: [PATCH] echo: fix octal escaping with \1...\7

On Oct 31, 2011, at 07:12, Eric Blake wrote:

> [adding bug-libtool]
>  
[removing, because I'm not registered.]

> On 10/30/2011 10:23 PM, Mike Frysinger wrote:
>> On Sunday 30 October 2011 23:41:58 Herbert Xu wrote:
>>> Mike Frysinger wrote:
>>>> POSIX states that octal escape sequences should take the form \0num
>>>> when using echo.  dash however additionally treats \num as an octal
>>>> sequence.  This breaks some packages (like libtool) who attempt to
>>>> use strings with these escape sequences via variables to execute sed
>>>> (since sed ends up getting passed a byte instead of a literal \1).
>  
> That's a bug in libtool for using "echo '\1'" and expecting sane behavior.  Can you provide more details on
this libtool bug, so we can get it fixed in libtool?  Or perhaps it has already been fixed in modern libtool,
and you are just encountering it in an older version?
>  
Yes, there's value in coding defensively.  However:

I used to know a statement in POSIX that builtins should behave
identically to the executables in /bin (or perhaps /usr/bin)
except for performance.  So, testing with dash on Ubuntu:

\! $ echo "\1"

\! $ /bin/echo "\1"
\1
\! $

(Continue reading)

Eric Blake | 31 Oct 2011 15:03

Picon

Re: [PATCH] echo: fix octal escaping with \1...\7

On 10/31/2011 07:35 AM, Paul Gilmartin wrote:
> I used to know a statement in POSIX that builtins should behave
> identically to the executables in /bin (or perhaps /usr/bin)
> except for performance.

Only in regards to standardized use of those utilities.  'echo "\1"' is 
not standardized, so it is allowed to differ between the dash built-in 
and /bin/echo.

--

-- 
Eric Blake   eblake <at> redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org
--
To unsubscribe from this list: send the line "unsubscribe dash" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Stephane CHAZELAS | 31 Oct 2011 15:56

Picon

Re: [PATCH] echo: fix octal escaping with \1...\7

2011-10-31, 07:35(-06), Paul Gilmartin:
> On Oct 31, 2011, at 07:12, Eric Blake wrote:
>
>> [adding bug-libtool]
>>  
> [removing, because I'm not registered.]
>
>> On 10/30/2011 10:23 PM, Mike Frysinger wrote:
>>> On Sunday 30 October 2011 23:41:58 Herbert Xu wrote:
>>>> Mike Frysinger wrote:
>>>>> POSIX states that octal escape sequences should take the form \0num
>>>>> when using echo.  dash however additionally treats \num as an octal
>>>>> sequence.  This breaks some packages (like libtool) who attempt to
>>>>> use strings with these escape sequences via variables to execute sed
>>>>> (since sed ends up getting passed a byte instead of a literal \1).
>>  
>> That's a bug in libtool for using "echo '\1'" and expecting sane behavior.  Can you provide more details on
this libtool bug, so we can get it fixed in libtool?  Or perhaps it has already been fixed in modern libtool,
and you are just encountering it in an older version?
>>  
> Yes, there's value in coding defensively.  However:
>
> I used to know a statement in POSIX that builtins should behave
> identically to the executables in /bin (or perhaps /usr/bin)
> except for performance.  So, testing with dash on Ubuntu:
[...]

This can only reasonably be done on systems where the shell and
utilities are maintained by the same persons. On my system,
/bin/echo '\FS' starts a flight simulator. Should I ask dash to

(Continue reading)

Harald van Dijk | 31 Oct 2011 19:07

Picon

Re: [PATCH] echo: fix octal escaping with \1...\7

On Mon, 2011-10-31 at 07:12 -0600, Eric Blake wrote:
> Scripts that rely on a certain interpretation of "echo '\1'" are broken 
> regardless of how dash behaves; but that said, since POSIX doesn't 
> require dash's current behavior, and since the proposed patch makes dash 
> both smaller and more like other shells in treating it as an extension 
> that means a literal 1 rather than an octal escape, I would be in favor 
> of making the change in dash.

The problem with that is that the patch makes dash behave differently
from other shells in its interpretation of printf's %b format. bash, for
instance, does accept \1 escape sequences there, as an extension to
POSIX. Currently, so does dash. With this patch, it stops working in
dash. And unsharing the code between echo and printf would probably make
dash larger.

There's also another problem: the patch doesn't work as advertised
anyway. It should continue to make echo "\0101" print a single character
followed by newline. Instead, with that patch dash takes \010 as an
escape sequence, and then follows it by 1 and a newline.

Cheers,
Harald

--
To unsubscribe from this list: send the line "unsubscribe dash" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Mike Frysinger | 31 Oct 2011 19:39

Picon

Re: [PATCH] echo: fix octal escaping with \1...\7

On Monday 31 October 2011 09:12:43 Eric Blake wrote:
> On 10/30/2011 10:23 PM, Mike Frysinger wrote:
> > On Sunday 30 October 2011 23:41:58 Herbert Xu wrote:
> >> Mike Frysinger wrote:
> >>> POSIX states that octal escape sequences should take the form \0num
> >>> when using echo.  dash however additionally treats \num as an octal
> >>> sequence.  This breaks some packages (like libtool) who attempt to
> >>> use strings with these escape sequences via variables to execute sed
> >>> (since sed ends up getting passed a byte instead of a literal \1).
> 
> That's a bug in libtool for using "echo '\1'" and expecting sane
> behavior.  Can you provide more details on this libtool bug, so we can
> get it fixed in libtool?  Or perhaps it has already been fixed in modern
> libtool, and you are just encountering it in an older version?

i plan on digging through the relevant packages and posting patches where 
applicable.  i might be wrong about the libtool side, but do know of at least 
one ax m4 file using it (which is what started this rat hole in the first 
place).  but i consider that a parallel issue :).

> >> OK this is a bit of problem.  From our conversation I had the
> >> impression that you were referring to the lack of support of
> >> escape codes, rather than unwanted support.
> >> 
> >> If it was the former I could easily add it if POSIX said so,
> >> however, as this is an existing feature there may well be scripts
> >> out there that depend on it.  So removing it is not an option
> >> unless it is explicitly forbidden by POSIX.
> > 
> > i'm not seeing how this jives with dash's goal.  if it intends to be a

(Continue reading)

Harald van Dijk | 31 Oct 2011 19:48

Picon

Re: [PATCH] echo: fix octal escaping with \1...\7

On Mon, 2011-10-31 at 14:39 -0400, Mike Frysinger wrote:
> i plan on digging through the relevant packages and posting patches where 
> applicable.  i might be wrong about the libtool side, but do know of at least 
> one ax m4 file using it (which is what started this rat hole in the first 
> place).  but i consider that a parallel issue :).

That one m4 file was ax_prefix_config_h.m4, for which I submitted a
patch upstream and which got accepted over a year ago:
<http://savannah.gnu.org/patch/index.php?7317>.

Cheers,
Harald

--
To unsubscribe from this list: send the line "unsubscribe dash" in
the body of a message to majordomo <at> vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Return

Return to gmane.comp.shells.dash.

Search Archive

Language

Change language

Options

Current view: Threads only / Showing only 20 lines / Not hiding cited text.
Change to All messages, whole messages, or hide cited text.

Post a message
NNTP Newsgroup
Classic Gmane web interface
XML

XML

RSS Feed
List Information

About Gmane