can't invoke sed properly...

can't invoke sed properly...

[Ned]

Hi all,
I have problemof invoking the sed text editor properly.
How, let's say, would I invoke sed in order to make a substitution in all lines of my_file:
sed :1,$ s/red/blue/g my_file
I am writing a bash script and need to test each line outside it first to make sure it works.


Many thanks in advance!
Ned

Re: can't invoke sed properly...

[Laura Eaves]

Hi Ned --
Well you should probably use

sed -e pattern filename >outfile

But beware of sed.
I once was chasing down an obscure bug in a very large script and between me 
and another person we found that a call to sed was implicitly truncating a 
long line to a built in maximum.
The resulting behavior of the script was quite confusing to say the least.
Sed will give no error message about this, and for must apps you won't run 
into it, but be aware that sed will truncate lines it can't load into its 
buffer.

Now as for your pattern, I believe you can delete the 1,$ as sed should 
cycle through the lines as it is and pass the updated liine to output.
So your line should be:

sed -e 's/red/blue/g' filename >outfile

HTH
--le


----- Original Message ----- 
From: "Ned" <ngranic@cox.net>
To: "Speakup is a screen review system for Linux." <speakup@braille.uwo.ca>
Sent: Monday, May 02, 2005 4:23 PM
Subject: can't invoke sed properly...


Hi all,
I have problemof invoking the sed text editor properly.
How, let's say, would I invoke sed in order to make a substitution in all 
lines of my_file:
sed :1,$ s/red/blue/g my_file
I am writing a bash script and need to test each line outside it first to 
make sure it works.


Many thanks in advance!
Ned
_______________________________________________
Speakup mailing list
Speakup@braille.uwo.ca
http://speech.braille.uwo.ca/mailman/listinfo/speakup 

Re: can't invoke sed properly...

[Lorenzo Taylor]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Someone tell me if there is an easier way, but this worked for me:

cat my-file | sed "s/red/blue/g" > my-file

This line replaced red with blue every time it appeared in the file.

I am using GNU sed version 4.1.4 here.  Your mileage may vary.

Lorenzo
- -- 
"We decided that we should evaluate the Microsoft offerings first. Once we
realised what a powerful set of tools they were, it became self-evident this was
the right way to go down."

Microsoft: the right way to go down
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (GNU/Linux)

iD8DBQFCdpKPG9IpekrhBfIRAg8oAJ0WeiXYLHbneHy2fzhSDq79r/23YACglvjU
j3rSTXu9XzKF+eUximAMCB0=
=sm17
-----END PGP SIGNATURE-----

Re: can't invoke sed properly...

[Laura Eaves]

Be careful putting the output into the same file as the input.
Note that I/O redirection takes place before the programs start running, so 
cat will open myfile and the shell will redirect the output to the input of 
sed, and before sed or cat ever start running, the shell will clobber the 
output file if it exists to make way for the output of sed.
Just use different names and later move the file back to the original if you 
want it updated.
Oh and you don't use the -e option -- I don't remember what version of sed I 
was using when I ran that.
But the -e option is useful if you have several patterns to apply to each 
input line.  sed will read the -e options left to right and treat each 
pattern as being on one line.  Of course you could always just use a quoted 
pattern that ran over multiple lines, but it is cleaner to use -e.

Finally, be careful using sed or any other pattern matching command, that 
you use single quotes instead of double quotes on the command line, as using 
double quotes will result in some undesired translation.  For example:

sed "1,$d"

will actually translate to

sed "1,xyz"

where xyz is the value of the environment variable d.
Lots of little gotchas in pattern matching.
Have fun!
--le

----- Original Message ----- 
From: "Lorenzo Taylor" <lorenzo@taylor.homelinux.net>
To: "Speakup is a screen review system for Linux." <speakup@braille.uwo.ca>
Sent: Monday, May 02, 2005 4:50 PM
Subject: Re: can't invoke sed properly...


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Someone tell me if there is an easier way, but this worked for me:

cat my-file | sed "s/red/blue/g" > my-file

This line replaced red with blue every time it appeared in the file.

I am using GNU sed version 4.1.4 here.  Your mileage may vary.

Lorenzo
- -- 
"We decided that we should evaluate the Microsoft offerings first. Once we
realised what a powerful set of tools they were, it became self-evident this 
was
the right way to go down."

Microsoft: the right way to go down
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (GNU/Linux)

iD8DBQFCdpKPG9IpekrhBfIRAg8oAJ0WeiXYLHbneHy2fzhSDq79r/23YACglvjU
j3rSTXu9XzKF+eUximAMCB0=
=sm17
-----END PGP SIGNATURE-----

_______________________________________________
Speakup mailing list
Speakup@braille.uwo.ca
http://speech.braille.uwo.ca/mailman/listinfo/speakup 

Re: can't invoke sed properly...

[Lorenzo Taylor]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Oh yeah.  Didn't think about the single quote vs. double quote thing there.

And the cat my-file |sed >my-file thing seems to have at best a 50% success rate.
Some times it works, and some times it clobbers the file.  Oh well.  I didn't
want to have to use a temporary file.  But I guess there are many places where
one just can't take shortcuts. <smile>

Lorenzo
- -- 
"We decided that we should evaluate the Microsoft offerings first. Once we
realised what a powerful set of tools they were, it became self-evident this was
the right way to go down."

Microsoft: the right way to go down
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (GNU/Linux)

iD8DBQFCdqc0G9IpekrhBfIRAkTzAKCZaevSG8nlmJzwRGMu5geVQdC6/ACfdKJ0
ILLqD17L9ZZXfG2+ze5eyes=
=cOFq
-----END PGP SIGNATURE-----

Re: can't invoke sed properly...

[Charles Hallenbeck]

Sed has an option to edit a file "in place" with an optional backup 
created if you need one. The option is -i and it has worked fine for me. 
Sed has also been able to edit files with lines that are 32K in length, 
although I used it as a filter in that application and did not try to 
"in place" option.


On Mon, 2 May 2005, Laura Eaves wrote:

> Be careful putting the output into the same file as the input.
> Note that I/O redirection takes place before the programs start running, so
> cat will open myfile and the shell will redirect the output to the input of
> sed, and before sed or cat ever start running, the shell will clobber the
> output file if it exists to make way for the output of sed.
> Just use different names and later move the file back to the original if you
> want it updated.
> Oh and you don't use the -e option -- I don't remember what version of sed I
> was using when I ran that.
> But the -e option is useful if you have several patterns to apply to each
> input line.  sed will read the -e options left to right and treat each
> pattern as being on one line.  Of course you could always just use a quoted
> pattern that ran over multiple lines, but it is cleaner to use -e.
>
> Finally, be careful using sed or any other pattern matching command, that
> you use single quotes instead of double quotes on the command line, as using
> double quotes will result in some undesired translation.  For example:
>
> sed "1,$d"
>
> will actually translate to
>
> sed "1,xyz"
>
> where xyz is the value of the environment variable d.
> Lots of little gotchas in pattern matching.
> Have fun!
> --le
>
> ----- Original Message -----
> From: "Lorenzo Taylor" <lorenzo@taylor.homelinux.net>
> To: "Speakup is a screen review system for Linux." <speakup@braille.uwo.ca>
> Sent: Monday, May 02, 2005 4:50 PM
> Subject: Re: can't invoke sed properly...
>
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Someone tell me if there is an easier way, but this worked for me:
>
> cat my-file | sed "s/red/blue/g" > my-file
>
> This line replaced red with blue every time it appeared in the file.
>
> I am using GNU sed version 4.1.4 here.  Your mileage may vary.
>
> Lorenzo
> - --
> "We decided that we should evaluate the Microsoft offerings first. Once we
> realised what a powerful set of tools they were, it became self-evident this
> was
> the right way to go down."
>
> Microsoft: the right way to go down
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.0 (GNU/Linux)
>
> iD8DBQFCdpKPG9IpekrhBfIRAg8oAJ0WeiXYLHbneHy2fzhSDq79r/23YACglvjU
> j3rSTXu9XzKF+eUximAMCB0=
> =sm17
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> Speakup mailing list
> Speakup@braille.uwo.ca
> http://speech.braille.uwo.ca/mailman/listinfo/speakup
>
>
> _______________________________________________
> Speakup mailing list
> Speakup@braille.uwo.ca
> http://speech.braille.uwo.ca/mailman/listinfo/speakup
>

-- 
The Moon is Waning Crescent (31% of Full)
"Things are in the saddle, and they ride mankind." Ralph Waldo Emerson
Visit my download site at http://www.mhcable.com/~chuckh

Re: can't invoke sed properly...

[Laura Eaves]

This may be a newer implementation of sed.  The one I ran while working on 
the said project (no pun intended) did indeed have a silent buffer limit and 
the bug was a real bear to find but easy to fix -- we just changed to awk, 
which is more flexible and powerful than sed.
I hadn't heard of the -i option, but I don't deny it exists.
But my question is this: don't you have to put the file on the command line 
when using -i rather than just redirecting with the shell > sign?
So you'd presumably have
    sed -i pattern filename
and sed would edit filename in place.
Nice feature.

But note that clobbering an input file by naming it also as the output file 
is a common mistake newbies make with many commands and it is a good idea to 
point it out so the user will beware.

Cheers.
--le

----- Original Message ----- 
From: "Charles Hallenbeck" <chuckh@hhs48.com>
To: "Speakup is a screen review system for Linux." <speakup@braille.uwo.ca>
Sent: Monday, May 02, 2005 6:48 PM
Subject: Re: can't invoke sed properly...


Sed has an option to edit a file "in place" with an optional backup
created if you need one. The option is -i and it has worked fine for me.
Sed has also been able to edit files with lines that are 32K in length,
although I used it as a filter in that application and did not try to
"in place" option.


On Mon, 2 May 2005, Laura Eaves wrote:

> Be careful putting the output into the same file as the input.
> Note that I/O redirection takes place before the programs start running, 
> so
> cat will open myfile and the shell will redirect the output to the input 
> of
> sed, and before sed or cat ever start running, the shell will clobber the
> output file if it exists to make way for the output of sed.
> Just use different names and later move the file back to the original if 
> you
> want it updated.
> Oh and you don't use the -e option -- I don't remember what version of sed 
> I
> was using when I ran that.
> But the -e option is useful if you have several patterns to apply to each
> input line.  sed will read the -e options left to right and treat each
> pattern as being on one line.  Of course you could always just use a 
> quoted
> pattern that ran over multiple lines, but it is cleaner to use -e.
>
> Finally, be careful using sed or any other pattern matching command, that
> you use single quotes instead of double quotes on the command line, as 
> using
> double quotes will result in some undesired translation.  For example:
>
> sed "1,$d"
>
> will actually translate to
>
> sed "1,xyz"
>
> where xyz is the value of the environment variable d.
> Lots of little gotchas in pattern matching.
> Have fun!
> --le
>
> ----- Original Message -----
> From: "Lorenzo Taylor" <lorenzo@taylor.homelinux.net>
> To: "Speakup is a screen review system for Linux." 
> <speakup@braille.uwo.ca>
> Sent: Monday, May 02, 2005 4:50 PM
> Subject: Re: can't invoke sed properly...
>
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Someone tell me if there is an easier way, but this worked for me:
>
> cat my-file | sed "s/red/blue/g" > my-file
>
> This line replaced red with blue every time it appeared in the file.
>
> I am using GNU sed version 4.1.4 here.  Your mileage may vary.
>
> Lorenzo
> - --
> "We decided that we should evaluate the Microsoft offerings first. Once we
> realised what a powerful set of tools they were, it became self-evident 
> this
> was
> the right way to go down."
>
> Microsoft: the right way to go down
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.0 (GNU/Linux)
>
> iD8DBQFCdpKPG9IpekrhBfIRAg8oAJ0WeiXYLHbneHy2fzhSDq79r/23YACglvjU
> j3rSTXu9XzKF+eUximAMCB0=
> =sm17
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> Speakup mailing list
> Speakup@braille.uwo.ca
> http://speech.braille.uwo.ca/mailman/listinfo/speakup
>
>
> _______________________________________________
> Speakup mailing list
> Speakup@braille.uwo.ca
> http://speech.braille.uwo.ca/mailman/listinfo/speakup
>

-- 
The Moon is Waning Crescent (31% of Full)
"Things are in the saddle, and they ride mankind." Ralph Waldo Emerson
Visit my download site at http://www.mhcable.com/~chuckh

_______________________________________________
Speakup mailing list
Speakup@braille.uwo.ca
http://speech.braille.uwo.ca/mailman/listinfo/speakup 

Re: can't invoke sed properly...

[Charles Hallenbeck]

Yes, you must mention the filename on the command line instead of 
redirecting or piping it. My version is 4.1.2, and I am not sure how 
long this option has been available. You are right about the dangers of 
redirecting both input and output using the same file, but I guess if 
you know about the -i option you might not be tempted to do that.

On Tue, 3 May 2005, Laura Eaves wrote:

> This may be a newer implementation of sed.  The one I ran while working on
> the said project (no pun intended) did indeed have a silent buffer limit and
> the bug was a real bear to find but easy to fix -- we just changed to awk,
> which is more flexible and powerful than sed.
> I hadn't heard of the -i option, but I don't deny it exists.
> But my question is this: don't you have to put the file on the command line
> when using -i rather than just redirecting with the shell > sign?
> So you'd presumably have
>    sed -i pattern filename
> and sed would edit filename in place.
> Nice feature.
>
> But note that clobbering an input file by naming it also as the output file
> is a common mistake newbies make with many commands and it is a good idea to
> point it out so the user will beware.
>
> Cheers.
> --le

-- 
The Moon is Waning Crescent (25% of Full)
"Things are in the saddle, and they ride mankind." Ralph Waldo Emerson
Visit my download site at http://www.mhcable.com/~chuckh