SEO

Do you know the right syntax for the TR command to strip characters from a
file ?


my file looks like this :

rfq=ADDRFQ%7ER-050427%3A033325%7ECARQUEST%7E9%7E1.1%7E2005-04-27T12%3A33%3A25.00
0-07%3A00%0D%0ARFQ%7ER-050427%3A033325%7ER-050427%3A033325%7E2005-04-27T12%3A33%
3A25.000-07%3A00%7E0%7E0%7E57603%7Etest%7E654654+test+ave%7 E%7E%7Esanta+ana%7ECA
%7E92704%7EUS%7E%28714%29+850-1544%7E32423%7ENew+York+DC%7E32+Apple+Lane%7E%7E%7
ENew+York%7ENY%7E10010%7EUS%7E%28212%29+212-2122%7E%0D%0ALINE%7ER-050427%3A03332
5%7E1%7ER-050427%3A033325%7EDF01%7EDF01%7E1.0%7EEA%7EFREGHT% 7E%0D%0A


and I want to change all the %7E for an ~

and I've tried

cat file.txt | tr "%7E" "~" but only changes the % for an ~ and leaves
the 7E.

Then I want to change the %0D%0A for a Line feed or carriage return.

cat file.txt | tr "%0D%0A" "/012"


Thanks

E Arredondo typed (on Wed, Apr 27, 2005 at 09:48:21PM +0000):
| Do you know the right syntax for the TR command to strip characters from a
| file ?
|
|
| my file looks like this :
|
| rfq=ADDRFQ%7ER-050427%3A033325%7ECARQUEST%7E9%7E1.1%7E2005-04-27T12%3A33%3A25.00
| 0-07%3A00%0D%0ARFQ%7ER-050427%3A033325%7ER-050427%3A033325%7E2005-04-27T12%3A33%
| 3A25.000-07%3A00%7E0%7E0%7E57603%7Etest%7E654654+test+ave%7 E%7E%7Esanta+ana%7ECA
| %7E92704%7EUS%7E%28714%29+850-1544%7E32423%7ENew+York+DC%7E32+Apple+Lane%7E%7E%7
| ENew+York%7ENY%7E10010%7EUS%7E%28212%29+212-2122%7E%0D%0ALINE%7ER-050427%3A03332
| 5%7E1%7ER-050427%3A033325%7EDF01%7EDF01%7E1.0%7EEA%7EFREGHT% 7E%0D%0A
|
|
| and I want to change all the %7E for an ~
|
| and I've tried
|
| cat file.txt | tr "%7E" "~" but only changes the % for an ~ and leaves
| the 7E.
|
| Then I want to change the %0D%0A for a Line feed or carriage return.
|
| cat file.txt | tr "%0D%0A" "/012"
|

Wrong tool. Use sed.

--
JP

----- Original Message -----
From: "E Arredondo" <atk@sbcglobal.net>
Newsgroups: comp.unix.sco.misc
To: <distro@jpr.com>
Sent: Wednesday, April 27, 2005 5:48 PM
Subject: tr syntax


> Do you know the right syntax for the TR command to strip characters from a
> file ?
>
>
> my file looks like this :
>
> rfq=ADDRFQ%7ER-050427%3A033325%7ECARQUEST%7E9%7E1.1%7E2005-04-27T12%3A33%3A25.00
> 0-07%3A00%0D%0ARFQ%7ER-050427%3A033325%7ER-050427%3A033325%7E2005-04-27T12%3A33%
> 3A25.000-07%3A00%7E0%7E0%7E57603%7Etest%7E654654+test+ave%7 E%7E%7Esanta+ana%7ECA
> %7E92704%7EUS%7E%28714%29+850-1544%7E32423%7ENew+York+DC%7E32+Apple+Lane%7E%7E%7
> ENew+York%7ENY%7E10010%7EUS%7E%28212%29+212-2122%7E%0D%0ALINE%7ER-050427%3A03332
> 5%7E1%7ER-050427%3A033325%7EDF01%7EDF01%7E1.0%7EEA%7EFREGHT% 7E%0D%0A
>
>
> and I want to change all the %7E for an ~
>
> and I've tried
>
> cat file.txt | tr "%7E" "~" but only changes the % for an ~ and
> leaves the 7E.
>
> Then I want to change the %0D%0A for a Line feed or carriage return.
>
> cat file.txt | tr "%0D%0A" "/012"
>
>
> Thanks

tr can not do that job, for that you want sed, or....

This is a good example of what we mean when we sometimes tell people "tell
us what you really want" instead of trying to abstract out the part you
think you need solved out of the real whole problem.

This appears to be url-encoded content, you may be interested to know there
are utilities and code snippets in various languages already designed just
for the purpose of url en/de-coding. There are a lot of other codes besides
the couple you asked about, and you should really be prepared to decode any
that might appear in the data, including ones that you haven't just happened
to encounter yet.

I found a couple of snippets of c code that do this and made them into bare
bones stand-alone executables.
http://www.aljex.com/bkw/sco/index.html#urlenc

Unfortunately, your data also includes "+" in place of spaces, which is
another common url hack, and unfortunately, it wouldn't be safe/sane in my
opinion to wholesale replace all + with " " (which tr can do) because surely
a "+" is a legitimate character that might appear in the data?

The spaces should have been submitted to the cgi as %20 instead of + in the
first place. By the time you recieve the data above, it's too late for you
to be able to reliably convert the +'s back to spaces without also
converting +'s that were supposed to actually be +'s in the data.

Here's a command to use the binary above and tr to decode all the url codes
and convert all +'s to spaces and just hope no + ever comes in as data.
deurl "$rfq" |tr "+" " "

My guess is you are ultimately getting this data into filepro. You could
leave off the tr and use xlate in filepro almost as easily. In that case,
you could decide to only xlate certain fields, leaving the possibility of
real data +'s in some fields. The tr command will convert every + in the
record and has no concept of fields.

Brian K. White -- brian@aljex.com -- http://www.aljex.com/bkw/
+++++[>+++[>+++++>+++++++<<-]<-]>>+.>.+++++.+++++++.-.[>+<---]>++.
filePro BBx Linux SCO Prosper/FACTS AutoCAD #callahans Satriani

"Brian K. White" <brian@aljex.com> wrote in message
news:046501c54b96$18014080$6500000a@venti...
>
> ----- Original Message -----
> From: "E Arredondo" <atk@sbcglobal.net>
> Newsgroups: comp.unix.sco.misc
> To: <distro@jpr.com>
> Sent: Wednesday, April 27, 2005 5:48 PM
> Subject: tr syntax
>
>
>> Do you know the right syntax for the TR command to strip characters from
>> a
>> file ?
>>
>>
>> my file looks like this :
>>
>> rfq=ADDRFQ%7ER-050427%3A033325%7ECARQUEST%7E9%7E1.1%7E2005-04-27T12%3A33%3A25.00
>> 0-07%3A00%0D%0ARFQ%7ER-050427%3A033325%7ER-050427%3A033325%7E2005-04-27T12%3A33%
>> 3A25.000-07%3A00%7E0%7E0%7E57603%7Etest%7E654654+test+ave%7 E%7E%7Esanta+ana%7ECA
>> %7E92704%7EUS%7E%28714%29+850-1544%7E32423%7ENew+York+DC%7E32+Apple+Lane%7E%7E%7
>> ENew+York%7ENY%7E10010%7EUS%7E%28212%29+212-2122%7E%0D%0ALINE%7ER-050427%3A03332
>> 5%7E1%7ER-050427%3A033325%7EDF01%7EDF01%7E1.0%7EEA%7EFREGHT% 7E%0D%0A
>>
>>
>> and I want to change all the %7E for an ~
>>
>> and I've tried
>>
>> cat file.txt | tr "%7E" "~" but only changes the % for an ~ and
>> leaves the 7E.
>>
>> Then I want to change the %0D%0A for a Line feed or carriage return.
>>
>> cat file.txt | tr "%0D%0A" "/012"
>>
>>
>> Thanks
>
> tr can not do that job, for that you want sed, or....
>
> This is a good example of what we mean when we sometimes tell people "tell
> us what you really want" instead of trying to abstract out the part you
> think you need solved out of the real whole problem.
>
> This appears to be url-encoded content, you may be interested to know
> there
> are utilities and code snippets in various languages already designed just
> for the purpose of url en/de-coding. There are a lot of other codes
> besides
> the couple you asked about, and you should really be prepared to decode
> any
> that might appear in the data, including ones that you haven't just
> happened
> to encounter yet.
>
> I found a couple of snippets of c code that do this and made them into
> bare
> bones stand-alone executables.
> http://www.aljex.com/bkw/sco/index.html#urlenc
>
> Unfortunately, your data also includes "+" in place of spaces, which is
> another common url hack, and unfortunately, it wouldn't be safe/sane in my
> opinion to wholesale replace all + with " " (which tr can do) because
> surely
> a "+" is a legitimate character that might appear in the data?
>
> The spaces should have been submitted to the cgi as %20 instead of + in
> the
> first place. By the time you recieve the data above, it's too late for you
> to be able to reliably convert the +'s back to spaces without also
> converting +'s that were supposed to actually be +'s in the data.
>
> Here's a command to use the binary above and tr to decode all the url
> codes
> and convert all +'s to spaces and just hope no + ever comes in as data.
> deurl "$rfq" |tr "+" " "
>
> My guess is you are ultimately getting this data into filepro. You could
> leave off the tr and use xlate in filepro almost as easily. In that case,
> you could decide to only xlate certain fields, leaving the possibility of
> real data +'s in some fields. The tr command will convert every + in the
> record and has no concept of fields.
>
> Brian K. White -- brian@aljex.com -- http://www.aljex.com/bkw/
> +++++[>+++[>+++++>+++++++<<-]<-]>>+.>.+++++.+++++++.-.[>+<---]>++.
> filePro BBx Linux SCO Prosper/FACTS AutoCAD #callahans Satriani
>

I can't sed my file because I found out that, after I receive the data from
a http call and then *cat*ing it to a file from a cgi-bin script it makes a
file like this whit the message "Incomplete Last line" and neither *tr* or
*sed* would work unless I *vi* the file and then save the file to get rid of
that message "Imcomplete last line", then tr and sed will work fine:

------------- start ---------------------
rfq=ADDRFQ%7ER-050523%3A020823%7ECARQUEST%7E9%7E1.1%7E2005-05-23T11%3A08%3A23.00
0-07%3A00%0D%0ARFQ%7ER-050523%3A020823%7ER-050523%3A020823%7E2005-05-23T11%3A08%
3A23.000-07%3A00%7E0%7E0%7E283642%7EATK+North+America%7E321 0+S.+Croddy+Way%7E%7E
%7ESanta+Ana%7ECA%7E92704%7EUS%7E%28714%29+850-1544%7E343992%7EJBS+Logistics%7E2
00+Regency+Dr%7E%7E%7EGlendale+Heights%7EIL%7E6013 9%7EUS%7E630-924-2000%7E%0D%0A
LINE%7ER-050523%3A020823%7E1%7ER-050523%3A020823%7EDF01%7EDF01%7E1.0%7EEA%7EFREG
HT%7E%0D%0A
~
~
~
"cq18821.txt" [Incomplete last line] 1 line, 491 characters
----------- cut here ------------------------

Here's what my cgi script does :

car.cgi----------------------------------
#!/bin/ksh
umask 0
.. ./fpsetenv.bat
FILE=/usr2/appl/fpmerge/cq$$.txt
cat > $FILE

echo Content-type: text/plain
echo

......etc (script keeps doing more things)
----------------------------------------------------


How can I fix this issue with the "Incomplete last line" ?

Thanks so much in advance

E Arredondo wrote:

> #!/bin/ksh
> umask 0
> . ./fpsetenv.bat
> FILE=/usr2/appl/fpmerge/cq$$.txt
> cat > $FILE

echo >> $FILE


> echo Content-type: text/plain
> echo



JS

In article <9wGle.2067$rY6.354@newssvr13.news.prodigy.com>,
E Arredondo <atk@sbcglobal.net> wrote:
>I can't sed my file because I found out that, after I receive the data from
>a http call and then *cat*ing it to a file from a cgi-bin script it makes a
>file like this whit the message "Incomplete Last line" and neither *tr* or
>*sed* would work unless I *vi* the file and then save the file to get rid of
>that message "Imcomplete last line", then tr and sed will work fine:
>How can I fix this issue with the "Incomplete last line" ?

tr should work fine. sed does insist on a trailing newline. You can do:

{ cat file; echo ""; } | sed ...

to append a newline to its input. Or use awk, which doesn't have this problem;
replace

sed 's/foo/bar/'

with

awk '{sub(/foo/,"bar")}1'

John
--
John DuBois spcecdt@armory.com KC6QKZ/AE http://www.armory.com/~spcecdt/

"John DuBois" <spcecdt@armory.com> wrote in message
news:119ejq3b7mr18c6@corp.supernews.com...
> In article <9wGle.2067$rY6.354@newssvr13.news.prodigy.com>,
> E Arredondo <atk@sbcglobal.net> wrote:
>>I can't sed my file because I found out that, after I receive the data
>>from
>>a http call and then *cat*ing it to a file from a cgi-bin script it makes
>>a
>>file like this whit the message "Incomplete Last line" and neither *tr* or
>>*sed* would work unless I *vi* the file and then save the file to get rid
>>of
>>that message "Imcomplete last line", then tr and sed will work fine:
>>How can I fix this issue with the "Incomplete last line" ?
>
> tr should work fine. sed does insist on a trailing newline. You can do:
>
> { cat file; echo ""; } | sed ...
>
> to append a newline to its input. Or use awk, which doesn't have this
> problem;
> replace
>
> sed 's/foo/bar/'
>
> with
>
> awk '{sub(/foo/,"bar")}1'
>
> John
> --

My script is working now but I was just wondering if there's a way to
compress the sed into 1 line to make it simpler to save some milliseconds or
maybe seconds of processing time, Does it accepts semi colons or commas
between searches ?, here's what my final script looks like, :

-------------begin here --------------------
FILE=/usr2/appl/fpmerge/cq$$.txt
FILE2=/usr2/appl/fpmerge/cqa$$.txt
cat > $FILE
echo >> $FILE

sed -e "s/%7E/\~/g" < $FILE > $FILE2
sed -e "s/%3A/\:/g" < $FILE2 > $FILE
sed -e "s/%0D%0A/\|/g" < $FILE > $FILE2 # this will let tr down below
change all |'s for LF's
sed -e "s/+/ /g" < $FILE2 > $FILE
sed -e "s/%28/\(/g" < $FILE > $FILE2
sed -e "s/%29/\)/g" < $FILE2 > $FILE
cat $FILE | tr "|" "[\012*]" > $FILE2

------------cut here-------------

E Arredondo wrote:
> "John DuBois" <spcecdt@armory.com> wrote in message
> news:119ejq3b7mr18c6@corp.supernews.com...
>
>>In article <9wGle.2067$rY6.354@newssvr13.news.prodigy.com>,
>>E Arredondo <atk@sbcglobal.net> wrote:
>>
>>>I can't sed my file because I found out that, after I receive the data
>>>from
>>>a http call and then *cat*ing it to a file from a cgi-bin script it makes
>>>a
>>>file like this whit the message "Incomplete Last line" and neither *tr* or
>>>*sed* would work unless I *vi* the file and then save the file to get rid
>>>of
>>>that message "Imcomplete last line", then tr and sed will work fine:
>>>How can I fix this issue with the "Incomplete last line" ?
>>
>>tr should work fine. sed does insist on a trailing newline. You can do:
>>
>>{ cat file; echo ""; } | sed ...
>>
>>to append a newline to its input. Or use awk, which doesn't have this
>>problem;
>>replace
>>
>> sed 's/foo/bar/'
>>
>>with
>>
>> awk '{sub(/foo/,"bar")}1'
>>
>>John
>>--
>
>
> My script is working now but I was just wondering if there's a way to
> compress the sed into 1 line to make it simpler to save some milliseconds or
> maybe seconds of processing time, Does it accepts semi colons or commas
> between searches ?, here's what my final script looks like, :
>
> -------------begin here --------------------
> FILE=/usr2/appl/fpmerge/cq$$.txt
> FILE2=/usr2/appl/fpmerge/cqa$$.txt
> cat > $FILE
> echo >> $FILE
>
> sed -e "s/%7E/\~/g" < $FILE > $FILE2
> sed -e "s/%3A/\:/g" < $FILE2 > $FILE
> sed -e "s/%0D%0A/\|/g" < $FILE > $FILE2 # this will let tr down below
> change all |'s for LF's
> sed -e "s/+/ /g" < $FILE2 > $FILE
> sed -e "s/%28/\(/g" < $FILE > $FILE2
> sed -e "s/%29/\)/g" < $FILE2 > $FILE
> cat $FILE | tr "|" "[\012*]" > $FILE2
>
> ------------cut here-------------
>
>
>
>

You can do multiple operations by separating sed commands with ";"

Just watch the order as each section sees the results of the previous:

date | sed "s/2005/2004/;s/May/June/;s/2004/2003/"

works as you think it would.



--
Tony Lawrence
Unix/Linux/Mac OS X resources: http://aplawrence.com

Tony Lawrence typed (on Tue, May 31, 2005 at 12:36:17PM -0400):
| E Arredondo wrote:
| >
| >My script is working now but I was just wondering if there's a way to
| >compress the sed into 1 line to make it simpler to save some milliseconds
| >or maybe seconds of processing time, Does it accepts semi colons or commas
| >between searches ?, here's what my final script looks like, :
| >
| >-------------begin here --------------------
| >FILE=/usr2/appl/fpmerge/cq$$.txt
| >FILE2=/usr2/appl/fpmerge/cqa$$.txt
| >cat > $FILE
| >echo >> $FILE
| >
| >sed -e "s/%7E/\~/g" < $FILE > $FILE2
| >sed -e "s/%3A/\:/g" < $FILE2 > $FILE
| >sed -e "s/%0D%0A/\|/g" < $FILE > $FILE2 # this will let tr down below
| >change all |'s for LF's
| >sed -e "s/+/ /g" < $FILE2 > $FILE
| >sed -e "s/%28/\(/g" < $FILE > $FILE2
| >sed -e "s/%29/\)/g" < $FILE2 > $FILE
| >cat $FILE | tr "|" "[\012*]" > $FILE2
| >
| >------------cut here-------------
|
| You can do multiple operations by separating sed commands with ";"
|
| Just watch the order as each section sees the results of the previous:
|
| date | sed "s/2005/2004/;s/May/June/;s/2004/2003/"
|
| works as you think it would.
|

I find it easier to read (and *much* easier to edit) if you
write each 'sed' command on its own line:

sed '
s/2005/2004/g
s;May;June;g
s-2004-2003-g
'
--
JP

Jean-Pierre Radley wrote:
> Tony Lawrence typed (on Tue, May 31, 2005 at 12:36:17PM -0400):
> | E Arredondo wrote:
> | >
> | >My script is working now but I was just wondering if there's a way to
> | >compress the sed into 1 line to make it simpler to save some milliseconds
> | >or maybe seconds of processing time, Does it accepts semi colons or commas
> | >between searches ?, here's what my final script looks like, :
> | >
> | >-------------begin here --------------------
> | >FILE=/usr2/appl/fpmerge/cq$$.txt
> | >FILE2=/usr2/appl/fpmerge/cqa$$.txt
> | >cat > $FILE
> | >echo >> $FILE
> | >
> | >sed -e "s/%7E/\~/g" < $FILE > $FILE2
> | >sed -e "s/%3A/\:/g" < $FILE2 > $FILE
> | >sed -e "s/%0D%0A/\|/g" < $FILE > $FILE2 # this will let tr down below
> | >change all |'s for LF's
> | >sed -e "s/+/ /g" < $FILE2 > $FILE
> | >sed -e "s/%28/\(/g" < $FILE > $FILE2
> | >sed -e "s/%29/\)/g" < $FILE2 > $FILE
> | >cat $FILE | tr "|" "[\012*]" > $FILE2
> | >
> | >------------cut here-------------
> |
> | You can do multiple operations by separating sed commands with ";"
> |
> | Just watch the order as each section sees the results of the previous:
> |
> | date | sed "s/2005/2004/;s/May/June/;s/2004/2003/"
> |
> | works as you think it would.
> |
>
> I find it easier to read (and *much* easier to edit) if you
> write each 'sed' command on its own line:
>
> sed '
> s/2005/2004/g
> s;May;June;g
> s-2004-2003-g
> '

Funny - I dislike that type of thing strongly. If I'm going to do that,
I'll put the commands in a file and call it with -f, but I just won't do
that on the command line - no reason, no rationality, just for some
reason I don't like it :-)



--
Tony Lawrence
Unix/Linux/Mac OS X resources: http://aplawrence.com