Hi all,

I have to mirror an http site using wget but I got a file not found
error, and I think this is due to a firewall.

I used the command:
wget -cr http:/.......

Can you suggest me another program to mirror this site?


Thank you

On 4 Sep 2003 10:26:57 -0700, mrlinux <mr_linux@tiscali.it> wrote:
> I have to mirror an http site using wget
>
> wget -cr http:/.......

You might like to look at the '-m' option (instead of '-r') in the wget
man page.


--
Simon <simon@no-dns-yet.org.uk> **** GPG: F4A23C69
"We demand rigidly defined areas of doubt and uncertainty."
- Douglas Adams

On 2003-09-04,
mrlinux <mr_linux@tiscali.it> wrote:
>
> I have to mirror an http site using wget but I got a file not found
> error, and I think this is due to a firewall.

What's the exact error you're getting?

If it turns out to be a robots.txt issue, then IIRC you'd have to
recompile wget for it not to honour this file.

> Can you suggest me another program to mirror this site?

A few people like a program called mirror. I've heard it's good.
I *think* this is the correct link to the project home page.

<URL:http://sunsite.org.uk/packages/mirror/>


--
Mark Hill <mark_usenet@yahoo.co.uk>
"Remind me not to fix mtrr.c after half a litre of wine in future."
- Alan Cox

mrlinux wrote:
> Hi all,
>
> I have to mirror an http site using wget but I got a file not found
> error, and I think this is due to a firewall.
>
> I used the command:
> wget -cr http:/.......
>
> Can you suggest me another program to mirror this site?

If you are wanting to mirror a site you might take a look at
"rsync" to do it.

# rsync -av --delete ftp.server-name.com:ath/to/directory

--
Confucius: He who play in root, eventually kill tree.
Registered with The Linux Counter. http://counter.li.org/
Slackware 9.0 Kernel 2.4.22 i686 (GCC) 3.3
Uptime: 5 days, 15:49, 1 user, load average: 1.55, 1.42, 1.26

On Thursday 04 September 2003 6:47 pm in alt.os.linux.slackware Mark Hill
wrote:

> On 2003-09-04,
> mrlinux <mr_linux@tiscali.it> wrote:
>>
>> I have to mirror an http site using wget but I got a file not found
>> error, and I think this is due to a firewall.
>
> What's the exact error you're getting?
>
> If it turns out to be a robots.txt issue, then IIRC you'd have to
> recompile wget for it not to honour this file.
>
>> Can you suggest me another program to mirror this site?
>
> A few people like a program called mirror. I've heard it's good.
> I *think* this is the correct link to the project home page.
>
> <URL:http://sunsite.org.uk/packages/mirror/>

BTDT - mirror uses ftp so it is not as efficient as rsync.
rsync is fast, works well and is easy to set up, but it does need the
site that you are mirroring to be running an rsync server.

On 2003-09-04,
C. Newport <crn@NOSPAM.netunix.com> wrote:

> BTDT - mirror uses ftp so it is not as efficient as rsync.
> rsync is fast, works well and is easy to set up, but it does need the
> site that you are mirroring to be running an rsync server.

I just realised that the OP was trying to mirror over http, so perhaps
neither of these options is available to him.

Yes, rsync looks *much* better. I'm only just starting to get in to it.


--
Mark Hill <mark_usenet@yahoo.co.uk>

In news:3de60166.0309040926.4af6dc90@posting.google.c om,
mrlinux <mr_linux@tiscali.it> rambled:

>
> I have to mirror an http site using wget but I got a file not found
> error, and I think this is due to a firewall.
>
> I used the command:
> wget -cr http:/.......
>
> Can you suggest me another program to mirror this site?
>

You could try HTTrack.

Billy

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Thu, Sep 04, 2003 at 10:26:57AM -0700, mrlinux wrote:

> I have to mirror an http site using wget but I got a file not found
> error, and I think this is due to a firewall.

> Can you suggest me another program to mirror this site?

ask the people who are running the site that you *HAVE* to mirror what
they recommand.


Jurgen.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.2 (GNU/Linux)

iD8DBQE/V94l1ucXIiwNwbURAikAAJ9wB2sdfUe/JEOAmyrMbAHEEY8uVgCeOAor
sAWIPYsYRb//axBdcZQ/19A=
=z9b0
-----END PGP SIGNATURE-----