Wednesday, April 21, 2010

Wget Vs. libcurl

People still use Wget as a quick command, sometimes from cosole and even embedded in their code.Please take a few precautions while using WGET

1/ WGET’s default behavior is retry, which can be killing at times. So use it with retry option if you presume, it to fail,

Ex.

wget -t 10 -w 5 --waitretry=10 2/ Try using libcurl instead of WGET, if possible,

As curl uses somewhat single-shot transfers of data mode, and WGET works on recursive mode. Hence, a particular http connection in an apache web server, which maintains Persistency, can prove fatal at times, that too when no retry count is specified, as we will be kind of misusing a valuable http connection slot, as it will linger for ever, or till when apache gets restarted.

Wget still does its HTTP operations using HTTP 1.0. In contrast curl which is basiccaly NOT a commnad rather a a cross-platform library with a stable API, supports multiple protocols.


Cheers! Happy piping !

No comments:

Post a Comment

AWS re:Invent summary 2017

As you might know, this year’s AWS re:Invent was held @ Las Vegas 11/27 – 12/1.   Below links could be a good start to explore all ...