Tool to download large HTML file

Sometimes when you need to download catalog data using a web browser (Chrome, Firefox, etc.), it will take some time and might cause crash or lag. Here is some ways to do it:

I. Using wget
+ Install if you do not have in terminal:

yum install wget

or

sudo apt-get install wget

+ Basic commands:

wget –output-document= {output file}  {link to download}

E.g:wget ‐‐output-document=filename.html example.com

Details of how to use it can be find here or here.

II. A simple python code

All you need to do it install python and tqdm package using

pip install tqdm

Or run the sh file which I have already written the code to install tqdm package.

Run ./download.sh or python download.py to run, example below:

You can download this small utility here.

—Nguyễn Công Nghĩa (IES, Academia Sinica)