Mirroring a website where you need to log in first

Mirroring a website where you need to log in first

I needed to make an complete offline copy of a website the other day but was having difficulties as you needed to login to the site to get the content.  Luckily, an firefox add-on plus the wonderful wget came to the rescue.

Export cookies (https://addons.mozilla.org/en-US/firefox/addon/export-cookies/?src=api) is a firefox add-on that lets you export cookies in a format that wget understands.

So here is the procedure:
1) Log in to the website
2) Export the cookies into a cookies.txt file
3) Run wget
wget --load-cookies cookies.txt -mkEp  http://website.com
4) Hey-presto
https://addons.mozilla.org/en-US/firefox/addon/export-cookies/?src=api

Comments

Popular posts from this blog

A census of amplified and overexpressed human cancer genes : Nature Reviews Cancer

RT @CancerInNorwich: 📢 Join us on Monday at 1pm for the next talk in our virtual seminar series. We will be hearing from Dr Wafa Al-Jamal from @QUBelfast, who will be talking about "Smart Nanomedicimes for Pancreatic Cancer". All are welcome to join via the MS Teams link: https://t.co/4xCYpjGuBZ https://t.co/EGyD4NAgeP

An excellent opportunity to join the cancer genetics team as a diagnostic lab bioinformatician in Norwich. Lots of exciting projects to get involved in. Deadline: 30/05/2023. Happy to chat about this role. https://t.co/SltGsxYF6Y https://t.co/zh08bqZFDb