Download script for images

×

Status message

You are not a member of this team. If you want to be part of this team, click on 'Subscribe to this team'.
MichaelKnight's picture
Submitted by MichaelKnight on Tue, 11/10/2020 - 19:59
Team(s): 
Coffee Shop

Hi guys,

a friend if mine and myself were building a small home server for the last months.
Said friend also wrote a python script to automatically download the install disc images from the FTP once a night.
This way, I have the actual versions at hand all the time and do not need to wait for a hour (or day-) long download when I find out I need a new image.
So I asked the dev of the script if he would like to open source the script and in return he said "sure but only if somebody also wants to have it, as I would need to tidy up the code before in this case".

So here you go, do you want to have the script? :)

Comments

jprisab's picture
Submitted by jprisab on Tue, 11/10/2020 - 20:38

Hello all,

please be aware that the FTP server is hosted by the icat institute and is not does not run in limitless-resource Hardware

having many people downloading 1xnight 500Gbs may overload the system, impacting other users and finally impacting as well the performance of other systems running in the icat Datacenter...

though it is a great idea, I would like to discourage implementing it in a large scale...

Hope you understand my concerns

BR

Javier

Rhein-Ruhr-Hub's picture
Submitted by Rhein-Ruhr-Hub on Wed, 11/11/2020 - 05:51

Hi Sebastian,
nice idea, but following Javiers concern there are other ready-to-use tools, which work more sustainable, e.g. Filezilla or rsync. You can run those tools in the background and/or as a cron job (time controlled) . They can be set in a way that it will only download those files, being updated since the last download.
See also https://www.labdoo.org/content/new-folder-structure-ftp-server-neue-ordn...
CU
Ralf

MichaelKnight's picture
Submitted by MichaelKnight on Wed, 11/11/2020 - 14:57

Hi Ralf and Javier,

sorry for my misleading comment. Of course our script does only donwload each image once. If nothing changed between local and remote image, it will not be downloaded again.
Additionally, it is easily configurable which languages you want. So for example, I am only downloading EN, FR and ES as these are the ones that I frequently use.
So I think your concerns are already covered by our current solution.

One more thing that I currently think of:
We are automating our script using Jenkins. Jenkins offers a wide range of possibilities for time based triggers. So I think I will set up that it should run once a week, on a random day of the week.
If everyone would do it this way, it would balance the load quite well.

Gruß,
seb