Configuring Automatic Download

As we've discussed, one of the significant features of Wacs is it's ability to monitor subscription web sites for updates to your favourite models, and while these tasks can be done manually on-demand, they can also be automated in conjunction with your operating system's timed execution facilities. For this to work of course, you do need to make sure that you've added your account details for each site that you want monitored into the vendor database using wacsvendmgr. You also need to make sure that each model's identity on that site has been entered as well; this can be done using the Identity Management mode of the wacsmodelmgr which will be discussed in the next chapter (See Chapter 15, More About Model Manager).

Defining Sites

If you're using a pre-defined site record for the Vendor site you wish to download from, you should have an easy time; just enable the site, add your password, and then add a few model details using the Wacs Model Manager and it should all start working. If you're creating a new Vendor site description, it can be rather more difficult.

The first thing you need to understand is how to set the Image and Video download id types to the correct values (vidtimg and vidtvid in the vendor schema - see section 3 of the programming manual). The actions take place in two places - first the chkmodel script must actually find the desired zip or video file name and create a download record requesting it be fetched. Second, the getarc script must actually perform the necessary retrieval and saving actions. The following table details how the various values currently work:

TypeValueDescription
Image Sets1Default - just get the URL specified in the download record modulo the addition of the specified image server URL if the download record URL doesn't start with http://
Video Clips1Default - just get the URL specified in the download record modulo the addition of the specified video server URL if the download record URL doesn't start with http://

Alternative values you may see invoke custom code designed for specific sites only which are not expected to be applicable to more general use. These typically include work-arounds such as re-navigating the lead-in web pages as the final target URLs are liable to change, or guess that given a certain stem such as jennifer1.zip, additional zip files in this set will be called jennifer2.zip, jennifer3.zip and so on.

Using cron For Unattended Download

On Linux, MacOS X and other Unix like systems, the automatic execution system is known as cron and it's operation is managed by a somewhat cryptic program also called crontab. Normally cron will be running already since it is used by the operating system itself for housekeeping. It is important to understand that crontab does not "wake up" a machine that is switched off or suspended, and so setting to run an overnight job when you turn it off each evening as you go to bed will not prove useful because the job will never run. If the machine is left on, as most servers probably will be, a time in the early hours of the morning will probably be fine; if the machine is normally switched off overnight, choose a time when the machine has a good chance of being switched on. The tasks will run in the background and will not affect other users of the system particularly.

To make use of the automatic download feature, you need to enable two more programs to run at a suitable time: refresh and getarc. refresh scans registered models and checks for updates; getarc fetches a number of the updates that refresh has found. To enable the automatic scanning feature, you need to open a terminal window as a suitable Wacs administrator user and type:

% crontab -e

and add the following line to the crontab file:

 
22 0 * * *      /usr/local/bin/refresh

[Tip]Tip

Crontab usually calls up the vi editor - you can recognise this by a tilde (~) symbol repeated all the way down the left hand side of the screen. This editor is a bit cryptic if you don't know it, but basically you need to press the lower case letter o, type the line above, press Enter, then press Escape and then type :wq. That should get the job done.

This will cause the refresh command to run at twenty-two minutes past midnight each day. Note that by default refresh will only check those models the Wacs system currently considers to be Active, ie those who have been found to have new sets within the last three months or so. An additional option to refresh, -a allows it to scan all models, including those marked as dormant. It is recommended to run this occasionally so that it will double check the dormant models at a much slower rate; you might wish for instance to check dormant models twice a week, and to do this you would add a second crontab entry of:

42 0 * * 2,5    /usr/local/bin/refresh -a

Once refresh has been run, any newly discovered sets will be registered in the WACS system - they can be examined using the Download list wacsdnllist or are listed per-model in the detailed model page wacsmodelpage. In order for them to be downloaded automatically you will also have to invoke the getarc command at some suitable point - I choose 6:45 in the morning. To set this up, you add the following to the crontab in the same way as above:

45 6 * * *      /usr/local/bin/getarc

The cron command when it runs will automatically email you with the details of what happened for each of the commands you've set it up to run in the crontab file.