Components Of The Download System

chkmodel

The chkmodel takes a single argument, a model number and proceeds to look at all the IDmaps for that model - for each IDmap, it checks if it knows a username and password for the relevant site and if it does, downloaded the relevant model page from the site. It then checks through that model page for the image set and video clip links, and adds any new ones it finds to the collection of download records for that model. Depending upon what it finds, it will update the models IDmap to show when it was last checked and when it last found a new set for that model. This way those models with an actively growing portfolio are checked more regularly than those who are no longer working and producing new sets.

The chkmodel is also capable of working on a saved web page in an off-line mode for those web sites whose pages are marked as unusable (ie the vmpaguse field in the vendor database entry for that site is set to N for no). Here it will look through the model index page seeking links to set pages and recording the details of what it finds that it did not already know about - in most cases this will probably be an incomplete (download status X) download record. The model's record for that site will be updated to show that it was last checked on the day on which the model index page it processed was saved to disc. The additional details needed can be added using the wacsdnlframe web app at a later time (see wacsdnlframe for more information).

[Note]Note

The enhancements to chkmodel to allow it to use saved files well were added at Wacs release 0.8.4; additional work to make the naming of saved files easier is expected to be added at a later release.

To save a page for chkmodel to read, it needs to go in a very specific directory - the modelpage-cache directory of the site directory in the download area. In a default package installation, and for a site called wacsdemo, this would be: /var/spool/wacs/download/wacsdemo/modelpage-cache/. The file then needs to be called a very specific name - the first part is the site code (for wacsdemo this might be WACSD), then a hyphen (-), then the model's key (model number/reference on that site) such as 123, and then another hyphen (-) and finally her model number on this Wacs installation such as 345. Thus the page would need to be saved as WACSD-123-345 for this model.

You would then run chkmodel 345 to scan that saved page for new sets. Any icons found in WACSD-123-345_files would also be transfered to the model's icon directory in icons in the site download directory (eg /var/spool/wacs/download/wacsdemo/). This naming convention is a little cumbersome and exposes more of Wacs's internal workings than is strictly desirable, and we do plan on addressing that in a future release. Right now we want to get the technology available to people who want to use it on sites where automatic download is not supported.

refresh

The refresh command is designed to be run by a nightly automatic job (aka crontab and calls chkmodel with the appropriate arguments to check a specific number of models each night. It limits itself to just a few so as to not be a "bad citizen" and outstay it's welcome. Refresh does actually use some fairly carefully designed metrics to optimise the way it looks for models, biasing it's checks towards those who are receiving regular updates and within those selecting those that have gone longest since the last check. It is generally pretty thorough in what it does.

getarc

The getarc command completes the picture by actually downloading the sets previously highlighted by the combination of refresh and chkmodel. Once getarc has obtained the necessary zip archive or movie file, it marks the set as pending at which point it can be unpacked using the standard Wacs unpack and placement tools invoked from either the download list or the model page (full version).

Again it is normally expected that getarc will be run at a quiet time of night by the crontab timed execution system.

wacsdnlnext

The wacsdnlnext web app is designed to assist you and guide you through downloading sets from those sites where getarc is not suitable for use. Detailed instructions on using it can be found in the reference section, wacsdnlnext.

[Warning]Warning

We do plan to overhaul the download system in the relatively near future to better cope with cookies and other techniques that sites use to ensure their security. We also currently invoke the command line based web download tool, wget for all downloads and feel it would be better if it did the work internally using the web download library for perl (LWP). Doing this should also improve the portability of Wacs to other platforms because expecting Unix/Linux command line semantics in this is a significant problem on other platforms, especially Windows.

Wacs Download manager

The Wacs Download Manager, wacsdnlmgr allows the resolution of some of the most common problems with download records. It's primary purpose is to allow manual changes to the status and set associations of download records. If you find a record you've downloaded has a broken zip file, you can use the download manager to list that download as Failed so it gets re-downloaded at some point in the future. If you have both models in a lesbian set having a download record in their own right for the set, you can use the download manager to mark the second download record as either successfully resolved or as a relationship entry. This usually depends on which model's name comes first - if the model upon the strength of whose record we unpacked it is the first model mentioned, the other(s) is a relationship entry; if not, both are marked as being successfull. It can also mark a set as being in error if the model featured in it is patently not the model to whom it has been attributed.

The easiest way to access it is via either the link on the download number on the detailed form of the model page, or on the entry in the download list. The first option via the model page only works while the download record is marked as unresolved (ie not Successful, Relationship Entry or Error.) To access a model record that is considered resolved, administrators will find a link on the download number listed in the details in the set info page.