As we were seeking to develop WACS 0.9.1, we decided to confront one of the ongoing dilemmas inherant in the dual use scenario we've always had for WACS. Are we an archive tool for collectors or a content management system for web site developers? Generally on the download side of things we'd stuck to our original model as an archive tool, avoiding duplication of data at a cost of greater processor activity and content delivery delays when downloads were requested. With the new tools and features of Wacs 0.9.1, we now provide a choice where none existed before. You can now choose to cache content for quicker and more accurate downloads, and in doing so improve the download experience at the cost of using significantly more disc space.
This emphasis on choice is important as we have not changed the defaults at all - we've just made new tools available to provide an option that didn't exist before. You need to use the tools to create and manage your content cache if you wish to reap the benefits.
The main changes to allow for content and thumbnail caching involve configuring values for the content cache areas within the WACS configuration file. If these values have non-empty values, they will be checked for the existence of a cache prior to falling back to using the conventional cgi-bin program normally used by WACS.
The login and logout programs will update the .htaccess files in the
appropriate toplevel of the content cache tree. There can be up to three
areas in here: images, videos
and thumbnails
depending upon the configuration. The simplest of these is the
thumbnails
which is automatically used if the
server configuration section attribute serverturl
is set to a suitable value. If no cached version of the thumbnail
is available, the wacsthumb application will create one
for the next time. Use of these depends on your application making use of
the thumblink
WACS API function when you add a thumbnail
to your page.
For image sets, the caching process works a little differently -
Nothing is automatically cached - you have to manually
decide to cache a given set using the the section called “The wacscachectl command”
command. This command will create a suitable .zip
file
for the set on demand.
% wacscachectl --create
123
[...]
%
At the simplest level, it works in much the same way for standard videos except that generally the stored video clip is exactly what we want to present to the user (no need to make a zip file) and so in this case the wacscachectl command will create an appropriate symbolic link to the video file within the stored video tree. The system does however provide for rather more complex features if required, but you do not need to use them if you don't want to.
Note | |
---|---|
We've started using the long-present but thus far unused
|
A recent change to the Wacs Placement Manager (wacsplacemgr ) has added a tick box that invokes wacscachectl with the appropriate options for a video file from the web interface when desired. This will be improved and expanded in future releases, but it has already made the process of creating a download cache much simpler.
A lot of commercial sites out there have got into the habit of dividing up their videos into multiple parts in order to keep the download sizes more manageable. The same occasionally also happens for image sets, particularly amongst those sites who offer particularly high resolution images for download. When a set has been marked up with continuation parts, a whole new aspect of the content cache comes into play.
The WACS content cache system includes the concept of having both raw and cooked versions of the same set available. In a video, the cooked version of the primary set would be a compilation of all the parts, while the raw version would be just that - the unchanged first part of the video itself. In many of the standard WACS apps, this distinction is understood and the link to the video clip is appropriate to the context in which it is presented. The main links go to the cooked compilation of all the parts, while the numbered parts would link to the raw versions of the respective videos.
Where a video is multipart, the wacscachectl command will look in your process area for the necessary combilation video. Your process area is adjacent to the unpack area used in the unpacking and placement of photo set and video files within the download area on your server. The WACS system includes a tool to make the appropriate compilation video called wacsvidcomb for WACS VIDeo COMBiner. We'll discuss this tool a little further in a momnet, but lets just review what we have so far in the caching system.
Let's start with the very simple case of a zip file for a single self-contained photo set:
Next let's consider a simple video clip that is fully self contained -
here the symbolic link points to the real video file
sarah124-solo-1a.wmv
within the main /video area but we have used
the sdownload
field to rename the downloaded file:
set124/ Sarah_Movie01_PinkBikini.wmv -> /video/blondes/Sarah/sarah124-solo-1a.wmv
For our third example, we've got a much more complex scenario where this set number is the first part of a three part movie of which we have made a compilation using the wacsvidcomb application. Here we have both raw and cooked versions:
set125/ cooked/ SarahAndJane_Movie01_KitchenCuties.wmv raw/ SarahAndJane_Movie01-part1.wmv -> /video/blondes/Sarah/sarah-jane-gg1-1a.wmv
The wacsvidcomb command retrieves data from the
Wacs set record and uses it to create the compilation video by way of the
mencoder
application that is part of the Linux
mplayer
video player project. Before the
wacsvidcomb program is used, it's a good idea to check that
it's mplayer companion can find all the necessary codecs to play the
types of video files you plan on converting. They share a common architecture
and codec library, so it is a good indication.
At it's simplest level, wacsvidcomb can simply make the compilation using the clips that make up the video sequence as recorded in the database. You must give it the set number of the Primary (first) video in the sequence in order to work. It will produce the resulting compilation, often in better than real time, and place it in your process area within the WACS download directory structure - exactly where a subsequent run of wacscachectl on the same set number would look.
However many of these multipart videos feature 18USC2257 declaration
slates on the front of each part of the video that would make an annoying
and unnecessary interuption on the compilation video. To this end there is
a field within the sets database called sskipfr
for
skip frame. You can now set this value for video
files using the wacsinfomgr as from WACS 0.9.1.
Unfortunately it's not quite as simple as you might hope due to the way
in which movie files on computers are constructed.
If the movie were uncompressed (which would make it HUGE), then you
would specify the number of frames to skip, so if it was a 30 frame per
second video, and the slate lasted six seconds, you would put a value of
180 into sskipfr
. Simple enough. However it's very
rarely that easy and the majority of video compression algorithms start
from a special complete frame called an I-frame and then follow it with only
the modifications to it. The cut can only be made at one of these I-frames
and so the counting is based on the number of I-frames rather than the
number of actual frames.
Unfortunately compression systems vary wildly in how frequently they
force an I-frame; some every second, some every two seconds and some (like
WMV) only on a major scene change. Generally the skip feature of
mencoder
counts in I-frames and trial and error is the only real
way to find the correct value to put into the sskipfr
value for any given video clip. Of course you can always do the edit manually
with any one of a multitude of video editing applications and place the
result into your process directory for wacscachectl to
collect when creating the content cache for the set.