Mercurial > mplayer.hg
view DOCS/tech/mirrors/mirror_howto.txt @ 22887:dc3ff1498f69
Mention DNS round-robin system in the mirroring HOWTO.
author | diego |
---|---|
date | Tue, 03 Apr 2007 11:03:14 +0000 |
parents | 92e4f4b5828f |
children | 0aaf6309be25 |
line wrap: on
line source
------------------------------ How to build an MPlayer mirror ------------------------------ This document might be inacurate or even incomplete, please send feedback + corrections to the mplayer-mirror mailing list. About this document ~~~~~~~~~~~~~~~~~~~ Mirroring MPlayer is quite easy but requires a few steps to be taken and a few things taken care of. This document describes these steps in detail so that anyone wishing to build an official or an unofficial mirror can do that without much trouble. A note on performance issues ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ A few of the commands used here will generate some load on our main server. Too many clients executing them at once will overload our server and cause a performance degradation for all our users. Thus we kindly ask you to be considerate about what you do. We do not want to restrict mirroring to a few select people, but this requires everyone using the system described here to behave. Outline of the mirroring system ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The mirroring system uses rsync to transfer the data and to perform updates. A script (update_mplayer_rsync) is provided to call the rsync client with the right set of parameters. This script should be run periodically via cron. Additionally, official mirrors should set up an ssh account so that updates can be triggered when important changes on the main server are performed. Mirrors should provide their data over HTTP or FTP or both. Each official mirror will be assigned a mirror number (wwwXXX.mplayerhq.hu or ftpXXX.mplayerhq.hu where XXX is the mirror number). This mirror number determines the hostname over which it can be reached directly. All mirrors are put into our DNS round-robin for the www.mplayerhq.hu and ftp.mplayerhq.hu names and should be set up to respond to these names as well. Getting the data, mirroring script and cron setup ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The mirroring script is located in our Subversion repository at svn://svn.mplayerhq.hu/mplayer/trunk/DOCS/tech/mirrors/update_mplayer_rsync or on the web at http://svn.mplayerhq.hu/*checkout*/mplayer/trunk/DOCS/tech/mirrors/update_mplayer_rsync This script requires a working `rsync` client. The handling of the lock file is done by using `lockfile` from the procmail package. Using a lock file is recommended but not necessary. The temporary file generation is handled by `mktemp` which is available from http://www.mktemp.org/mktemp/ . The script contains a few configuration variables at the begining that can and should be set: PATH: The $PATH to be used within the script (recommended). LOCK: The full path to the lock file to be used (/var/lock/mplayer-mirror-lock or something similar, recommended). MIRROR_ROOT: The root of the mirror. This is the directory where all files are downloaded to (required). MAILADR: The mail address where reports should be sent to (required). TMPDIR: The directory where temporary files should be created. If you set this explicitly, you have to uncomment the export below (defaults to /tmp if not set). Install this script and set the variables according to your setup. Then run it once to get the first checkout of the mirror. This will require (at the time of this writing - 2006-06) about 500MB of disk space. You should get two directories in your $MIRROR_ROOT: homepage and MPlayer. The former containing the HTML pages for the mirror and the latter the downloadable files. If this worked out OK, you should set up a cron job that periodically updates the files. If you run an official mirror you should run the script every 6h to 12h (6h recommended). If you do not run an official mirror, you should not run the script more often than once a day. Please use an "odd" time to run the script when it is unlikely that any other cron job is running. Bad times are e.g. full hours, or minutes that are divisible by 5. An example crontab line would look like this: --- 17 1,8,13,19 * * * /path/to/update_mplayer_rsync --- (please change the minute and hours to something random) You can change the rest of the script as you see fit, although it is not recommended. Please DO NOT CHANGE the options of the rsync commands. Especially DO NOT REMOVE the -t and -W options. These prevent calculating checksums on the server side which are very expensive. Setting up an SSH account for update triggers ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Official mirrors should provide an ssh-based trigger to run the update script on request. This makes it possible to distribute releases and other important files immediately to all mirrors. The setup does not need a special user other than the one as which the update script is run and does not allow running any other command. First you need to create an ssh key pair by running: --- ssh-keygen -t dsa -C MPHQ_rsync_trigger -f www#_sshkey --- (replace the '#' by your mirror number) You should send the private key to us by mail and specify the host and user to be used. Please use a private mail of one of the admins and DO NOT send the private key to the mirror mailing list. The public key should be placed in the ~/.ssh/authorized_keys file of the user running the updates. To restrict the ssh key to only one command place the following directives at the begining of the line with the key: from="*.mplayerhq.hu",command="<path_to_update_mplayer_rsync>" e.g.: --- from="*.mplayerhq.hu",command="/path/to/update_mplayer_rsync" ssh-dss AAAA B3NzaC1kc3MAAAEBAI20yhE3/bRjzojUhhMz4DHnGhcJUiPWOfoP9CygnFOYOxJTFlxgqM3iJiHWRxgK FJ/Uw40eV9K4Ww4fp2pe1guXJzKna8+6vBXaPPVEVxSyaxgtt4Xt3zpUuCnNljgArcEhwcNyOyH2RVln yhyxsrKhuq5ZoNHD3caBGjZu3eOR2atPGS1NOdeN/hytIoh8T8DicPqPI29yWX9yAjnHv6wdPutwMLu6 [...] n0Fs3CJY6/1UpgDGH7VPey0SdpJEDewltRLA+buP++2vJD/NUOeGzcRydo2NdZ1wiiaytXxkaec928JC NABTeBh6NKAg4vnPvcRLKEBVdSrar/fARSbOmf3HOcsw3uZoAIE9jDGhnMKcnXfHjPZ2tZP9CHs6Wo4n yDOxIfDZmJ7VJqMRc6//p5k81pkkGvawbPA63StI/Dkv/648l4XONuJc2z5gaUdjrTA8TsD/VJGiGcHl mlGj3IWCBz7e4+XB3L64kFZwLCYN8kwDUAaHq4EtcMVOnQ== MPHQ_rsync_trigger --- (lines split for readability) Setting up a webserver ~~~~~~~~~~~~~~~~~~~~~~ Set up Apache or whatever web server you prefer. We just have static pages, so no fancy configuration is necessary. However, we need a few aliases so that links on our pages work correctly. /MPlayer and /DOCS should redirect to the directory with the downloadable files and /homepage should redirect to the directory with the HTML pages. Here is an example stanza to paste into your Apache configuration: <VirtualHost www#.mplayerhq.hu> DocumentRoot /path/to/htdocs Options FollowSymLinks Indexes Alias /homepage /path/to/htdocs Alias /MPlayer /path/to/MPlayer Alias /DOCS /path/to/MPlayer/DOCS AddDefaultCharset off </VirtualHost> <VirtualHost www.mplayerhq.hu> DocumentRoot /path/to/htdocs Options FollowSymLinks Indexes Alias /homepage /path/to/htdocs Alias /MPlayer /path/to/MPlayer Alias /DOCS /path/to/MPlayer/DOCS AddDefaultCharset off </VirtualHost> The AddDefaultCharset directive is necessary because newer versions of Apache appear to default to defining a standard charset. This breaks our documentation translations, which are written in different encodings. The second VirtualHost is necessary for the DNS round-robin address. Setting up an FTP server ~~~~~~~~~~~~~~~~~~~~~~~~ Any FTP server will do. We use vsftpd, it's fast and secure. You should have the same directory layout as we do on our server, so there should be a subdirectory named 'MPlayer' (notice the capitals) that contains the downloadable files. Mailinglist ~~~~~~~~~~~ TO BE DONE