Mercurial > mplayer.hg
annotate DOCS/tech/mirrors/mirror_howto.txt @ 22887:dc3ff1498f69
Mention DNS round-robin system in the mirroring HOWTO.
author | diego |
---|---|
date | Tue, 03 Apr 2007 11:03:14 +0000 |
parents | 92e4f4b5828f |
children | 0aaf6309be25 |
rev | line source |
---|---|
18674 | 1 ------------------------------ |
2 How to build an MPlayer mirror | |
3 ------------------------------ | |
4 | |
22774 | 5 This document might be inacurate or even incomplete, please |
6 send feedback + corrections to the mplayer-mirror mailing list. | |
18674 | 7 |
8 About this document | |
9 ~~~~~~~~~~~~~~~~~~~ | |
10 | |
11 Mirroring MPlayer is quite easy but requires a few steps to be taken | |
12 and a few things taken care of. This document describes these steps | |
13 in detail so that anyone wishing to build an official or an unofficial | |
14 mirror can do that without much trouble. | |
15 | |
16 | |
17 | |
18 A note on performance issues | |
19 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
20 | |
21 A few of the commands used here will generate some load on our main server. | |
18725 | 22 Too many clients executing them at once will overload our server and cause |
23 a performance degradation for all our users. Thus we kindly ask you to be | |
24 considerate about what you do. We do not want to restrict mirroring to a | |
25 few select people, but this requires everyone using the system described | |
26 here to behave. | |
18674 | 27 |
28 | |
29 Outline of the mirroring system | |
30 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
31 | |
18725 | 32 The mirroring system uses rsync to transfer the data and to perform |
33 updates. A script (update_mplayer_rsync) is provided to call the rsync | |
34 client with the right set of parameters. This script should be run | |
35 periodically via cron. | |
36 Additionally, official mirrors should set up an ssh account so that | |
37 updates can be triggered when important changes on the main server | |
18674 | 38 are performed. |
18725 | 39 Mirrors should provide their data over HTTP or FTP or both. Each official |
40 mirror will be assigned a mirror number (wwwXXX.mplayerhq.hu or | |
41 ftpXXX.mplayerhq.hu where XXX is the mirror number). This mirror | |
22887
dc3ff1498f69
Mention DNS round-robin system in the mirroring HOWTO.
diego
parents:
22774
diff
changeset
|
42 number determines the hostname over which it can be reached directly. |
dc3ff1498f69
Mention DNS round-robin system in the mirroring HOWTO.
diego
parents:
22774
diff
changeset
|
43 |
dc3ff1498f69
Mention DNS round-robin system in the mirroring HOWTO.
diego
parents:
22774
diff
changeset
|
44 All mirrors are put into our DNS round-robin for the www.mplayerhq.hu and |
dc3ff1498f69
Mention DNS round-robin system in the mirroring HOWTO.
diego
parents:
22774
diff
changeset
|
45 ftp.mplayerhq.hu names and should be set up to respond to these names as well. |
18674 | 46 |
47 | |
48 Getting the data, mirroring script and cron setup | |
49 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
50 | |
18725 | 51 The mirroring script is located in our Subversion repository at |
52 | |
53 svn://svn.mplayerhq.hu/mplayer/trunk/DOCS/tech/mirrors/update_mplayer_rsync | |
54 | |
55 or on the web at | |
56 | |
57 http://svn.mplayerhq.hu/*checkout*/mplayer/trunk/DOCS/tech/mirrors/update_mplayer_rsync | |
18674 | 58 |
59 This script requires a working `rsync` client. The handling of the | |
60 lock file is done by using `lockfile` from the procmail package. | |
61 Using a lock file is recommended but not necessary. The temporary file | |
18725 | 62 generation is handled by `mktemp` which is available from |
63 http://www.mktemp.org/mktemp/ . | |
18674 | 64 |
65 The script contains a few configuration variables at the begining that | |
66 can and should be set: | |
67 PATH: The $PATH to be used within the script (recommended). | |
18725 | 68 LOCK: The full path to the lock file to be used (/var/lock/mplayer-mirror-lock |
69 or something similar, recommended). | |
18674 | 70 MIRROR_ROOT: The root of the mirror. This is the directory where all files |
18725 | 71 are downloaded to (required). |
72 MAILADR: The mail address where reports should be sent to (required). | |
73 TMPDIR: The directory where temporary files should be created. | |
74 If you set this explicitly, you have to uncomment the export below | |
75 (defaults to /tmp if not set). | |
18674 | 76 |
77 Install this script and set the variables according to your setup. Then run | |
18725 | 78 it once to get the first checkout of the mirror. This will require (at the |
79 time of this writing - 2006-06) about 500MB of disk space. | |
18674 | 80 You should get two directories in your $MIRROR_ROOT: homepage and MPlayer. |
18725 | 81 The former containing the HTML pages for the mirror and the latter the |
82 downloadable files. | |
18674 | 83 |
18725 | 84 If this worked out OK, you should set up a cron job that periodically updates |
18674 | 85 the files. If you run an official mirror you should run the script every |
86 6h to 12h (6h recommended). If you do not run an official mirror, you should | |
87 not run the script more often than once a day. Please use an "odd" time | |
88 to run the script when it is unlikely that any other cron job is running. | |
89 Bad times are e.g. full hours, or minutes that are divisible by 5. | |
90 An example crontab line would look like this: | |
91 --- | |
92 17 1,8,13,19 * * * /path/to/update_mplayer_rsync | |
93 --- | |
94 (please change the minute and hours to something random) | |
95 | |
96 You can change the rest of the script as you see fit, although it is not | |
97 recommended. Please DO NOT CHANGE the options of the rsync commands. | |
98 Especially DO NOT REMOVE the -t and -W options. These prevent calculating | |
99 checksums on the server side which are very expensive. | |
100 | |
101 | |
18725 | 102 Setting up an SSH account for update triggers |
18674 | 103 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
104 | |
18725 | 105 Official mirrors should provide an ssh-based trigger to run the update script |
106 on request. This makes it possible to distribute releases and other important | |
107 files immediately to all mirrors. | |
18674 | 108 |
18725 | 109 The setup does not need a special user other than the one as which the update |
110 script is run and does not allow running any other command. | |
18674 | 111 |
18725 | 112 First you need to create an ssh key pair by running: |
18674 | 113 --- |
114 ssh-keygen -t dsa -C MPHQ_rsync_trigger -f www#_sshkey | |
115 --- | |
116 (replace the '#' by your mirror number) | |
117 | |
118 You should send the private key to us by mail and specify the host and | |
18725 | 119 user to be used. Please use a private mail of one of the admins and |
120 DO NOT send the private key to the mirror mailing list. | |
18674 | 121 |
18725 | 122 The public key should be placed in the ~/.ssh/authorized_keys file of the |
123 user running the updates. To restrict the ssh key to only one command place | |
124 the following directives at the begining of the line with the key: | |
125 from="*.mplayerhq.hu",command="<path_to_update_mplayer_rsync>" | |
18674 | 126 e.g.: |
127 --- | |
18725 | 128 from="*.mplayerhq.hu",command="/path/to/update_mplayer_rsync" ssh-dss AAAA |
18674 | 129 B3NzaC1kc3MAAAEBAI20yhE3/bRjzojUhhMz4DHnGhcJUiPWOfoP9CygnFOYOxJTFlxgqM3iJiHWRxgK |
130 FJ/Uw40eV9K4Ww4fp2pe1guXJzKna8+6vBXaPPVEVxSyaxgtt4Xt3zpUuCnNljgArcEhwcNyOyH2RVln | |
131 yhyxsrKhuq5ZoNHD3caBGjZu3eOR2atPGS1NOdeN/hytIoh8T8DicPqPI29yWX9yAjnHv6wdPutwMLu6 | |
132 [...] | |
133 n0Fs3CJY6/1UpgDGH7VPey0SdpJEDewltRLA+buP++2vJD/NUOeGzcRydo2NdZ1wiiaytXxkaec928JC | |
134 NABTeBh6NKAg4vnPvcRLKEBVdSrar/fARSbOmf3HOcsw3uZoAIE9jDGhnMKcnXfHjPZ2tZP9CHs6Wo4n | |
135 yDOxIfDZmJ7VJqMRc6//p5k81pkkGvawbPA63StI/Dkv/648l4XONuJc2z5gaUdjrTA8TsD/VJGiGcHl | |
136 mlGj3IWCBz7e4+XB3L64kFZwLCYN8kwDUAaHq4EtcMVOnQ== MPHQ_rsync_trigger | |
137 --- | |
138 (lines split for readability) | |
139 | |
140 | |
141 Setting up a webserver | |
142 ~~~~~~~~~~~~~~~~~~~~~~ | |
143 | |
18726 | 144 Set up Apache or whatever web server you prefer. We just have static pages, |
145 so no fancy configuration is necessary. However, we need a few aliases so that | |
146 links on our pages work correctly. /MPlayer and /DOCS should redirect to the | |
147 directory with the downloadable files and /homepage should redirect to the | |
148 directory with the HTML pages. | |
149 | |
150 Here is an example stanza to paste into your Apache configuration: | |
151 | |
152 <VirtualHost www#.mplayerhq.hu> | |
153 DocumentRoot /path/to/htdocs | |
19268 | 154 Options FollowSymLinks Indexes |
18726 | 155 Alias /homepage /path/to/htdocs |
156 Alias /MPlayer /path/to/MPlayer | |
157 Alias /DOCS /path/to/MPlayer/DOCS | |
20458
038e29652be8
Add a note about charset configuration to the Apache section.
diego
parents:
19268
diff
changeset
|
158 AddDefaultCharset off |
18726 | 159 </VirtualHost> |
18674 | 160 |
22887
dc3ff1498f69
Mention DNS round-robin system in the mirroring HOWTO.
diego
parents:
22774
diff
changeset
|
161 <VirtualHost www.mplayerhq.hu> |
dc3ff1498f69
Mention DNS round-robin system in the mirroring HOWTO.
diego
parents:
22774
diff
changeset
|
162 DocumentRoot /path/to/htdocs |
dc3ff1498f69
Mention DNS round-robin system in the mirroring HOWTO.
diego
parents:
22774
diff
changeset
|
163 Options FollowSymLinks Indexes |
dc3ff1498f69
Mention DNS round-robin system in the mirroring HOWTO.
diego
parents:
22774
diff
changeset
|
164 Alias /homepage /path/to/htdocs |
dc3ff1498f69
Mention DNS round-robin system in the mirroring HOWTO.
diego
parents:
22774
diff
changeset
|
165 Alias /MPlayer /path/to/MPlayer |
dc3ff1498f69
Mention DNS round-robin system in the mirroring HOWTO.
diego
parents:
22774
diff
changeset
|
166 Alias /DOCS /path/to/MPlayer/DOCS |
dc3ff1498f69
Mention DNS round-robin system in the mirroring HOWTO.
diego
parents:
22774
diff
changeset
|
167 AddDefaultCharset off |
dc3ff1498f69
Mention DNS round-robin system in the mirroring HOWTO.
diego
parents:
22774
diff
changeset
|
168 </VirtualHost> |
dc3ff1498f69
Mention DNS round-robin system in the mirroring HOWTO.
diego
parents:
22774
diff
changeset
|
169 |
20458
038e29652be8
Add a note about charset configuration to the Apache section.
diego
parents:
19268
diff
changeset
|
170 The AddDefaultCharset directive is necessary because newer versions of Apache |
038e29652be8
Add a note about charset configuration to the Apache section.
diego
parents:
19268
diff
changeset
|
171 appear to default to defining a standard charset. This breaks our documentation |
038e29652be8
Add a note about charset configuration to the Apache section.
diego
parents:
19268
diff
changeset
|
172 translations, which are written in different encodings. |
038e29652be8
Add a note about charset configuration to the Apache section.
diego
parents:
19268
diff
changeset
|
173 |
22887
dc3ff1498f69
Mention DNS round-robin system in the mirroring HOWTO.
diego
parents:
22774
diff
changeset
|
174 The second VirtualHost is necessary for the DNS round-robin address. |
dc3ff1498f69
Mention DNS round-robin system in the mirroring HOWTO.
diego
parents:
22774
diff
changeset
|
175 |
18674 | 176 |
177 Setting up an FTP server | |
178 ~~~~~~~~~~~~~~~~~~~~~~~~ | |
179 | |
18726 | 180 Any FTP server will do. We use vsftpd, it's fast and secure. You should have |
181 the same directory layout as we do on our server, so there should be a | |
182 subdirectory named 'MPlayer' (notice the capitals) that contains the | |
183 downloadable files. | |
18674 | 184 |
185 | |
186 Mailinglist | |
187 ~~~~~~~~~~~ | |
188 | |
189 TO BE DONE |