Mercurial > hgbook
comparison en/ch04-concepts.xml @ 658:b90b024729f1
WIP DocBook snapshot that all compiles. Mirabile dictu!
author | Bryan O'Sullivan <bos@serpentine.com> |
---|---|
date | Wed, 18 Feb 2009 00:22:09 -0800 |
parents | en/ch04-concepts.tex@f72b7e6cbe90 |
children | 13513d2a128d |
comparison
equal
deleted
inserted
replaced
657:8631da51309b | 658:b90b024729f1 |
---|---|
1 <!-- vim: set filetype=docbkxml shiftwidth=2 autoindent expandtab tw=77 : --> | |
2 | |
3 <chapter id="chap:concepts"> | |
4 <title>Behind the scenes</title> | |
5 | |
6 <para>Unlike many revision control systems, the concepts upon which | |
7 Mercurial is built are simple enough that it's easy to understand | |
8 how the software really works. Knowing this certainly isn't | |
9 necessary, but I find it useful to have a <quote>mental | |
10 model</quote> of what's going on.</para> | |
11 | |
12 <para>This understanding gives me confidence that Mercurial has been | |
13 carefully designed to be both <emphasis>safe</emphasis> and | |
14 <emphasis>efficient</emphasis>. And just as importantly, if it's | |
15 easy for me to retain a good idea of what the software is doing | |
16 when I perform a revision control task, I'm less likely to be | |
17 surprised by its behaviour.</para> | |
18 | |
19 <para>In this chapter, we'll initially cover the core concepts | |
20 behind Mercurial's design, then continue to discuss some of the | |
21 interesting details of its implementation.</para> | |
22 | |
23 <sect1> | |
24 <title>Mercurial's historical record</title> | |
25 | |
26 <sect2> | |
27 <title>Tracking the history of a single file</title> | |
28 | |
29 <para>When Mercurial tracks modifications to a file, it stores | |
30 the history of that file in a metadata object called a | |
31 <emphasis>filelog</emphasis>. Each entry in the filelog | |
32 contains enough information to reconstruct one revision of the | |
33 file that is being tracked. Filelogs are stored as files in | |
34 the <filename role="special" | |
35 class="directory">.hg/store/data</filename> directory. A | |
36 filelog contains two kinds of information: revision data, and | |
37 an index to help Mercurial to find a revision | |
38 efficiently.</para> | |
39 | |
40 <para>A file that is large, or has a lot of history, has its | |
41 filelog stored in separate data | |
42 (<quote><literal>.d</literal></quote> suffix) and index | |
43 (<quote><literal>.i</literal></quote> suffix) files. For | |
44 small files without much history, the revision data and index | |
45 are combined in a single <quote><literal>.i</literal></quote> | |
46 file. The correspondence between a file in the working | |
47 directory and the filelog that tracks its history in the | |
48 repository is illustrated in figure <xref | |
49 linkend="fig:concepts:filelog"/>.</para> | |
50 | |
51 <informalfigure id="fig:concepts:filelog"> | |
52 <mediaobject><imageobject><imagedata | |
53 fileref="filelog"/></imageobject><textobject><phrase>XXX | |
54 add text</phrase></textobject> | |
55 <caption><para>Relationships between files in working | |
56 directory and filelogs in | |
57 repository</para></caption></mediaobject> | |
58 </informalfigure> | |
59 | |
60 </sect2> | |
61 <sect2> | |
62 <title>Managing tracked files</title> | |
63 | |
64 <para>Mercurial uses a structure called a | |
65 <emphasis>manifest</emphasis> to collect together information | |
66 about the files that it tracks. Each entry in the manifest | |
67 contains information about the files present in a single | |
68 changeset. An entry records which files are present in the | |
69 changeset, the revision of each file, and a few other pieces | |
70 of file metadata.</para> | |
71 | |
72 </sect2> | |
73 <sect2> | |
74 <title>Recording changeset information</title> | |
75 | |
76 <para>The <emphasis>changelog</emphasis> contains information | |
77 about each changeset. Each revision records who committed a | |
78 change, the changeset comment, other pieces of | |
79 changeset-related information, and the revision of the | |
80 manifest to use.</para> | |
81 | |
82 </sect2> | |
83 <sect2> | |
84 <title>Relationships between revisions</title> | |
85 | |
86 <para>Within a changelog, a manifest, or a filelog, each | |
87 revision stores a pointer to its immediate parent (or to its | |
88 two parents, if it's a merge revision). As I mentioned above, | |
89 there are also relationships between revisions | |
90 <emphasis>across</emphasis> these structures, and they are | |
91 hierarchical in nature.</para> | |
92 | |
93 <para>For every changeset in a repository, there is exactly one | |
94 revision stored in the changelog. Each revision of the | |
95 changelog contains a pointer to a single revision of the | |
96 manifest. A revision of the manifest stores a pointer to a | |
97 single revision of each filelog tracked when that changeset | |
98 was created. These relationships are illustrated in figure | |
99 <xref linkend="fig:concepts:metadata"/>.</para> | |
100 | |
101 <informalfigure id="fig:concepts:metadata"> | |
102 <mediaobject><imageobject><imagedata | |
103 fileref="metadata"/></imageobject><textobject><phrase>XXX | |
104 add text</phrase></textobject><caption><para>Metadata | |
105 relationships</para></caption> | |
106 </mediaobject> | |
107 </informalfigure> | |
108 | |
109 <para>As the illustration shows, there is | |
110 <emphasis>not</emphasis> a <quote>one to one</quote> | |
111 relationship between revisions in the changelog, manifest, or | |
112 filelog. If the manifest hasn't changed between two | |
113 changesets, the changelog entries for those changesets will | |
114 point to the same revision of the manifest. If a file that | |
115 Mercurial tracks hasn't changed between two changesets, the | |
116 entry for that file in the two revisions of the manifest will | |
117 point to the same revision of its filelog.</para> | |
118 | |
119 </sect2> | |
120 </sect1> | |
121 <sect1> | |
122 <title>Safe, efficient storage</title> | |
123 | |
124 <para>The underpinnings of changelogs, manifests, and filelogs are | |
125 provided by a single structure called the | |
126 <emphasis>revlog</emphasis>.</para> | |
127 | |
128 <sect2> | |
129 <title>Efficient storage</title> | |
130 | |
131 <para>The revlog provides efficient storage of revisions using a | |
132 <emphasis>delta</emphasis> mechanism. Instead of storing a | |
133 complete copy of a file for each revision, it stores the | |
134 changes needed to transform an older revision into the new | |
135 revision. For many kinds of file data, these deltas are | |
136 typically a fraction of a percent of the size of a full copy | |
137 of a file.</para> | |
138 | |
139 <para>Some obsolete revision control systems can only work with | |
140 deltas of text files. They must either store binary files as | |
141 complete snapshots or encoded into a text representation, both | |
142 of which are wasteful approaches. Mercurial can efficiently | |
143 handle deltas of files with arbitrary binary contents; it | |
144 doesn't need to treat text as special.</para> | |
145 | |
146 </sect2> | |
147 <sect2 id="sec:concepts:txn"> | |
148 <title>Safe operation</title> | |
149 | |
150 <para>Mercurial only ever <emphasis>appends</emphasis> data to | |
151 the end of a revlog file. It never modifies a section of a | |
152 file after it has written it. This is both more robust and | |
153 efficient than schemes that need to modify or rewrite | |
154 data.</para> | |
155 | |
156 <para>In addition, Mercurial treats every write as part of a | |
157 <emphasis>transaction</emphasis> that can span a number of | |
158 files. A transaction is <emphasis>atomic</emphasis>: either | |
159 the entire transaction succeeds and its effects are all | |
160 visible to readers in one go, or the whole thing is undone. | |
161 This guarantee of atomicity means that if you're running two | |
162 copies of Mercurial, where one is reading data and one is | |
163 writing it, the reader will never see a partially written | |
164 result that might confuse it.</para> | |
165 | |
166 <para>The fact that Mercurial only appends to files makes it | |
167 easier to provide this transactional guarantee. The easier it | |
168 is to do stuff like this, the more confident you should be | |
169 that it's done correctly.</para> | |
170 | |
171 </sect2> | |
172 <sect2> | |
173 <title>Fast retrieval</title> | |
174 | |
175 <para>Mercurial cleverly avoids a pitfall common to all earlier | |
176 revision control systems: the problem of <emphasis>inefficient | |
177 retrieval</emphasis>. Most revision control systems store | |
178 the contents of a revision as an incremental series of | |
179 modifications against a <quote>snapshot</quote>. To | |
180 reconstruct a specific revision, you must first read the | |
181 snapshot, and then every one of the revisions between the | |
182 snapshot and your target revision. The more history that a | |
183 file accumulates, the more revisions you must read, hence the | |
184 longer it takes to reconstruct a particular revision.</para> | |
185 | |
186 <informalfigure id="fig:concepts:snapshot"> | |
187 <mediaobject><imageobject><imagedata | |
188 fileref="snapshot"/></imageobject><textobject><phrase>XXX | |
189 add text</phrase></textobject><caption><para>Snapshot of | |
190 a revlog, with incremental | |
191 deltas</para></caption></mediaobject> | |
192 </informalfigure> | |
193 | |
194 <para>The innovation that Mercurial applies to this problem is | |
195 simple but effective. Once the cumulative amount of delta | |
196 information stored since the last snapshot exceeds a fixed | |
197 threshold, it stores a new snapshot (compressed, of course), | |
198 instead of another delta. This makes it possible to | |
199 reconstruct <emphasis>any</emphasis> revision of a file | |
200 quickly. This approach works so well that it has since been | |
201 copied by several other revision control systems.</para> | |
202 | |
203 <para>Figure <xref linkend="fig:concepts:snapshot"/> illustrates | |
204 the idea. In an entry in a revlog's index file, Mercurial | |
205 stores the range of entries from the data file that it must | |
206 read to reconstruct a particular revision.</para> | |
207 | |
208 <sect3> | |
209 <title>Aside: the influence of video compression</title> | |
210 | |
211 <para>If you're familiar with video compression or have ever | |
212 watched a TV feed through a digital cable or satellite | |
213 service, you may know that most video compression schemes | |
214 store each frame of video as a delta against its predecessor | |
215 frame. In addition, these schemes use <quote>lossy</quote> | |
216 compression techniques to increase the compression ratio, so | |
217 visual errors accumulate over the course of a number of | |
218 inter-frame deltas.</para> | |
219 | |
220 <para>Because it's possible for a video stream to <quote>drop | |
221 out</quote> occasionally due to signal glitches, and to | |
222 limit the accumulation of artefacts introduced by the lossy | |
223 compression process, video encoders periodically insert a | |
224 complete frame (called a <quote>key frame</quote>) into the | |
225 video stream; the next delta is generated against that | |
226 frame. This means that if the video signal gets | |
227 interrupted, it will resume once the next key frame is | |
228 received. Also, the accumulation of encoding errors | |
229 restarts anew with each key frame.</para> | |
230 | |
231 </sect3> | |
232 </sect2> | |
233 <sect2> | |
234 <title>Identification and strong integrity</title> | |
235 | |
236 <para>Along with delta or snapshot information, a revlog entry | |
237 contains a cryptographic hash of the data that it represents. | |
238 This makes it difficult to forge the contents of a revision, | |
239 and easy to detect accidental corruption.</para> | |
240 | |
241 <para>Hashes provide more than a mere check against corruption; | |
242 they are used as the identifiers for revisions. The changeset | |
243 identification hashes that you see as an end user are from | |
244 revisions of the changelog. Although filelogs and the | |
245 manifest also use hashes, Mercurial only uses these behind the | |
246 scenes.</para> | |
247 | |
248 <para>Mercurial verifies that hashes are correct when it | |
249 retrieves file revisions and when it pulls changes from | |
250 another repository. If it encounters an integrity problem, it | |
251 will complain and stop whatever it's doing.</para> | |
252 | |
253 <para>In addition to the effect it has on retrieval efficiency, | |
254 Mercurial's use of periodic snapshots makes it more robust | |
255 against partial data corruption. If a revlog becomes partly | |
256 corrupted due to a hardware error or system bug, it's often | |
257 possible to reconstruct some or most revisions from the | |
258 uncorrupted sections of the revlog, both before and after the | |
259 corrupted section. This would not be possible with a | |
260 delta-only storage model.</para> | |
261 | |
262 </sect2> | |
263 </sect1> | |
264 <sect1> | |
265 <title>Revision history, branching, and merging</title> | |
266 | |
267 <para>Every entry in a Mercurial revlog knows the identity of its | |
268 immediate ancestor revision, usually referred to as its | |
269 <emphasis>parent</emphasis>. In fact, a revision contains room | |
270 for not one parent, but two. Mercurial uses a special hash, | |
271 called the <quote>null ID</quote>, to represent the idea | |
272 <quote>there is no parent here</quote>. This hash is simply a | |
273 string of zeroes.</para> | |
274 | |
275 <para>In figure <xref linkend="fig:concepts:revlog"/>, you can see | |
276 an example of the conceptual structure of a revlog. Filelogs, | |
277 manifests, and changelogs all have this same structure; they | |
278 differ only in the kind of data stored in each delta or | |
279 snapshot.</para> | |
280 | |
281 <para>The first revision in a revlog (at the bottom of the image) | |
282 has the null ID in both of its parent slots. For a | |
283 <quote>normal</quote> revision, its first parent slot contains | |
284 the ID of its parent revision, and its second contains the null | |
285 ID, indicating that the revision has only one real parent. Any | |
286 two revisions that have the same parent ID are branches. A | |
287 revision that represents a merge between branches has two normal | |
288 revision IDs in its parent slots.</para> | |
289 | |
290 <informalfigure id="fig:concepts:revlog"> | |
291 <mediaobject><imageobject><imagedata | |
292 fileref="revlog"/></imageobject><textobject><phrase>XXX | |
293 add text</phrase></textobject></mediaobject> | |
294 </informalfigure> | |
295 | |
296 </sect1> | |
297 <sect1> | |
298 <title>The working directory</title> | |
299 | |
300 <para>In the working directory, Mercurial stores a snapshot of the | |
301 files from the repository as of a particular changeset.</para> | |
302 | |
303 <para>The working directory <quote>knows</quote> which changeset | |
304 it contains. When you update the working directory to contain a | |
305 particular changeset, Mercurial looks up the appropriate | |
306 revision of the manifest to find out which files it was tracking | |
307 at the time that changeset was committed, and which revision of | |
308 each file was then current. It then recreates a copy of each of | |
309 those files, with the same contents it had when the changeset | |
310 was committed.</para> | |
311 | |
312 <para>The <emphasis>dirstate</emphasis> contains Mercurial's | |
313 knowledge of the working directory. This details which | |
314 changeset the working directory is updated to, and all of the | |
315 files that Mercurial is tracking in the working | |
316 directory.</para> | |
317 | |
318 <para>Just as a revision of a revlog has room for two parents, so | |
319 that it can represent either a normal revision (with one parent) | |
320 or a merge of two earlier revisions, the dirstate has slots for | |
321 two parents. When you use the <command role="hg-cmd">hg | |
322 update</command> command, the changeset that you update to is | |
323 stored in the <quote>first parent</quote> slot, and the null ID | |
324 in the second. When you <command role="hg-cmd">hg | |
325 merge</command> with another changeset, the first parent | |
326 remains unchanged, and the second parent is filled in with the | |
327 changeset you're merging with. The <command role="hg-cmd">hg | |
328 parents</command> command tells you what the parents of the | |
329 dirstate are.</para> | |
330 | |
331 <sect2> | |
332 <title>What happens when you commit</title> | |
333 | |
334 <para>The dirstate stores parent information for more than just | |
335 book-keeping purposes. Mercurial uses the parents of the | |
336 dirstate as <emphasis>the parents of a new | |
337 changeset</emphasis> when you perform a commit.</para> | |
338 | |
339 <informalfigure id="fig:concepts:wdir"> | |
340 <mediaobject><imageobject><imagedata | |
341 fileref="wdir"/></imageobject><textobject><phrase>XXX | |
342 add text</phrase></textobject><caption><para>The working | |
343 directory can have two | |
344 parents</para></caption></mediaobject> | |
345 </informalfigure> | |
346 | |
347 <para>Figure <xref linkend="fig:concepts:wdir"/> shows the | |
348 normal state of the working directory, where it has a single | |
349 changeset as parent. That changeset is the | |
350 <emphasis>tip</emphasis>, the newest changeset in the | |
351 repository that has no children.</para> | |
352 | |
353 <informalfigure id="fig:concepts:wdir-after-commit"> | |
354 <mediaobject><imageobject><imagedata | |
355 fileref="wdir-after-commit"/></imageobject><textobject><phrase>XXX | |
356 add text</phrase></textobject><caption><para>The working | |
357 directory gains new parents after a | |
358 commit</para></caption></mediaobject> | |
359 </informalfigure> | |
360 | |
361 <para>It's useful to think of the working directory as | |
362 <quote>the changeset I'm about to commit</quote>. Any files | |
363 that you tell Mercurial that you've added, removed, renamed, | |
364 or copied will be reflected in that changeset, as will | |
365 modifications to any files that Mercurial is already tracking; | |
366 the new changeset will have the parents of the working | |
367 directory as its parents.</para> | |
368 | |
369 <para>After a commit, Mercurial will update the parents of the | |
370 working directory, so that the first parent is the ID of the | |
371 new changeset, and the second is the null ID. This is shown | |
372 in figure <xref linkend="fig:concepts:wdir-after-commit"/>. | |
373 Mercurial | |
374 doesn't touch any of the files in the working directory when | |
375 you commit; it just modifies the dirstate to note its new | |
376 parents.</para> | |
377 | |
378 </sect2> | |
379 <sect2> | |
380 <title>Creating a new head</title> | |
381 | |
382 <para>It's perfectly normal to update the working directory to a | |
383 changeset other than the current tip. For example, you might | |
384 want to know what your project looked like last Tuesday, or | |
385 you could be looking through changesets to see which one | |
386 introduced a bug. In cases like this, the natural thing to do | |
387 is update the working directory to the changeset you're | |
388 interested in, and then examine the files in the working | |
389 directory directly to see their contents as they were when you | |
390 committed that changeset. The effect of this is shown in | |
391 figure <xref linkend="fig:concepts:wdir-pre-branch"/>.</para> | |
392 | |
393 <informalfigure id="fig:concepts:wdir-pre-branch"> | |
394 <mediaobject><imageobject><imagedata | |
395 fileref="wdir-pre-branch"/></imageobject><textobject><phrase>XXX | |
396 add text</phrase></textobject><caption><para>The working | |
397 directory, updated to an older | |
398 changeset</para></caption></mediaobject> | |
399 </informalfigure> | |
400 | |
401 <para>Having updated the working directory to an older | |
402 changeset, what happens if you make some changes, and then | |
403 commit? Mercurial behaves in the same way as I outlined | |
404 above. The parents of the working directory become the | |
405 parents of the new changeset. This new changeset has no | |
406 children, so it becomes the new tip. And the repository now | |
407 contains two changesets that have no children; we call these | |
408 <emphasis>heads</emphasis>. You can see the structure that | |
409 this creates in figure <xref | |
410 linkend="fig:concepts:wdir-branch"/>.</para> | |
411 | |
412 <informalfigure id="fig:concepts:wdir-branch"> | |
413 <mediaobject><imageobject><imagedata | |
414 fileref="wdir-branch"/></imageobject><textobject><phrase>XXX | |
415 add text</phrase></textobject><caption><para>After a | |
416 commit made while synced to an older | |
417 changeset</para></caption></mediaobject> | |
418 </informalfigure> | |
419 | |
420 <note> | |
421 <para> If you're new to Mercurial, you should keep in mind a | |
422 common <quote>error</quote>, which is to use the <command | |
423 role="hg-cmd">hg pull</command> command without any | |
424 options. By default, the <command role="hg-cmd">hg | |
425 pull</command> command <emphasis>does not</emphasis> | |
426 update the working directory, so you'll bring new changesets | |
427 into your repository, but the working directory will stay | |
428 synced at the same changeset as before the pull. If you | |
429 make some changes and commit afterwards, you'll thus create | |
430 a new head, because your working directory isn't synced to | |
431 whatever the current tip is.</para> | |
432 | |
433 <para> I put the word <quote>error</quote> in quotes because | |
434 all that you need to do to rectify this situation is | |
435 <command role="hg-cmd">hg merge</command>, then <command | |
436 role="hg-cmd">hg commit</command>. In other words, this | |
437 almost never has negative consequences; it just surprises | |
438 people. I'll discuss other ways to avoid this behaviour, | |
439 and why Mercurial behaves in this initially surprising way, | |
440 later on.</para> | |
441 </note> | |
442 | |
443 </sect2> | |
444 <sect2> | |
445 <title>Merging heads</title> | |
446 | |
447 <para>When you run the <command role="hg-cmd">hg merge</command> | |
448 command, Mercurial leaves the first parent of the working | |
449 directory unchanged, and sets the second parent to the | |
450 changeset you're merging with, as shown in figure <xref | |
451 linkend="fig:concepts:wdir-merge"/>.</para> | |
452 | |
453 <informalfigure id="fig:concepts:wdir-merge"> | |
454 <mediaobject><imageobject><imagedata | |
455 fileref="wdir-merge"/></imageobject><textobject><phrase>XXX | |
456 add text</phrase></textobject><caption><para>Merging two | |
457 heads</para></caption></mediaobject> | |
458 </informalfigure> | |
459 | |
460 <para>Mercurial also has to modify the working directory, to | |
461 merge the files managed in the two changesets. Simplified a | |
462 little, the merging process goes like this, for every file in | |
463 the manifests of both changesets.</para> | |
464 <itemizedlist> | |
465 <listitem><para>If neither changeset has modified a file, do | |
466 nothing with that file.</para> | |
467 </listitem> | |
468 <listitem><para>If one changeset has modified a file, and the | |
469 other hasn't, create the modified copy of the file in the | |
470 working directory.</para> | |
471 </listitem> | |
472 <listitem><para>If one changeset has removed a file, and the | |
473 other hasn't (or has also deleted it), delete the file | |
474 from the working directory.</para> | |
475 </listitem> | |
476 <listitem><para>If one changeset has removed a file, but the | |
477 other has modified the file, ask the user what to do: keep | |
478 the modified file, or remove it?</para> | |
479 </listitem> | |
480 <listitem><para>If both changesets have modified a file, | |
481 invoke an external merge program to choose the new | |
482 contents for the merged file. This may require input from | |
483 the user.</para> | |
484 </listitem> | |
485 <listitem><para>If one changeset has modified a file, and the | |
486 other has renamed or copied the file, make sure that the | |
487 changes follow the new name of the file.</para> | |
488 </listitem></itemizedlist> | |
489 <para>There are more details&emdash;merging has plenty of corner | |
490 cases&emdash;but these are the most common choices that are | |
491 involved in a merge. As you can see, most cases are | |
492 completely automatic, and indeed most merges finish | |
493 automatically, without requiring your input to resolve any | |
494 conflicts.</para> | |
495 | |
496 <para>When you're thinking about what happens when you commit | |
497 after a merge, once again the working directory is <quote>the | |
498 changeset I'm about to commit</quote>. After the <command | |
499 role="hg-cmd">hg merge</command> command completes, the | |
500 working directory has two parents; these will become the | |
501 parents of the new changeset.</para> | |
502 | |
503 <para>Mercurial lets you perform multiple merges, but you must | |
504 commit the results of each individual merge as you go. This | |
505 is necessary because Mercurial only tracks two parents for | |
506 both revisions and the working directory. While it would be | |
507 technically possible to merge multiple changesets at once, the | |
508 prospect of user confusion and making a terrible mess of a | |
509 merge immediately becomes overwhelming.</para> | |
510 | |
511 </sect2> | |
512 </sect1> | |
513 <sect1> | |
514 <title>Other interesting design features</title> | |
515 | |
516 <para>In the sections above, I've tried to highlight some of the | |
517 most important aspects of Mercurial's design, to illustrate that | |
518 it pays careful attention to reliability and performance. | |
519 However, the attention to detail doesn't stop there. There are | |
520 a number of other aspects of Mercurial's construction that I | |
521 personally find interesting. I'll detail a few of them here, | |
522 separate from the <quote>big ticket</quote> items above, so that | |
523 if you're interested, you can gain a better idea of the amount | |
524 of thinking that goes into a well-designed system.</para> | |
525 | |
526 <sect2> | |
527 <title>Clever compression</title> | |
528 | |
529 <para>When appropriate, Mercurial will store both snapshots and | |
530 deltas in compressed form. It does this by always | |
531 <emphasis>trying to</emphasis> compress a snapshot or delta, | |
532 but only storing the compressed version if it's smaller than | |
533 the uncompressed version.</para> | |
534 | |
535 <para>This means that Mercurial does <quote>the right | |
536 thing</quote> when storing a file whose native form is | |
537 compressed, such as a <literal>zip</literal> archive or a JPEG | |
538 image. When these types of files are compressed a second | |
539 time, the resulting file is usually bigger than the | |
540 once-compressed form, and so Mercurial will store the plain | |
541 <literal>zip</literal> or JPEG.</para> | |
542 | |
543 <para>Deltas between revisions of a compressed file are usually | |
544 larger than snapshots of the file, and Mercurial again does | |
545 <quote>the right thing</quote> in these cases. It finds that | |
546 such a delta exceeds the threshold at which it should store a | |
547 complete snapshot of the file, so it stores the snapshot, | |
548 again saving space compared to a naive delta-only | |
549 approach.</para> | |
550 | |
551 <sect3> | |
552 <title>Network recompression</title> | |
553 | |
554 <para>When storing revisions on disk, Mercurial uses the | |
555 <quote>deflate</quote> compression algorithm (the same one | |
556 used by the popular <literal>zip</literal> archive format), | |
557 which balances good speed with a respectable compression | |
558 ratio. However, when transmitting revision data over a | |
559 network connection, Mercurial uncompresses the compressed | |
560 revision data.</para> | |
561 | |
562 <para>If the connection is over HTTP, Mercurial recompresses | |
563 the entire stream of data using a compression algorithm that | |
564 gives a better compression ratio (the Burrows-Wheeler | |
565 algorithm from the widely used <literal>bzip2</literal> | |
566 compression package). This combination of algorithm and | |
567 compression of the entire stream (instead of a revision at a | |
568 time) substantially reduces the number of bytes to be | |
569 transferred, yielding better network performance over almost | |
570 all kinds of network.</para> | |
571 | |
572 <para>(If the connection is over <command>ssh</command>, | |
573 Mercurial <emphasis>doesn't</emphasis> recompress the | |
574 stream, because <command>ssh</command> can already do this | |
575 itself.)</para> | |
576 | |
577 </sect3> | |
578 </sect2> | |
579 <sect2> | |
580 <title>Read/write ordering and atomicity</title> | |
581 | |
582 <para>Appending to files isn't the whole story when it comes to | |
583 guaranteeing that a reader won't see a partial write. If you | |
584 recall figure <xref linkend="fig:concepts:metadata"/>, | |
585 revisions in the | |
586 changelog point to revisions in the manifest, and revisions in | |
587 the manifest point to revisions in filelogs. This hierarchy | |
588 is deliberate.</para> | |
589 | |
590 <para>A writer starts a transaction by writing filelog and | |
591 manifest data, and doesn't write any changelog data until | |
592 those are finished. A reader starts by reading changelog | |
593 data, then manifest data, followed by filelog data.</para> | |
594 | |
595 <para>Since the writer has always finished writing filelog and | |
596 manifest data before it writes to the changelog, a reader will | |
597 never read a pointer to a partially written manifest revision | |
598 from the changelog, and it will never read a pointer to a | |
599 partially written filelog revision from the manifest.</para> | |
600 | |
601 </sect2> | |
602 <sect2> | |
603 <title>Concurrent access</title> | |
604 | |
605 <para>The read/write ordering and atomicity guarantees mean that | |
606 Mercurial never needs to <emphasis>lock</emphasis> a | |
607 repository when it's reading data, even if the repository is | |
608 being written to while the read is occurring. This has a big | |
609 effect on scalability; you can have an arbitrary number of | |
610 Mercurial processes safely reading data from a repository | |
611 safely all at once, no matter whether it's being written to or | |
612 not.</para> | |
613 | |
614 <para>The lockless nature of reading means that if you're | |
615 sharing a repository on a multi-user system, you don't need to | |
616 grant other local users permission to | |
617 <emphasis>write</emphasis> to your repository in order for | |
618 them to be able to clone it or pull changes from it; they only | |
619 need <emphasis>read</emphasis> permission. (This is | |
620 <emphasis>not</emphasis> a common feature among revision | |
621 control systems, so don't take it for granted! Most require | |
622 readers to be able to lock a repository to access it safely, | |
623 and this requires write permission on at least one directory, | |
624 which of course makes for all kinds of nasty and annoying | |
625 security and administrative problems.)</para> | |
626 | |
627 <para>Mercurial uses locks to ensure that only one process can | |
628 write to a repository at a time (the locking mechanism is safe | |
629 even over filesystems that are notoriously hostile to locking, | |
630 such as NFS). If a repository is locked, a writer will wait | |
631 for a while to retry if the repository becomes unlocked, but | |
632 if the repository remains locked for too long, the process | |
633 attempting to write will time out after a while. This means | |
634 that your daily automated scripts won't get stuck forever and | |
635 pile up if a system crashes unnoticed, for example. (Yes, the | |
636 timeout is configurable, from zero to infinity.)</para> | |
637 | |
638 <sect3> | |
639 <title>Safe dirstate access</title> | |
640 | |
641 <para>As with revision data, Mercurial doesn't take a lock to | |
642 read the dirstate file; it does acquire a lock to write it. | |
643 To avoid the possibility of reading a partially written copy | |
644 of the dirstate file, Mercurial writes to a file with a | |
645 unique name in the same directory as the dirstate file, then | |
646 renames the temporary file atomically to | |
647 <filename>dirstate</filename>. The file named | |
648 <filename>dirstate</filename> is thus guaranteed to be | |
649 complete, not partially written.</para> | |
650 | |
651 </sect3> | |
652 </sect2> | |
653 <sect2> | |
654 <title>Avoiding seeks</title> | |
655 | |
656 <para>Critical to Mercurial's performance is the avoidance of | |
657 seeks of the disk head, since any seek is far more expensive | |
658 than even a comparatively large read operation.</para> | |
659 | |
660 <para>This is why, for example, the dirstate is stored in a | |
661 single file. If there were a dirstate file per directory that | |
662 Mercurial tracked, the disk would seek once per directory. | |
663 Instead, Mercurial reads the entire single dirstate file in | |
664 one step.</para> | |
665 | |
666 <para>Mercurial also uses a <quote>copy on write</quote> scheme | |
667 when cloning a repository on local storage. Instead of | |
668 copying every revlog file from the old repository into the new | |
669 repository, it makes a <quote>hard link</quote>, which is a | |
670 shorthand way to say <quote>these two names point to the same | |
671 file</quote>. When Mercurial is about to write to one of a | |
672 revlog's files, it checks to see if the number of names | |
673 pointing at the file is greater than one. If it is, more than | |
674 one repository is using the file, so Mercurial makes a new | |
675 copy of the file that is private to this repository.</para> | |
676 | |
677 <para>A few revision control developers have pointed out that | |
678 this idea of making a complete private copy of a file is not | |
679 very efficient in its use of storage. While this is true, | |
680 storage is cheap, and this method gives the highest | |
681 performance while deferring most book-keeping to the operating | |
682 system. An alternative scheme would most likely reduce | |
683 performance and increase the complexity of the software, each | |
684 of which is much more important to the <quote>feel</quote> of | |
685 day-to-day use.</para> | |
686 | |
687 </sect2> | |
688 <sect2> | |
689 <title>Other contents of the dirstate</title> | |
690 | |
691 <para>Because Mercurial doesn't force you to tell it when you're | |
692 modifying a file, it uses the dirstate to store some extra | |
693 information so it can determine efficiently whether you have | |
694 modified a file. For each file in the working directory, it | |
695 stores the time that it last modified the file itself, and the | |
696 size of the file at that time.</para> | |
697 | |
698 <para>When you explicitly <command role="hg-cmd">hg | |
699 add</command>, <command role="hg-cmd">hg remove</command>, | |
700 <command role="hg-cmd">hg rename</command> or <command | |
701 role="hg-cmd">hg copy</command> files, Mercurial updates the | |
702 dirstate so that it knows what to do with those files when you | |
703 commit.</para> | |
704 | |
705 <para>When Mercurial is checking the states of files in the | |
706 working directory, it first checks a file's modification time. | |
707 If that has not changed, the file must not have been modified. | |
708 If the file's size has changed, the file must have been | |
709 modified. If the modification time has changed, but the size | |
710 has not, only then does Mercurial need to read the actual | |
711 contents of the file to see if they've changed. Storing these | |
712 few extra pieces of information dramatically reduces the | |
713 amount of data that Mercurial needs to read, which yields | |
714 large performance improvements compared to other revision | |
715 control systems.</para> | |
716 | |
717 </sect2> | |
718 </sect1> | |
719 </chapter> | |
720 | |
721 <!-- | |
722 local variables: | |
723 sgml-parent-document: ("00book.xml" "book" "chapter") | |
724 end: | |
725 --> |