annotate en/examples/run-example @ 192:06ab90119fa6

Be more conservative in timeouts. Needed on my new, slower (!?) laptop.
author Bryan O'Sullivan <bos@serpentine.com>
date Mon, 16 Apr 2007 14:39:51 -0700
parents 754312dc23d5
children 1a55ba6ceca1
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
67
a2f0b010d6e3 Don't assume that python is in /usr/bin.
Bryan O'Sullivan <bos@serpentine.com>
parents: 45
diff changeset
1 #!/usr/bin/env python
4
33a2e7b9978d Make it possible to include example input and output from real programs.
Bryan O'Sullivan <bos@serpentine.com>
parents: 3
diff changeset
2 #
33a2e7b9978d Make it possible to include example input and output from real programs.
Bryan O'Sullivan <bos@serpentine.com>
parents: 3
diff changeset
3 # This program takes something that resembles a shell script and runs
33a2e7b9978d Make it possible to include example input and output from real programs.
Bryan O'Sullivan <bos@serpentine.com>
parents: 3
diff changeset
4 # it, spitting input (commands from the script) and output into text
33a2e7b9978d Make it possible to include example input and output from real programs.
Bryan O'Sullivan <bos@serpentine.com>
parents: 3
diff changeset
5 # files, for use in examples.
3
906d9021f9e5 Making progress on autogenerated example output.
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
6
906d9021f9e5 Making progress on autogenerated example output.
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
7 import cStringIO
73
9604dd885616 Fix run-example script on Debian.
Bryan O'Sullivan <bos@serpentine.com>
parents: 72
diff changeset
8 import errno
78
a893de25bc24 Add -v option to run-example, to assist with debugging example scripts.
Bryan O'Sullivan <bos@serpentine.com>
parents: 77
diff changeset
9 import getopt
3
906d9021f9e5 Making progress on autogenerated example output.
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
10 import os
906d9021f9e5 Making progress on autogenerated example output.
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
11 import pty
906d9021f9e5 Making progress on autogenerated example output.
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
12 import re
79
53427f786a0f Make run-example time out if shell seems to get stuck.
Bryan O'Sullivan <bos@serpentine.com>
parents: 78
diff changeset
13 import select
4
33a2e7b9978d Make it possible to include example input and output from real programs.
Bryan O'Sullivan <bos@serpentine.com>
parents: 3
diff changeset
14 import shutil
6
69d90ab9fd80 Really run example command sequences under a single shell.
Bryan O'Sullivan <bos@serpentine.com>
parents: 4
diff changeset
15 import signal
36
5cee64874312 Require examples to be executable, so it's easier to see them with "ls".
Bryan O'Sullivan <bos@serpentine.com>
parents: 24
diff changeset
16 import stat
3
906d9021f9e5 Making progress on autogenerated example output.
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
17 import sys
4
33a2e7b9978d Make it possible to include example input and output from real programs.
Bryan O'Sullivan <bos@serpentine.com>
parents: 3
diff changeset
18 import tempfile
33a2e7b9978d Make it possible to include example input and output from real programs.
Bryan O'Sullivan <bos@serpentine.com>
parents: 3
diff changeset
19 import time
3
906d9021f9e5 Making progress on autogenerated example output.
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
20
77
773f4a9e7975 Fix escaping of backslashes. Finally!
Bryan O'Sullivan <bos@serpentine.com>
parents: 75
diff changeset
21 tex_subs = {
773f4a9e7975 Fix escaping of backslashes. Finally!
Bryan O'Sullivan <bos@serpentine.com>
parents: 75
diff changeset
22 '\\': '\\textbackslash{}',
773f4a9e7975 Fix escaping of backslashes. Finally!
Bryan O'Sullivan <bos@serpentine.com>
parents: 75
diff changeset
23 '{': '\\{',
773f4a9e7975 Fix escaping of backslashes. Finally!
Bryan O'Sullivan <bos@serpentine.com>
parents: 75
diff changeset
24 '}': '\\}',
773f4a9e7975 Fix escaping of backslashes. Finally!
Bryan O'Sullivan <bos@serpentine.com>
parents: 75
diff changeset
25 }
773f4a9e7975 Fix escaping of backslashes. Finally!
Bryan O'Sullivan <bos@serpentine.com>
parents: 75
diff changeset
26
773f4a9e7975 Fix escaping of backslashes. Finally!
Bryan O'Sullivan <bos@serpentine.com>
parents: 75
diff changeset
27 def gensubs(s):
773f4a9e7975 Fix escaping of backslashes. Finally!
Bryan O'Sullivan <bos@serpentine.com>
parents: 75
diff changeset
28 start = 0
773f4a9e7975 Fix escaping of backslashes. Finally!
Bryan O'Sullivan <bos@serpentine.com>
parents: 75
diff changeset
29 for i, c in enumerate(s):
773f4a9e7975 Fix escaping of backslashes. Finally!
Bryan O'Sullivan <bos@serpentine.com>
parents: 75
diff changeset
30 sub = tex_subs.get(c)
773f4a9e7975 Fix escaping of backslashes. Finally!
Bryan O'Sullivan <bos@serpentine.com>
parents: 75
diff changeset
31 if sub:
773f4a9e7975 Fix escaping of backslashes. Finally!
Bryan O'Sullivan <bos@serpentine.com>
parents: 75
diff changeset
32 yield s[start:i]
773f4a9e7975 Fix escaping of backslashes. Finally!
Bryan O'Sullivan <bos@serpentine.com>
parents: 75
diff changeset
33 start = i + 1
773f4a9e7975 Fix escaping of backslashes. Finally!
Bryan O'Sullivan <bos@serpentine.com>
parents: 75
diff changeset
34 yield sub
773f4a9e7975 Fix escaping of backslashes. Finally!
Bryan O'Sullivan <bos@serpentine.com>
parents: 75
diff changeset
35 yield s[start:]
773f4a9e7975 Fix escaping of backslashes. Finally!
Bryan O'Sullivan <bos@serpentine.com>
parents: 75
diff changeset
36
4
33a2e7b9978d Make it possible to include example input and output from real programs.
Bryan O'Sullivan <bos@serpentine.com>
parents: 3
diff changeset
37 def tex_escape(s):
77
773f4a9e7975 Fix escaping of backslashes. Finally!
Bryan O'Sullivan <bos@serpentine.com>
parents: 75
diff changeset
38 return ''.join(gensubs(s))
4
33a2e7b9978d Make it possible to include example input and output from real programs.
Bryan O'Sullivan <bos@serpentine.com>
parents: 3
diff changeset
39
137
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
40 def maybe_unlink(name):
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
41 try:
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
42 os.unlink(name)
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
43 return True
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
44 except OSError, err:
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
45 if err.errno != errno.ENOENT:
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
46 raise
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
47 return False
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
48
172
5f305adeb584 Try to tighten up the run environment to make things more reproducible.
Bryan O'Sullivan <bos@serpentine.com>
parents: 168
diff changeset
49 def find_path_to(program):
5f305adeb584 Try to tighten up the run environment to make things more reproducible.
Bryan O'Sullivan <bos@serpentine.com>
parents: 168
diff changeset
50 for p in os.environ.get('PATH', os.defpath).split(os.pathsep):
5f305adeb584 Try to tighten up the run environment to make things more reproducible.
Bryan O'Sullivan <bos@serpentine.com>
parents: 168
diff changeset
51 name = os.path.join(p, program)
5f305adeb584 Try to tighten up the run environment to make things more reproducible.
Bryan O'Sullivan <bos@serpentine.com>
parents: 168
diff changeset
52 if os.access(name, os.X_OK):
5f305adeb584 Try to tighten up the run environment to make things more reproducible.
Bryan O'Sullivan <bos@serpentine.com>
parents: 168
diff changeset
53 return p
5f305adeb584 Try to tighten up the run environment to make things more reproducible.
Bryan O'Sullivan <bos@serpentine.com>
parents: 168
diff changeset
54 return None
5f305adeb584 Try to tighten up the run environment to make things more reproducible.
Bryan O'Sullivan <bos@serpentine.com>
parents: 168
diff changeset
55
3
906d9021f9e5 Making progress on autogenerated example output.
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
56 class example:
70
365b41b6a15e bash is not necessarily in /bin.
Bryan O'Sullivan <bos@serpentine.com>
parents: 68
diff changeset
57 shell = '/usr/bin/env bash'
103
5b80c922ebdd More merge content.
Bryan O'Sullivan <bos@serpentine.com>
parents: 83
diff changeset
58 ps1 = '__run_example_ps1__ '
5b80c922ebdd More merge content.
Bryan O'Sullivan <bos@serpentine.com>
parents: 83
diff changeset
59 ps2 = '__run_example_ps2__ '
168
f8b5b782e150 Backed out changeset 08a4467f489162c14cf17b94aa771a23dd6f02dc
Bryan O'Sullivan <bos@serpentine.com>
parents: 166
diff changeset
60 pi_re = re.compile(r'#\$\s*(name|ignore):\s*(.*)$')
4
33a2e7b9978d Make it possible to include example input and output from real programs.
Bryan O'Sullivan <bos@serpentine.com>
parents: 3
diff changeset
61
192
06ab90119fa6 Be more conservative in timeouts.
Bryan O'Sullivan <bos@serpentine.com>
parents: 173
diff changeset
62 timeout = 10
79
53427f786a0f Make run-example time out if shell seems to get stuck.
Bryan O'Sullivan <bos@serpentine.com>
parents: 78
diff changeset
63
78
a893de25bc24 Add -v option to run-example, to assist with debugging example scripts.
Bryan O'Sullivan <bos@serpentine.com>
parents: 77
diff changeset
64 def __init__(self, name, verbose):
3
906d9021f9e5 Making progress on autogenerated example output.
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
65 self.name = name
78
a893de25bc24 Add -v option to run-example, to assist with debugging example scripts.
Bryan O'Sullivan <bos@serpentine.com>
parents: 77
diff changeset
66 self.verbose = verbose
79
53427f786a0f Make run-example time out if shell seems to get stuck.
Bryan O'Sullivan <bos@serpentine.com>
parents: 78
diff changeset
67 self.poll = select.poll()
3
906d9021f9e5 Making progress on autogenerated example output.
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
68
906d9021f9e5 Making progress on autogenerated example output.
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
69 def parse(self):
4
33a2e7b9978d Make it possible to include example input and output from real programs.
Bryan O'Sullivan <bos@serpentine.com>
parents: 3
diff changeset
70 '''yield each hunk of input from the file.'''
3
906d9021f9e5 Making progress on autogenerated example output.
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
71 fp = open(self.name)
906d9021f9e5 Making progress on autogenerated example output.
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
72 cfp = cStringIO.StringIO()
906d9021f9e5 Making progress on autogenerated example output.
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
73 for line in fp:
906d9021f9e5 Making progress on autogenerated example output.
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
74 cfp.write(line)
906d9021f9e5 Making progress on autogenerated example output.
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
75 if not line.rstrip().endswith('\\'):
906d9021f9e5 Making progress on autogenerated example output.
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
76 yield cfp.getvalue()
906d9021f9e5 Making progress on autogenerated example output.
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
77 cfp.seek(0)
906d9021f9e5 Making progress on autogenerated example output.
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
78 cfp.truncate()
906d9021f9e5 Making progress on autogenerated example output.
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
79
906d9021f9e5 Making progress on autogenerated example output.
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
80 def status(self, s):
906d9021f9e5 Making progress on autogenerated example output.
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
81 sys.stdout.write(s)
906d9021f9e5 Making progress on autogenerated example output.
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
82 if not s.endswith('\n'):
906d9021f9e5 Making progress on autogenerated example output.
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
83 sys.stdout.flush()
906d9021f9e5 Making progress on autogenerated example output.
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
84
6
69d90ab9fd80 Really run example command sequences under a single shell.
Bryan O'Sullivan <bos@serpentine.com>
parents: 4
diff changeset
85 def send(self, s):
78
a893de25bc24 Add -v option to run-example, to assist with debugging example scripts.
Bryan O'Sullivan <bos@serpentine.com>
parents: 77
diff changeset
86 if self.verbose:
a893de25bc24 Add -v option to run-example, to assist with debugging example scripts.
Bryan O'Sullivan <bos@serpentine.com>
parents: 77
diff changeset
87 print >> sys.stderr, '>', self.debugrepr(s)
73
9604dd885616 Fix run-example script on Debian.
Bryan O'Sullivan <bos@serpentine.com>
parents: 72
diff changeset
88 while s:
9604dd885616 Fix run-example script on Debian.
Bryan O'Sullivan <bos@serpentine.com>
parents: 72
diff changeset
89 count = os.write(self.cfd, s)
9604dd885616 Fix run-example script on Debian.
Bryan O'Sullivan <bos@serpentine.com>
parents: 72
diff changeset
90 s = s[count:]
6
69d90ab9fd80 Really run example command sequences under a single shell.
Bryan O'Sullivan <bos@serpentine.com>
parents: 4
diff changeset
91
78
a893de25bc24 Add -v option to run-example, to assist with debugging example scripts.
Bryan O'Sullivan <bos@serpentine.com>
parents: 77
diff changeset
92 def debugrepr(self, s):
a893de25bc24 Add -v option to run-example, to assist with debugging example scripts.
Bryan O'Sullivan <bos@serpentine.com>
parents: 77
diff changeset
93 rs = repr(s)
a893de25bc24 Add -v option to run-example, to assist with debugging example scripts.
Bryan O'Sullivan <bos@serpentine.com>
parents: 77
diff changeset
94 limit = 60
a893de25bc24 Add -v option to run-example, to assist with debugging example scripts.
Bryan O'Sullivan <bos@serpentine.com>
parents: 77
diff changeset
95 if len(rs) > limit:
a893de25bc24 Add -v option to run-example, to assist with debugging example scripts.
Bryan O'Sullivan <bos@serpentine.com>
parents: 77
diff changeset
96 return ('%s%s ... [%d bytes]' % (rs[:limit], rs[0], len(s)))
a893de25bc24 Add -v option to run-example, to assist with debugging example scripts.
Bryan O'Sullivan <bos@serpentine.com>
parents: 77
diff changeset
97 else:
a893de25bc24 Add -v option to run-example, to assist with debugging example scripts.
Bryan O'Sullivan <bos@serpentine.com>
parents: 77
diff changeset
98 return rs
a893de25bc24 Add -v option to run-example, to assist with debugging example scripts.
Bryan O'Sullivan <bos@serpentine.com>
parents: 77
diff changeset
99
79
53427f786a0f Make run-example time out if shell seems to get stuck.
Bryan O'Sullivan <bos@serpentine.com>
parents: 78
diff changeset
100 timeout = 5
53427f786a0f Make run-example time out if shell seems to get stuck.
Bryan O'Sullivan <bos@serpentine.com>
parents: 78
diff changeset
101
173
754312dc23d5 If something times out, try to tell what it was.
Bryan O'Sullivan <bos@serpentine.com>
parents: 172
diff changeset
102 def read(self, hint):
79
53427f786a0f Make run-example time out if shell seems to get stuck.
Bryan O'Sullivan <bos@serpentine.com>
parents: 78
diff changeset
103 events = self.poll.poll(self.timeout * 1000)
53427f786a0f Make run-example time out if shell seems to get stuck.
Bryan O'Sullivan <bos@serpentine.com>
parents: 78
diff changeset
104 if not events:
173
754312dc23d5 If something times out, try to tell what it was.
Bryan O'Sullivan <bos@serpentine.com>
parents: 172
diff changeset
105 print >> sys.stderr, ('[%stimed out after %d seconds]' %
754312dc23d5 If something times out, try to tell what it was.
Bryan O'Sullivan <bos@serpentine.com>
parents: 172
diff changeset
106 (hint, self.timeout))
79
53427f786a0f Make run-example time out if shell seems to get stuck.
Bryan O'Sullivan <bos@serpentine.com>
parents: 78
diff changeset
107 os.kill(self.pid, signal.SIGHUP)
53427f786a0f Make run-example time out if shell seems to get stuck.
Bryan O'Sullivan <bos@serpentine.com>
parents: 78
diff changeset
108 return ''
53427f786a0f Make run-example time out if shell seems to get stuck.
Bryan O'Sullivan <bos@serpentine.com>
parents: 78
diff changeset
109 return os.read(self.cfd, 1024)
53427f786a0f Make run-example time out if shell seems to get stuck.
Bryan O'Sullivan <bos@serpentine.com>
parents: 78
diff changeset
110
173
754312dc23d5 If something times out, try to tell what it was.
Bryan O'Sullivan <bos@serpentine.com>
parents: 172
diff changeset
111 def receive(self, hint):
6
69d90ab9fd80 Really run example command sequences under a single shell.
Bryan O'Sullivan <bos@serpentine.com>
parents: 4
diff changeset
112 out = cStringIO.StringIO()
4
33a2e7b9978d Make it possible to include example input and output from real programs.
Bryan O'Sullivan <bos@serpentine.com>
parents: 3
diff changeset
113 while True:
73
9604dd885616 Fix run-example script on Debian.
Bryan O'Sullivan <bos@serpentine.com>
parents: 72
diff changeset
114 try:
78
a893de25bc24 Add -v option to run-example, to assist with debugging example scripts.
Bryan O'Sullivan <bos@serpentine.com>
parents: 77
diff changeset
115 if self.verbose:
a893de25bc24 Add -v option to run-example, to assist with debugging example scripts.
Bryan O'Sullivan <bos@serpentine.com>
parents: 77
diff changeset
116 sys.stderr.write('< ')
173
754312dc23d5 If something times out, try to tell what it was.
Bryan O'Sullivan <bos@serpentine.com>
parents: 172
diff changeset
117 s = self.read(hint)
73
9604dd885616 Fix run-example script on Debian.
Bryan O'Sullivan <bos@serpentine.com>
parents: 72
diff changeset
118 except OSError, err:
9604dd885616 Fix run-example script on Debian.
Bryan O'Sullivan <bos@serpentine.com>
parents: 72
diff changeset
119 if err.errno == errno.EIO:
103
5b80c922ebdd More merge content.
Bryan O'Sullivan <bos@serpentine.com>
parents: 83
diff changeset
120 return '', ''
73
9604dd885616 Fix run-example script on Debian.
Bryan O'Sullivan <bos@serpentine.com>
parents: 72
diff changeset
121 raise
78
a893de25bc24 Add -v option to run-example, to assist with debugging example scripts.
Bryan O'Sullivan <bos@serpentine.com>
parents: 77
diff changeset
122 if self.verbose:
a893de25bc24 Add -v option to run-example, to assist with debugging example scripts.
Bryan O'Sullivan <bos@serpentine.com>
parents: 77
diff changeset
123 print >> sys.stderr, self.debugrepr(s)
6
69d90ab9fd80 Really run example command sequences under a single shell.
Bryan O'Sullivan <bos@serpentine.com>
parents: 4
diff changeset
124 out.write(s)
73
9604dd885616 Fix run-example script on Debian.
Bryan O'Sullivan <bos@serpentine.com>
parents: 72
diff changeset
125 s = out.getvalue()
103
5b80c922ebdd More merge content.
Bryan O'Sullivan <bos@serpentine.com>
parents: 83
diff changeset
126 if s.endswith(self.ps1):
5b80c922ebdd More merge content.
Bryan O'Sullivan <bos@serpentine.com>
parents: 83
diff changeset
127 return self.ps1, s.replace('\r\n', '\n')[:-len(self.ps1)]
5b80c922ebdd More merge content.
Bryan O'Sullivan <bos@serpentine.com>
parents: 83
diff changeset
128 if s.endswith(self.ps2):
5b80c922ebdd More merge content.
Bryan O'Sullivan <bos@serpentine.com>
parents: 83
diff changeset
129 return self.ps2, s.replace('\r\n', '\n')[:-len(self.ps2)]
4
33a2e7b9978d Make it possible to include example input and output from real programs.
Bryan O'Sullivan <bos@serpentine.com>
parents: 3
diff changeset
130
173
754312dc23d5 If something times out, try to tell what it was.
Bryan O'Sullivan <bos@serpentine.com>
parents: 172
diff changeset
131 def sendreceive(self, s, hint):
6
69d90ab9fd80 Really run example command sequences under a single shell.
Bryan O'Sullivan <bos@serpentine.com>
parents: 4
diff changeset
132 self.send(s)
173
754312dc23d5 If something times out, try to tell what it was.
Bryan O'Sullivan <bos@serpentine.com>
parents: 172
diff changeset
133 ps, r = self.receive(hint)
6
69d90ab9fd80 Really run example command sequences under a single shell.
Bryan O'Sullivan <bos@serpentine.com>
parents: 4
diff changeset
134 if r.startswith(s):
69d90ab9fd80 Really run example command sequences under a single shell.
Bryan O'Sullivan <bos@serpentine.com>
parents: 4
diff changeset
135 r = r[len(s):]
103
5b80c922ebdd More merge content.
Bryan O'Sullivan <bos@serpentine.com>
parents: 83
diff changeset
136 return ps, r
6
69d90ab9fd80 Really run example command sequences under a single shell.
Bryan O'Sullivan <bos@serpentine.com>
parents: 4
diff changeset
137
3
906d9021f9e5 Making progress on autogenerated example output.
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
138 def run(self):
906d9021f9e5 Making progress on autogenerated example output.
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
139 ofp = None
4
33a2e7b9978d Make it possible to include example input and output from real programs.
Bryan O'Sullivan <bos@serpentine.com>
parents: 3
diff changeset
140 basename = os.path.basename(self.name)
33a2e7b9978d Make it possible to include example input and output from real programs.
Bryan O'Sullivan <bos@serpentine.com>
parents: 3
diff changeset
141 self.status('running %s ' % basename)
33a2e7b9978d Make it possible to include example input and output from real programs.
Bryan O'Sullivan <bos@serpentine.com>
parents: 3
diff changeset
142 tmpdir = tempfile.mkdtemp(prefix=basename)
78
a893de25bc24 Add -v option to run-example, to assist with debugging example scripts.
Bryan O'Sullivan <bos@serpentine.com>
parents: 77
diff changeset
143
136
7b5894fffc37 Don't falsely signal success to make.
Bryan O'Sullivan <bos@serpentine.com>
parents: 124
diff changeset
144 # remove the marker file that we tell make to use to see if
7b5894fffc37 Don't falsely signal success to make.
Bryan O'Sullivan <bos@serpentine.com>
parents: 124
diff changeset
145 # this run succeeded
137
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
146 maybe_unlink(self.name + '.run')
136
7b5894fffc37 Don't falsely signal success to make.
Bryan O'Sullivan <bos@serpentine.com>
parents: 124
diff changeset
147
78
a893de25bc24 Add -v option to run-example, to assist with debugging example scripts.
Bryan O'Sullivan <bos@serpentine.com>
parents: 77
diff changeset
148 rcfile = os.path.join(tmpdir, '.hgrc')
a893de25bc24 Add -v option to run-example, to assist with debugging example scripts.
Bryan O'Sullivan <bos@serpentine.com>
parents: 77
diff changeset
149 rcfp = open(rcfile, 'w')
a893de25bc24 Add -v option to run-example, to assist with debugging example scripts.
Bryan O'Sullivan <bos@serpentine.com>
parents: 77
diff changeset
150 print >> rcfp, '[ui]'
a893de25bc24 Add -v option to run-example, to assist with debugging example scripts.
Bryan O'Sullivan <bos@serpentine.com>
parents: 77
diff changeset
151 print >> rcfp, "username = Bryan O'Sullivan <bos@serpentine.com>"
a893de25bc24 Add -v option to run-example, to assist with debugging example scripts.
Bryan O'Sullivan <bos@serpentine.com>
parents: 77
diff changeset
152
6
69d90ab9fd80 Really run example command sequences under a single shell.
Bryan O'Sullivan <bos@serpentine.com>
parents: 4
diff changeset
153 rcfile = os.path.join(tmpdir, '.bashrc')
69d90ab9fd80 Really run example command sequences under a single shell.
Bryan O'Sullivan <bos@serpentine.com>
parents: 4
diff changeset
154 rcfp = open(rcfile, 'w')
103
5b80c922ebdd More merge content.
Bryan O'Sullivan <bos@serpentine.com>
parents: 83
diff changeset
155 print >> rcfp, 'PS1="%s"' % self.ps1
5b80c922ebdd More merge content.
Bryan O'Sullivan <bos@serpentine.com>
parents: 83
diff changeset
156 print >> rcfp, 'PS2="%s"' % self.ps2
6
69d90ab9fd80 Really run example command sequences under a single shell.
Bryan O'Sullivan <bos@serpentine.com>
parents: 4
diff changeset
157 print >> rcfp, 'unset HISTFILE'
172
5f305adeb584 Try to tighten up the run environment to make things more reproducible.
Bryan O'Sullivan <bos@serpentine.com>
parents: 168
diff changeset
158 path = ['/usr/bin', '/bin']
5f305adeb584 Try to tighten up the run environment to make things more reproducible.
Bryan O'Sullivan <bos@serpentine.com>
parents: 168
diff changeset
159 hg = find_path_to('hg')
5f305adeb584 Try to tighten up the run environment to make things more reproducible.
Bryan O'Sullivan <bos@serpentine.com>
parents: 168
diff changeset
160 if hg and hg not in path:
5f305adeb584 Try to tighten up the run environment to make things more reproducible.
Bryan O'Sullivan <bos@serpentine.com>
parents: 168
diff changeset
161 path.append(hg)
5f305adeb584 Try to tighten up the run environment to make things more reproducible.
Bryan O'Sullivan <bos@serpentine.com>
parents: 168
diff changeset
162 def re_export(envar):
5f305adeb584 Try to tighten up the run environment to make things more reproducible.
Bryan O'Sullivan <bos@serpentine.com>
parents: 168
diff changeset
163 v = os.getenv(envar)
5f305adeb584 Try to tighten up the run environment to make things more reproducible.
Bryan O'Sullivan <bos@serpentine.com>
parents: 168
diff changeset
164 if v is not None:
5f305adeb584 Try to tighten up the run environment to make things more reproducible.
Bryan O'Sullivan <bos@serpentine.com>
parents: 168
diff changeset
165 print >> rcfp, 'export ' + envar + '=' + v
5f305adeb584 Try to tighten up the run environment to make things more reproducible.
Bryan O'Sullivan <bos@serpentine.com>
parents: 168
diff changeset
166 print >> rcfp, 'export PATH=' + ':'.join(path)
5f305adeb584 Try to tighten up the run environment to make things more reproducible.
Bryan O'Sullivan <bos@serpentine.com>
parents: 168
diff changeset
167 re_export('PYTHONPATH')
19
187702df428b Piles of new content for MQ chapter - cookbook stuff.
Bryan O'Sullivan <bos@serpentine.com>
parents: 6
diff changeset
168 print >> rcfp, 'export EXAMPLE_DIR="%s"' % os.getcwd()
124
c9aad709bd3a Document the backout command.
Bryan O'Sullivan <bos@serpentine.com>
parents: 103
diff changeset
169 print >> rcfp, 'export HGMERGE=merge'
6
69d90ab9fd80 Really run example command sequences under a single shell.
Bryan O'Sullivan <bos@serpentine.com>
parents: 4
diff changeset
170 print >> rcfp, 'export LANG=C'
69d90ab9fd80 Really run example command sequences under a single shell.
Bryan O'Sullivan <bos@serpentine.com>
parents: 4
diff changeset
171 print >> rcfp, 'export LC_ALL=C'
69d90ab9fd80 Really run example command sequences under a single shell.
Bryan O'Sullivan <bos@serpentine.com>
parents: 4
diff changeset
172 print >> rcfp, 'export TZ=GMT'
69d90ab9fd80 Really run example command sequences under a single shell.
Bryan O'Sullivan <bos@serpentine.com>
parents: 4
diff changeset
173 print >> rcfp, 'export HGRC="%s/.hgrc"' % tmpdir
69d90ab9fd80 Really run example command sequences under a single shell.
Bryan O'Sullivan <bos@serpentine.com>
parents: 4
diff changeset
174 print >> rcfp, 'export HGRCPATH=$HGRC'
69d90ab9fd80 Really run example command sequences under a single shell.
Bryan O'Sullivan <bos@serpentine.com>
parents: 4
diff changeset
175 print >> rcfp, 'cd %s' % tmpdir
69d90ab9fd80 Really run example command sequences under a single shell.
Bryan O'Sullivan <bos@serpentine.com>
parents: 4
diff changeset
176 rcfp.close()
68
c574ce277a2b Mostly random attempt to see if fiddling with the child will help the parent.
Bryan O'Sullivan <bos@serpentine.com>
parents: 67
diff changeset
177 sys.stdout.flush()
c574ce277a2b Mostly random attempt to see if fiddling with the child will help the parent.
Bryan O'Sullivan <bos@serpentine.com>
parents: 67
diff changeset
178 sys.stderr.flush()
79
53427f786a0f Make run-example time out if shell seems to get stuck.
Bryan O'Sullivan <bos@serpentine.com>
parents: 78
diff changeset
179 self.pid, self.cfd = pty.fork()
53427f786a0f Make run-example time out if shell seems to get stuck.
Bryan O'Sullivan <bos@serpentine.com>
parents: 78
diff changeset
180 if self.pid == 0:
172
5f305adeb584 Try to tighten up the run environment to make things more reproducible.
Bryan O'Sullivan <bos@serpentine.com>
parents: 168
diff changeset
181 cmdline = ['/usr/bin/env', '-i', 'bash', '--noediting',
5f305adeb584 Try to tighten up the run environment to make things more reproducible.
Bryan O'Sullivan <bos@serpentine.com>
parents: 168
diff changeset
182 '--noprofile', '--norc']
68
c574ce277a2b Mostly random attempt to see if fiddling with the child will help the parent.
Bryan O'Sullivan <bos@serpentine.com>
parents: 67
diff changeset
183 try:
c574ce277a2b Mostly random attempt to see if fiddling with the child will help the parent.
Bryan O'Sullivan <bos@serpentine.com>
parents: 67
diff changeset
184 os.execv(cmdline[0], cmdline)
c574ce277a2b Mostly random attempt to see if fiddling with the child will help the parent.
Bryan O'Sullivan <bos@serpentine.com>
parents: 67
diff changeset
185 except OSError, err:
c574ce277a2b Mostly random attempt to see if fiddling with the child will help the parent.
Bryan O'Sullivan <bos@serpentine.com>
parents: 67
diff changeset
186 print >> sys.stderr, '%s: %s' % (cmdline[0], err.strerror)
c574ce277a2b Mostly random attempt to see if fiddling with the child will help the parent.
Bryan O'Sullivan <bos@serpentine.com>
parents: 67
diff changeset
187 sys.stderr.flush()
c574ce277a2b Mostly random attempt to see if fiddling with the child will help the parent.
Bryan O'Sullivan <bos@serpentine.com>
parents: 67
diff changeset
188 os._exit(0)
79
53427f786a0f Make run-example time out if shell seems to get stuck.
Bryan O'Sullivan <bos@serpentine.com>
parents: 78
diff changeset
189 self.poll.register(self.cfd, select.POLLIN | select.POLLERR |
53427f786a0f Make run-example time out if shell seems to get stuck.
Bryan O'Sullivan <bos@serpentine.com>
parents: 78
diff changeset
190 select.POLLHUP)
103
5b80c922ebdd More merge content.
Bryan O'Sullivan <bos@serpentine.com>
parents: 83
diff changeset
191
5b80c922ebdd More merge content.
Bryan O'Sullivan <bos@serpentine.com>
parents: 83
diff changeset
192 prompts = {
5b80c922ebdd More merge content.
Bryan O'Sullivan <bos@serpentine.com>
parents: 83
diff changeset
193 '': '',
5b80c922ebdd More merge content.
Bryan O'Sullivan <bos@serpentine.com>
parents: 83
diff changeset
194 self.ps1: '$',
5b80c922ebdd More merge content.
Bryan O'Sullivan <bos@serpentine.com>
parents: 83
diff changeset
195 self.ps2: '>',
5b80c922ebdd More merge content.
Bryan O'Sullivan <bos@serpentine.com>
parents: 83
diff changeset
196 }
5b80c922ebdd More merge content.
Bryan O'Sullivan <bos@serpentine.com>
parents: 83
diff changeset
197
137
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
198 ignore = [
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
199 r'\d+:[0-9a-f]{12}', # changeset number:hash
141
627effec9d4e More attempts to tidy up regexps for ignoring.
Bryan O'Sullivan <bos@serpentine.com>
parents: 138
diff changeset
200 r'[0-9a-f]{40}', # long changeset hash
627effec9d4e More attempts to tidy up regexps for ignoring.
Bryan O'Sullivan <bos@serpentine.com>
parents: 138
diff changeset
201 r'[0-9a-f]{12}', # short changeset hash
137
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
202 r'^(?:---|\+\+\+) .*', # diff header with dates
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
203 r'^date:.*', # date
141
627effec9d4e More attempts to tidy up regexps for ignoring.
Bryan O'Sullivan <bos@serpentine.com>
parents: 138
diff changeset
204 #r'^diff -r.*', # "diff -r" is followed by hash
138
d374685eb7fa Handle a few more common regexps. Clean up on exit nicely.
Bryan O'Sullivan <bos@serpentine.com>
parents: 137
diff changeset
205 r'^# Date \d+ \d+', # hg patch header
137
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
206 ]
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
207
138
d374685eb7fa Handle a few more common regexps. Clean up on exit nicely.
Bryan O'Sullivan <bos@serpentine.com>
parents: 137
diff changeset
208 err = False
173
754312dc23d5 If something times out, try to tell what it was.
Bryan O'Sullivan <bos@serpentine.com>
parents: 172
diff changeset
209 read_hint = ''
138
d374685eb7fa Handle a few more common regexps. Clean up on exit nicely.
Bryan O'Sullivan <bos@serpentine.com>
parents: 137
diff changeset
210
4
33a2e7b9978d Make it possible to include example input and output from real programs.
Bryan O'Sullivan <bos@serpentine.com>
parents: 3
diff changeset
211 try:
71
ddf533d41c09 Propagate errors correctly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 70
diff changeset
212 try:
73
9604dd885616 Fix run-example script on Debian.
Bryan O'Sullivan <bos@serpentine.com>
parents: 72
diff changeset
213 # eat first prompt string from shell
173
754312dc23d5 If something times out, try to tell what it was.
Bryan O'Sullivan <bos@serpentine.com>
parents: 172
diff changeset
214 self.read(read_hint)
71
ddf533d41c09 Propagate errors correctly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 70
diff changeset
215 # setup env and prompt
173
754312dc23d5 If something times out, try to tell what it was.
Bryan O'Sullivan <bos@serpentine.com>
parents: 172
diff changeset
216 ps, output = self.sendreceive('source %s\n' % rcfile,
754312dc23d5 If something times out, try to tell what it was.
Bryan O'Sullivan <bos@serpentine.com>
parents: 172
diff changeset
217 read_hint)
71
ddf533d41c09 Propagate errors correctly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 70
diff changeset
218 for hunk in self.parse():
ddf533d41c09 Propagate errors correctly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 70
diff changeset
219 # is this line a processing instruction?
ddf533d41c09 Propagate errors correctly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 70
diff changeset
220 m = self.pi_re.match(hunk)
ddf533d41c09 Propagate errors correctly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 70
diff changeset
221 if m:
ddf533d41c09 Propagate errors correctly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 70
diff changeset
222 pi, rest = m.groups()
ddf533d41c09 Propagate errors correctly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 70
diff changeset
223 if pi == 'name':
ddf533d41c09 Propagate errors correctly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 70
diff changeset
224 self.status('.')
ddf533d41c09 Propagate errors correctly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 70
diff changeset
225 out = rest
155
914babdc99c8 run-example: better error if bogus section name found.
Bryan O'Sullivan <bos@serpentine.com>
parents: 154
diff changeset
226 if out in ('err', 'lxo', 'out', 'run', 'tmp'):
914babdc99c8 run-example: better error if bogus section name found.
Bryan O'Sullivan <bos@serpentine.com>
parents: 154
diff changeset
227 print >> sys.stderr, ('%s: illegal section '
914babdc99c8 run-example: better error if bogus section name found.
Bryan O'Sullivan <bos@serpentine.com>
parents: 154
diff changeset
228 'name %r' %
914babdc99c8 run-example: better error if bogus section name found.
Bryan O'Sullivan <bos@serpentine.com>
parents: 154
diff changeset
229 (self.name, out))
914babdc99c8 run-example: better error if bogus section name found.
Bryan O'Sullivan <bos@serpentine.com>
parents: 154
diff changeset
230 return 1
71
ddf533d41c09 Propagate errors correctly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 70
diff changeset
231 assert os.sep not in out
137
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
232 if ofp is not None:
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
233 ofp.close()
160
745ff473c8c4 Catch more errors.
Bryan O'Sullivan <bos@serpentine.com>
parents: 155
diff changeset
234 err |= self.rename_output(ofp_basename, ignore)
71
ddf533d41c09 Propagate errors correctly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 70
diff changeset
235 if out:
137
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
236 ofp_basename = '%s.%s' % (self.name, out)
173
754312dc23d5 If something times out, try to tell what it was.
Bryan O'Sullivan <bos@serpentine.com>
parents: 172
diff changeset
237 read_hint = ofp_basename + ' '
137
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
238 ofp = open(ofp_basename + '.tmp', 'w')
71
ddf533d41c09 Propagate errors correctly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 70
diff changeset
239 else:
ddf533d41c09 Propagate errors correctly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 70
diff changeset
240 ofp = None
137
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
241 elif pi == 'ignore':
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
242 ignore.append(rest)
71
ddf533d41c09 Propagate errors correctly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 70
diff changeset
243 elif hunk.strip():
ddf533d41c09 Propagate errors correctly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 70
diff changeset
244 # it's something we should execute
173
754312dc23d5 If something times out, try to tell what it was.
Bryan O'Sullivan <bos@serpentine.com>
parents: 172
diff changeset
245 newps, output = self.sendreceive(hunk, read_hint)
168
f8b5b782e150 Backed out changeset 08a4467f489162c14cf17b94aa771a23dd6f02dc
Bryan O'Sullivan <bos@serpentine.com>
parents: 166
diff changeset
246 if not ofp:
71
ddf533d41c09 Propagate errors correctly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 70
diff changeset
247 continue
ddf533d41c09 Propagate errors correctly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 70
diff changeset
248 # first, print the command we ran
ddf533d41c09 Propagate errors correctly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 70
diff changeset
249 if not hunk.startswith('#'):
ddf533d41c09 Propagate errors correctly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 70
diff changeset
250 nl = hunk.endswith('\n')
103
5b80c922ebdd More merge content.
Bryan O'Sullivan <bos@serpentine.com>
parents: 83
diff changeset
251 hunk = ('%s \\textbf{%s}' %
5b80c922ebdd More merge content.
Bryan O'Sullivan <bos@serpentine.com>
parents: 83
diff changeset
252 (prompts[ps],
5b80c922ebdd More merge content.
Bryan O'Sullivan <bos@serpentine.com>
parents: 83
diff changeset
253 tex_escape(hunk.rstrip('\n'))))
71
ddf533d41c09 Propagate errors correctly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 70
diff changeset
254 if nl: hunk += '\n'
ddf533d41c09 Propagate errors correctly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 70
diff changeset
255 ofp.write(hunk)
ddf533d41c09 Propagate errors correctly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 70
diff changeset
256 # then its output
ddf533d41c09 Propagate errors correctly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 70
diff changeset
257 ofp.write(tex_escape(output))
103
5b80c922ebdd More merge content.
Bryan O'Sullivan <bos@serpentine.com>
parents: 83
diff changeset
258 ps = newps
71
ddf533d41c09 Propagate errors correctly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 70
diff changeset
259 self.status('\n')
ddf533d41c09 Propagate errors correctly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 70
diff changeset
260 except:
72
12df31afb4e1 Propagate exceptions more correctly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 71
diff changeset
261 print >> sys.stderr, '(killed)'
79
53427f786a0f Make run-example time out if shell seems to get stuck.
Bryan O'Sullivan <bos@serpentine.com>
parents: 78
diff changeset
262 os.kill(self.pid, signal.SIGKILL)
72
12df31afb4e1 Propagate exceptions more correctly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 71
diff changeset
263 pid, rc = os.wait()
71
ddf533d41c09 Propagate errors correctly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 70
diff changeset
264 raise
72
12df31afb4e1 Propagate exceptions more correctly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 71
diff changeset
265 else:
71
ddf533d41c09 Propagate errors correctly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 70
diff changeset
266 try:
173
754312dc23d5 If something times out, try to tell what it was.
Bryan O'Sullivan <bos@serpentine.com>
parents: 172
diff changeset
267 ps, output = self.sendreceive('exit\n', read_hint)
138
d374685eb7fa Handle a few more common regexps. Clean up on exit nicely.
Bryan O'Sullivan <bos@serpentine.com>
parents: 137
diff changeset
268 if ofp is not None:
71
ddf533d41c09 Propagate errors correctly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 70
diff changeset
269 ofp.write(output)
138
d374685eb7fa Handle a few more common regexps. Clean up on exit nicely.
Bryan O'Sullivan <bos@serpentine.com>
parents: 137
diff changeset
270 ofp.close()
160
745ff473c8c4 Catch more errors.
Bryan O'Sullivan <bos@serpentine.com>
parents: 155
diff changeset
271 err |= self.rename_output(ofp_basename, ignore)
73
9604dd885616 Fix run-example script on Debian.
Bryan O'Sullivan <bos@serpentine.com>
parents: 72
diff changeset
272 os.close(self.cfd)
71
ddf533d41c09 Propagate errors correctly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 70
diff changeset
273 except IOError:
ddf533d41c09 Propagate errors correctly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 70
diff changeset
274 pass
79
53427f786a0f Make run-example time out if shell seems to get stuck.
Bryan O'Sullivan <bos@serpentine.com>
parents: 78
diff changeset
275 os.kill(self.pid, signal.SIGTERM)
72
12df31afb4e1 Propagate exceptions more correctly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 71
diff changeset
276 pid, rc = os.wait()
160
745ff473c8c4 Catch more errors.
Bryan O'Sullivan <bos@serpentine.com>
parents: 155
diff changeset
277 err = err or rc
745ff473c8c4 Catch more errors.
Bryan O'Sullivan <bos@serpentine.com>
parents: 155
diff changeset
278 if err:
72
12df31afb4e1 Propagate exceptions more correctly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 71
diff changeset
279 if os.WIFEXITED(rc):
12df31afb4e1 Propagate exceptions more correctly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 71
diff changeset
280 print >> sys.stderr, '(exit %s)' % os.WEXITSTATUS(rc)
12df31afb4e1 Propagate exceptions more correctly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 71
diff changeset
281 elif os.WIFSIGNALED(rc):
12df31afb4e1 Propagate exceptions more correctly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 71
diff changeset
282 print >> sys.stderr, '(signal %s)' % os.WTERMSIG(rc)
136
7b5894fffc37 Don't falsely signal success to make.
Bryan O'Sullivan <bos@serpentine.com>
parents: 124
diff changeset
283 else:
7b5894fffc37 Don't falsely signal success to make.
Bryan O'Sullivan <bos@serpentine.com>
parents: 124
diff changeset
284 open(self.name + '.run', 'w')
160
745ff473c8c4 Catch more errors.
Bryan O'Sullivan <bos@serpentine.com>
parents: 155
diff changeset
285 return err
72
12df31afb4e1 Propagate exceptions more correctly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 71
diff changeset
286 finally:
4
33a2e7b9978d Make it possible to include example input and output from real programs.
Bryan O'Sullivan <bos@serpentine.com>
parents: 3
diff changeset
287 shutil.rmtree(tmpdir)
3
906d9021f9e5 Making progress on autogenerated example output.
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
288
137
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
289 def rename_output(self, base, ignore):
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
290 mangle_re = re.compile('(?:' + '|'.join(ignore) + ')')
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
291 def mangle(s):
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
292 return mangle_re.sub('', s)
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
293 def matchfp(fp1, fp2):
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
294 while True:
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
295 s1 = mangle(fp1.readline())
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
296 s2 = mangle(fp2.readline())
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
297 if cmp(s1, s2):
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
298 break
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
299 if not s1:
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
300 return True
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
301 return False
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
302
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
303 oldname = base + '.out'
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
304 tmpname = base + '.tmp'
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
305 errname = base + '.err'
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
306 errfp = open(errname, 'w+')
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
307 for line in open(tmpname):
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
308 errfp.write(mangle_re.sub('', line))
146
65f6f9d18fa1 Oops! I forgot that I need the undoctored output files in the book!
Bryan O'Sullivan <bos@serpentine.com>
parents: 142
diff changeset
309 os.rename(tmpname, base + '.lxo')
137
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
310 errfp.seek(0)
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
311 try:
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
312 oldfp = open(oldname)
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
313 except IOError, err:
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
314 if err.errno != errno.ENOENT:
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
315 raise
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
316 os.rename(errname, oldname)
160
745ff473c8c4 Catch more errors.
Bryan O'Sullivan <bos@serpentine.com>
parents: 155
diff changeset
317 return False
137
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
318 if matchfp(oldfp, errfp):
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
319 os.unlink(errname)
160
745ff473c8c4 Catch more errors.
Bryan O'Sullivan <bos@serpentine.com>
parents: 155
diff changeset
320 return False
137
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
321 else:
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
322 print >> sys.stderr, '\nOutput of %s has changed!' % base
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
323 os.system('diff -u %s %s 1>&2' % (oldname, errname))
138
d374685eb7fa Handle a few more common regexps. Clean up on exit nicely.
Bryan O'Sullivan <bos@serpentine.com>
parents: 137
diff changeset
324 return True
137
9d7dffe74b2c Save "good" example output so we can see if something has broken.
Bryan O'Sullivan <bos@serpentine.com>
parents: 136
diff changeset
325
3
906d9021f9e5 Making progress on autogenerated example output.
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
326 def main(path='.'):
78
a893de25bc24 Add -v option to run-example, to assist with debugging example scripts.
Bryan O'Sullivan <bos@serpentine.com>
parents: 77
diff changeset
327 opts, args = getopt.getopt(sys.argv[1:], 'v', ['verbose'])
a893de25bc24 Add -v option to run-example, to assist with debugging example scripts.
Bryan O'Sullivan <bos@serpentine.com>
parents: 77
diff changeset
328 verbose = False
a893de25bc24 Add -v option to run-example, to assist with debugging example scripts.
Bryan O'Sullivan <bos@serpentine.com>
parents: 77
diff changeset
329 for o, a in opts:
a893de25bc24 Add -v option to run-example, to assist with debugging example scripts.
Bryan O'Sullivan <bos@serpentine.com>
parents: 77
diff changeset
330 if o in ('-v', '--verbose'):
a893de25bc24 Add -v option to run-example, to assist with debugging example scripts.
Bryan O'Sullivan <bos@serpentine.com>
parents: 77
diff changeset
331 verbose = True
71
ddf533d41c09 Propagate errors correctly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 70
diff changeset
332 errs = 0
3
906d9021f9e5 Making progress on autogenerated example output.
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
333 if args:
906d9021f9e5 Making progress on autogenerated example output.
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
334 for a in args:
75
2bfa2499e971 Make run-example print better error messages when things go wrong.
Bryan O'Sullivan <bos@serpentine.com>
parents: 73
diff changeset
335 try:
2bfa2499e971 Make run-example print better error messages when things go wrong.
Bryan O'Sullivan <bos@serpentine.com>
parents: 73
diff changeset
336 st = os.lstat(a)
2bfa2499e971 Make run-example print better error messages when things go wrong.
Bryan O'Sullivan <bos@serpentine.com>
parents: 73
diff changeset
337 except OSError, err:
2bfa2499e971 Make run-example print better error messages when things go wrong.
Bryan O'Sullivan <bos@serpentine.com>
parents: 73
diff changeset
338 print >> sys.stderr, '%s: %s' % (a, err.strerror)
2bfa2499e971 Make run-example print better error messages when things go wrong.
Bryan O'Sullivan <bos@serpentine.com>
parents: 73
diff changeset
339 errs += 1
2bfa2499e971 Make run-example print better error messages when things go wrong.
Bryan O'Sullivan <bos@serpentine.com>
parents: 73
diff changeset
340 continue
2bfa2499e971 Make run-example print better error messages when things go wrong.
Bryan O'Sullivan <bos@serpentine.com>
parents: 73
diff changeset
341 if stat.S_ISREG(st.st_mode) and st.st_mode & 0111:
78
a893de25bc24 Add -v option to run-example, to assist with debugging example scripts.
Bryan O'Sullivan <bos@serpentine.com>
parents: 77
diff changeset
342 if example(a, verbose).run():
75
2bfa2499e971 Make run-example print better error messages when things go wrong.
Bryan O'Sullivan <bos@serpentine.com>
parents: 73
diff changeset
343 errs += 1
2bfa2499e971 Make run-example print better error messages when things go wrong.
Bryan O'Sullivan <bos@serpentine.com>
parents: 73
diff changeset
344 else:
2bfa2499e971 Make run-example print better error messages when things go wrong.
Bryan O'Sullivan <bos@serpentine.com>
parents: 73
diff changeset
345 print >> sys.stderr, '%s: not a file, or not executable' % a
71
ddf533d41c09 Propagate errors correctly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 70
diff changeset
346 errs += 1
ddf533d41c09 Propagate errors correctly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 70
diff changeset
347 return errs
164
8f4c9ae918af Run examples in predictable order if we're doing the lot.
Bryan O'Sullivan <bos@serpentine.com>
parents: 160
diff changeset
348 names = os.listdir(path)
8f4c9ae918af Run examples in predictable order if we're doing the lot.
Bryan O'Sullivan <bos@serpentine.com>
parents: 160
diff changeset
349 names.sort()
8f4c9ae918af Run examples in predictable order if we're doing the lot.
Bryan O'Sullivan <bos@serpentine.com>
parents: 160
diff changeset
350 for name in names:
3
906d9021f9e5 Making progress on autogenerated example output.
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
351 if name == 'run-example' or name.startswith('.'): continue
906d9021f9e5 Making progress on autogenerated example output.
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
352 if name.endswith('.out') or name.endswith('~'): continue
45
6b7b0339e7d6 Don't rerun examples unnecessarily.
Bryan O'Sullivan <bos@serpentine.com>
parents: 36
diff changeset
353 if name.endswith('.run'): continue
19
187702df428b Piles of new content for MQ chapter - cookbook stuff.
Bryan O'Sullivan <bos@serpentine.com>
parents: 6
diff changeset
354 pathname = os.path.join(path, name)
142
d9f332f85673 Handle YA corner case.
Bryan O'Sullivan <bos@serpentine.com>
parents: 141
diff changeset
355 try:
d9f332f85673 Handle YA corner case.
Bryan O'Sullivan <bos@serpentine.com>
parents: 141
diff changeset
356 st = os.lstat(pathname)
d9f332f85673 Handle YA corner case.
Bryan O'Sullivan <bos@serpentine.com>
parents: 141
diff changeset
357 except OSError, err:
d9f332f85673 Handle YA corner case.
Bryan O'Sullivan <bos@serpentine.com>
parents: 141
diff changeset
358 # could be an output file that was removed while we ran
d9f332f85673 Handle YA corner case.
Bryan O'Sullivan <bos@serpentine.com>
parents: 141
diff changeset
359 if err.errno != errno.ENOENT:
d9f332f85673 Handle YA corner case.
Bryan O'Sullivan <bos@serpentine.com>
parents: 141
diff changeset
360 raise
d9f332f85673 Handle YA corner case.
Bryan O'Sullivan <bos@serpentine.com>
parents: 141
diff changeset
361 continue
36
5cee64874312 Require examples to be executable, so it's easier to see them with "ls".
Bryan O'Sullivan <bos@serpentine.com>
parents: 24
diff changeset
362 if stat.S_ISREG(st.st_mode) and st.st_mode & 0111:
78
a893de25bc24 Add -v option to run-example, to assist with debugging example scripts.
Bryan O'Sullivan <bos@serpentine.com>
parents: 77
diff changeset
363 if example(pathname, verbose).run():
71
ddf533d41c09 Propagate errors correctly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 70
diff changeset
364 errs += 1
4
33a2e7b9978d Make it possible to include example input and output from real programs.
Bryan O'Sullivan <bos@serpentine.com>
parents: 3
diff changeset
365 print >> open(os.path.join(path, '.run'), 'w'), time.asctime()
71
ddf533d41c09 Propagate errors correctly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 70
diff changeset
366 return errs
3
906d9021f9e5 Making progress on autogenerated example output.
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
367
906d9021f9e5 Making progress on autogenerated example output.
Bryan O'Sullivan <bos@serpentine.com>
parents:
diff changeset
368 if __name__ == '__main__':
71
ddf533d41c09 Propagate errors correctly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 70
diff changeset
369 sys.exit(main())