Mercurial > mplayer.hg
annotate DOCS/tech/libmpcodecs.txt @ 12110:a34dc5a369ca
restrictions to ensure that O(log n) seeking and error recovery is possible
author | michael |
---|---|
date | Sat, 03 Apr 2004 13:23:44 +0000 |
parents | 2e1bd9192ce2 |
children | d6f9191f4f82 |
rev | line source |
---|---|
7397 | 1 The libMPcodecs API details, hints - by A'rpi |
2 ================================== | |
3 | |
7399 | 4 See also: colorspaces.txt, codec-devel.txt, dr-methods.txt, codecs.conf.txt |
7397 | 5 |
6 The VIDEO path: | |
7 =============== | |
8 | |
9 [MPlayer core] | |
10 | (1) | |
11 _____V______ (2) /~~~~~~~~~~\ (3,4) |~~~~~~| | |
12 | | -----> | vd_XXX.c | -------> | vd.c | | |
13 | decvideo | \__________/ <-(3a)-- |______| | |
14 | | -----, ,.............(3a,4a).....: | |
15 ~~~~~~~~~~~~ (6) V V | |
16 /~~~~~~~~\ /~~~~~~~~\ (8) | |
17 | vf_X.c | --> | vf_Y.c | ----> vf_vo.c / ve_XXX.c | |
18 \________/ \________/ | |
19 | ^ | |
20 (7) | |~~~~~~| : (7a) | |
21 `-> | vf.c |...: | |
22 |______| | |
23 | |
24 Short description of video path: | |
25 1. mplayer/mencoder core requests the decoding of a compressed video frame: | |
26 calls decvideo.c::decode_video() | |
27 | |
28 2. decode_video() calls the previously ( init_video() ) selected video codec | |
29 (vd_XXXX.c file, where XXXX == vfm name, see the 'driver' line of codecs.conf) | |
30 | |
31 3. the codec should initialize output device before decoding the first frame, | |
32 it may happen in init() or at the middle of the first decode(). see 3.a. | |
33 it means calling vd.c::mpcodecs_config_vo() with the image dimensions, | |
34 and the _preferred_ (mean: internal, native, best) colorspace. | |
35 NOTE: this colorspace may not be equal to the actually used colorspace, it's | |
36 just a _hint_ for the csp matching algorithm, and mainly used _only_ when | |
37 csp conversion is required, as input format of the converter. | |
38 | |
39 3.a. selecting the best output colorspace: | |
40 the vd.c::mpcodecs_config_vo() function will go through the outfmt list | |
41 defined by codecs.conf's 'out' lines, and query both vd (codec) and vo | |
42 (output device/filter/encoder) if it's supported or not. | |
43 | |
44 For the vo, it calls the query_format() func of vf_XXX.c or ve_XXX.c. | |
45 It should return a set of feature flags, the most important ons for this | |
46 stage are: VFCAP_CSP_SUPPORTED (csp supported directly or by conversion) | |
47 and VFCAP_CSP_SUPPORTED_BY_HW (csp supported WITHOUT any conversion). | |
48 | |
49 For the vd (codec), control() with VDCTRL_QUERY_FORMAT will be called. | |
50 If it doesn't implement VDCTRL_QUERY_FORMAT, (ie answers CONTROL_UNKNOWN | |
51 or CONTROL_NA) it will be assumed to be CONTROL_TRUE (csp supported)! | |
52 | |
53 So, by default, if the list of supported colorspaces is constant, doesn't | |
54 depend on the actual file's/stream's header, it's enough to list them | |
55 in codecs.conf ('out' field), and don't implement VDCTRL_QUERY_FORMAT. | |
56 It's the case for the most codecs. | |
57 | |
58 If the supported csp list depends on the file being decoded, list the | |
59 possible out formats (colorspaces) in codecs.conf, and implement the | |
60 VDCTRL_QUERY_FORMAT to test the availability of the given csp for the | |
61 given video file/stream. | |
62 | |
63 The vd.c core will find the best matching colorspace, depending on the | |
64 VFCAP_CSP_SUPPORTED_BY_HW flag (see vfcap.h). If no match at all, it | |
65 will try again with the 'scale' filter inserted between vd and vo. | |
66 If still no match, it will fail :( | |
67 | |
68 4. requesting buffer for the decoded frame: | |
69 The codec have to call mpcodecs_get_image() with proper imgtype & imgflag. | |
70 It will find the optimal buffering setup (preferred stride, alignment etc) | |
71 and return a pointer to the allocated and filled up mpi (mp_image_t*) struct. | |
72 The 'imgtype' controls the buffering setup, ie STATIC (just one buffer, | |
73 it 'remembers' its contents between frames), TEMP (write-only, full update), | |
74 EXPORT (memory allocation is done by the codec, not recommended) and so on. | |
75 The 'imgflags' set up the limits for the buffer, ie stride limitations, | |
76 readability, remembering content etc. See mp_image.h for the short descr. | |
77 See dr-methods.txt for the explanation of buffer importing and mpi imgtypes. | |
78 | |
79 Always try to implement stride support! (stride == bytes per line) | |
80 If no stride support, then stride==bytes_per_pixel*image_width. | |
81 If you have stride supoprt in your decoder, use the mpi->stride[] value | |
82 for the byte_per_line for each plane. | |
83 Also take care of other imgflags, like MP_IMGFLAG_PRESERVE and | |
84 MP_IMGFLAG_READABLE, MP_IMGFLAG_COMMON_STRIDE and MP_IMGFLAG_COMMON_PLANE! | |
85 The file mp_image.h contains description of flags in comments, read it! | |
86 Ask for help on -dev-eng, describing the behaviour your codec, if unsure. | |
87 | |
88 4.a. buffer allocation, vd.c::mpcodecs_get_image(): | |
89 If the requested buffer imgtype!=EXPORT, then vd.c will try to do | |
90 direct rendering, ie. asks the next filter/vo for the buffer allocation. | |
91 It's done by calling get_image() of the vf_XXX.c file. | |
92 If it was successful, the imgflag MP_IMGFLAG_DIRECT will be set, and one | |
93 memcpy() will be saved when passing the data from vd to the next filter/vo. | |
94 See dr-methods.txt for details and examples. | |
95 | |
96 5. decode the frame, to the mpi structure requested at 4, then return the mpi | |
97 to decvideo.c. Return NULL if the decoding failed or skipped frame. | |
98 | |
99 6. decvideo.c::decode_video() will now pass the 'mpi' to the next filter (vf_X). | |
100 | |
101 7. the filter's (vf_X) put_image() then requests a new mpi buffer by calling | |
102 vf.c::vf_get_image(). | |
103 | |
104 7.a. vf.c::vf_get_image() will try to get direct rendering, by asking the | |
105 next fliter to do the buffer allocation (calls vf_Y's get_image()). | |
106 if it fails, it will fallback to normal system memory allocation. | |
107 | |
108 8. when we're over the whole filter chain (multiple filters can be connected, | |
109 even the same filter multiple times) then the last, 'leaf' filter will be | |
110 called. The only difference between leaf and non-leaf filter is that leaf | |
111 filter have to implement the whole filter api. | |
112 Leaf filters are now: vf_vo.c (wrapper over libvo) and ve_XXX.c (video | |
113 encoders used by mencoder). | |
9687 | 114 |
7397 | 115 |
9687 | 116 Video Filters |
117 ============= | |
118 | |
119 Video filters are plugin-like code modules implementing the interface | |
120 defined in vf.h. | |
121 | |
122 Basically it means video output manipulation, i.e. these plugins can | |
123 modify the image and the image properties (size, colorspace etc) between | |
124 the video decoders (vd.h) and the output layer (libvo or video encoders). | |
125 | |
126 The actual API is a mixture of the video decoder (vd.h) and libvo | |
127 (video_out.h) APIs. | |
128 | |
129 main differences: | |
130 - vf plugins may be "loaded" multiple times, with different parameters | |
131 and context - it's new in MPlayer, old APIs weren't reentrant. | |
132 - vf plugins don't have to implement all functions - all functions have a | |
133 'fallback' version, so the plugins only override these if wanted. | |
134 - Each vf plugin has its own get_image context, and they can interchange | |
135 images/buffers using these get_image/put_image calls. | |
136 | |
137 | |
8001 | 138 The VIDEO FILTER API: |
139 ===================== | |
140 filename: vf_FILTERNAME.c | |
141 | |
142 vf_info_t* info; | |
143 pointer to the filter description structure: | |
144 | |
145 const char *info; // description of the filter | |
146 const char *name; // short name of the filter, must be FILTERNAME | |
147 const char *author; // name and email/url of the author(s) | |
148 const char *comment;// comment, url to papers describing algo etc. | |
149 int (*open)(struct vf_instance_s* vf,char* args); | |
150 // pointer to the open() function: | |
151 | |
8020 | 152 Sample: |
153 | |
154 vf_info_t vf_info_foobar = { | |
155 "Universal Foo and Bar filter", | |
156 "foobar", | |
157 "Ms. Foo Bar", | |
158 "based on algo described at http://www.foo-bar.org", | |
159 open | |
160 }; | |
161 | |
8001 | 162 The open() function: |
163 | |
164 open() is called when the filter is appended/inserted to the filter chain. | |
165 it'll receive the handler (vf) and the optional filter parameters as | |
166 char* string. Note, that encoders (ve_*) and vo wrapper (vf_vo.c) have | |
167 non-string arg, but it's specially handled by mplayer/mencoder. | |
168 | |
169 The open() function should fill the vf_instance_t structure, with the | |
170 implemented functions' pointers (see bellow). | |
171 It can optinally allocate memory for its internal data (vf_priv_t) and | |
172 store the pointer in vf->priv. | |
173 | |
174 The open() func should parse (or at least check syntax) of parameters, | |
175 and fail (return 0) if error. | |
176 | |
177 Sample: | |
178 | |
179 static int open(vf_instance_t *vf, char* args){ | |
180 vf->query_format=query_format; | |
181 vf->config=config; | |
182 vf->put_image=put_image; | |
183 // allocate local storage: | |
184 vf->priv=malloc(sizeof(struct vf_priv_s)); | |
185 vf->priv->w= | |
186 vf->priv->h=-1; | |
187 if(args) // parse args: | |
188 if(sscanf(args, "%d:%d", &vf->priv->w, &vf->priv->h)!=2) return 0; | |
189 return 1; | |
190 } | |
191 | |
192 Functions in vf_instance_s: | |
193 | |
194 NOTE: All these are optional, their func pointer is either NULL or points to | |
195 a default implementation. If you implement them, don't forget to set | |
196 vf->FUNCNAME in your open() ! | |
197 | |
198 int (*query_format)(struct vf_instance_s* vf, | |
199 unsigned int fmt); | |
200 | |
201 The query_format() func. is called one or more times before the config(), | |
202 to find out the capabilities and/or support status of a given colorspace (fmt). | |
203 For the return values, see vfcap.h! | |
204 Normally, a filter should return at least VFCAP_CSP_SUPPORTED for all supported | |
8020 | 205 colorspaces it accepts as input, and 0 for the unsupported ones. |
206 If your filter does linear conversion, it should query the next filter, | |
207 and merge in its capability flags. Note: you should always ensure that the | |
208 next filter will accept at least one of your possible output colorspaces! | |
8001 | 209 |
210 Sample: | |
211 | |
212 static int query_format(struct vf_instance_s* vf, unsigned int fmt){ | |
213 switch(fmt){ | |
214 case IMGFMT_YV12: | |
215 case IMGFMT_I420: | |
216 case IMGFMT_IYUV: | |
217 case IMGFMT_422P: | |
218 return vf_next_query_format(vf,IMGFMT_YUY2) & (~VFCAP_CSP_SUPPORTED_BY_HW); | |
219 } | |
220 return 0; | |
221 } | |
222 | |
8020 | 223 For the more complex case, when you have an N->M colorspace mapping matrix, |
224 see vf_scale or vf_rgb2bgr for examples. | |
225 | |
8001 | 226 |
227 int (*config)(struct vf_instance_s* vf, | |
228 int width, int height, int d_width, int d_height, | |
229 unsigned int flags, unsigned int outfmt); | |
230 | |
231 The config() is called to initialize/confugre the filter before using it. | |
232 Its parameters are already well-known from libvo: | |
233 width, height: size of the coded image | |
234 d_width, d_height: wanted display size (usually aspect corrected w/h) | |
8020 | 235 Filters should use width,height as input image dimension, but the |
236 resizing filters (crop, expand, scale, rotate, etc) should update | |
237 d_width/d_height (display size) to preserve the correct aspect ratio! | |
238 Filters should not rely on d_width, d_height as input parameters, | |
239 the only exception is when a filter replaces some libvo functionality | |
9644
0fe056bdb135
vop -> vf change, small fixes. The Polish documentation should be checked for correctness.
jonas
parents:
9399
diff
changeset
|
240 (like -vf scale with -zoom, or OSD rendering wiht -vf expand). |
8001 | 241 flags: the "good" old flags set of libvo: |
242 0x01 - force fullscreen (-fs) | |
243 0x02 - allow mode switching (-vm) | |
244 0x04 - allow software scaling (-zoom) | |
245 0x08 - flipping (-flip) | |
246 (Usually you don't have to worry about flags, just pass it to next config.) | |
247 outfmt: the selected colorspace/pixelformat. You'll receive images in this | |
248 format. | |
249 | |
250 Sample: | |
251 | |
252 static int config(struct vf_instance_s* vf, | |
253 int width, int height, int d_width, int d_height, | |
254 unsigned int flags, unsigned int outfmt){ | |
255 // use d_width/d_height if not set by user: | |
256 if(vf->priv->w==-1) vf->priv->w=d_width; | |
257 if(vf->priv->h==-1) vf->priv->h=d_width; | |
258 // initialize your filter code | |
259 ... | |
260 // ok now config the rest of the filter chain, with our output parameters: | |
261 return vf_next_config(vf,vf->priv->w,vf->priv->h,d_width,d_height,flags,outfmt); | |
262 } | |
263 | |
264 void (*uninit)(struct vf_instance_s* vf); | |
265 | |
266 Okey, uninit() is the simplest, it's called at the end. You can free your | |
267 private buffers etc here. | |
268 | |
269 int (*put_image)(struct vf_instance_s* vf, | |
270 mp_image_t *mpi); | |
271 | |
272 Ah, put_image(). This is the main filter function it should convert/filter/ | |
273 transform the image data from one format/size/color/whatever to another. | |
274 Its input parameter is an mpi (mplayer image) structure, see mp_image.h. | |
275 Your filter has to request a new image buffer for the output, using the | |
276 vf_get_image() function. NOTE: even if you don't want to modify the image, | |
277 just pass it to the next filter, you have to either | |
278 - do not implement put_image() at all - then it will be skipped | |
279 - request a new image with type==EXPORT and copy the pointers | |
280 NEVER pass the mpi as-is, it's local to filters and may cause trouble. | |
281 | |
282 If you completely copy/transform the image, then you probably want this: | |
283 | |
284 dmpi=vf_get_image(vf->next,mpi->imgfmt, | |
285 MP_IMGTYPE_TEMP, MP_IMGFLAG_ACCEPT_STRIDE, | |
286 vf->priv->w, vf->priv->h); | |
287 | |
288 It will allocate a new image, and return an mp_image structure filled by | |
289 buffer pointers and stride (bytes per line) values, in size of vf->priv->w | |
290 times vf->priv->h. If your filter cannot handle stride, then left out | |
291 MP_IMGFLAG_ACCEPT_STRIDE. Note, that you can do this, but it isn't recommended, | |
292 the whole video path is designed to use strides to get optimal throughput. | |
293 If your filter allocates output image buffers, then use MP_IMGTYPE_EXPORT, | |
294 and fill the returned dmpi's planes[], stride[] with your buffer parameters. | |
295 Note, it is not recommended (no direct rendering), so if you can, use | |
296 vf_get_image() for buffer allocation! | |
297 For other image types and flags see mp_image.h, it has comments. | |
298 If you are unsure, feel free to ask on the -dev-eng maillist. Please | |
299 describe the behaviour of your filter, an dits limitations, so we can | |
300 suggest the optimal buffer type + flags for your code. | |
301 | |
302 Now, that you have the input (mpi) and output (dmpi) buffers, you can do | |
303 the conversion. If you didn't notice yet, mp_image has some useful info | |
304 fields, may help you a lot creating if() or for() structures: | |
305 flags: MP_IMGFLAG_PLANAR, MP_IMGFLAG_YUV, MP_IMGFLAG_SWAPPED | |
306 helps you to handle various pixel formats in single code. | |
307 bpp: bits per pixel | |
308 WARNING! It's number of bits _allocated_ to store a pixel, | |
309 it is not the number of bits actually used to keep colors! | |
310 So, it's 16 for both 15 and 16bit color depth, and is 32 for | |
311 32bpp (actually 24bit color depth) mode! | |
312 It's 1 for 1bpp, 9 for YVU9, and is 12 for YV12 mode. Get it? | |
313 For planar formats, you also have chroma_width, chroma_height and | |
8020 | 314 chroma_x_shift, chroma_y_shift too, they specify the chroma subsampling |
315 for yuv formats: | |
316 chroma_width = luma_width >>chroma_x_shift; | |
317 chroma_height= luma_height>>chroma_y_shift; | |
8001 | 318 |
319 If you're done, call the rest of the filter chain to process your output | |
320 image: | |
321 return vf_next_put_image(vf,dmpi); | |
322 | |
323 | |
324 Ok, the rest is for advanced functionality only: | |
325 | |
326 int (*control)(struct vf_instance_s* vf, | |
327 int request, void* data); | |
328 | |
329 You can control the filter in runtime from mplayer/mencoder/dec_video: | |
330 #define VFCTRL_QUERY_MAX_PP_LEVEL 4 /* test for postprocessing support (max level) */ | |
331 #define VFCTRL_SET_PP_LEVEL 5 /* set postprocessing level */ | |
332 #define VFCTRL_SET_EQUALIZER 6 /* set color options (brightness,contrast etc) */ | |
333 #define VFCTRL_GET_EQUALIZER 8 /* gset color options (brightness,contrast etc) */ | |
334 #define VFCTRL_DRAW_OSD 7 | |
335 #define VFCTRL_CHANGE_RECTANGLE 9 /* Change the rectangle boundaries */ | |
336 | |
8020 | 337 |
8001 | 338 void (*get_image)(struct vf_instance_s* vf, |
339 mp_image_t *mpi); | |
340 | |
341 This is for direct rendering support, works the same way as in libvo drivers. | |
342 It makes in-place pixel modifications possible. | |
343 If you implement it (vf->get_image!=NULL) then it will be called to do the | |
344 buffer allocation. You SHOULD check the buffer restrictions (stride, type, | |
345 readability etc) and if all OK, then allocate the requested buffer using | |
346 the vf_get_image() func and copying the buffer pointers. | |
347 | |
8020 | 348 NOTE: You HAVE TO save dmpi pointer, as you'll need it in put_image() later. |
349 It is not guaranteed that you'll get the same mpi for put_image() as in | |
350 get_image() (think of out-of-order decoding, get_image is called in decoding | |
351 order, while put_image is called for display) so the only safe place to save | |
352 it is in the mpi struct itself: mpi->priv=(void*)dmpi; | |
353 | |
354 | |
8001 | 355 void (*draw_slice)(struct vf_instance_s* vf, |
356 unsigned char** src, int* stride, int w,int h, int x, int y); | |
357 | |
358 It's the good old draw_slice callback, already known from libvo. | |
8020 | 359 If your filter can operate on partial images, you can implement this one |
8001 | 360 to improve performance (cache utilization). |
361 | |
8020 | 362 Ah, and there is two set of capability/requirement flags (vfcap.h type) |
8001 | 363 in vf_instance_t, used by default query_format() implementation, and by |
364 the automatic colorspace/stride matching code (vf_next_config()). | |
365 | |
366 // caps: | |
367 unsigned int default_caps; // used by default query_format() | |
368 unsigned int default_reqs; // used by default config() | |
369 | |
9399 | 370 btw, u should avoid using global or static variables, to store filter instance |
371 specific stuff, as filters might be used multiple times & in the future even | |
372 multiple streams might be possible | |
7397 | 373 |
9687 | 374 |
7397 | 375 The AUDIO path: |
376 =============== | |
377 TODO!!! |