]> diplodocus.org Git - nmh/blob - man/mhfixmsg.man
Only get a write lock on the sequences file if we're performing a pack
[nmh] / man / mhfixmsg.man
1 .TH MHFIXMSG %manext1% "March 12, 2016" "%nmhversion%"
2 .\"
3 .\" %nmhwarning%
4 .\"
5 .SH NAME
6 mhfixmsg \- rewrite MIME messages with various transformations
7 .SH SYNOPSIS
8 .HP 5
9 .na
10 .B mhfixmsg
11 .RI [ +folder ]
12 .RI [ msgs " | "
13 .IR "absolute pathname" " | "
14 .RB \-file
15 .IR file ]
16 .RB [ \-decodetext
17 8bit/7bit |
18 .BR \-nodecodetext ]
19 .RB [ \-decodetypes
20 .IR "type/[subtype][,...]" ]
21 .RB [ \-crlflinebreaks " | " \-nocrlflinebreaks ]
22 .RB [ \-textcharset
23 .I charset
24 .RB "| " \-notextcharset ]
25 .RB [ \-reformat " | " \-noreformat ]
26 .RB [ \-replacetextplain " | " \-noreplacetextplain ]
27 .RB [ \-fixboundary " | " \-nofixboundary ]
28 .RB [ \-fixcte " | " \-nofixcte ]
29 .RB [ \-fixtype
30 .IR mimetype ]
31 .RB [ \-outfile
32 .IR outfile ]
33 .RB [ \-rmmproc
34 .IR program ]
35 .RB [ \-normmproc ]
36 .RB [ \-changecur " | " \-nochangecur ]
37 .RB [ \-verbose " | " \-noverbose ]
38 .RB [ \-version ]
39 .RB [ \-help ]
40 .ad
41 .SH DESCRIPTION
42 .B mhfixmsg
43 rewrites MIME messages, applying specific transformations such as
44 decoding of MIME-encoded message parts and repairing invalid MIME
45 headers.
46 .PP
47 MIME messages are specified in RFC 2045 to RFC 2049
48 (see
49 .IR mhbuild (1)).
50 The
51 .B mhlist
52 command is invaluable for viewing the content structure of MIME
53 messages.
54 .B mhfixmsg
55 passes non-MIME messages through without any transformations. If no
56 transformations apply to a MIME message, the original message or file
57 is not modified or removed. Thus,
58 .B mhfixmsg
59 can safely be run multiple times on a message.
60 .PP
61 The
62 .B \-decodetext
63 switch enables a transformation to decode each base64 and
64 quoted-printable text message part to the selected 8bit or 7bit
65 encoding. If 7bit is selected for a base64 part but it will only fit
66 8bit, as defined by RFC 2045, then it will be decoded to 8bit
67 quoted-printable. Otherwise, if the decoded text would not fit the
68 selected encoding, the part is not decoded (and a message will be
69 displayed if
70 .B \-verbose
71 is enabled).
72 .PP
73 When the
74 .B \-decodetext
75 switch is enabled, each carriage return character that precedes a
76 linefeed character is removed from text parts encoded in ASCII,
77 ISO-8859-x, UTF-8, or Windows-12xx.
78 .PP
79 The
80 .B \-decodetypes
81 switch specifies the message parts, by type and optionally subtype,
82 to which
83 .B \-decodetext
84 applies. Its argument is a comma-separated list of type/subtype
85 elements. If an element does not contain a subtype, then
86 .B \-decodetext
87 applies to all subtypes of the type. The default is
88 .B \-decodetypes
89 .IR text ;
90 it can be overridden, e.g., with
91 .B \-decodetypes
92 .I text/plain
93 to restrict
94 .B \-decodetext
95 to just text/plain parts.
96 .PP
97 By default, carriage return characters are preserved or inserted at
98 the end of each line of text content. The
99 .B \-crlflinebreaks
100 switch selects this behavior and is enabled by default. The
101 .B \-nocrlflinebreaks
102 switch causes carriage return characters to be stripped from, and not
103 inserted in, text content when it is decoded and encoded. Note that
104 its use can cause the generation of MIME messages that do not conform
105 with RFC 2046, §4.1.1, paragraph 1.
106 .PP
107 The
108 .B \-textcharset
109 switch specifies that all text/plain parts of the message(s)
110 should be converted to
111 .IR charset .
112 Charset conversions require that
113 .B nmh
114 be built with
115 .IR iconv (3);
116 see the
117 .BR mhparam (1)
118 man page for how determine whether your
119 .B nmh
120 installation includes that.
121 To convert text parts other than text/plain, an external program can
122 be used, via the
123 .B \-reformat
124 switch.
125 .PP
126 The
127 .B \-reformat
128 switch enables a transformation for text parts in the message. For
129 each text part that is not text/plain and that does not have a
130 corresponding text/plain in a multipart/alternative part,
131 .B mhfixmsg
132 looks for a mhfixmsg-format-text/subtype profile entry that matches
133 the subtype of the part. If one is found and can be used to
134 successfully convert the part to text/plain,
135 .B mhfixmsg
136 inserts that text/plain part at the beginning of the containing
137 multipart/alternative part, if present. If not, it creates a
138 multipart/alternative part.
139 .PP
140 With the
141 .B \-reformat
142 switch, multipart/related parts are handled differently than
143 multipart/alternative. If the multipart/related has only a single
144 part that is not text/plain and can be converted to text/plain, a
145 text/plain part is added and the type of the part is changed to
146 multipart/alternative. If the multipart/related has more than one
147 part but does not have a text/plain part,
148 .B mhfixmsg
149 tries to add one.
150 .PP
151 The
152 .B \-replacetextplain
153 switch broadens the applicability of
154 .B \-reformat
155 by always replacing a corresponding text/plain part, if one exists.
156 If
157 .B \-verbose
158 if enabled, the replacement will be shown as two steps: a removal of
159 the text/plain part followed by the usual insertion of a new part.
160 .PP
161 .B \-reformat
162 requires a profile entry for each text part subtype to be reformatted.
163 The mhfixmsg-format-text/subtype profile entries are based on external
164 conversion programs, and are used the same way that
165 .B mhshow
166 uses its mhshow-show-text/subtype entries. When
167 .B nmh
168 is installed, it searches for a conversion program for text/html
169 content, and if one is found, inserts a mhfixmsg-format-text/html
170 entry in %nmhetcdir%/mhn.defaults. An entry of the same name in the
171 user's profile takes precedence. The user can add entries for
172 other text subtypes to their profile.
173 .PP
174 The
175 .B \-fixboundary
176 switch enables a transformation to repair the boundary portion of the
177 Content-Type header field of the message to match the boundaries of
178 the outermost multipart part of the message, if it does not. That
179 condition is indicated by a \*(lqbogus multipart content in
180 message\*(rq error message from
181 .B mhlist
182 and other
183 .B nmh
184 programs that parse MIME messages.
185 .PP
186 The
187 .B \-fixcte
188 switch enables a transformation to change the
189 Content-Transfer-Encoding from an invalid value to 8bit in message
190 parts with a Content-Type of multipart, as required by RFC 2045,
191 §6.4. That condition is indicated by a \*(lqmust be encoded in
192 7bit, 8bit, or binary\*(rq error message from
193 .B mhlist
194 and other
195 .B nmh
196 programs that parse MIME messages.
197 .PP
198 The
199 .B \-fixtype
200 switch ensures that each part of the message has the correct MIME type
201 shown in its Content-Type header. It may be repeated. It is
202 typically used to replace \*(lqapplication/octet-stream\*(rq with a
203 more descriptive MIME type. It may not be used for multipart and
204 message types.
205 .PP
206 .B mhfixmsg
207 applies one transformation unconditionally: it removes an extraneous
208 trailing semicolon from the parameter lists of MIME header fields.
209 .PP
210 The
211 .B \-verbose
212 switch directs
213 .B mhfixmsg
214 to output informational message for each transformation applied.
215 .PP
216 The return status of
217 .B mhfixmsg
218 is 0 if all of the requested transformations are performed, or
219 non-zero otherwise.
220 .RB ( mhfixmsg
221 will not decode to binary content, but a request to do so is
222 not considered a failure, and is noted with
223 .BR \-verbose .)
224 If a problem is detected with any one of multiple messages such that
225 the return status is non-zero, then none of the messages will be
226 modified.
227 .PP
228 The
229 .B \-file
230 .I file
231 switch directs
232 .B mhfixmsg
233 to use the specified
234 file as the source message, rather than a message from a folder.
235 Only one file argument may be provided. The
236 .B \-file
237 switch is implied if
238 .I file
239 is an absolute pathname.
240 If the file is \*(lq-\*(rq, then
241 .B mhfixmsg
242 accepts the source message on the standard input stream. If
243 the
244 .B \-outfile
245 switch is not enabled when using the standard input stream,
246 .B mhfixmsg
247 will not produce a transformed output message.
248 .PP
249 .BR mhfixmsg ,
250 by default, transforms the message in place. If the
251 .B \-outfile
252 switch is enabled, then
253 .B mhfixmsg
254 does not modify the input message or file, but instead places its
255 output in the specified file. An outfile name of \*(lq-\*(rq
256 specifies the standard output stream.
257 .PP
258 Combined with the
259 .B \-verbose
260 switch, the
261 .B \-outfile
262 switch can be used to show what transformations
263 .B mhfixmsg
264 would apply without actually applying them, e.g.,
265 .PP
266 .RS 5
267 mhfixmsg -outfile /dev/null -verbose
268 .RE
269 .PP
270 As always, this usage obeys any
271 .B mhfixmsg
272 switches in the user's profile.
273 .PP
274 .B \-outfile
275 can be combined with
276 .B rcvstore
277 to add a single transformed message to a different folder, e.g.,
278 .PP
279 .RS 5
280 mhfixmsg -outfile - | \\
281 .RS 0
282 %nmhlibexecdir%/rcvstore +folder
283 .RE
284 .RE
285 .SS Summary of Applicability
286 The transformations apply to the parts of a message depending on
287 content type and/or encoding as follows:
288 .PP
289 .RS 5
290 .nf
291 .ta \w'\-crlflinebreaks 'u
292 \-decodetext base64 and quoted-printable encoded text parts
293 \-decodetypes limits parts to which -decodetext applies
294 \-crlflinebreaks text parts
295 \-textcharset text/plain parts
296 \-reformat text parts that are not text/plain
297 \-fixboundary outermost multipart part
298 \-fixcte multipart part
299 \-fixtype all except multipart and message parts
300 .fi
301 .RE
302 .SS "Backup of Original Message/File"
303 If it applies any transformations to a message or file,
304 and the
305 .B \-outfile
306 switch is not used,
307 .B mhfixmsg
308 backs up the original the same way as
309 .BR rmm .
310 That is, it uses the
311 .I rmmproc
312 profile component, if present. If not present,
313 .B mhfixmsg
314 moves the original message to a backup file.
315 The
316 .B \-rmmproc
317 switch may be used to override this profile component. The
318 .B \-normmproc
319 switch disables the use of any
320 .I rmmproc
321 profile component and negates all prior
322 .B \-rmmproc
323 switches.
324 .SS "Integration with inc"
325 .B mhfixmsg
326 can be used as an add-hook, as described in %docdir%/README-HOOKS.
327 Note that add-hooks are called from all
328 .B nmh
329 programs that add a message to a folder, not just
330 .BR inc .
331 Alternatively, a simple shell alias or function can be used to
332 call
333 .B mhfixmsg
334 immediately after a successful invocation of
335 .BR inc .
336 One approach could be based on:
337 .PP
338 .RS 5
339 msgs=`inc -format '%(msg)'` && [ -n "$msgs" ] && scan $msgs && \
340 mhfixmsg -nochangecur $msgs
341 .RE
342 .PP
343 Another approach would rely on adding a sequence to Unseen-Sequence,
344 which
345 .B inc
346 sets with the newly incorporated messages. Those could then be
347 supplied to
348 .BR mhfixmsg .
349 An example is shown below.
350 .SS "Integration with procmail"
351 By way of example, here is an excerpt from a procmailrc file
352 that filters messages through
353 .B mhfixmsg
354 before storing them in the user's
355 .I nmh-workers
356 folder. It also stores the incoming message in the
357 .I Backups
358 folder in a filename generated by
359 .BR mkstemp ,
360 which is a non-POSIX utility to generate a temporary file.
361 Alternatively,
362 .B mhfixmsg
363 could be called on the message after it is stored.
364 .PP
365 .RS 5
366 .nf
367 .ta \w'\-fixboundary 'u
368 PATH = %bindir%:$PATH
369 MAILDIR = `mhparam path`
370 #### The Backups directory is relative to MAILDIR.
371 MKSTEMP = 'mkstemp -directory Backups -prefix mhfixmsg'
372 MHFIXMSG = 'mhfixmsg -noverbose -file - -outfile -'
373 STORE = %nmhlibexecdir%/rcvstore
374
375 :0 w: nmh-workers/procmail.$LOCKEXT
376 * ^TOnmh-workers@nongnu.org
377 | tee `$MKSTEMP` | $MHFIXMSG | $STORE +nmh-workers
378 .fi
379 .RE
380 .SH "EXAMPLES"
381 .PP
382 .SS Basic usage
383 To run
384 .B mhfixmsg
385 on the current message in the current folder, with default transformations to
386 fix MIME boundaries and Content-Transfer-Encoding, to decode text and
387 application/ics content parts to 8 bit, and to add a corresponding text/plain
388 part where lacking:
389 .PP
390 .RS
391 .nf
392 mhfixmsg -verbose
393 .fi
394 .RE
395 .SS Specified folder and messages
396 To run
397 .B mhfixmsg
398 on specified messages, without its informational output:
399 .PP
400 .RS
401 .nf
402 mhfixmsg +inbox last:4
403 .fi
404 .RE
405 .SS View without modification
406 By default,
407 .B mhfixmsg
408 transforms the message in place. To view the MIME structure that would result from running
409 .B mhfixmsg
410 on the current message, without modifying the message:
411 .PP
412 .RS
413 .nf
414 mhfixmsg -outfile - | mhlist -file -
415 .fi
416 .RE
417 .SS Search message without modification
418 To search the current message, which possibly contains base64 or quoted printable encoded text parts,
419 without modifiying it, use the -outfile switch:
420 .PP
421 .RS
422 .nf
423 mhfixmsg -outfile - | grep \fIpattern\fR
424 .fi
425 .RE
426 .PP
427 -outfile can be abbreviated in usual MH fashion, e.g., to -o. The search will be
428 on the entire message, not just text parts.
429 .SS Translate text/plain parts to UTF-8
430 To translate all text/plain parts in the current message to UTF-8, in addition
431 to all of the default transformations:
432 .PP
433 .RS
434 .nf
435 mhfixmsg -textcharset utf-8
436 .fi
437 .RE
438 .SS Fix all messages in a folder
439 To run
440 .B mhfixmsg
441 on all of the messages in a folder, using a Bourne shell loop:
442 .PP
443 .RS
444 .nf
445 for msg in `pick +folder`; do mhfixmsg $msg; done
446 .fi
447 .RE
448 .PP
449 .B mhfixmsg
450 can be run on more than one message, but on a large number of messages
451 may attempt to open too many files.
452 .SS Run on newly incorporated messages
453 To run
454 .B mhfixmsg
455 on messages as they are incorporated:
456 .PP
457 .RS
458 .nf
459 inc && mhfixmsg -nochangecur unseen
460 .fi
461 .RE
462 .PP
463 This assumes that the Unseen-Sequence profile entry is set to
464 .BR unseen ,
465 as shown in the mh\-profile(5) man page.
466 .SH FILES
467 .B mhfixmsg
468 looks for mhn.defaults in multiple locations: absolute pathnames are
469 accessed directly, tilde expansion is done on usernames, and files are
470 searched for in the user's
471 .I Mail
472 directory as specified in their profile. If not found there, the directory
473 .RI \*(lq %nmhetcdir% \*(rq
474 is checked.
475 .PP
476 .fc ^ ~
477 .nf
478 .ta \w'%nmhetcdir%/mhn.defaults 'u
479 ^$HOME/\&.mh\(ruprofile~^The user profile
480 ^%nmhetcdir%/mhn.defaults~^Default mhfixmsg conversion entries
481 .fi
482 .SH "PROFILE COMPONENTS"
483 .fc ^ ~
484 .nf
485 .ta 2.4i
486 .ta \w'ExtraBigProfileName 'u
487 ^Path:~^To determine the user's nmh directory
488 ^Current\-Folder:~^To find the default current folder
489 ^rmmproc:~^Program to delete original messages or files
490 .fi
491 .SH "SEE ALSO"
492 .IR inc (1),
493 .IR iconv (3),
494 .IR mh-profile (5),
495 .IR mhbuild (1),
496 .IR mhlist (1),
497 .IR mhparam (1),
498 .IR mhshow (1),
499 .IR mh-mkstemp (1),
500 .IR procmail (1),
501 .IR procmailrc (5),
502 .IR rcvstore (1),
503 .IR rmm (1)
504 .SH DEFAULTS
505 .nf
506 .RB ` +folder "' defaults to the current folder"
507 .RB ` msgs "' defaults to cur"
508 .RB ` "\-decodetext 8bit"'
509 .RB ` "\-decodetypes text,application/ics"'
510 .RB ` \-crlflinebreaks '
511 .RB ` \-notextcharset '
512 .RB ` \-reformat '
513 .RB ` \-noreplacetextplain '
514 .RB ` \-fixboundary '
515 .RB ` \-fixcte '
516 .RB ` \-changecur '
517 .RB ` \-noverbose '
518 .fi
519 .SH CONTEXT
520 If a folder is given, it will become the current folder. The last
521 message selected from a folder will become the current message, unless
522 the
523 .B \-nochangecur
524 switch is enabled. If the
525 .B \-file
526 switch or an absolute pathname is used, the context will not be
527 modified.
528 .SH BUGS
529 .B mhfixmsg
530 opens files internally for decoding and character set conversion, and apparently does not
531 close them expeditiously. Until that is resolved, it is recommended that
532 .B mhfixmsg
533 not be run on a large number of messages at once, as noted in the EXAMPLES above.
534 .PP
535 As noted in the DESCRIPTION above,
536 .B mhfixmsg
537 will not decode to binary content. This restriction should be removed at some point. It's
538 not due to any issue in
539 .BR mhfixmsg ,
540 but rather an observation of incorrect behavior by other nmh tools on messages with binary
541 content.