]> diplodocus.org Git - nmh/blob - man/mhfixmsg.man
Move things around so you can still initialize netsec with SASL
[nmh] / man / mhfixmsg.man
1 .TH MHFIXMSG %manext1% "September 22, 2016" "%nmhversion%"
2 .\"
3 .\" %nmhwarning%
4 .\"
5 .SH NAME
6 mhfixmsg \- rewrite MIME messages with various transformations
7 .SH SYNOPSIS
8 .HP 5
9 .na
10 .B mhfixmsg
11 .RI [ +folder ]
12 .RI [ msgs " | "
13 .IR "absolute pathname" " | "
14 .RB \-file
15 .IR file ]
16 .RB [ \-decodetext
17 8bit|7bit|binary |
18 .BR \-nodecodetext ]
19 .RB [ \-decodetypes
20 .IR "type/[subtype][,...]" ]
21 .RB [ \-crlflinebreaks " | " \-nocrlflinebreaks ]
22 .RB [ \-textcharset
23 .I charset
24 .RB "| " \-notextcharset ]
25 .RB [ \-reformat " | " \-noreformat ]
26 .RB [ \-replacetextplain " | " \-noreplacetextplain ]
27 .RB [ \-fixboundary " | " \-nofixboundary ]
28 .RB [ \-fixcte " | " \-nofixcte ]
29 .RB [ \-fixtype
30 .IR mimetype ]
31 .RB [ \-outfile
32 .IR outfile ]
33 .RB [ \-rmmproc
34 .IR program ]
35 .RB [ \-normmproc ]
36 .RB [ \-changecur " | " \-nochangecur ]
37 .RB [ \-verbose " | " \-noverbose ]
38 .RB [ \-version ]
39 .RB [ \-help ]
40 .ad
41 .SH DESCRIPTION
42 .B mhfixmsg
43 rewrites MIME messages, applying specific transformations such as
44 decoding of MIME-encoded message parts and repairing invalid MIME
45 headers.
46 .PP
47 MIME messages are specified in RFC 2045 to RFC 2049
48 (see
49 .IR mhbuild (1)).
50 The
51 .B mhlist
52 command is invaluable for viewing the content structure of MIME
53 messages.
54 .B mhfixmsg
55 passes non-MIME messages through without any transformations. If no
56 transformations apply to a MIME message, the original message or file
57 is not modified or removed. Thus,
58 .B mhfixmsg
59 can safely be run multiple times on a message.
60 .PP
61 The
62 .B \-decodetext
63 switch enables a transformation to decode each base64 and
64 quoted-printable text message part to the selected 8bit, 7bit, or
65 binary encoding.
66 If 7bit is selected for a base64 part but it will only fit
67 8bit, as defined by RFC 2045, then it will be decoded to 8bit
68 quoted-printable.
69 Similarly, with 8bit, if the decoded text would be binary,
70 then the part is not decoded (and a message will be
71 displayed if
72 .B \-verbose
73 is enabled). Note that
74 .B \-decodetext
75 binary can produce messages that are not RFC 2045 compliant.
76 .PP
77 When the
78 .B \-decodetext
79 switch is enabled, each carriage return character that precedes a
80 linefeed character is removed from text parts encoded in ASCII,
81 ISO-8859-x, UTF-8, or Windows-12xx.
82 .PP
83 The
84 .B \-decodetypes
85 switch specifies the message parts, by type and optionally subtype,
86 to which
87 .B \-decodetext
88 applies. Its argument is a comma-separated list of type/subtype
89 elements. If an element does not contain a subtype, then
90 .B \-decodetext
91 applies to all subtypes of the type. The default is
92 .B \-decodetypes
93 .IR text ;
94 it can be overridden, e.g., with
95 .B \-decodetypes
96 .I text/plain
97 to restrict
98 .B \-decodetext
99 to just text/plain parts.
100 .PP
101 By default, carriage return characters are preserved or inserted at
102 the end of each line of text content. The
103 .B \-crlflinebreaks
104 switch selects this behavior and is enabled by default. The
105 .B \-nocrlflinebreaks
106 switch causes carriage return characters to be stripped from, and not
107 inserted in, text content when it is decoded and encoded. Note that
108 its use can cause the generation of MIME messages that do not conform
109 with RFC 2046, §4.1.1, paragraph 1.
110 .PP
111 The
112 .B \-textcharset
113 switch specifies that all text/plain parts of the message(s)
114 should be converted to
115 .IR charset .
116 Charset conversions require that
117 .B nmh
118 be built with
119 .IR iconv (3);
120 see the
121 .BR mhparam (1)
122 man page for how determine whether your
123 .B nmh
124 installation includes that.
125 To convert text parts other than text/plain, an external program can
126 be used, via the
127 .B \-reformat
128 switch.
129 .PP
130 The
131 .B \-reformat
132 switch enables a transformation for text parts in the message. For
133 each text part that is not text/plain and that does not have a
134 corresponding text/plain in a multipart/alternative part,
135 .B mhfixmsg
136 looks for a mhfixmsg-format-text/subtype profile entry that matches
137 the subtype of the part. If one is found and can be used to
138 successfully convert the part to text/plain,
139 .B mhfixmsg
140 inserts that text/plain part at the beginning of the containing
141 multipart/alternative part, if present. If not, it creates a
142 multipart/alternative part.
143 .PP
144 With the
145 .B \-reformat
146 switch, multipart/related parts are handled differently than
147 multipart/alternative. If the multipart/related has only a single
148 part that is not text/plain and can be converted to text/plain, a
149 text/plain part is added and the type of the part is changed to
150 multipart/alternative. If the multipart/related has more than one
151 part but does not have a text/plain part,
152 .B mhfixmsg
153 tries to add one.
154 .PP
155 The
156 .B \-replacetextplain
157 switch broadens the applicability of
158 .B \-reformat
159 by always replacing a corresponding text/plain part, if one exists.
160 If
161 .B \-verbose
162 if enabled, the replacement will be shown as two steps: a removal of
163 the text/plain part followed by the usual insertion of a new part.
164 .PP
165 .B \-reformat
166 requires a profile entry for each text part subtype to be reformatted.
167 The mhfixmsg-format-text/subtype profile entries are based on external
168 conversion programs, and are used the same way that
169 .B mhshow
170 uses its mhshow-show-text/subtype entries. When
171 .B nmh
172 is installed, it searches for a conversion program for text/html
173 content, and if one is found, inserts a mhfixmsg-format-text/html
174 entry in %nmhetcdir%/mhn.defaults. An entry of the same name in the
175 user's profile takes precedence. The user can add entries for
176 other text subtypes to their profile.
177 .PP
178 The
179 .B \-fixboundary
180 switch enables a transformation to repair the boundary portion of the
181 Content-Type header field of the message to match the boundaries of
182 the outermost multipart part of the message, if it does not. That
183 condition is indicated by a \*(lqbogus multipart content in
184 message\*(rq error message from
185 .B mhlist
186 and other
187 .B nmh
188 programs that parse MIME messages.
189 .PP
190 The
191 .B \-fixcte
192 switch enables a transformation to change the
193 Content-Transfer-Encoding from an invalid value to 8bit in message
194 parts with a Content-Type of multipart, as required by RFC 2045,
195 §6.4. That condition is indicated by a \*(lqmust be encoded in
196 7bit, 8bit, or binary\*(rq error message from
197 .B mhlist
198 and other
199 .B nmh
200 programs that parse MIME messages.
201 .PP
202 The
203 .B \-fixtype
204 switch ensures that each part of the message has the correct MIME type
205 shown in its Content-Type header. It may be repeated. It is
206 typically used to replace \*(lqapplication/octet-stream\*(rq with a
207 more descriptive MIME type. It may not be used for multipart and
208 message types.
209 .PP
210 .B mhfixmsg
211 applies one transformation unconditionally: it removes an extraneous
212 trailing semicolon from the parameter lists of MIME header fields.
213 .PP
214 The
215 .B \-verbose
216 switch directs
217 .B mhfixmsg
218 to output informational message for each transformation applied.
219 .PP
220 The return status of
221 .B mhfixmsg
222 is 0 if all of the requested transformations are performed, or
223 non-zero otherwise.
224 .RB ( mhfixmsg
225 will not decode to binary content with the default
226 .B \-decodetext
227 setting, but a request to do so is not considered a failure, and is noted
228 with
229 .BR \-verbose .)
230 If a problem is detected with any one of multiple messages such that
231 the return status is non-zero, then none of the messages will be
232 modified.
233 .PP
234 The
235 .B \-file
236 .I file
237 switch directs
238 .B mhfixmsg
239 to use the specified
240 file as the source message, rather than a message from a folder.
241 Only one file argument may be provided. The
242 .B \-file
243 switch is implied if
244 .I file
245 is an absolute pathname.
246 If the file is \*(lq-\*(rq, then
247 .B mhfixmsg
248 accepts the source message on the standard input stream. If
249 the
250 .B \-outfile
251 switch is not enabled when using the standard input stream,
252 .B mhfixmsg
253 will not produce a transformed output message.
254 .PP
255 .BR mhfixmsg ,
256 by default, transforms the message in place. If the
257 .B \-outfile
258 switch is enabled, then
259 .B mhfixmsg
260 does not modify the input message or file, but instead places its
261 output in the specified file. An outfile name of \*(lq-\*(rq
262 specifies the standard output stream.
263 .PP
264 Combined with the
265 .B \-verbose
266 switch, the
267 .B \-outfile
268 switch can be used to show what transformations
269 .B mhfixmsg
270 would apply without actually applying them, e.g.,
271 .PP
272 .RS 5
273 mhfixmsg -outfile /dev/null -verbose
274 .RE
275 .PP
276 As always, this usage obeys any
277 .B mhfixmsg
278 switches in the user's profile.
279 .PP
280 .B \-outfile
281 can be combined with
282 .B rcvstore
283 to add a single transformed message to a different folder, e.g.,
284 .PP
285 .RS 5
286 mhfixmsg -outfile - | \\
287 .RS 0
288 %nmhlibexecdir%/rcvstore +folder
289 .RE
290 .RE
291 .SS Summary of Applicability
292 The transformations apply to the parts of a message depending on
293 content type and/or encoding as follows:
294 .PP
295 .RS 5
296 .nf
297 .ta \w'\-crlflinebreaks 'u
298 \-decodetext base64 and quoted-printable encoded text parts
299 \-decodetypes limits parts to which -decodetext applies
300 \-crlflinebreaks text parts
301 \-textcharset text/plain parts
302 \-reformat text parts that are not text/plain
303 \-fixboundary outermost multipart part
304 \-fixcte multipart part
305 \-fixtype all except multipart and message parts
306 .fi
307 .RE
308 .SS "Backup of Original Message/File"
309 If it applies any transformations to a message or file,
310 and the
311 .B \-outfile
312 switch is not used,
313 .B mhfixmsg
314 backs up the original the same way as
315 .BR rmm .
316 That is, it uses the
317 .I rmmproc
318 profile component, if present. If not present,
319 .B mhfixmsg
320 moves the original message to a backup file.
321 The
322 .B \-rmmproc
323 switch may be used to override this profile component. The
324 .B \-normmproc
325 switch disables the use of any
326 .I rmmproc
327 profile component and negates all prior
328 .B \-rmmproc
329 switches.
330 .SS "Integration with inc"
331 .B mhfixmsg
332 can be used as an add-hook, as described in %docdir%/README-HOOKS.
333 Note that add-hooks are called from all
334 .B nmh
335 programs that add a message to a folder, not just
336 .BR inc .
337 Alternatively, a simple shell alias or function can be used to
338 call
339 .B mhfixmsg
340 immediately after a successful invocation of
341 .BR inc .
342 One approach could be based on:
343 .PP
344 .RS 5
345 msgs=`inc -format '%(msg)'` && [ -n "$msgs" ] && scan $msgs && \
346 mhfixmsg -nochangecur $msgs
347 .RE
348 .PP
349 Another approach would rely on adding a sequence to Unseen-Sequence,
350 which
351 .B inc
352 sets with the newly incorporated messages. Those could then be
353 supplied to
354 .BR mhfixmsg .
355 An example is shown below.
356 .SS "Integration with procmail"
357 By way of example, here is an excerpt from a procmailrc file
358 that filters messages through
359 .B mhfixmsg
360 before storing them in the user's
361 .I nmh-workers
362 folder. It also stores the incoming message in the
363 .I Backups
364 folder in a filename generated by
365 .BR mkstemp ,
366 which is a non-POSIX utility to generate a temporary file.
367 Alternatively,
368 .B mhfixmsg
369 could be called on the message after it is stored.
370 .PP
371 .RS 5
372 .nf
373 .ta \w'\-fixboundary 'u
374 PATH = %bindir%:$PATH
375 MAILDIR = `mhparam path`
376 #### The Backups directory is relative to MAILDIR.
377 MKSTEMP = 'mkstemp -directory Backups -prefix mhfixmsg'
378 MHFIXMSG = 'mhfixmsg -noverbose -file - -outfile -'
379 STORE = %nmhlibexecdir%/rcvstore
380
381 :0 w: nmh-workers/procmail.$LOCKEXT
382 * ^TOnmh-workers@nongnu.org
383 | tee `$MKSTEMP` | $MHFIXMSG | $STORE +nmh-workers
384 .fi
385 .RE
386 .SH "EXAMPLES"
387 .PP
388 .SS Basic usage
389 To run
390 .B mhfixmsg
391 on the current message in the current folder, with default transformations to
392 fix MIME boundaries and Content-Transfer-Encoding, to decode text and
393 application/ics content parts to 8 bit, and to add a corresponding text/plain
394 part where lacking:
395 .PP
396 .RS
397 .nf
398 mhfixmsg -verbose
399 .fi
400 .RE
401 .SS Specified folder and messages
402 To run
403 .B mhfixmsg
404 on specified messages, without its informational output:
405 .PP
406 .RS
407 .nf
408 mhfixmsg +inbox last:4
409 .fi
410 .RE
411 .SS View without modification
412 By default,
413 .B mhfixmsg
414 transforms the message in place. To view the MIME structure that would result from running
415 .B mhfixmsg
416 on the current message, without modifying the message:
417 .PP
418 .RS
419 .nf
420 mhfixmsg -outfile - | mhlist -file -
421 .fi
422 .RE
423 .SS Search message without modification
424 To search the current message, which possibly contains base64 or quoted printable encoded text parts,
425 without modifiying it, use the -outfile switch:
426 .PP
427 .RS
428 .nf
429 mhfixmsg -outfile - | grep \fIpattern\fR
430 .fi
431 .RE
432 .PP
433 -outfile can be abbreviated in usual MH fashion, e.g., to -o. The search will be
434 on the entire message, not just text parts.
435 .SS Translate text/plain parts to UTF-8
436 To translate all text/plain parts in the current message to UTF-8, in addition
437 to all of the default transformations:
438 .PP
439 .RS
440 .nf
441 mhfixmsg -textcharset utf-8
442 .fi
443 .RE
444 .SS Fix all messages in a folder
445 To run
446 .B mhfixmsg
447 on all of the messages in a folder, using a Bourne shell loop:
448 .PP
449 .RS
450 .nf
451 for msg in `pick +folder`; do mhfixmsg $msg; done
452 .fi
453 .RE
454 .PP
455 .B mhfixmsg
456 can be run on more than one message, but on a large number of messages
457 may attempt to open too many files.
458 .SS Run on newly incorporated messages
459 To run
460 .B mhfixmsg
461 on messages as they are incorporated:
462 .PP
463 .RS
464 .nf
465 inc && mhfixmsg -nochangecur unseen
466 .fi
467 .RE
468 .PP
469 This assumes that the Unseen-Sequence profile entry is set to
470 .BR unseen ,
471 as shown in the mh\-profile(5) man page.
472 .SH FILES
473 .B mhfixmsg
474 looks for mhn.defaults in multiple locations: absolute pathnames are
475 accessed directly, tilde expansion is done on usernames, and files are
476 searched for in the user's
477 .I Mail
478 directory as specified in their profile. If not found there, the directory
479 .RI \*(lq %nmhetcdir% \*(rq
480 is checked.
481 .PP
482 .fc ^ ~
483 .nf
484 .ta \w'%nmhetcdir%/mhn.defaults 'u
485 ^$HOME/\&.mh\(ruprofile~^The user profile
486 ^%nmhetcdir%/mhn.defaults~^Default mhfixmsg conversion entries
487 .fi
488 .SH "PROFILE COMPONENTS"
489 .fc ^ ~
490 .nf
491 .ta 2.4i
492 .ta \w'ExtraBigProfileName 'u
493 ^Path:~^To determine the user's nmh directory
494 ^Current\-Folder:~^To find the default current folder
495 ^rmmproc:~^Program to delete original messages or files
496 .fi
497 .SH "SEE ALSO"
498 .IR inc (1),
499 .IR iconv (3),
500 .IR mh-profile (5),
501 .IR mhbuild (1),
502 .IR mhlist (1),
503 .IR mhparam (1),
504 .IR mhshow (1),
505 .IR mh-mkstemp (1),
506 .IR procmail (1),
507 .IR procmailrc (5),
508 .IR rcvstore (1),
509 .IR rmm (1)
510 .SH DEFAULTS
511 .nf
512 .RB ` +folder "' defaults to the current folder"
513 .RB ` msgs "' defaults to cur"
514 .RB ` "\-decodetext 8bit"'
515 .RB ` "\-decodetypes text,application/ics"'
516 .RB ` \-crlflinebreaks '
517 .RB ` \-notextcharset '
518 .RB ` \-reformat '
519 .RB ` \-noreplacetextplain '
520 .RB ` \-fixboundary '
521 .RB ` \-fixcte '
522 .RB ` \-changecur '
523 .RB ` \-noverbose '
524 .fi
525 .SH CONTEXT
526 If a folder is given, it will become the current folder. The last
527 message selected from a folder will become the current message, unless
528 the
529 .B \-nochangecur
530 switch is enabled. If the
531 .B \-file
532 switch or an absolute pathname is used, the context will not be
533 modified.
534 .SH BUGS
535 .B mhfixmsg
536 opens files internally for decoding and character set conversion, and apparently does not
537 close them expeditiously. Until that is resolved, it is recommended that
538 .B mhfixmsg
539 not be run on a large number of messages at once, as noted in the EXAMPLES above.