Alright, things compile now, but don't link yet; need to write the

[nmh] / man / mhfixmsg.man

diff --git a/man/mhfixmsg.man b/man/mhfixmsg.man
index 68f0687194f1991ad790ac3806354e469f547f36..eb9595de9547912fb4c7fa49162b4b465abe0112 100644 (file)
--- a/man/mhfixmsg.man
+++ b/man/mhfixmsg.man
@@ -1,4 +1,4 @@
-.TH MHFIXMSG %manext1% "March 21, 2013" "%nmhversion%"
+.TH MHFIXMSG %manext1% "March 12, 2016" "%nmhversion%"

 .\"
 .\" %nmhwarning%
 .\"
 .\"
 .\" %nmhwarning%
 .\"


@@ -9,23 +9,31 @@ mhfixmsg \- rewrite MIME messages with various transformations
 .na
 .B mhfixmsg
 .RI [ +folder ]
 .na
 .B mhfixmsg
 .RI [ +folder ]

-.RI [ msgs ]
+.RI [ msgs " | "
+.IR "absolute pathname" " | "
+.RB \-file
+.IR file ]

 .RB [ \-decodetext
 8bit/7bit |
 .BR \-nodecodetext ]
 .RB [ \-decodetext
 8bit/7bit |
 .BR \-nodecodetext ]

-.RB [ \-textcodeset
-.I codeset
-.RB "| " \-notextcodeset ]
+.RB [ \-decodetypes
+.IR "type/[subtype][,...]" ]
+.RB [ \-crlflinebreaks " | " \-nocrlflinebreaks ]
+.RB [ \-textcharset
+.I charset
+.RB "| " \-notextcharset ]

 .RB [ \-reformat " | " \-noreformat ]
 .RB [ \-reformat " | " \-noreformat ]

+.RB [ \-replacetextplain " | " \-noreplacetextplain ]

 .RB [ \-fixboundary " | " \-nofixboundary ]
 .RB [ \-fixcte " | " \-nofixcte ]
 .RB [ \-fixboundary " | " \-nofixboundary ]
 .RB [ \-fixcte " | " \-nofixcte ]

-.RB [ \-file
-.IR file ]
+.RB [ \-fixtype
+.IR mimetype ]

 .RB [ \-outfile
 .IR outfile ]
 .RB [ \-rmmproc
 .IR program ]
 .RB [ \-normmproc ]
 .RB [ \-outfile
 .IR outfile ]
 .RB [ \-rmmproc
 .IR program ]
 .RB [ \-normmproc ]

+.RB [ \-changecur " | " \-nochangecur ]

 .RB [ \-verbose " | " \-noverbose ]
 .RB [ \-version ]
 .RB [ \-help ]
 .RB [ \-verbose " | " \-noverbose ]
 .RB [ \-version ]
 .RB [ \-help ]


@@ -36,7 +44,7 @@ rewrites MIME messages, applying specific transformations such as
 decoding of MIME-encoded message parts and repairing invalid MIME
 headers.
 .PP
 decoding of MIME-encoded message parts and repairing invalid MIME
 headers.
 .PP

-MIME messages are specified in RFC\-2045 to RFC\-2049
+MIME messages are specified in RFC 2045 to RFC 2049

 (see
 .IR mhbuild (1)).
 The
 (see
 .IR mhbuild (1)).
 The


@@ -46,29 +54,70 @@ messages.
 .B mhfixmsg
 passes non-MIME messages through without any transformations.  If no
 transformations apply to a MIME message, the original message or file
 .B mhfixmsg
 passes non-MIME messages through without any transformations.  If no
 transformations apply to a MIME message, the original message or file

-is not modified or removed.
+is not modified or removed.  Thus,
+.B mhfixmsg
+can safely be run multiple times on a message.

 .PP
 The
 .B \-decodetext
 switch enables a transformation to decode each base64 and
 quoted-printable text message part to the selected 8bit or 7bit
 encoding.  If 7bit is selected for a base64 part but it will only fit
 .PP
 The
 .B \-decodetext
 switch enables a transformation to decode each base64 and
 quoted-printable text message part to the selected 8bit or 7bit
 encoding.  If 7bit is selected for a base64 part but it will only fit

-8bit, as defined by RFC-2045, then it will be decoded to 8bit
+8bit, as defined by RFC 2045, then it will be decoded to 8bit

 quoted-printable.  Otherwise, if the decoded text would not fit the
 selected encoding, the part is not decoded (and a message will be
 displayed if
 .B \-verbose
 is enabled).
 .PP
 quoted-printable.  Otherwise, if the decoded text would not fit the
 selected encoding, the part is not decoded (and a message will be
 displayed if
 .B \-verbose
 is enabled).
 .PP

+When the
+.B \-decodetext
+switch is enabled, each carriage return character that precedes a
+linefeed character is removed from text parts encoded in ASCII,
+ISO-8859-x, UTF-8, or Windows-12xx.
+.PP
+The
+.B \-decodetypes
+switch specifies the message parts, by type and optionally subtype,
+to which
+.B \-decodetext
+applies.  Its argument is a comma-separated list of type/subtype
+elements.  If an element does not contain a subtype, then
+.B \-decodetext
+applies to all subtypes of the type.  The default is
+.B \-decodetypes
+.IR text ;
+it can be overridden, e.g., with
+.B \-decodetypes
+.I text/plain
+to restrict
+.B \-decodetext
+to just text/plain parts.
+.PP
+By default, carriage return characters are preserved or inserted at
+the end of each line of text content.  The
+.B \-crlflinebreaks
+switch selects this behavior and is enabled by default.  The
+.B \-nocrlflinebreaks
+switch causes carriage return characters to be stripped from, and not
+inserted in, text content when it is decoded and encoded.  Note that
+its use can cause the generation of MIME messages that do not conform
+with RFC 2046, §4.1.1, paragraph 1.
+.PP

 The
 The

-.B \-textcodeset
+.B \-textcharset

 switch specifies that all text/plain parts of the message(s)
 should be converted to
 switch specifies that all text/plain parts of the message(s)
 should be converted to

-.IR codeset .
-Codeset conversions require that
+.IR charset .
+Charset conversions require that

 .B nmh
 be built with
 .B nmh
 be built with

-.IR iconv (3).
+.IR iconv (3);
+see the
+.BR mhparam (1)
+man page for how determine whether your
+.B nmh
+installation includes that.

 To convert text parts other than text/plain, an external program can
 be used, via the
 .B \-reformat
 To convert text parts other than text/plain, an external program can
 be used, via the
 .B \-reformat


@@ -88,6 +137,27 @@ inserts that text/plain part at the beginning of the containing
 multipart/alternative part, if present.  If not, it creates a
 multipart/alternative part.
 .PP
 multipart/alternative part, if present.  If not, it creates a
 multipart/alternative part.
 .PP

+With the
+.B \-reformat
+switch, multipart/related parts are handled differently than
+multipart/alternative.  If the multipart/related has only a single
+part that is not text/plain and can be converted to text/plain, a
+text/plain part is added and the type of the part is changed to
+multipart/alternative.  If the multipart/related has more than one
+part but does not have a text/plain part,
+.B mhfixmsg
+tries to add one.
+.PP
+The
+.B \-replacetextplain
+switch broadens the applicability of
+.B \-reformat
+by always replacing a corresponding text/plain part, if one exists.
+If
+.B \-verbose
+if enabled, the replacement will be shown as two steps:  a removal of
+the text/plain part followed by the usual insertion of a new part.
+.PP

 .B \-reformat
 requires a profile entry for each text part subtype to be reformatted.
 The mhfixmsg-format-text/subtype profile entries are based on external
 .B \-reformat
 requires a profile entry for each text part subtype to be reformatted.
 The mhfixmsg-format-text/subtype profile entries are based on external


@@ -97,7 +167,7 @@ uses its mhshow-show-text/subtype entries.  When
 .B nmh
 is installed, it searches for a conversion program for text/html
 content, and if one is found, inserts a mhfixmsg-format-text/html
 .B nmh
 is installed, it searches for a conversion program for text/html
 content, and if one is found, inserts a mhfixmsg-format-text/html

-entry in %etcdir%/mhn.defaults.  An entry of the same name in the
+entry in %nmhetcdir%/mhn.defaults.  An entry of the same name in the

 user's profile takes precedence.  The user can add entries for
 other text subtypes to their profile.
 .PP
 user's profile takes precedence.  The user can add entries for
 other text subtypes to their profile.
 .PP


@@ -118,7 +188,7 @@ The
 switch enables a transformation to change the
 Content-Transfer-Encoding from an invalid value to 8bit in message
 parts with a Content-Type of multipart, as required by RFC 2045,
 switch enables a transformation to change the
 Content-Transfer-Encoding from an invalid value to 8bit in message
 parts with a Content-Type of multipart, as required by RFC 2045,

-Section 6.4.  That condition is indicated by a \*(lqmust be encoded in
+§6.4.  That condition is indicated by a \*(lqmust be encoded in

 7bit, 8bit, or binary\*(rq error message from
 .B mhlist
 and other
 7bit, 8bit, or binary\*(rq error message from
 .B mhlist
 and other


@@ -126,11 +196,35 @@ and other
 programs that parse MIME messages.
 .PP
 The
 programs that parse MIME messages.
 .PP
 The

+.B \-fixtype
+switch ensures that each part of the message has the correct MIME type
+shown in its Content-Type header.  It may be repeated.  It is
+typically used to replace \*(lqapplication/octet-stream\*(rq with a
+more descriptive MIME type.  It may not be used for multipart and
+message types.
+.PP
+.B mhfixmsg
+applies one transformation unconditionally:  it removes an extraneous
+trailing semicolon from the parameter lists of MIME header fields.
+.PP
+The

 .B \-verbose
 switch directs
 .B mhfixmsg
 to output informational message for each transformation applied.
 .PP
 .B \-verbose
 switch directs
 .B mhfixmsg
 to output informational message for each transformation applied.
 .PP

+The return status of
+.B mhfixmsg
+is 0 if all of the requested transformations are performed, or
+non-zero otherwise.
+.RB ( mhfixmsg
+will not decode to binary content, but a request to do so is
+not considered a failure, and is noted with
+.BR \-verbose .)
+If a problem is detected with any one of multiple messages such that
+the return status is non-zero, then none of the messages will be
+modified.
+.PP

 The
 .B \-file
 .I file
 The
 .B \-file
 .I file


@@ -138,7 +232,12 @@ switch directs
 .B mhfixmsg
 to use the specified
 file as the source message, rather than a message from a folder.
 .B mhfixmsg
 to use the specified
 file as the source message, rather than a message from a folder.

-If this file is \*(lq-\*(rq, then
+Only one file argument may be provided.  The
+.B \-file
+switch is implied if
+.I file
+is an absolute pathname.
+If the file is \*(lq-\*(rq, then

 .B mhfixmsg
 accepts the source message on the standard input stream.  If
 the
 .B mhfixmsg
 accepts the source message on the standard input stream.  If
 the


@@ -180,7 +279,7 @@ to add a single transformed message to a different folder, e.g.,
 .RS 5
 mhfixmsg -outfile - | \\
 .RS 0
 .RS 5
 mhfixmsg -outfile - | \\
 .RS 0

-%libdir%/rcvstore +folder
+%nmhlibexecdir%/rcvstore +folder

 .RE
 .RE
 .SS Summary of Applicability
 .RE
 .RE
 .SS Summary of Applicability


@@ -189,15 +288,17 @@ content type and/or encoding as follows:
 .PP
 .RS 5
 .nf
 .PP
 .RS 5
 .nf

-.ta \w'\-fixboundary 'u
-\-decodetext   base64 and quoted-printable encoded text parts
-\-textcodeset  text/plain parts
-\-reformat     text parts that are not text/plain
-\-fixboundary  outermost multipart part
-\-fixcte       multipart part
+.ta \w'\-crlflinebreaks 'u
+\-decodetext         base64 and quoted-printable encoded text parts
+\-decodetypes        limits parts to which -decodetext applies
+\-crlflinebreaks     text parts
+\-textcharset        text/plain parts
+\-reformat           text parts that are not text/plain
+\-fixboundary        outermost multipart part
+\-fixcte             multipart part
+\-fixtype            all except multipart and message parts

 .fi
 .RE
 .fi
 .RE

-.PP

 .SS "Backup of Original Message/File"
 If it applies any transformations to a message or file,
 and the
 .SS "Backup of Original Message/File"
 If it applies any transformations to a message or file,
 and the


@@ -220,7 +321,32 @@ switch disables the use of any
 profile component and negates all prior
 .B \-rmmproc
 switches.
 profile component and negates all prior
 .B \-rmmproc
 switches.

+.SS "Integration with inc"
+.B mhfixmsg
+can be used as an add-hook, as described in %docdir%/README-HOOKS.
+Note that add-hooks are called from all
+.B nmh
+programs that add a message to a folder, not just
+.BR inc .
+Alternatively, a simple shell alias or function can be used to
+call
+.B mhfixmsg
+immediately after a successful invocation of
+.BR inc .
+One approach could be based on:
+.PP
+.RS 5
+msgs=`inc -format '%(msg)'`  &&  [ -n "$msgs" ]  &&  scan $msgs  &&  \
+mhfixmsg -nochangecur $msgs
+.RE

 .PP
 .PP

+Another approach would rely on adding a sequence to Unseen-Sequence,
+which
+.B inc
+sets with the newly incorporated messages.  Those could then be
+supplied to
+.BR mhfixmsg .
+An example is shown below.

 .SS "Integration with procmail"
 By way of example, here is an excerpt from a procmailrc file
 that filters messages through
 .SS "Integration with procmail"
 By way of example, here is an excerpt from a procmailrc file
 that filters messages through


@@ -230,11 +356,9 @@ before storing them in the user's
 folder.  It also stores the incoming message in the
 .I Backups
 folder in a filename generated by
 folder.  It also stores the incoming message in the
 .I Backups
 folder in a filename generated by

-.BR mktemp ,
+.BR mkstemp ,

 which is a non-POSIX utility to generate a temporary file.
 which is a non-POSIX utility to generate a temporary file.

-If you do not have that utility, then the
-.BR mkstemp (3)
-function could form the basis for a substitute.  Or,
+Alternatively,

 .B mhfixmsg
 could be called on the message after it is stored.
 .PP
 .B mhfixmsg
 could be called on the message after it is stored.
 .PP


@@ -243,16 +367,102 @@ could be called on the message after it is stored.
 .ta \w'\-fixboundary 'u
 PATH = %bindir%:$PATH
 MAILDIR = `mhparam path`
 .ta \w'\-fixboundary 'u
 PATH = %bindir%:$PATH
 MAILDIR = `mhparam path`

-MKTEMP = 'mktemp Backups/mhfixmsg.XXXXXXXX'
+#### The Backups directory is relative to MAILDIR.
+MKSTEMP = 'mkstemp -directory Backups -prefix mhfixmsg'

 MHFIXMSG = 'mhfixmsg -noverbose -file - -outfile -'
 MHFIXMSG = 'mhfixmsg -noverbose -file - -outfile -'

-STORE = %libdir%/rcvstore
+STORE = %nmhlibexecdir%/rcvstore

 
 

-:0 w: nmh-worker/procmail.$LOCKEXT
+:0 w: nmh-workers/procmail.$LOCKEXT

 * ^TOnmh-workers@nongnu.org
 * ^TOnmh-workers@nongnu.org

-| tee `$MKTEMP` | $MHFIXMSG | $STORE +nmh-workers
+| tee `$MKSTEMP` | $MHFIXMSG | $STORE +nmh-workers
+.fi
+.RE
+.SH "EXAMPLES"
+.PP
+.SS Basic usage
+To run
+.B mhfixmsg
+on the current message in the current folder, with default transformations to
+fix MIME boundaries and Content-Transfer-Encoding, to decode text and
+application/ics content parts to 8 bit, and to add a corresponding text/plain
+part where lacking:
+.PP
+.RS
+.nf
+mhfixmsg -verbose
+.fi
+.RE
+.SS Specified folder and messages
+To run
+.B mhfixmsg
+on specified messages, without its informational output:
+.PP
+.RS
+.nf
+mhfixmsg +inbox last:4
+.fi
+.RE
+.SS View without modification
+By default,
+.B mhfixmsg
+transforms the message in place.  To view the MIME structure that would result from running
+.B mhfixmsg
+on the current message, without modifying the message:
+.PP
+.RS
+.nf
+mhfixmsg -outfile - | mhlist -file -
+.fi
+.RE
+.SS Search message without modification
+To search the current message, which possibly contains base64 or quoted printable encoded text parts,
+without modifiying it, use the -outfile switch:
+.PP
+.RS
+.nf
+mhfixmsg -outfile - | grep \fIpattern\fR

 .fi
 .RE
 .PP
 .fi
 .RE
 .PP

+-outfile can be abbreviated in usual MH fashion, e.g., to -o.  The search will be
+on the entire message, not just text parts.
+.SS Translate text/plain parts to UTF-8
+To translate all text/plain parts in the current message to UTF-8, in addition
+to all of the default transformations:
+.PP
+.RS
+.nf
+mhfixmsg -textcharset utf-8
+.fi
+.RE
+.SS Fix all messages in a folder
+To run
+.B mhfixmsg
+on all of the messages in a folder, using a Bourne shell loop:
+.PP
+.RS
+.nf
+for msg in `pick +folder`; do mhfixmsg $msg; done
+.fi
+.RE
+.PP
+.B mhfixmsg
+can be run on more than one message, but on a large number of messages
+may attempt to open too many files.
+.SS Run on newly incorporated messages
+To run
+.B mhfixmsg
+on messages as they are incorporated:
+.PP
+.RS
+.nf
+inc  &&  mhfixmsg -nochangecur unseen
+.fi
+.RE
+.PP
+This assumes that the Unseen-Sequence profile entry is set to
+.BR unseen ,
+as shown in the mh\-profile(5) man page.

 .SH FILES
 .B mhfixmsg
 looks for mhn.defaults in multiple locations: absolute pathnames are
 .SH FILES
 .B mhfixmsg
 looks for mhn.defaults in multiple locations: absolute pathnames are


@@ -260,14 +470,14 @@ accessed directly, tilde expansion is done on usernames, and files are
 searched for in the user's
 .I Mail
 directory as specified in their profile.  If not found there, the directory
 searched for in the user's
 .I Mail
 directory as specified in their profile.  If not found there, the directory

-.RI \*(lq %etcdir% \*(rq
+.RI \*(lq %nmhetcdir% \*(rq

 is checked.
 .PP
 .fc ^ ~
 .nf
 is checked.
 .PP
 .fc ^ ~
 .nf

-.ta \w'%etcdir%/mhn.defaults  'u
+.ta \w'%nmhetcdir%/mhn.defaults  'u

 ^$HOME/\&.mh\(ruprofile~^The user profile
 ^$HOME/\&.mh\(ruprofile~^The user profile

-^%etcdir%/mhn.defaults~^Default mhfixmsg conversion entries
+^%nmhetcdir%/mhn.defaults~^Default mhfixmsg conversion entries

 .fi
 .SH "PROFILE COMPONENTS"
 .fc ^ ~
 .fi
 .SH "PROFILE COMPONENTS"
 .fc ^ ~


@@ -279,11 +489,14 @@ is checked.
 ^rmmproc:~^Program to delete original messages or files
 .fi
 .SH "SEE ALSO"
 ^rmmproc:~^Program to delete original messages or files
 .fi
 .SH "SEE ALSO"

+.IR inc (1),
+.IR iconv (3),

 .IR mh-profile (5),
 .IR mhbuild (1),
 .IR mhlist (1),
 .IR mh-profile (5),
 .IR mhbuild (1),
 .IR mhlist (1),

+.IR mhparam (1),

 .IR mhshow (1),
 .IR mhshow (1),

-.IR mkstemp (3),
+.IR mh-mkstemp (1),

 .IR procmail (1),
 .IR procmailrc (5),
 .IR rcvstore (1),
 .IR procmail (1),
 .IR procmailrc (5),
 .IR rcvstore (1),


@@ -293,15 +506,36 @@ is checked.
 .RB ` +folder "' defaults to the current folder"
 .RB ` msgs "' defaults to cur"
 .RB ` "\-decodetext 8bit"'
 .RB ` +folder "' defaults to the current folder"
 .RB ` msgs "' defaults to cur"
 .RB ` "\-decodetext 8bit"'

-.RB ` \-notextcodeset '
+.RB ` "\-decodetypes text,application/ics"'
+.RB ` \-crlflinebreaks '
+.RB ` \-notextcharset '

 .RB ` \-reformat '
 .RB ` \-reformat '

+.RB ` \-noreplacetextplain '

 .RB ` \-fixboundary '
 .RB ` \-fixcte '
 .RB ` \-fixboundary '
 .RB ` \-fixcte '

+.RB ` \-changecur '

 .RB ` \-noverbose '
 .fi
 .SH CONTEXT
 If a folder is given, it will become the current folder.  The last
 .RB ` \-noverbose '
 .fi
 .SH CONTEXT
 If a folder is given, it will become the current folder.  The last

-message selected from a folder will become the current message.  If
+message selected from a folder will become the current message, unless

 the
 the

+.B \-nochangecur
+switch is enabled.  If the

 .B \-file
 .B \-file

-switch is used, the context will not be modified.
+switch or an absolute pathname is used, the context will not be
+modified.
+.SH BUGS
+.B mhfixmsg
+opens files internally for decoding and character set conversion, and apparently does not
+close them expeditiously.  Until that is resolved, it is recommended that
+.B mhfixmsg
+not be run on a large number of messages at once, as noted in the EXAMPLES above.
+.PP
+As noted in the DESCRIPTION above,
+.B mhfixmsg
+will not decode to binary content.  This restriction should be removed at some point.  It's
+not due to any issue in
+.BR mhfixmsg ,
+but rather an observation of incorrect behavior by other nmh tools on messages with binary
+content.