configured to use less and vim by default but options allow use of
more and emacs, respectively."
-HOMEPAGE="http://savannah.nongnu.org/projects/nmh"
-SRC_URI="http://download.savannah.nongnu.org/releases/nmh/nmh-${VERSION}.tar.gz"
-CYGCONF_ARGS="--with-cyrus-sasl --with-tls --with-oauth"
-
+HOMEPAGE="http://savannah.nongnu.org/projects/${PN}"
+SRC_URI="http://download.savannah.nongnu.org/releases/${PN}/${P}.tar.gz"
+CYGCONF_ARGS="--sysconfdir=${prefix}/etc/${PN} --with-cyrus-sasl --with-tls --with-oauth"
- Support for SMTPUTF8 (RFC 6531) has been added. mhshow(1) already supported
RFC 6532, assuming all 8-bit message header field bodies are UTF-8 and use
of a UTF-8 locale.
+- mhfixmsg now replaces RFC 2047 encoding with RFC 2231 encoding of name and
+ filename parameters in Content-Type and Content-Disposition headers,
+ respectively.
+- If a message body contains 8-bit bytes, post(8) uses SMTP 8BITMIME if the
+ server supports it. If not, post fails with a message to the user to
+ encode the message for 7-bit transport.
- Added welcome message when nmh detects that its version changed.
-----------------
scan(1), inc(1), and the other programs that rely on the format scanner.
- The first character of some very short (less than 4 characters) message
bodies is no longer dropped.
+- mhfixmsg now adds a Content-Transfer-Encoding header at the message level,
+ if needed after decoding text parts.
+- mhbuild now checks whether all text parts need a Content-Transfer-Encoding
+ header, not just those with a character set not specified.
* At the top level, the c_first_hf list has a list of all message
* headers. If the content-type is multipart (c_type == CT_MULTIPART)
* then c_ctparams will contain a pointer to a struct multipart.
- * A struct multipart contains (among other trhings) a linked list
+ * A struct multipart contains (among other things) a linked list
* of struct part elements, and THOSE contain a pointer to the sub-part's
* Content structure.
*/
char *output_params(size_t initialwidth, PM params, int *offsetout,
int external);
+/*
+ * Encode a parameter value using RFC 2231 encode.
+ *
+ * Arguments are:
+ *
+ * pm - PM containing the parameter value and related info.
+ * output - Output buffer.
+ * len - Size, in octets, of output buffer.
+ * valuelen - Number of characters in the value
+ * valueoff - Offset into value field (pm->pm_value).
+ * index - If 0, output character set and language tag.
+ */
+size_t encode_param(PM pm, char *output, size_t len, size_t valuelen,
+ size_t valueoff, int index);
+
/*
* Add a parameter to the parameter linked list.
*
* Retrieve the various entries for the OAuth mechanism
*/
-boolean
+boolean
mh_oauth_get_service_info(const char *svc_name, mh_oauth_service_info *svcinfo,
char *errbuf, size_t errbuflen);
-
-char *mh_oauth_get_svc_name(mh_oauth_ctx *ctx);
-void mh_oauth_set_cred_fn(mh_oauth_ctx *ctx, char *filename);
switch, then
.B mhbuild
will encode each content with
-a transfer encoding, even it the content contains only 7\-bit data.
+a transfer encoding, even if the content contains only 7\-bit data.
This
is to increase the likelihood that the content is not changed while in
transport.
-.TH MHFIXMSG %manext1% "September 22, 2016" "%nmhversion%"
+.TH MHFIXMSG %manext1% "October 3, 2016" "%nmhversion%"
.\"
.\" %nmhwarning%
.\"
.B \-fixcte
switch enables a transformation to change the
Content-Transfer-Encoding from an invalid value to 8bit in message
-parts with a Content-Type of multipart, as required by RFC 2045,
-§6.4. That condition is indicated by a \*(lqmust be encoded in
-7bit, 8bit, or binary\*(rq error message from
+parts with a Content-Type of multipart and message, as required by
+RFC 2045, §6.4. That condition is indicated by a \*(lqmust be
+encoded in 7bit, 8bit, or binary\*(rq error message from
.B mhlist
and other
.B nmh
message types.
.PP
.B mhfixmsg
-applies one transformation unconditionally: it removes an extraneous
-trailing semicolon from the parameter lists of MIME header fields.
+applies two transformations unconditionally.
+The first removes an extraneous trailing semicolon from the parameter
+lists of MIME header field values.
+The second replaces RFC 2047 encoding with RFC 2231 encoding of name
+and filename parameters in Content-Type and Content-Disposition header
+field values, respectively.
.PP
The
.B \-verbose
\-textcharset text/plain parts
\-reformat text parts that are not text/plain
\-fixboundary outermost multipart part
-\-fixcte multipart part
+\-fixcte multipart or message part
\-fixtype all except multipart and message parts
.fi
.RE
.RB [ \-sasl ]
.RB [ \-saslmech
.IR mechanism ]
+.RB [ \-initialtls ]
+.RB [ \-notls ]
.RB [ \-authservice
.IR service ]
.RB [ \-snoop ]
.B mhlogin
man page for more details.
.PP
-Gmail only supports POP3 over TLS, but
-.B msgchk
-has no TLS support. To work around this, use something like
-.B -proxy 'openssl s_client -connect %h:995 -CAfile /etc/ssl/certs/ca-certificates.crt -quiet'
+If
+.B nmh
+has been compiled with TLS support, the
+.B \-initialtls
+switch will require the negotiation of TLS when connecting
+to the remote POP server. The
+.B \-initialtls
+switch will negotiate TLS immediately after the connection has taken place,
+before any POP commands are sent or received. Data encrypted by TLS is
+labeled `(tls-encrypted)' and `(tls-decrypted)` with viewing the POP
+transaction with the
+.B \-snoop
+switch. The
+.B \-notls
+switch will disable all attempts to negotiate TLS.
.SH FILES
.fc ^ ~
.nf
}
int
-sm_winit (char *from, int smtputf8)
+sm_winit (char *from, int smtputf8, int eightbit)
{
- const char *const mail_parameters = smtputf8
- ? " BODY=8BITMIME SMTPUTF8"
- : "";
-
- /* Just for information, if an attempt is made to send to an 8-bit
- address without specifying SMTPUTF8, Gmail responds with
- 555 5.5.2 Syntax error.
- Gmail doesn't require the 8BITMIME, but RFC 6531 Sec. 1.2 does. */
- if (smtputf8 && (! EHLOset("8BITMIME") || ! EHLOset("SMTPUTF8"))) {
- sm_end (NOTOK);
- return RP_UCMD;
+ const char *mail_parameters = "";
+
+ if (smtputf8) {
+ /* Just for information, if an attempt is made to send to an 8-bit
+ address without specifying SMTPUTF8, Gmail responds with
+ 555 5.5.2 Syntax error.
+ Gmail doesn't require the 8BITMIME, but RFC 6531 Sec. 1.2 does. */
+ if (EHLOset ("8BITMIME") && EHLOset ("SMTPUTF8")) {
+ mail_parameters = " BODY=8BITMIME SMTPUTF8";
+ } else {
+ sm_end (NOTOK);
+ return RP_UCMD;
+ }
+ } else if (eightbit) {
+ /* Comply with RFC 6152, for messages that have any 8-bit characters
+ in their body. */
+ if (EHLOset ("8BITMIME")) {
+ mail_parameters = " BODY=8BITMIME";
+ } else {
+ advise (NULL, "SMTP server does not support 8BITMIME, not sending.\n"
+ "Suggest encoding message for 7-bit transport by setting your\n"
+ "locale to C, and/or specifying *b64 in mhbuild directives.\n");
+ sm_end (NOTOK);
+ return RP_UCMD;
+ }
}
switch (smtalk (SM_MAIL, "MAIL FROM:<%s>%s", from, mail_parameters)) {
/* int client (); */
int sm_init (char *, char *, char *, int, int, int, int, const char *,
const char *, const char *, int);
-int sm_winit (char *, int);
+int sm_winit (char *, int, int);
int sm_wadr (char *, char *, char *);
int sm_waend (void);
int sm_wtxt (char *, int);
/*
* How this code works, in general.
*
- * _If_ we are using no encryption or SASL encryption, then we buffer the
- * network data through ns_inbuffer and ns_outbuffer. That should be
- * relatively self-explanatory.
+ * _If_ we are using no encryption then we buffer the network data
+ * through ns_inbuffer and ns_outbuffer. That should be relatively
+ * self-explanatory.
*
- * If we are using SSL for encryption, then use a buffering BIO for output
- * (that just easier). Still do buffering for reads; when we need more
- * data we call the BIO_read() function to fill our local buffer.
+ * If we use encryption, then ns_inbuffer and ns_outbuffer contain the
+ * cleartext data. When it comes time to send the encrypted data on the
+ * (either from a flush or the buffer is full) we either use BIO_write()
+ * for TLS or sasl_encode() (followed by a write() for Cyrus-SASL. For
+ * reads we either use BIO_read() (TLS) or do a network read into a
+ * temporary buffer and use sasl_decode() (Cyrus-SASL). Note that if
+ * negotiate TLS then we disable SASL encryption.
*
- * For SASL, we make use of (for now) the Cyrus-SASL library. For some
- * mechanisms, we implement those mechanisms directly since the Cyrus SASL
- * library doesn't support them (like OAuth).
+ * We used to use a buffering BIO for the reads/writes for TLS, but it
+ * ended up being complicated to special-case the buffering for everything
+ * except TLS, so the buffering is now unified, no matter which encryption
+ * method is being used (even none).
+ *
+ * For SASL authentication, we make use of (for now) the Cyrus-SASL
+ * library. For some mechanisms, we implement those mechanisms directly
+ * since the Cyrus SASL library doesn't support them (like OAuth).
*/
/*
if (decodeBase64(string, &decoded, &decodedlen, 1, NULL) == OK) {
/*
- * Some mechanisms preoduce large binary tokens, which aren't really
+ * Some mechanisms produce large binary tokens, which aren't really
* readable. So let's do a simple heuristic. If the token is greater
* than 100 characters _and_ the first 100 bytes are more than 50%
* non-ASCII, then don't print the decoded buffer, just the
nsc->ns_inptr = nsc->ns_inbuffer;
}
-#ifdef CYRUS_SASL
+#if defined(CYRUS_SASL) || defined(TLS_SUPPORT)
retry:
-#endif /* CYRUS_SASL */
+#endif /* CYRUS_SASL || TLS_SUPPORT */
/*
* If we are using TLS and there's anything pending, then skip the
* select call
*/
}
+ /*
+ * Some explanation:
+ *
+ * startoffset is the offset from the beginning of the input
+ * buffer to data that is in our input buffer, but has not yet
+ * been consumed. This can be non-zero if functions like
+ * netsec_readline() leave leftover data.
+ *
+ * remaining is the remaining amount of unconsumed data in the input
+ * buffer.
+ *
+ * end is a pointer to the end of the valid data + 1; it's where
+ * the next read should go.
+ */
+
startoffset = nsc->ns_inptr - nsc->ns_inbuffer;
remaining = nsc->ns_inbufsize - (startoffset + nsc->ns_inbuflen);
end = nsc->ns_inptr + nsc->ns_inbuflen;
+ /*
+ * If we're past the halfway point in our read buffers, shuffle everything
+ * back to the beginning.
+ */
+
+ if (startoffset > nsc->ns_inbufsize / 2) {
+ memmove(nsc->ns_inbuffer, nsc->ns_inptr, nsc->ns_inbuflen);
+ nsc->ns_inptr = nsc->ns_inbuffer;
+ startoffset = 0;
+ remaining = nsc->ns_inbufsize - nsc->ns_inbuflen;
+ end = nsc->ns_inptr + nsc->ns_inbuflen;
+ }
+
/*
* If we are using TLS, then just read via the BIO. But we still
* use our local buffer.
if (nsc->tls_active) {
rc = BIO_read(nsc->ssl_io, end, remaining);
if (rc == 0) {
+ SSL *ssl;
+ int errcode;
+
+ /*
+ * Check to see if we're supposed to retry; if so,
+ * then go back and read again.
+ */
+
+ if (BIO_should_retry(nsc->ssl_io))
+ goto retry;
+
/*
- * Either EOF, or possibly an error. Either way, it was probably
- * unexpected, so treat as error.
+ * Okay, fine. Get the real error out of the SSL context.
*/
- netsec_err(errstr, "TLS peer aborted connection");
+
+ if (BIO_get_ssl(nsc->ssl_io, &ssl) < 1) {
+ netsec_err(errstr, "SSL_read() returned 0, but cannot "
+ "retrieve SSL context");
+ return NOTOK;
+ }
+
+ errcode = SSL_get_error(ssl, rc);
+ if (errcode == SSL_ERROR_ZERO_RETURN) {
+ netsec_err(errstr, "TLS peer closed remote connection");
+ } else {
+ netsec_err(errstr, "TLS network read failed: %s",
+ ERR_error_string(ERR_peek_last_error(), NULL));
+ }
+ if (nsc->ns_snoop)
+ ERR_print_errors_fp(stderr);
return NOTOK;
} else if (rc < 0) {
/* Definitely an error */
#endif /* CYRUS_SASL */
nsc->ns_inbuflen += rc;
- /*
- * If we're past the halfway point in our read buffers, shuffle everything
- * back to the beginning.
- */
-
- if (startoffset > nsc->ns_inbufsize / 2) {
- memmove(nsc->ns_inbuffer, nsc->ns_inptr, nsc->ns_inbuflen);
- nsc->ns_inptr = nsc->ns_inbuffer;
- }
-
return OK;
}
if (nsc->ns_userid == NULL) {
/*
* Pass the 1 third argument to nmh_get_credentials() so that
- * a defauly user if the -user switch wasn't supplied, and so
+ * a default user if the -user switch wasn't supplied, and so
* that a default password will be supplied. That's used when
* those values really don't matter, and only with legacy/.netrc,
* i.e., with a credentials profile entry.
#define JSON_TYPE "application/json"
-/* We pretend access tokens expire 30 seconds earlier than they actually do to
+/* We pretend access tokens expire 60 seconds earlier than they actually do to
* allow for separate processes to use and refresh access tokens. The process
* that uses the access token (post) has an error if the token is expired; the
* process that refreshes the access token (send) must have already refreshed if
* the expiration is close.
*
- * 30s is arbitrary, and hopefully is enough to allow for clock skew.
+ * 60s is arbitrary, and hopefully is enough to allow for clock skew.
* Currently only Gmail supports XOAUTH2, and seems to always use a token
* life-time of 3600s, but that is not guaranteed. It is possible for Gmail to
* issue an access token with a life-time so short that even after send
* (not counting header and not null-terminated) */
#define RESPONSE_BODY_MAX 8192
-/* Maxium size for URLs and URI-encoded query strings, null-terminated.
+/* Maximum size for URLs and URI-encoded query strings, null-terminated.
*
* Actual maximum we need is based on the size of tokens (limited by
* RESPONSE_BODY_MAX), code user copies from a web page (arbitrarily large), and
export HOME
unset MAILDROP MHBUILD MHCONTEXT MHMTSUSERCONF MHN MHSHOW MHSTORE
-unset MHLDEBUG MHPDEBUG MHWDEBUG PAGER XOAUTH
+unset MHLDEBUG MHPDEBUG MHWDEBUG PAGER XOAUTH SMTPUTF8
#### Set LC_ALL in individual tests as needed. Unset these so
#### that we don't depend on user's settings in other tests.
unset LANG LC_ALL LC_CTYPE
int rc, conn, smtp_state;
FILE *f;
const char *xoauth = getenv("XOAUTH");
+ const char *smtputf8 = getenv("SMTPUTF8");
if (argc != 3) {
fprintf(stderr, "Usage: %s output-filename port\n", argv[0]);
smtp_state = SMTP_DATA;
continue;
}
- if (xoauth == NULL) {
+ if (strncmp(line, "EHLO", 4) == 0) {
putcrlf(conn, "250-ready");
- putcrlf(conn, "250-8BITMIME");
- putcrlf(conn, "250-SMTPUTF8");
- } else {
- /* XOAUTH2 support enabled; handle EHLO and AUTH. */
- if (strncmp(line, "EHLO", 4) == 0) {
- putcrlf(conn, "250-ready");
- putcrlf(conn, "250 AUTH XOAUTH2");
- continue;
+ if (smtputf8 != NULL) {
+ putcrlf(conn, "250-8BITMIME");
+ putcrlf(conn, "250-SMTPUTF8");
+ }
+ if (xoauth != NULL) {
+ putcrlf(conn, "250-AUTH XOAUTH2");
}
+ putcrlf(conn, "250 I'll buy that for a dollar!");
+ continue;
+ }
+ if (xoauth != NULL) {
+ /* XOAUTH2 support enabled; handle AUTH (and EHLO above). */
if (strncmp(line, "AUTH", 4) == 0) {
if (strncmp(line, "AUTH XOAUTH2", 12) == 0
&& strstr(line, xoauth) != NULL) {
check "${testname}.actual" "$1"
}
+
# check that 7-bit body isn't encoded
+start_test "7-bit body isn't encoded"
cat > "${MH_TEST_DIR}/attachment.txt" <<EOF
this is an attachment
EOF
test_attachment "${testname}.expected"
+
# check that 8-bit body is encoded as quoted-printable text/plain,
# even with an attachment
-
+start_test '8-bit body is encoded as quoted-printable text/plain'
cat > "${MH_TEST_DIR}/Mail/draft" <<EOF
From: Mr Nobody <nobody@example.com>
To: Somebody <somebody@example.com>
test_attachment "${testname}.expected"
-# Repeat the previous test, but make sure that the locale is set to C, which
-# should cause mhbuild to fail
+# check that 8-bit attachment gets C-T-E
+start_test '8-bit attachment gets C-T-E'
+rm -f "${MH_TEST_DIR}/attachment.txt"
+cat > "${MH_TEST_DIR}/attachment.txt" <<EOF
+8-bit attachment, ¡Ay, caramba!
+EOF
+
+cat > "${MH_TEST_DIR}/Mail/draft" <<EOF
+From: Mr Nobody <nobody@example.com>
+To: Somebody <somebody@example.com>
+Subject: Test
+Attach: $MH_TEST_DIR/attachment.txt
+EOF
+
+cat > "${testname}.expected" <<EOF
+From: Mr Nobody <nobody@example.com>
+To: Somebody <somebody@example.com>
+Subject: Test
+MIME-Version: 1.0
+Content-Type: text/plain; name="attachment.txt"
+Content-Description: attachment.txt
+Content-Disposition: attachment; filename="attachment.txt"
+Content-Transfer-Encoding: 8bit
+Date:
+
+8-bit attachment, ¡Ay, caramba!
+EOF
+
+test_attachment "${testname}.expected"
+
+
+# check 8-bit body but make sure that the locale is set to C, which
+# should cause mhbuild to fail
+start_test '8-bit body with C locale'
cat > "${MH_TEST_DIR}/Mail/draft" <<EOF
From: Mr Nobody <nobody@example.com>
To: Somebody <somebody@example.com>
check "$expected" "$actual"
+
rm -f ${MHMTSCONF} "${MH_TEST_DIR}/attachment.txt"
+finish_test
exit ${failed:-0}
# emacs hackage to ensure that it writes the inverted exclamation
------=_nmh-multipart
Content-Type: text/plain; charset="Windows-1252"
-Content-Transfer-Encoding: 8bit
+Content-Transfer-Encoding: 7bit
Need to go! Need ... to ... go!
------=_nmh-multipart
Content-Type: text/html; charset="Windows-1252"
-Content-Transfer-Encoding: 8bit
+Content-Transfer-Encoding: 7bit
<html>
<head>
------=_nmh-multipart-3
Content-Type: text/plain; charset="Windows-1252"
-Content-Transfer-Encoding: 8bit
+Content-Transfer-Encoding: 7bit
------=_nmh-multipart
------=_nmh-multipart-1
------=_nmh-multipart-3
Content-Type: text/html; charset="Windows-1252"
-Content-Transfer-Encoding: 8bit
+Content-Transfer-Encoding: 7bit
<html>
<head>
------=_nmh-multipart
Content-Type: text/plain; charset="Windows-1252"
-Content-Transfer-Encoding: 8bit
+Content-Transfer-Encoding: 7bit
Need to go! Need ... to ... go!
------- =_aaaaaaaaaa0
Content-Type: text/plain; charset="iso-8859-1"; name="test4.txt"
Content-Disposition: attachment; filename="test4.txt"
-Content-Transfer-Encoding: 8bit
+Content-Transfer-Encoding: 7bit
This is a text/plain part.
Content-Type: text/plain; charset="iso-8859-1"
Content-Disposition: attachment; filename="test2.txt"
Content-MD5: kq+Hnc2SD/eKwAnkFBDuEA==
-Content-Transfer-Encoding: 8bit
+Content-Transfer-Encoding: 7bit
This is the second text/plain part.
------- =_aaaaaaaaaa0
Content-Type: text/plain; charset="iso-8859-1"; name="test4.txt"
Content-Disposition: attachment; filename="test4.txt"
-Content-Transfer-Encoding: 8bit
+Content-Transfer-Encoding: 7bit
This is the fourth text/plain part.
Content-Type: multipart/mixed; boundary="----- =_aaaaaaaaaa0"
------- =_aaaaaaaaaa0
-Content-Type: text/plain; charset=UTF-8; name="nul+square.txt"
+Content-Type: text/plain; charset="UTF-8"; name="nul+square.txt"
Content-Transfer-Encoding: base64
vbI9vAAK
------- =_aaaaaaaaaa0
Content-Type: text/plain; charset="iso-8859-1"
-Content-Transfer-Encoding: 8bit
+Content-Transfer-Encoding: 7bit
This is a text plain part
Subject: mhfixmsg successful decode of text/plain with failed binary decode
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="----- =_aaaaaaaaaa0"
+Content-Transfer-Encoding: binary
------- =_aaaaaaaaaa0
Content-Type: text/plain; charset="iso-8859-1"
-Content-Transfer-Encoding: 8bit
+Content-Transfer-Encoding: 7bit
This is a text plain part
Subject: mhfixmsg binary decode test
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary=\"----- =_aaaaaaaaaa0\"
+Content-Transfer-Encoding: binary
------- =_aaaaaaaaaa0
-Content-Type: text/plain; charset=UTF-8; name=\"nul+square.txt\"
+Content-Type: text/plain; charset=\"UTF-8\"; name=\"nul+square.txt\"
Content-Transfer-Encoding: binary
" "
------=_nmh-multipart
Content-Type: text/plain; charset="ISO-8859-1"
-Content-Transfer-Encoding: 8bit
+Content-Transfer-Encoding: 7bit
Mile $0.00
Time $78.71
Subject: mhfixmsg textcharset test
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="----- =_aaaaaaaaaa0"
+Content-Transfer-Encoding: 8bit
------- =_aaaaaaaaaa0
Content-Type: text/plain; charset="utf-8"; name="square.txt"
------=_Part_876302
Content-Type: text/plain; charset="UTF-8"
-Content-Transfer-Encoding: 8bit
+Content-Transfer-Encoding: 7bit
Yes, the text/plain part really was empty.
------=_Part_876302
Content-Type: text/html; charset="UTF-8"
-Content-Transfer-Encoding: 8bit
Content-Disposition: inline
<html><head>
------=_Part_876302
Content-Type: text/html; charset="UTF-8"
-Content-Transfer-Encoding: 8bit
Content-Disposition: inline
<html><head>
------=_Part_876302
Content-Type: text/html; charset="UTF-8"
-Content-Transfer-Encoding: 8bit
Content-Disposition: inline
<html><head>
Content-Type: text/plain; charset="iso-8859-1"
Content-Disposition: attachment; filename="test1.txt"
Content-MD5: kq+Hnc2SD/eKwAnkFBDuEA==
-Content-Transfer-Encoding: 8bit
+Content-Transfer-Encoding: 7bit
This is the text/plain part.
------- =_aaaaaaaaaa0
Content-Type: text/html; charset="iso-8859-1"; name="test2.txt"
Content-Disposition: attachment; filename="test2.txt"
-Content-Transfer-Encoding: 8bit
+Content-Transfer-Encoding: 7bit
<html>
<head>
Content-Type: text/plain; charset="iso-8859-1"
Content-Disposition: attachment; filename="test1.txt"
Content-MD5: kq+Hnc2SD/eKwAnkFBDuEA==
-Content-Transfer-Encoding: 8bit
+Content-Transfer-Encoding: 7bit
This is the text/plain part.
Content-Type: text/plain; charset="iso-8859-1"
Content-Disposition: attachment; filename="test1.txt"
Content-MD5: kq+Hnc2SD/eKwAnkFBDuEA==
-Content-Transfer-Encoding: 8bit
+Content-Transfer-Encoding: 7bit
This is the text/plain part.
------- =_aaaaaaaaaa0
Content-Type: application/ics; charset="iso-8859-1"; name="invite.ics"
Content-Disposition: attachment; filename="invite.ics"
-Content-Transfer-Encoding: 8bit
+Content-Transfer-Encoding: 7bit
BEGIN:VCALENDAR\r
VERSION:2.0\r
Content-Type: text/plain; charset="iso-8859-1"
Content-Disposition: attachment; filename="test1.txt"
Content-MD5: kq+Hnc2SD/eKwAnkFBDuEA==
-Content-Transfer-Encoding: 8bit
+Content-Transfer-Encoding: 7bit
This is the text/plain part.
------- =_aaaaaaaaaa0
Content-Type: application/ics; charset="iso-8859-1"; name="invite.ics"
Content-Disposition: attachment; filename="invite.ics"
-Content-Transfer-Encoding: 8bit
+Content-Transfer-Encoding: 7bit
BEGIN:VCALENDAR
VERSION:2.0
check `mhpath last` "$actual"
+start_test "fix RFC 2047 encoded header parameters"
+cat >"$expected" <<EOF
+To: recipient@example.com
+From: sender@example.com
+Date: Wed, 28 Sep 2016 11:24:28 -0400
+Subject: invalid header parameter encoding
+MIME-Version: 1.0
+Content-Type: multipart/mixed; boundary=001a114dd3e8fe9c56053d92f414
+Content-Transfer-Encoding: 8bit
+
+--001a114dd3e8fe9c56053d92f414
+Content-Type: text/plain; charset=UTF-8
+
+This is a test.
+
+--001a114dd3e8fe9c56053d92f414
+Content-Type: text/plain; charset="UTF-8"; name*=UTF-8''test%C3%B8.txt
+Content-Disposition: attachment; filename*=UTF-8''test%C3%B8.txt
+Content-Transfer-Encoding: 8bit
+
+This is the first text/plain part, in a subpart. The file name
+is testø.txt.
+
+--001a114dd3e8fe9c56053d92f414--
+EOF
+
+cat >`mhpath new` <<EOF
+To: recipient@example.com
+From: sender@example.com
+Date: Wed, 28 Sep 2016 11:24:28 -0400
+Subject: invalid header parameter encoding
+MIME-Version: 1.0
+Content-Type: multipart/mixed; boundary=001a114dd3e8fe9c56053d92f414
+
+--001a114dd3e8fe9c56053d92f414
+Content-Type: text/plain; charset=UTF-8
+
+This is a test.
+
+--001a114dd3e8fe9c56053d92f414
+Content-Type: text/plain; charset=UTF-8; name="=?UTF-8?B?dGVzdMO4LnR4dA==?="
+Content-Disposition: attachment; filename="=?UTF-8?B?dGVzdMO4LnR4dA==?="
+Content-Transfer-Encoding: 8bit
+
+This is the first text/plain part, in a subpart. The file name
+is testø.txt.
+
+--001a114dd3e8fe9c56053d92f414--
+EOF
+run_prog mhfixmsg -file - -outfile - <`mhpath last` >"$actual" 2>/dev/null
+check "$expected" "$actual"
+
+
# make sure there are no tmp files left over
find "$MH_TEST_DIR/Mail" \( -name 'mhfix*' -o -name ',mhfix*' \) -print \
>"$actual"
From: sender27@example.com
MIME-Version: 1.0
Content-Type: text/plain;charset=utf-8
-Content-Transfer-Encoding: 8bit
+Content-Transfer-Encoding: 7bit
Date:
with added header fields
test_mhmail "$expected" \
"-from sender27@example.com -headerfield MIME-Version:1.0 \
-headerfield Content-Type:text/plain;charset=utf-8 \
--headerfield Content-Transfer-Encoding:8bit" \
+-headerfield Content-Transfer-Encoding:7bit" \
-b 'with added header fields'
[ ${failed:-0} -eq 0 ] || exit ${failed:-0}
# Basic test - Simple message, single user, single recipient. Note that
# we test dot-stuffing here as well.
#
-
+start_test 'simple message'
cat > "${MH_TEST_DIR}/Mail/draft" <<EOF
From: Mr Nobody <nobody@example.com>
To: Somebody Else <somebody@example.com>
#
# Make sure a draft without a From: is rejected
#
-
+start_test 'reject draft without a From:'
cat > "${MH_TEST_DIR}/Mail/draft" <<EOF
To: Somebody Else <somebody@example.com>
Subject: Blank Test
# Make sure that empty Nmh-* header lines are ignored, and that post
# warns about non-empty ones.
#
+start_test 'ignore Nmh-* header lines'
cat > "${MH_TEST_DIR}/Mail/draft" <<EOF
From: Mr Nobody <nobody@example.com>
To: Somebody Else <somebody@example.com>
check "${testname}.send_output" "${testname}.expected_send_output"
+#
+# 8-bit without 8BITMIME support
+#
+start_test '8-bit without 8BITMIME support'
+cat > "${MH_TEST_DIR}/Mail/draft" <<EOF
+From: Mr Nobody <nobody@example.com>
+To: Somebody Else <somebody@example.com>
+Subject: Test
+MIME-Version: 1.0
+Content-Type: text/plain; charset="utf-8"
+Content-Transfer-Encoding: 8bit
+
+This is a test
+.
+EOF
+
+cat > "${testname}.expected" <<EOF
+EHLO nosuchhost.example.com
+RSET
+QUIT
+EOF
+
+cat > "${testname}.err.expected" <<EOF
+post: SMTP server does not support 8BITMIME, not sending.
+Suggest encoding message for 7-bit transport by setting your
+locale to C, and/or specifying *b64 in mhbuild directives.
+
+post: problem initializing server; [BHST] ready; I'll buy that for a dollar!
+send: message not delivered to anyone
+EOF
+
+set +e
+test_post "${testname}.actual" "${testname}.expected" 2>"${testname}.err"
+set +e
+check "${testname}.err" "${testname}.err.expected"
+
+#
+# 8-bit with 8BITMIME support
+#
+start_test '8-bit with 8BITMIME support'
+# Cheat: SMTPUTF8 enables 8BITMIME in fakestmp
+SMTPUTF8=1; export SMTPUTF8
+cat > "${testname}.expected" <<EOF
+EHLO nosuchhost.example.com
+MAIL FROM:<nobody@example.com> BODY=8BITMIME
+RCPT TO:<somebody@example.com>
+DATA
+From: Mr Nobody <nobody@example.com>
+To: Somebody Else <somebody@example.com>
+Subject: Test
+MIME-Version: 1.0
+Content-Type: text/plain; charset="utf-8"
+Content-Transfer-Encoding: 8bit
+Date:
+
+This is a test
+..
+.
+QUIT
+EOF
+test_post "${testname}.actual" "${testname}.expected"
+
+
+#
+# 8-bit with 8BITMIME support, inferred from content
+#
+start_test '8-bit, inferred, with 8BITMIME support'
+cat > "${MH_TEST_DIR}/Mail/draft" <<EOF
+From: Mr Nobody <nobody@example.com>
+To: Somebody Else <somebody@example.com>
+Subject: Test
+MIME-Version: 1.0
+Content-Type: text/plain; charset="utf-8"
+
+This is a test, with a non-ascii character: §
+.
+EOF
+
+cat > "${testname}.expected" <<EOF
+EHLO nosuchhost.example.com
+MAIL FROM:<nobody@example.com> BODY=8BITMIME
+RCPT TO:<somebody@example.com>
+DATA
+From: Mr Nobody <nobody@example.com>
+To: Somebody Else <somebody@example.com>
+Subject: Test
+MIME-Version: 1.0
+Content-Type: text/plain; charset="utf-8"
+Date:
+
+This is a test, with a non-ascii character: §
+..
+.
+QUIT
+EOF
+test_post "${testname}.actual" "${testname}.expected"
+
+finish_test
exit ${failed:-0}
test_post ()
{ pid=`"${MH_OBJ_DIR}/test/fakesmtp" "$1" $localport`
- run_prog send -draft -server 127.0.0.1 -port $localport $3 || exit 1
+ run_prog send -draft -server 127.0.0.1 -port $localport $3
#
# It's hard to calculate the exact Date: header post is going to
setup_test
mhl="${MH_LIBEXEC_DIR}/mhl"
+SMTPUTF8=1; export SMTPUTF8
#### Make sure that html-to-text conversion is what we expect.
require_locale en_US.utf-8 en_US.utf8
test_post "${testname}.actual" "${testname}.expected"
+start_test "smtp server doesn't support SMTPUTF8"
+unset SMTPUTF8
+cat > "${testname}.expected" <<EOF
+EHLO nosuchhost.example.com
+RSET
+QUIT
+EOF
+cat > "${testname}.expected.err" <<EOF
+post: problem initializing server; [BHST] ready; I'll buy that for a dollar!
+send: message not delivered to anyone
+EOF
+mv -f "${MH_TEST_DIR}/Mail/,draft" "${MH_TEST_DIR}/Mail/draft"
+
+set +e
+test_post "${testname}.actual" "${testname}.expected" 2>"${testname}.err"
+set -e
+check "${testname}.err" "${testname}.expected.err"
+
+
finish_test
exit $failed
# check -convertarg with multiple parts and additional text in draft
+start_test '-convertarg with multiple parts and additional text in draft'
cat >"$expected" <<'EOF'
From: recipient@example.com
To: sender@example.com
#### Make sure that this works with 8-bit encoding.
+finish_test
require_locale en_US.utf-8 en_US.utf8
LC_ALL=en_US.UTF-8; export LC_ALL
-# check -convertarg with multiple parts and no additional text in draft
+# check -convertarg with multiple parts, 7 bit
+start_test '-convertarg with multiple parts, 7 bit'
cat >"$expected" <<'EOF'
From: recipient@example.com
To: sender@example.com
message dated "Thu, 11 Dec 2014 08:19:02 -0600."
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
-Content-Transfer-Encoding: 8bit
sender@example.com writes:
This is part 1.
--_001_
-Content-Type: text/plain
+Content-Type: text/plain; charset="UTF-8"
This is part 2.
check "$actual" "$expected"
+# check -convertarg with multiple parts, 8 bit
+start_test '-convertarg with multiple parts, 8 bit'
+cat >"$expected" <<'EOF'
+From: recipient@example.com
+To: sender@example.com
+cc:
+Fcc: +outbox
+Subject: Re: test
+Comments: In-reply-to sender@example.com
+ message dated "Thu, 11 Dec 2014 08:19:02 -0600."
+MIME-Version: 1.0
+Content-Type: text/plain; charset="UTF-8"
+Content-Transfer-Encoding: 8bit
+
+sender@example.com writes:
+
+> This is part 1.
+
+> This is §2, with a non-ASCII character.
+EOF
+
+cat >`mhpath new` <<'EOF'
+From: sender@example.com
+To: recipient@example.com
+Subject: test
+Date: Thu, 11 Dec 2014 08:19:02 -0600
+Content-Type: multipart/mixed; boundary="_001_"
+MIME-Version: 1.0
+
+--_001_
+Content-Type: text/plain
+
+This is part 1.
+
+--_001_
+Content-Type: text/plain; charset="UTF-8"
+
+This is §2, with a non-ASCII character.
+
+--_001_--
+EOF
+
+repl -filter mhl.replywithoutbody -convertarg text/plain '' -nowhatnowproc last
+mhbuild "$actual"
+check "$actual" "$expected"
+
+
# check message with text part in multipart/related
+start_test 'check message with text part in multipart/related'
cat >"$expected" <<'EOF'
From: recipient@example.com
To: sender@example.com
message dated "."
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
-Content-Transfer-Encoding: 8bit
sender@example.com writes:
Content-Type: multipart/related; type="text/plain"; boundary="_002_"
--_002_
-Content-Type: text/plain
+Content-Type: text/plain; charset="UTF-8"
This is a test.
# check reply to calendar request
+start_test 'check reply to calendar request'
cat >"$expected" <<'EOF'
From: recipient@example.com
To: sender@example.com
check "$actual" "$expected"
+finish_test
exit $failed
<= 220 Not really an ESMTP server
=> EHLO nosuchhost.example.com
<= 250-ready
-<= 250-8BITMIME
-<= 250-SMTPUTF8
<= 250 I'll buy that for a dollar!
=> MAIL FROM:<nobody@example.com>
-<= 250-ready
-<= 250-8BITMIME
-<= 250-SMTPUTF8
<= 250 I'll buy that for a dollar!
=> RCPT TO:<somebody@example.com>
-<= 250-ready
-<= 250-8BITMIME
-<= 250-SMTPUTF8
<= 250 I'll buy that for a dollar!
=> DATA
<= 354 Go ahead
* Decide what to check while scanning this content. Note that
* for text content we always check for 8bit characters if the
* charset is unspecified, because that controls whether or not the
- * character set is us-ascii or retrieved from the locale.
+ * character set is us-ascii or retrieved from the locale. And
+ * we check even if the charset is specified, to allow setting
+ * the proper Content-Transfer-Encoding.
*/
if (ct->c_type == CT_TEXT) {
t = (struct text *) ct->c_ctparams;
if (t->tx_charset == CHARSET_UNSPECIFIED) {
- check8bit = 1;
checknul = 1;
}
+ check8bit = 1;
}
switch (ct->c_reqencoding) {
} else {
set_charset (reply_ct, -1);
charset = get_param (reply_ct->c_ctinfo.ci_first_pm, "charset", '?', 1);
- if (reply_ct->c_reqencoding == CE_UNKNOWN) {
- /* Assume that 8bit is sufficient (for text). */
- reply_ct->c_reqencoding =
- strcasecmp (charset, "US-ASCII") ? CE_8BIT : CE_7BIT;
- }
}
/* Concatenate text/plain parts. */
/* Make sure that the charset is set in the text/plain
part. */
set_charset (*text_plain_ct, -1);
- if ((*text_plain_ct)->c_reqencoding == CE_UNKNOWN) {
- /* Assume that 8bit is sufficient (for text). */
- (*text_plain_ct)->c_reqencoding =
- strcasecmp (charset, "US-ASCII") ? CE_8BIT : CE_7BIT;
- }
}
if (*text_plain_ct) {
X("noreplacetextplain", 0, NREPLACETEXTPLAINSW) \
X("fixboundary", 0, FIXBOUNDARYSW) \
X("nofixboundary", 0, NFIXBOUNDARYSW) \
- X("fixcte", 0, FIXCTESW) \
- X("nofixcte", 0, NFIXCTESW) \
+ X("fixcte", 0, FIXCOMPOSITECTESW) \
+ X("nofixcte", 0, NFIXCOMPOSITECTESW) \
X("fixtype mimetype", 0, FIXTYPESW) \
X("file file", 0, FILESW) \
X("outfile file", 0, OUTFILESW) \
*/
typedef struct fix_transformations {
int fixboundary;
- int fixcte;
+ int fixcompositecte;
svector_t fixtypes;
int reformat;
int replacetextplain;
static int fix_types (CT, svector_t, int *);
static char *replace_substring (char **, const char *, const char *);
static char *remove_parameter (char *, const char *);
-static int fix_multipart_cte (CT, int *);
+static int fix_composite_cte (CT, int *);
static int set_ce (CT, int);
static int ensure_text_plain (CT *, CT, int *, int);
static int find_textplain_sibling (CT, int, int *);
static void copy_ctinfo (CI, CI);
static int decode_part (CT);
static int reformat_part (CT, char *, char *, char *, int);
-static int charset_encoding (CT);
static CT build_multipart_alt (CT, CT, int, int);
static int boundary_in_content (FILE **, char *, const char *);
static void transfer_noncontent_headers (CT, CT);
static int should_decode(const char *, const char *, const char *);
static int content_encoding (CT, const char **);
static int strip_crs (CT, int *);
+static void update_cte (CT);
+static int least_restrictive_encoding (CT);
+static int less_restrictive (int, int);
static int convert_charsets (CT, char *, int *);
static int fix_always (CT, int *);
+static int fix_filename_param (char *, char *, PM *, PM *);
+static int fix_filename_encoding (CT);
static int write_content (CT, const char *, char *, int, int);
static void set_text_ctparams(CT, char *, int);
static int remove_file (const char *);
int chgflag = 1;
int status = OK;
fix_transformations fx;
- fx.reformat = fx.fixcte = fx.fixboundary = 1;
+ fx.reformat = fx.fixcompositecte = fx.fixboundary = 1;
fx.fixtypes = NULL;
fx.replacetextplain = 0;
fx.decodetext = CE_8BIT;
case NFIXBOUNDARYSW:
fx.fixboundary = 0;
continue;
- case FIXCTESW:
- fx.fixcte = 1;
+ case FIXCOMPOSITECTESW:
+ fx.fixcompositecte = 1;
continue;
- case NFIXCTESW:
- fx.fixcte = 0;
+ case NFIXCOMPOSITECTESW:
+ fx.fixcompositecte = 0;
continue;
case FIXTYPESW:
if (! (cp = *argp++) || (*cp == '-' && cp[1])) {
}
+/*
+ * Apply transformations to one message.
+ */
int
mhfixmsgsbr (CT *ctp, const fix_transformations *fx, char *outfile) {
/* Store input filename in case one of the transformations, i.e.,
if (status == OK && fx->fixtypes != NULL) {
status = fix_types (*ctp, fx->fixtypes, &message_mods);
}
- if (status == OK && fx->fixcte) {
- status = fix_multipart_cte (*ctp, &message_mods);
+ if (status == OK && fx->fixcompositecte) {
+ status = fix_composite_cte (*ctp, &message_mods);
}
if (status == OK && fx->reformat) {
status =
ensure_text_plain (ctp, NULL, &message_mods, fx->replacetextplain);
}
if (status == OK && fx->decodetext) {
- status = decode_text_parts (*ctp, fx->decodetext, fx->decodetypes, &message_mods);
+ status = decode_text_parts (*ctp, fx->decodetext, fx->decodetypes,
+ &message_mods);
+ update_cte (*ctp);
}
if (status == OK && fx->textcharset != NULL) {
status = convert_charsets (*ctp, fx->textcharset, &message_mods);
}
-/* Copy input message to output. Assumes not modifying in place, so this
- might be running as part of a pipeline. */
+/*
+ * Copy input message to output. Assumes not modifying in place, so this
+ * might be running as part of a pipeline.
+ */
static int
copy_input_to_output (const char *input_filename, const char *output_filename) {
int in = open (input_filename, O_RDONLY);
}
+/*
+ * Fix mismatched outer level boundary.
+ */
static int
fix_boundary (CT *ct, int *message_mods) {
struct multipart *mp;
}
+/*
+ * Find boundary at end of multipart.
+ */
static int
get_multipart_boundary (CT ct, char **part_boundary) {
char buffer[BUFSIZ];
}
-/* Open and copy ct->c_file to file, replacing the multipart boundary. */
+/*
+ * Open and copy ct->c_file to file, replacing the multipart boundary.
+ */
static int
replace_boundary (CT ct, char *file, char *boundary) {
FILE *fpin, *fpout;
}
+/*
+ * Fix Content-Type header to reflect the content of its part.
+ */
static int
fix_types (CT ct, svector_t fixtypes, int *message_mods) {
int status = OK;
return status;
}
+
+/*
+ * Replace a substring, allocating space to hold the new one.
+ */
char *
replace_substring (char **str, const char *old, const char *new) {
char *cp;
}
}
+
/*
* Remove a name=value parameter, given just its name, from a header value.
*/
return str;
}
+
+/*
+ * Fix Content-Transfer-Encoding of composite,, e.g., message or multipart, part.
+ * According to RFC 2045 Sec. 6.4, it must be 7bit, 8bit, or binary. Set it to
+ * 8 bit.
+ */
static int
-fix_multipart_cte (CT ct, int *message_mods) {
+fix_composite_cte (CT ct, int *message_mods) {
int status = OK;
- if (ct->c_type == CT_MULTIPART) {
- struct multipart *m;
- struct part *part;
-
+ if (ct->c_type == CT_MESSAGE || ct->c_type == CT_MULTIPART) {
if (ct->c_encoding != CE_7BIT && ct->c_encoding != CE_8BIT &&
ct->c_encoding != CE_BINARY) {
HF hf;
set_ce (ct, CE_8BIT);
}
- m = (struct multipart *) ct->c_ctparams;
- for (part = m->mp_parts; part; part = part->mp_next) {
- if (fix_multipart_cte (part->mp_part, message_mods) != OK) {
- status = NOTOK;
- break;
+ if (ct->c_type == CT_MULTIPART) {
+ struct multipart *m;
+ struct part *part;
+
+ m = (struct multipart *) ct->c_ctparams;
+ for (part = m->mp_parts; part; part = part->mp_next) {
+ if (fix_composite_cte (part->mp_part, message_mods) != OK) {
+ status = NOTOK;
+ break;
+ }
}
}
}
}
+/*
+ * Set content encoding.
+ */
static int
set_ce (CT ct, int encoding) {
const char *ce = ce_str (encoding);
ct->c_cefile.ce_file to the name of the file containing
the contents. */
+ if (ct->c_ceclosefnx) {
+ (*ct->c_ceclosefnx) (ct);
+ }
+
/* Restore the cefile. */
ct->c_cefile = decoded_content_info;
}
-/* Make sure each text part has a corresponding text/plain part. */
+/*
+ * Make sure each text part has a corresponding text/plain part.
+ */
static int
ensure_text_plain (CT *ct, CT parent, int *message_mods, int replacetextplain) {
int status = OK;
}
-/* See if there is a sibling text/plain. */
+/*
+ * See if there is a sibling text/plain, and return its subpart number.
+ */
static int
find_textplain_sibling (CT parent, int replacetextplain,
int *new_subpart_number) {
}
+/*
+ * Insert a new text/plain part.
+ */
static int
insert_new_text_plain_part (CT ct, int new_subpart_number, CT parent) {
struct multipart *mp = (struct multipart *) parent->c_ctparams;
}
+/*
+ * Create a text/plain part to go along with non-plain sibling part.
+ */
static CT
build_text_plain_part (CT encoded_part) {
CT tp_part = divide_part (encoded_part);
}
-/* Slip new text/plain part into a new multipart/alternative. */
+/*
+ * Slip new text/plain part into a new multipart/alternative.
+ */
static int
insert_into_new_mp_alt (CT *ct, int *message_mods) {
CT tp_part = build_text_plain_part (*ct);
return status;
}
+
+/*
+ * Clone a MIME part.
+ */
static CT
divide_part (CT ct) {
CT new_part;
}
+/*
+ * Copy the content info from one part to another.
+ */
static void
copy_ctinfo (CI dest, CI src) {
PM s_pm, d_pm;
for (s_pm = src->ci_first_pm; s_pm; s_pm = s_pm->pm_next) {
d_pm = add_param(&dest->ci_first_pm, &dest->ci_last_pm, s_pm->pm_name,
s_pm->pm_value, 0);
- if (s_pm->pm_charset)
+ if (s_pm->pm_charset) {
d_pm->pm_charset = getcpy(s_pm->pm_charset);
- if (s_pm->pm_lang)
+ }
+ if (s_pm->pm_lang) {
d_pm->pm_lang = getcpy(s_pm->pm_lang);
+ }
}
dest->ci_comment = src->ci_comment ? add (src->ci_comment, NULL) : NULL;
}
+/*
+ * Decode content.
+ */
static int
decode_part (CT ct) {
char *tmp_decoded;
}
-/* Some of the arguments aren't really needed now, but maybe will
- be in the future for other than text types. */
+/*
+ * Reformat content as plain text.
+ * Some of the arguments aren't really needed now, but maybe will
+ * be in the future for other than text types.
+ */
static int
reformat_part (CT ct, char *file, char *type, char *subtype, int c_type) {
int output_subtype, output_encoding;
+ const char *reason = NULL;
char *cp, *cf;
int status;
/* Set subtype to 0, which is always an UNKNOWN subtype. */
output_subtype = 0;
}
- output_encoding = charset_encoding (ct);
+ output_encoding = content_encoding (ct, &reason);
if (set_ct_type (ct, c_type, output_subtype, output_encoding) == OK) {
ct->c_cefile.ce_file = file;
ct->c_cefile.ce_unlink = 1;
}
-/* Identifies 7bit or 8bit content based on charset. */
-static int
-charset_encoding (CT ct) {
- char *ct_charset = content_charset (ct);
- int encoding = strcasecmp (ct_charset, "US-ASCII") ? CE_8BIT : CE_7BIT;
-
- free (ct_charset);
-
- return encoding;
-}
-
-
+/*
+ * Fill in a multipart/alternative part.
+ */
static CT
build_multipart_alt (CT first_alt, CT new_part, int type, int subtype) {
char *boundary_prefix = "----=_nmh-multipart";
}
-/* Check that the boundary does not appear in the content. */
+/*
+ * Check that the boundary does not appear in the content.
+ */
static int
boundary_in_content (FILE **fp, char *file, const char *boundary) {
char buffer[BUFSIZ];
}
-/* Remove all non-Content headers. */
+/*
+ * Remove all non-Content headers.
+ */
static void
transfer_noncontent_headers (CT old, CT new) {
HF hp, hp_prev;
}
+/*
+ * Set content type.
+ */
static int
set_ct_type (CT ct, int type, int subtype, int encoding) {
char *typename = ct_type_str (type);
* that character set again after decoding."
*/
static int
-decode_text_parts (CT ct, int encoding, const char *decodetypes, int *message_mods) {
+decode_text_parts (CT ct, int encoding, const char *decodetypes,
+ int *message_mods) {
int status = OK;
int lf_line_endings = 0;
/* Should check to see if the body for this part is encoded?
For now, it gets passed along as-is by InitMultiPart(). */
for (part = m->mp_parts; status == OK && part; part = part->mp_next) {
- status = decode_text_parts (part->mp_part, encoding, decodetypes, message_mods);
+ status = decode_text_parts (part->mp_part, encoding, decodetypes,
+ message_mods);
}
break;
}
if (ct->c_subtype == MESSAGE_EXTERNAL) {
struct exbody *e = (struct exbody *) ct->c_ctparams;
- status = decode_text_parts (e->eb_content, encoding, decodetypes, message_mods);
+ status = decode_text_parts (e->eb_content, encoding, decodetypes,
+ message_mods);
}
break;
ct->c_cefile.ce_file = NULL;
} else {
int enc;
+
if (ct_encoding == CE_BINARY) {
enc = CE_BINARY;
} else if (ct_encoding == CE_8BIT && encoding == CE_7BIT) {
enc = CE_QUOTED;
} else {
- enc = charset_encoding (ct);
+ enc = ct_encoding;
}
if (set_ce (ct, enc) == OK) {
++*message_mods;
}
-/* Determine if the part with type[/subtype] should be decoded, according to
- decodetypes (which came from the -decodetypes switch). */
+/*
+ * Determine if the part with type[/subtype] should be decoded, according to
+ * decodetypes (which came from the -decodetypes switch).
+ */
static int
should_decode(const char *decodetypes, const char *type, const char *subtype) {
/* Quick search for matching type[/subtype] in decodetypes: bracket
}
-/* See if the decoded content is 7bit, 8bit, or binary. It's binary
- if it has any NUL characters, a CR not followed by a LF, or lines
- greater than 998 characters in length. If binary, reason is set
- to a string explaining why. */
+/*
+ * See if the decoded content is 7bit, 8bit, or binary. It's binary
+ * if it has any NUL characters, a CR not followed by a LF, or lines
+ * greater than 998 characters in length. If binary, reason is set
+ * to a string explaining why.
+ */
static int
content_encoding (CT ct, const char **reason) {
CE ce = &ct->c_cefile;
}
+/*
+ * Strip carriage returns from content.
+ */
static int
strip_crs (CT ct, int *message_mods) {
char *charset = content_charset (ct);
}
+/*
+ * Add/update, if necessary, the message C-T-E, based on the least restrictive
+ * of the part C-T-E's.
+ */
+static void
+update_cte (CT ct) {
+ const int least_restrictive_enc = least_restrictive_encoding (ct);
+
+ if (least_restrictive_enc != CE_UNKNOWN &&
+ least_restrictive_enc != CE_7BIT) {
+ char *cte = concat (" ", ce_str (least_restrictive_enc), "\n", NULL);
+ HF hf;
+ int found_cte = 0;
+
+ /* Update/add Content-Transfer-Encoding header field. */
+ for (hf = ct->c_first_hf; hf; hf = hf->next) {
+ if (! strcasecmp (ENCODING_FIELD, hf->name)) {
+ found_cte = 1;
+ free (hf->value);
+ hf->value = cte;
+ }
+ }
+ if (! found_cte) {
+ add_header (ct, add (ENCODING_FIELD, NULL), cte);
+ }
+ }
+}
+
+
+/*
+ * Find the least restrictive encoding (7bit, 8bit, binary) of the parts
+ * within a message.
+ */
+static int
+least_restrictive_encoding (CT ct) {
+ int encoding = CE_UNKNOWN;
+
+ switch (ct->c_type) {
+ case CT_MULTIPART: {
+ struct multipart *m = (struct multipart *) ct->c_ctparams;
+ struct part *part;
+
+ for (part = m->mp_parts; part; part = part->mp_next) {
+ const int part_encoding =
+ least_restrictive_encoding (part->mp_part);
+
+ if (less_restrictive (encoding, part_encoding)) {
+ encoding = part_encoding;
+ }
+ }
+ break;
+ }
+
+ case CT_MESSAGE:
+ if (ct->c_subtype == MESSAGE_EXTERNAL) {
+ struct exbody *e = (struct exbody *) ct->c_ctparams;
+ const int part_encoding =
+ least_restrictive_encoding (e->eb_content);
+
+ if (less_restrictive (encoding, part_encoding)) {
+ encoding = part_encoding;
+ }
+ }
+ break;
+
+ default: {
+ if (less_restrictive (encoding, ct->c_encoding)) {
+ encoding = ct->c_encoding;
+ }
+ }}
+
+ return encoding;
+}
+
+
+/*
+ * Return whether the second encoding is less restrictive than the first, where
+ * "less restrictive" is in the sense used by RFC 2045 Secs. 6.1 and 6.4. So,
+ * CE_BINARY is less restrictive than CE_8BIT and
+ * CE_8BIT is less restrictive than CE_7BIT.
+ */
+static int
+less_restrictive (int encoding, int second_encoding) {
+ switch (second_encoding) {
+ case CE_BINARY:
+ return encoding != CE_BINARY;
+ case CE_8BIT:
+ return encoding != CE_BINARY && encoding != CE_8BIT;
+ case CE_7BIT:
+ return encoding != CE_BINARY && encoding != CE_8BIT &&
+ encoding != CE_7BIT;
+ default :
+ return 0;
+ }
+}
+
+
+/*
+ * Convert character set of each part.
+ */
static int
convert_charsets (CT ct, char *dest_charset, int *message_mods) {
int status = OK;
/*
* Fix various problems that aren't handled elsewhere. These
* are fixed unconditionally: there are no switches to disable
- * them. (Currently, "problems" is just one: an extraneous
- * semicolon at the end of a header parameter list.)
+ * them. Currently, "problems" are these:
+ * 1) remove extraneous semicolon at the end of a header parameter list
+ * 2) replace RFC 2047 encoding with RFC 2231 encoding of name and
+ * filename parameters in Content-Type and Content-Disposition
+ * headers, respectively.
*/
static int
fix_always (CT ct, int *message_mods) {
default: {
HF hf;
+ if (ct->c_first_hf) {
+ fix_filename_encoding (ct);
+ }
+
for (hf = ct->c_first_hf; hf; hf = hf->next) {
size_t len = strlen (hf->value);
}
+/*
+ * Factor out common code for loops in fix_filename_encoding().
+ */
+static int
+fix_filename_param (char *name, char *value, PM *first_pm, PM *last_pm) {
+ size_t value_len;
+ int fixed = 0;
+
+ if (((value_len = strlen (value)) > 0) &&
+ strncmp (value, "=?", 2) == 0 &&
+ strncmp (&value[value_len - 2], "?=", 2) == 0) {
+ /* Looks like an RFC 2047 encoded parameter. */
+ char decoded[PATH_MAX + 1];
+
+ if (decode_rfc2047 (value, decoded, sizeof decoded)) {
+ /* Encode using RFC 2231. */
+ replace_param (first_pm, last_pm, name, decoded, 0);
+ fixed = 1;
+ } else {
+ advise (NULL, "failed to decode %s parameter %s", name, value);
+ }
+ }
+
+ return fixed;
+}
+
+
+/*
+ * Replace RFC 2047 encoding with RFC 2231 encoding of name and
+ * filename parameters in Content-Type and Content-Disposition
+ * headers, respectively.
+ */
+static int
+fix_filename_encoding (CT ct) {
+ PM pm;
+ HF hf;
+ int fixed = 0;
+
+ for (pm = ct->c_ctinfo.ci_first_pm; pm; pm = pm->pm_next) {
+ if (pm->pm_name && pm->pm_value &&
+ strcasecmp (pm->pm_name, "name") == 0) {
+ fixed = fix_filename_param (pm->pm_name, pm->pm_value,
+ &ct->c_ctinfo.ci_first_pm,
+ &ct->c_ctinfo.ci_last_pm);
+ }
+ }
+
+ for (pm = ct->c_dispo_first; pm; pm = pm->pm_next) {
+ if (pm->pm_name && pm->pm_value &&
+ strcasecmp (pm->pm_name, "filename") == 0) {
+ fixed = fix_filename_param (pm->pm_name, pm->pm_value,
+ &ct->c_dispo_first,
+ &ct->c_dispo_last);
+ }
+ }
+
+ /* Fix hf values to correspond. */
+ for (hf = ct->c_first_hf; fixed && hf; hf = hf->next) {
+ enum { OTHER, TYPE_HEADER, DISPO_HEADER } field = OTHER;
+
+ if (strcasecmp (hf->name, TYPE_FIELD) == 0) {
+ field = TYPE_HEADER;
+ } else if (strcasecmp (hf->name, DISPO_FIELD) == 0) {
+ field = DISPO_HEADER;
+ }
+
+ if (field != OTHER) {
+ const char *const semicolon_loc = strchr (hf->value, ';');
+
+ if (semicolon_loc) {
+ const size_t len =
+ strlen (hf->name) + 1 + semicolon_loc - hf->value;
+ const char *const params =
+ output_params (len,
+ field == TYPE_HEADER
+ ? ct->c_ctinfo.ci_first_pm
+ : ct->c_dispo_first,
+ NULL, 0);
+ const char *const new_params = concat (params, "\n", NULL);
+
+ replace_substring (&hf->value, semicolon_loc, new_params);
+ free ((char *) new_params);
+ free ((char *) params);
+ } else {
+ advise (NULL, "did not find semicolon in %s:%s\n",
+ hf->name, hf->value);
+ }
+ }
+ }
+
+ return OK;
+}
+
+
+/*
+ * Output content in input file to output file.
+ */
static int
write_content (CT ct, const char *input_filename, char *outfile, int modify_inplace,
int message_mods) {
/*
- * parse_mime() does not set lf_line_endings in struct text, so use this function to do it.
- * It touches the parts the decodetypes identifies.
+ * parse_mime() does not set lf_line_endings in struct text, so use this
+ * function to do it. It touches the parts the decodetypes identifies.
*/
static void
set_text_ctparams(CT ct, char *decodetypes, int lf_line_endings) {
}
+/*
+ * Output formatted message to user.
+ */
static void
report (char *what, char *partno, char *filename, char *message, ...) {
va_list args;
static int parse_header_attrs (const char *, const char *, char **, PM *,
PM *, char **);
static size_t param_len(PM, int, size_t, int *, int *, size_t *);
-static size_t encode_param(PM, char *, size_t, size_t, size_t, int);
static size_t normal_param(PM, char *, size_t, size_t, size_t);
static int get_dispo (char *, CT, int);
* Output an encoded parameter string.
*/
-static size_t
+size_t
encode_param(PM pm, char *output, size_t len, size_t valuelen,
size_t valueoff, int index)
{
# define SASLminc(a) 0
#endif
+#ifndef TLS_SUPPORT
+# define TLSminc(a) (a)
+#else
+# define TLSminc(a) 0
+#endif
+
#define MSGCHK_SWITCHES \
X("date", 0, DATESW) \
X("nodate", 0, NDATESW) \
X("sasl", SASLminc(4), SASLSW) \
X("saslmech", SASLminc(5), SASLMECHSW) \
X("authservice", SASLminc(0), AUTHSERVICESW) \
+ X("initialtls", TLSminc(-10), INITTLSSW) \
+ X("notls", TLSminc(-5), NOTLSSW) \
X("proxy command", 0, PROXYSW) \
#define X(sw, minchars, id) id,
static int donote (char *, int);
static int checkmail (char *, char *, int, int, int);
static int remotemail (char *, char *, char *, char *, int, int, int, int,
- char *, const char *);
+ char *, int, const char *);
int
main (int argc, char **argv)
{
int datesw = 1, notifysw = NT_ALL;
- int status = 0, sasl = 0;
+ int status = 0, sasl = 0, tls = 0;
int snoop = 0, vecp = 0;
char *cp, *host = NULL, *port = NULL, *user = NULL, *proxy = NULL;
char buf[BUFSIZ], *saslmech = NULL, *auth_svc = NULL;
adios (NULL, "missing argument to %s", argp[-2]);
continue;
+ case INITTLSSW:
+ tls++;
+ continue;
+
+ case NOTLSSW:
+ tls = 0;
+ continue;
+
case AUTHSERVICESW:
#ifdef OAUTH_SUPPORT
if (!(auth_svc = *argp++) || *auth_svc == '-')
if (host) {
if (vecp == 0) {
status = remotemail (host, port, user, proxy, notifysw, 1,
- snoop, sasl, saslmech, auth_svc);
+ snoop, sasl, saslmech, tls, auth_svc);
} else {
for (vecp = 0; vec[vecp]; vecp++)
status += remotemail (host, port, vec[vecp], proxy, notifysw, 0,
- snoop, sasl, saslmech, auth_svc);
+ snoop, sasl, saslmech, tls, auth_svc);
}
} else {
if (user == NULL) user = getusername ();
static int
remotemail (char *host, char *port, char *user, char *proxy, int notifysw,
- int personal, int snoop, int sasl, char *saslmech,
+ int personal, int snoop, int sasl, char *saslmech, int tls,
const char *auth_svc)
{
int nmsgs, nbytes, status;
/* open the POP connection */
if (pop_init (host, port, creds.user, creds.password, proxy, snoop, sasl,
- saslmech, 0, auth_svc) == NOTOK
+ saslmech, tls, auth_svc) == NOTOK
|| pop_stat (&nmsgs, &nbytes) == NOTOK /* check for messages */
|| pop_quit () == NOTOK) { /* quit POP connection */
advise (NULL, "%s", response);
#define HDCC 0x0400 /* another undocumented feature */
#define HONE 0x0800 /* Only (zero or) one address allowed */
#define HEFM 0x1000 /* Envelope-From: header */
+#define HMIM 0x2000 /* MIME-Version: header */
+#define HCTE 0x4000 /* Content-Transfer-Encoding: header */
/*
* flags for headers->set
#define MSND 0x0020 /* we've seen a Sender: */
#define MRSN 0x0040 /* We've seen a Resent-Sendr:*/
#define MEFM 0x0080 /* We've seen Envelope-From: */
-
+#define MMIM 0x0100 /* We've seen Mime-Version: */
static struct headers NHeaders[] = {
{ "Return-Path", HBAD, 0 },
{ "Message-ID", HBAD, 0 },
{ "Fcc", HFCC, 0 },
{ "Envelope-From", HADR|HONE|HEFM, MEFM },
+ { "MIME-Version", HMIM, MMIM },
+ { "Content-Transfer-Encoding", HCTE, 0 },
{ NULL, 0, 0 }
};
{ "Bcc", HADR|HTRY|HBCC|HNIL, 0 },
{ "Fcc", HIGN, 0 },
{ "Envelope-From", HADR|HONE|HEFM, MEFM },
+ { "MIME-Version", HMIM, MMIM },
+ { "Content-Transfer-Encoding", HCTE, 0 },
{ NULL, 0, 0 }
};
static char *filter = NULL; /* the filter for BCC'ing */
static char *subject = NULL; /* the subject field for BCC'ing */
static char *fccfold[FCCS]; /* foldernames for FCC'ing */
+enum encoding { UNKNOWN = 0, BINARY = 1, SEVENBIT = 7, EIGHTBIT = 8 };
+static enum encoding cte = UNKNOWN;
static struct headers *hdrtab; /* table for the message we're doing */
static void die (char *, char *, ...);
static void post (char *, int, int, int, char *, int, char *);
static void do_text (char *file, int fd);
+static int scan_input (int, int *);
static void do_an_address (struct mailname *, int);
static void do_addresses (int, int);
static int find_prefix (void);
insert_fcc (hdr, pp);
return;
}
-
+ if (hdr->flags & HCTE) {
+ if (strncasecmp (str, "7bit", 4) == 0) {
+ cte = SEVENBIT;
+ } else if (strncasecmp (str, "8bit", 4) == 0) {
+ cte = EIGHTBIT;
+ } else if (strncasecmp (str, "binary", 6) == 0) {
+ cte = BINARY;
+ }
+ }
if (!(hdr->flags & HADR)) {
fprintf (out, "%s: %s", name, str);
return;
post (char *file, int bccque, int talk, int eai, char *envelope,
int oauth_flag, char *auth_svc)
{
- int fd;
int retval, i;
pid_t child_id;
break;
}
} else {
+ const int fd = open (file, O_RDONLY);
+ int eightbit = 0;
+
+ if (fd == NOTOK) {
+ die (file, "unable to re-open");
+ }
+
+ if (msgflags & MMIM && cte != UNKNOWN) {
+ /* MIME message with C-T-E header. (BINARYMIME isn't
+ supported, use 8BITMIME instead for binary.) */
+ eightbit = cte != SEVENBIT;
+ } else {
+ if (scan_input (fd, &eightbit) == NOTOK) {
+ close (fd);
+ die (file, "problem reading from");
+ }
+ }
+
if (rp_isbad (retval = sm_init (clientsw, serversw, port, watch,
verbose, snoop, sasl, saslmech, user,
oauth_flag ? auth_svc : NULL, tls))
- || rp_isbad (retval = sm_winit (envelope, eai))) {
+ || rp_isbad (retval = sm_winit (envelope, eai, eightbit))) {
+ close (fd);
die (NULL, "problem initializing server; %s", rp_string (retval));
}
do_addresses (bccque, talk && verbose);
- if ((fd = open (file, O_RDONLY)) == NOTOK)
- die (file, "unable to re-open");
do_text (file, fd);
close (fd);
fflush (stdout);
sigon ();
if (!whomsw || checksw) {
+ /* Not sending message body, so don't need to use 8BITMIME. */
+ const int eightbit = 0;
+
if (rp_isbad (retval = sm_init (clientsw, serversw, port, watch,
verbose, snoop, sasl, saslmech, user,
oauth_flag ? auth_svc : NULL, tls))
- || rp_isbad (retval = sm_winit (envelope, eai))) {
+ || rp_isbad (retval = sm_winit (envelope, eai, eightbit))) {
die (NULL, "problem initializing server; %s", rp_string (retval));
}
}
}
+/*
+ * See if input has any 8-bit bytes.
+ */
+static int
+scan_input (int fd, int *eightbit) {
+ int state;
+ char buf[BUFSIZ];
+
+ lseek (fd, (off_t) 0, SEEK_SET);
+
+ while ((state = read (fd, buf, sizeof buf)) > 0) {
+ if (contains8bit (buf, buf + state)) {
+ *eightbit = 1;
+ return OK;
+ }
+ }
+
+ return state == NOTOK ? NOTOK : OK;
+}
+
+
/*
* SIGNAL HANDLING
*/