Ken Hornstein [Thu, 18 May 2017 01:00:47 +0000 (21:00 -0400)]
Switch from using script(1) to a new custom utility, runpty. It turns out
on some systems script(1) has a bug that results in hangs that is simply
too difficult to test for, and this custom utility is simpler.
Ralph Corderoy [Wed, 17 May 2017 14:15:18 +0000 (15:15 +0100)]
uip/mhbuildsbr.c: Fix long-line truncation when BUFSIZ is 1024.
a23477eb changed one char array from BUFSIZ to NMH_BUFSIZ elements, but
the long line read into it was passed to user_content() that
strncpy(3)'d it, silently truncating, to another char array, still
BUFSIZ long. This show up on platforms where BUFSIZ is 1024 versus
NMH_BUFSIZ's minimum of 8192.
Ralph Corderoy [Wed, 17 May 2017 11:58:15 +0000 (12:58 +0100)]
test/mhbuild/test-cte: Move run_prog() to just before check().
Prepare the input and output files, then run the program and check the
result. Makes it easier to see every run is being checked if a,
sometimes large, expected-output file isn't being produced in between.
Ralph Corderoy [Wed, 17 May 2017 11:45:26 +0000 (12:45 +0100)]
test/common.sh.in: Clarify test name in failure message.
Some test names don't stand out as that when suffixed with `failed'
amidst all the other output. Change `foo failed' to `first test
failure: foo' instead. Also makes clear there's possibly other tests
that failed later.
Ralph Corderoy [Tue, 16 May 2017 10:35:49 +0000 (11:35 +0100)]
Merge sbr/norm_charmap.c into sbr/check_charset.c.
Alter norm_charmap() to be static now its only caller is in the same
file. I've checked the upstream cam.ac.uk source and it's still the
same and hasn't changed in years, so I don't think there's much benefit
from having it standalone to ease checking.
Ralph Corderoy [Tue, 16 May 2017 10:29:17 +0000 (11:29 +0100)]
Part 1 of merging sbr/norm_charmap.c into sbr/check_charset.c.
This commit won't compile, but I'm attempting to get the rename into git
so the history after the next, merge, commit will continue through both
parents.
Ralph Corderoy [Mon, 15 May 2017 17:06:19 +0000 (18:06 +0100)]
uip/mhshowsbr.h: Move in declarations from h/mhparse.h.
Delete non-existent markerform parameter from show_all_messages()'s
comment. Alter show_all_messages()'s definition's concatsw parameter to
match comment and prototype.
Ralph Corderoy [Mon, 15 May 2017 16:14:01 +0000 (17:14 +0100)]
uip/mhshowsbr.c: Create header file with exports' declarations.
Create a single set of externs for mhshowsbr.c's global variables. Have
the other users of those use the new include file instead of their own
declarations.
Ralph Corderoy [Mon, 15 May 2017 15:09:28 +0000 (16:09 +0100)]
uip/mhcachesbr.c: Move exported symbols to existing h/mhcachesbr.h.
Remove the user's copies of the declarations.
A static struct swit caches[] was declared everywhere that included
h/mhcachesbr.h. The larger number of includers causes `variable not
used' for `caches'. Move that declaration to uip/mhcachesbr.c and have
an exported `cache_policy' symbol that points to it for the existing
users.
Ralph Corderoy [Mon, 15 May 2017 13:40:52 +0000 (14:40 +0100)]
sbr/m_mktemp.h: Create with missing prototypes.
Some of sbr/m_mktemp.c's functions didn't have prototypes in a header
file so callers were declaring prototypes themselves. Have them include
the new header file instead.
Ralph Corderoy [Mon, 15 May 2017 13:23:53 +0000 (14:23 +0100)]
uip/picksbr.c: Use function prototypes for `nexus' functions.
Prototypes for ORaction(), etc., now state the parameters they expect.
Removed the args() macro, instead explicitly listing the arguments being
passed to functions.
Ralph Corderoy [Mon, 15 May 2017 13:12:21 +0000 (14:12 +0100)]
Specify function parameters in prototypes, mainly void.
These prototypes are all in *.[cl] source compared with the *.h of the
earlier commit. All but one had void added, that one's parameter was a
char pointer.
Ralph Corderoy [Sun, 14 May 2017 18:35:13 +0000 (19:35 +0100)]
sbr/dtimep.l: Remove redundant wrapping parenthesis in definitions.
It's been decades since a lex(1) didn't follow POSIX and treat `{foo}'
as parenthesis surrounding foo's definition. Other nmh lex files work
without the extra wrapping.
If the profile component nmh-storage was a single character, e.g. the
relative directory `d', then it would be dropped from the output path so
instead of `d/foo' being written, `/foo' would probably fail. I think
this was due to a faulty check for nmh-storage being `/'. Add a test
for a non-/ single-character nmh-storage.
Ralph Corderoy [Sun, 14 May 2017 10:23:33 +0000 (11:23 +0100)]
Use new PLURALS(n) macro instead of variety of tests.
Whether to output a plural noun in a message was decided by ternary
operators that tested n==1, n!=1, n>1, etc. Make them all consistent by
using PLURALS(n).
Ralph Corderoy [Sat, 13 May 2017 17:39:06 +0000 (18:39 +0100)]
uip/folder.c: Rewrite plural test to common form.
The common form is a choice between "" and "s". This one occurrence was
using " " and "s" so either output was a fixed width. Alter the
printf(3) format string from `%s' to `%1s' to provide that instead.
Allows the new code to be altered by an upcoming `plural' change.
Ralph Corderoy [Sat, 13 May 2017 17:35:15 +0000 (18:35 +0100)]
Makefile.am: Alter long lists to be sorted, one entry per line.
It was hard to see what files were included in some variables when
searching for the best place for new content. Some variables had
definitions that were almost sorted, by weren't. Switch to a
one-entry-per-line list, in `LC_ALL=C sort' order, documented at the
start of the file. Makes it easy to run through the lists and see the
patterns.
The space padding was added at the start of the output buffer, not the
start of the component being formatted. Caused by 92128dac's move to
dynamic allocation for fmt_scan()'s output. Only shows if the component
isn't at the start of the buffer. Expand existing
test/format/test-rightjustify to cover this.
Ralph Corderoy [Tue, 9 May 2017 22:10:06 +0000 (23:10 +0100)]
config/version.sh: Rewrite. Use uname(1), git-describe(1), and UTC.
Not sure why it searched through PATH manually for uname(1) and
hostname(1). uname and its -n option are POSIX so just use those. It
was the preference over hostname anyway.
Use git-describe(1), not just git-branch(1), as it gives more detail,
including --dirty to show the built source differs from the commit.
Specify the format for the build date, and its timezone; +0000.
Ralph Corderoy [Sun, 7 May 2017 17:02:03 +0000 (18:02 +0100)]
uip/picksbr.c: Increase line-buffer size for "grep" action.
Alter LBSIZE from 1024 to NMH_BUFSIZ, e.g. 8192. This dominates the
size of the char array used to hold a single unfolded header line when
evaluating a "grep" action, e.g. `-to foo' becomes `^to[ ^I]*:.*foo'.
Ralph Corderoy [Sun, 7 May 2017 13:01:48 +0000 (14:01 +0100)]
uip/scansbr.c: Replace uses of SBUFSIZ with NMH_BUFSIZ.
Now that the former is defined as the latter, the uses of SBUFSIZ were
all in the same expression and it simplifies from a ternary expression
to NMH_BUFSIZ.
Ralph Corderoy [Sun, 7 May 2017 12:57:39 +0000 (13:57 +0100)]
uip/rcvdist.c: Replace SBUFSIZ with NMH_BUFSIZ.
The former was defined as the latter and had only one proper use: to
size a char array. The other uses should have been the sizeof operator
on that array, and now are.
Ralph Corderoy [Sun, 7 May 2017 12:03:01 +0000 (13:03 +0100)]
Make many m_getfld() buffer parameters NMH_BUFSIZ big.
char arrays were often stdio.h's BUFSIZ large, and that's 8192 with here
with glibc 2.25-1 on Linux x86_64, so using NMH_BUFSIZ is no different
as that's max(BUFSIZ, 8192). But some were 256 or 512 with a local
SBUFSIZ macro and they caused scan(1) to truncate a field, or pick(1) to
not spot text because the field was truncated.
Delete bad-input/test-header's test for m_getfld()'s detection for a
header without a colon that's under the header-length NAMESZ limit, but
longer than the passed in buffer. This no longer happens in scan as the
buffer is larger than the longest allowed header.
Ralph Corderoy [Fri, 5 May 2017 12:11:15 +0000 (13:11 +0100)]
configure.ac: Enable assert(3) by default.
Revert 167e542b that disabled assert(3) by default after private email
discussion with David Levine. I suggested it being enabled by default
in git so those running nmh from git were testing them for us, with it
being disabled as part of the release process so packagers continue with
it disabled. David pointed out that would make the release
intentionally different from what had been long tested so the default is
now enabled assertions everywhere unless disabled by a packager.
Ralph Corderoy [Fri, 28 Apr 2017 21:08:26 +0000 (22:08 +0100)]
sbr/oauth.c: Remove const to avoid curl's debug_callback warning.
The curl_debug_callback typedef doesn't use const for any of the
function's parameters. On one of the compilation platforms here,
oauth.c's debug_callback having const specifiers for some parameters
causes compilation warnings, and -Werror stops the compilation. Remove
them.
Ralph Corderoy [Fri, 28 Apr 2017 16:12:31 +0000 (17:12 +0100)]
sbr/folder_read.c: Don't malloc() once per message.
Instead of struct msgs having a pointer to a malloc'd array of pointers,
each to a malloc'd struct bvector, 1+N, it now has a pointer to a
malloc'd array of struct bvector; one malloc for all of them. This
avoids the large number of calls to malloc() and free() that's linear
with the size of the folder.
But there are some downsides. In order to step through an array of
struct bvector, code outside of sbr/vector.c needs to know the struct's
size. The simplest way to do this is to make the struct's definition
public, with a comment that access should be through vector.c.
New functions are needed to initialise the content of an already
allocated bvector, and to finish with its content prior to deallocation.
bvector_create() and bvector_free() now also use these new functions.
Before, it was the array of pointers to bvector that would be realloc'd.
That doesn't work for the array of bvectors as they may contain pointers
to within themselves. The solution is to malloc a new array and
bvector_copy() the ones to keep across, as folder_realloc() now does.
The other half of its logic that coped with growth at the end of the
array, has been deleted. Also deleted, is the code to clear the
bvectors before and after the old ones as they start in that state.
Ralph Corderoy [Fri, 28 Apr 2017 11:45:26 +0000 (12:45 +0100)]
sbr/vector.c: Only allocate bvector storage for set bits.
Now that a pointer to all the bit storage doesn't escape vector.c,
there's no need to allocate storage, that's initialised to 0, just to
then clear a bit in it. Only extend the bits's storage for set bits.
Remove the unused initial size parameter from bvector_create() as it
simplifies the implementation.
Ralph Corderoy [Wed, 26 Apr 2017 23:14:28 +0000 (00:14 +0100)]
sbr/vector.c: Zero the growth with memset(3), not loop.
When the resize functions grow the vectors, as they always do, zero the
new slots with memset(3) rather than a for loop. Particularly of note
for the bit vector where it was bvector_clear()ing one bit at a time.
Although a NULL pointer needn't have a representation of all-zero bits,
the code was already assuming that, e.g. on the initial allocation, so
we're no worse off.
Ralph Corderoy [Wed, 26 Apr 2017 12:41:27 +0000 (13:41 +0100)]
sbr/vector.c: Change bvector_bits() to return first word.
Rename it to bvector_first_bits() to represent its new behaviour. It
has only one caller that uses it to produce debug. Tighten the API so
other callers don't get access to the location of the bvector's bits.
Ralph Corderoy [Tue, 25 Apr 2017 22:21:21 +0000 (23:21 +0100)]
sbr/vector.c: Embed initial vector storage in header struct.
Instead of a malloc(3)'d struct bvector having a pointer to a separately
malloc'd area for the bits in the normal case of the default initial
size, have non-malloc'd storage in the struct itself, and set the
pointer to that. It's two unsigned longs, which is less than the
previous default of 256 bits for struct bvector, but still double the
pre-bvector norm of one word on 32 and 64-bit architectures.
This halves the mallocs needed to create a struct bvector in the common
case, but does mean that embedded memory is wasted should it not be
enough. That's probably an unusual case. It also means derefencing the
pointer to the bits probably hits the same cache line.
Have separate initial sizes for string and int vectors.