[Info-vax] TCPIP Services IMAP and POP resource consumption

[Rich Jordan]

On Wednesday, February 3, 2016 at 1:25:00 PM UTC-6, Rich Jordan wrote:
> An older off-support VMS sites is still using their Alpha DS15 for email (no budget for replacement).  VMS V7.3-2, TCPIP V5.4 ECO 7 and can't update.  About 15 remaining users have it as their mail server, 12 POP and 3 IMAP.
> 
> KZPEA U160 SCSI with dual 10KRPM 73GB drives (all mail / user directories are on the data drive).  768MB RAM, huge pagefiles.
> 
> It worked well for years.  The only other thing still running on it is Apache for one static website that gets 10-20 hits per day and is almost unmeasurable in resource usage compared to what is coming.
> 
> Lately the system is getting eaten alive between POP and IMAP.  As users are let go or move to alternate mail systems, their mail folders are ZIPped so no longer visible as mail to the IMAP/POP servers (if they check).  
> 
> The disks are not fragmented.  Split I/O rate is minimal.  
> 
> We run weekly cleanups (on POP users only) that stage mail out of 'NEWMAIL', to a holding folder, and delete it after 60 days because we've had too much trouble with outlook, outlook express, and livemail cleaning up up after themselves (Eudora works perfectly).
> 
> IMAP users are set to only sync back 14 days.  POP and IMAP are on 10 minute check intervals but with current behavior, even getting one email can take that long so I'm not sure if they disconnect (turned up the logging, haven't had time to check).
> 
> During the day the system is at constant 50% CPU, about 2/3rds IMAP, 1/3 POP.  Direct I/O rates are through the roof and constant, 5-600 for IMAP, 4-500 for POP.  At time the buffered rate goes over 1900 with POP leading the charge at 1800+.  It can take two or more minutes to even log on to the POP server during the day.
> 
> I ran a defrag on the disk (image backup/restore).  I've gone through all the active users mail files and done purge/reclaim/stat and compressed the mail files, I've used one of the old DECUS utilities that went through the mail file and the mail$*.mai mail files and 'cleaned up' any inconsistencies.  Nothing seems to make any difference.  It definitely did not used to do this even with 28 (max) email users and higher mail volume in the past.
> 
> The error logs show nothing relevant.  The LAN connection is fine and error free (100Mbit FDX, no mismatch).  The POP and IMAP logs don't show anything since we upped the timeouts on the clients to several minutes (before that we'd see the client drop the connection attempt)
> 
> Any thoughts?  I had suggested putting MX on it as a workaround (and now maybe Ruslan's IUPOP since its been updated) but can't get the go ahead without some indication it might help.
> 
> And no, there's no budget for replacement (and yes I think I brought an earlier phase of this up here before) but the system is just thrashing now and I can't seem to find out the base cause.  Over the next 6 months the remaining users will be migrating off to a cloud email service but in the meantime I've got to keep this system running.
> 
> 
> Thanks for any thoughts/suggestions.

Thanks all

The system is on a LAN, protected by a firewall.  We had run checks a couple of months ago when the problem was not as bad; no sign of external intruders (or internal); haven't had time to repeat it but we may have to.  If there's a rogue element on the LAN, then its a recent thing.

That last time we set up a mirror port on the switch the Alpha is plugged into and collected packets for 20 minutes or so.  Other than the Alpha being slow to respond (not /delay_ack slow, worse, and TCP was set to /nodelay_ack long ago) we didn't see errors (according to our network guy who knows this stuff)

However my directory just finished and what do you know; Dan was right.  One account (an IMAP, and unfortunately the owner of the company) has grown his stash of *.MAI files to 28000+.  Quick check shows he was and is getting spambombed but apparently never told us.  I'll work on getting that cleaned up tonight if they will let me, and see what happens.  His is the one IMAP account we are not allowed to automatically clean up...

Thanks Dan!  I definitely should have thought of that.

Next closest has about 4000.  Anyone on POP shouldn't have that (some users though still have email from back when they used VMS mail interactively, and its still in their folders; time to archive it out into a ZIP and get rid of the excess).