[Info-vax] ssh dictionary attacks, DDoS (was: Re: Anti-virus ?)

[Stephen Hoffman]

On 2023-08-14 21:10:31 +0000, Hunter Goatley said:

> On 8/14/2023 8:39 AM, Simon Clubley wrote:
>> 
>> Every so often, Eisner's network services (including SSH) simply stop
>> working. Sometimes, basic stuff such as ICMP continues to work, but
>> anything involving process creation is utterly stuffed.
> 
> Lately, it's been a problem of EISNER seeing an unprecedented (per 
> EISNER's history) level of dictionary attacks via SSH and SMTP. I've 
> had to increase quotas for MultiNet's Intrusion Prevention Service 
> process to try to keep up with the events. Each time, I've thought, 
> "Well, that should be enough," and then the number of attacks grows, 
> and it's not.

Put a three or so second delay ahead of each ssh connection prior to 
the password processing, and put a five or ten second delay after a 
failed password, and then a delay again before dropping the connection 
when disconnecting from a failed login.

Make the delays adjustable via configuration file or via (gag) logical 
names or such, if following OpenVMS app configuration UI norms.  I've 
met a few of these that build the delay within the text shown while 
stalling, so the characters will dribble back to the originating host.

Adding support for fail2ban into ssh would be a nice addition if not 
already present, but adding it is probably more work than adding 
delays, and less able to handle botnet brute-force and DDoS shenanigans.

Allowing the delays to be region or country specific is another 
longer-term option, if there are lots of problems in some regions and 
some blocks, and fewer in others.

Basically, adding greylisting, and tarpit support.

> Something in all of that is eating up paged memory, and when the system 
> runs out of that, pretty much everything stops, and the system has to 
> be rebooted.

That's usually either a resource leak, or resource exhaustion when 
things get too busy and all this as you are well aware, of course.

> I thought EISNER was getting hit hard before the recent relocation, but 
> the number of SSH and SMTP connections trying bogus usernames or trying 
> to guess passwords has shot up dramatically since the relocation. 
> Apparently, EISNER's new IP address makes it a bigger target than the 
> previous address for some reason.

The mail server should probably check the incoming mail server 
connection DNS for DANE or SPF or such, and force incoming connections 
to STARTTLS, and quite possibly add an RBL check.

This does block user connections via TCP port 25, but that's normal for 
most mail providers.

There are a half-dozen or so settings in POSTFIX related to this 
anti-spam and related processing that can really slow the malicious 
traffic. (I'd expect OpenSMTPd has some similarities, but haven't had 
the opportunity to implement that in production.)

> Over the past three days, over 21,000 IP address filters were 
> automatically created in response to the attempts. That's not the total 
> number of connections, just the connections that triggered IPS to 
> create a filter. While I was checking that number, I saw five more get 
> created in the 20 seconds I was looking.
> 
> If I could block certain countries, a lot of the problem would be 
> alleviated. But that doesn't really work for a system like EISNER, 
> which aims to be open to everyone.
> 
> So we learn, adjust, reboot, and repeat.
> 
> Oh, and since EISNER is no one's full-time job, that process is taking 
> longer than it might otherwise. I sometimes see that EISNER is not 
> answering before anyone else---but not always.

Another option I've used—may or may not be an option here, and that for 
various reasons—is to relay incoming and outgoing messages via whatever 
mail server VSI is using, depending on the anti-spam and related 
capabilities of the VSI mail server. (Yeah, my suggestion around all of 
this reply including using relay is probably impolitic here, and, yeah, 
VSI might not want to "share" their mail server.)


-- 
Pure Personal Opinion | HoffmanLabs LLC