Hello,

John Summerfield wrote:
> Eduardo Bach wrote:
> > Hello to all,
> > 
> > One of our servers was invaded. We just started the investigations, but the
> > main clue, plus some strange files copied and deleted, is that sshd binary
> > has changed. Its original size was ~313KB and moved to 1.18Mb. His version
> > was 3.9p1-11.e4_7. As we at the beginning of investigations, I wonder if
> > anyone had similar problem, or have any clue on how the intruder may have
> > entered?
> > Thanks in advance.
> 
> Almost certainly a password. Since the sshd binary was changed, it's 
> clear someone (not very expert!) obtained root access.
> 
> Best practice is to shut the system down immediately, to allow forensic 
> examination. Anything you do to repair the system, or even just to use 
> it, obscures the evidence of what was done and by whom. It's like 
> trampling the footprints outside the window!
> 
> Note that a clone of the disk isn't sufficient for forensic examination; 
> properly equipped, one can find contents of deleted files and even 
> overwritten sectors, and that information is lost when a disk is cloned.
> 
> Best practice is also to discard this server image, and to build a 
> replacement. Use care in copying any files, especially compiled programs 
> and scripts.
> 
> This is general advice, not particular to your circumstances.
> Google for best practice on securing the system, but these are good to 
> start with:
> 1. Disallow root login through ssh. Preferably, disable root logins 
> entirely, use sudo and/or su to do root things.
> 2. Use passwords that are hard to guess. "john" is bad, especially if 
> it's your son's name, "john#claudius" is better (takes a good dictionary 
> and a lot of guessing), some prefer completely arcane passwords such as 
> K2~Ouu3sk.
> 
> I prefer something I can recall and that's far down any dictionary 
> account, say "redcucumber." It's less likely to be written down than 
> something like "K2~Ouu3sk."
> 
> 3. Make it hard to enumerate accounts with login access: if you use 
> /etc/passwd to store accounts for email, ftp etc than give them a bogus 
> home directory (maybe /var/empty) and a shell of one of these: 
> /bin/{true,false,nologin}.
> 
> 4. Require use of keys, not passwords, with ssh. If my sshd 
> configuration does not allow the use of passwords, you can't use my 
> account even if you know my password. Keys can be protected with 
> passphrases.
> 
> A key is something you possess (it might be on your computer, or on your 
> USB drive), a passphrase is something you know: a bit like a POS card 
> and PIN.
> 
> 5. Limit the rate at which remote users can connect to ssh. I only allow 
> one ssh user (me), and there aren't that many places I might need to 
> login from, so rules like these are useful:
> [root@sysadmin ~]# cat /etc/sysconfig/iptables | grep -w 22
> -A INPUT -s 58.6.0.0/255.254.0.0 -p tcp -m tcp --dport 22 -j ACCEPT
> <snip>
> -A INPUT -i eth0 -p tcp -m tcp --dport 22 -m state --state NEW -m limit 
> --limit 5/hour -j LOG --log-prefix "SSH connexion "
> -A INPUT -i eth0 -p tcp -m tcp --dport 22 -m state --state NEW -m limit 
> --limit 5/hour -j ACCEPT
> -A INPUT -i eth0 -p tcp -m tcp --dport 22 -j LOG --log-prefix "SSH 
> connexion attack dropped "
> -A INPUT -i eth0 -p tcp -m tcp --dport 22 -j DROP
> 
> The basic idea is there are no restrictions on some areas connecting, 
> but just in case, I allow a small number of connexions from other 
> places. In practice, this has worked well for me, but wouldn't protect 
> me from a geographically-disperse attack. But, if someone's that 
> determined, selecting another port wouldn't help either. I find I block 
> 90% or so of connexions from outside my preferred zone: anyone making 
> more attempts from a single IP addres just increases the number of 
> attempts I block.
> 
> It's also possible (and prudent) to block some areas entirely: China, 
> Korea and USA come to mind
> 
> _I_ also restrict outgoing ports, if you visit me you might find your 
> use of port 2323 (a random example) blocked.
> 
> 
> I have seen a couple of crackers' tools; one sent the IP address of eth0 
> to an email address at one of the free email sites, another installed an 
> IRC bot and proceeded to try to crack other sites.
> 
> The IRC bot was licenced under the GPL and the criminal was fully 
> compliant, he gave me the source.
> 
> The first had root access, and promptly installed replacement binaries 
> for common programs (typical behaviour), Fortunately, they didn't work, 
> and so it was pretty trivial to spot that the server was broken (it 
> didn't boot).
> this command helped me see what was done.
> chattr +a .bash_history
> 
> The second criminal used ftp to enumerate accounts, I suspect many 
> sysadmins don't read their ftp logs.
> 
> 6. Configure syslog to log to another computer. Ideally, the other 
> computer doesn't run any other services, and is inaccessible from 
> everywhere dangerous.
> 
> 7. Some people recommend port tapping. This is a bit like knocking 
> (perhaps tapping out the first bar of a favourite melody) at the front 
> door, where the arrangement is that the back door is then opened. There 
> are various schemes and programs, including one that involves capturing 
> incoming IP traffic (like tcpdump) and looking for a magic code, and 
> then opening a port. See google.com and freshmeat.net for more ideas.
> 
> 8. As I noted elsewhere, I don't recommend simply using a different 
> port. A portscan will find it quite easily, and you won't be as secured 
> as you might think.
Portscans could be eliminated -- or at least made very difficult -- by
using portsentry (sourceforge.net/projects/sentrytools). I simply put
ssh from port 22 to 1234 (say), and arm portsentry to listen on 22 (and
others, where common services are usually listening, eg. telnet, ftp,
smtp, finger...). If someone does a portscan, portsentry immediately
locks that ip out (with iptables or route or hosts.deny). Watch out
however, as source ip addresses may be spoofed, so a "legitimate"
computer can also become blocked. I suggest to whitelist some hosts (eg.
your desktop machine on the LAN), in case you get locked out
accidentally.

Sincerely,

Andras Wacha

> 
> 
>