linux_log_viewer

A quite unglamous but realistic representation of what someone’s screen looks like when they’re analyzing hacker activity.

Q1: Is my site secure?

A1: There is no such thing as a website (or any internet-connected software system) secure from hackers. If this were true, there would not be nearly daily news reports of millions of credit cards stolen from major companies, medical records, and government data stored behind what most people believe are nearly impenetrable fortresses with multi million dollar IT budgets. The stack of software required to make a website work is so extensive – from the operating system the server runs on, the related software it uses, as well as the website software itself – that there is no way to know the security status of every piece. All of these applications are constantly being dissected for vulnerabilities by nefarious parties. Accepting this and acting accordingly is key to a solid security strategy.

Q2: What is the nature of most hacks you’ve seen?

A2: Most are automated and their purpose is to inject ads into sites pointing to outside URLs to manipulate the Google Adwords system or similar ad networks to make the hackers money. Occasionally we see “proof of concept” hacks which simply aim to demonstrate to the hackers friends that they can do it, but these seem less common now. Sometimes the purpose is to add the webserver to a botnet that can be utilized to send spam or perform other tasks as part of a network of compromised devices, which are sometimes sold as a service to others for activities like DDoSing. We have never had to handle a hack that was purposeful and targeted at a victim in such a way that the hackers indicated that they knew the target and intended to cause harm to them due to some kind of relationship or other issue they have with the site’s company. Hacking attempts are constant and omnipresent. They appear to be mostly automated attacks which work by probing websites located with the help of Google’s index or similar scanning tools for known vulnerabilities and then exploiting those vulnerabilities to post the advertisements or whatever modifications they intend to make.

Q3: What should I really care about?

A3: Because of A1, the primary thing you should care about is what data will be exposed if you are hacked, and the status of your backups. Secondarily, whether your software is up to date (covered further down.)

Q4: What are you already doing to make it more secure?

A4: First by taking a realistic approach to security. We realize that everything is vulnerable so we try to monitor everything as best we can, without forgetting that we cannot see everything. We run online and offline backups, since everything on the servers could be deleted at any time. We watch logfiles for trends or unusual activity, but we cannot see it all. We run various reports that extract and summarize large quantities of information that is impossible to collate by hand to spot irregularities, never forgetting that it’s impossible to watch all of the doors at once. We’re always trying to stay abreast what to look for. We run automated intrusion detection and blocking systems appropriate for the environment that we use, but we do not trust them to simply take care of everything, because they won’t. In summary – we’re doing everything we can to survive in a perpetual, unwinnable war between parties more powerful than any of us, simply due to their numbers – the armies of hackers versus the armies of software developers who make the hundreds (and possibly thousands) of individual pieces of software that are stitched together to powers everything we use. This attitude is critical order to accomplish the only goal that matters – maximizing your security by understanding the nature of the problem, and being prepared if it fails.

Q5: How can we make it more secure?

A5: We stress A1 because the primary factor is lack of realistic expectations. Beyond that, in our experience the primary factors in a typical website hack are low-quality passwords, outdated WordPress installations, outdated themes, unused themes, extraneous plugins, outdated plugins, lack of a security plugin, and lack of monitoring.

The primary defenses, aside from addressing those issues directly one by one, are 1. Backups and 2. Understanding and reducing the dataset that can be accessed if the site is indeed hacked. Almost all of the major hacks you read about in the news are due to a company-wide and really, nationwide) failure to accept that second tenet.

A common problem we see is that clients building website make feature requests without factoring in the security vulnerabilities that fulfilling that request will make by the nature of its requirements in terms of additional plugins which widen the site’s attack surface area. The goal should be to minimize this by reducing the site’s requirements to the least possible, therefore running the least amount of software possible to meet those requirements, thus reducing the surface area as well as reducing the maintenance work required to keep everything updated.

Remember that keeping your WordPress, all plugins, and themes up to date is important, but it will not make it impenetrable, because A1.

Q6: What is covered in my hosting plan?

A6: Installation and configuration of a security plugin is included, and is typically done on all WordPress sites. Some sites require special handling to make the security plugin work, and in those cases, there is additional cost. File-level change monitoring is a powerful tool – we will not monitor this for you, but we will set it up for you and teach you how at no cost. It is very simple but requires understanding some key concepts. Of course all the of things we are already doing covered in A4 are part of the service, but it is done at a group level for all sites and is limited in its ability to examine specific vulnerabilities of specific sites, given that they are constantly changing and the vulnerabilities available across all of that software is unknowable. Backups are automated and run daily, weekly, and monthly and are shuttled offsite periodically. Any work that must be done that is specific to your site, outside of these general things already done, either in response to a request from you or in response to observed hacker activity is not covered and is billed at the usual rate.

Q7: Do you do hack cleanups?

A7: Yes, even on other hosting environments, and occasionally on our servers. This is consulting work and comes at a cost because the amount of work will vary from hack to hack, and we are not responsible for the hack (because A1) therefore we cannot shoulder the cost of repairing it. Restoring backups is free as long as it doesn’t become excessive and repetitive, which is an option if you want to simply restore the site and see if the hackers go away on their own with no changes. This in fact does happen, but it’s not advisable to simply leave the door they used to get in open for them to return.

Q8: If my site gets hacked, what do we do?

A8: Call us first of course. What we normally do is make a snapshot to preserve and study, then restore a backup. Then we examine the modifications the hackers made to the site. Then examine the logs and other evidence to determine how they did it. Then we use the knowledge of how they did it to close that door, and ban every IP involved at the firewall level.

Q9: Why so rarely on your own servers? Are you better than others?

A9: Not saying that. We do not know whether we have simply been lucky, whether the security defenses we have in place work well, or whether there simply aren’t enough sites or the sites aren’t important enough to garner real attention. There is no way to know the answer to that question.

Q10: I have reason to believe we will be particularly targeted due to activity on other sites we own. What should we do?

A10: Aside from updating everything, reducing your attack surface area, and setting up the security plugin as discussed above in Q5, enable file-level monitoring and watch it. Most WordPress hacks will modify files that are part of your account’s filesystem and this will spot them. If any are unauthorized you should act immediately. Upon launch of the site, ongoing monitoring of http logs may be appropriate to precisely observe the site’s activity and manually deal with anything not being handled by the security plugin or other intrusion detection systems.

Q11: What about PCI compliance?

A11: For a period of time our servers were PCI-compliant and generally remain mostly so. However over time the requirements for PCI compliance, in our opinion, became unrealistic and therefore we withdrew from that program. It required locking down certain vital tools that simply couldn’t be locked down in the way they recommend while still being usable for our customers. Any failure to comply with any requirement of PCI is grounds for a failure status. However, most (virtually all) of what PCI intends to do is indeed helpful, and we do comply with much of it. Just not in any official capacity. While the things we do not comply with, in theory and in reality, do indeed expose us slightly more than a fully PCI compliant system, they are tradeoffs we are willing to accept – since even PCI compliance is no guarantee. Avoiding such self-delusion is key to our security strategy and therefore we are confident with this decision. Read more here if you’re curious about that subject.