site trouble ?

[FourBarR]

Good Morning Steve. Is the site running really slow or is it only me. I have high speed connection and have never seen it like this.

 

[Gary - K7GLD]

Extremely slow here today also - other sites are fine...

 

[Steve St.Laurent]

Yes, there is a major problem. A hacker has been able to inject some code onto the server to send out SPAM emails that I have been unable to find. Last night I shut down sendmail as a stop gap measure so that our IP wouldn't be shut off by the host for violating our TOS. I went to bed at 4am having still been unable to find it and shortly thereafter processes started piling up and bringing the server to it's knees. I just rebooted the server and I'm going at it again looking for it.

-Steve St. Laurent
(A very weary webmaster)

 

[Gary - K7GLD]

AH, the delights of being a site administrator - how I miss those sleepless nights, and rapid trips from family vacations and outings to repair whatever was busted this time...



Thanks Steve, - hope you find it all OK!

 

[JasonCzerak]

You do have a Tripwire /AIDE report generated every so often don't you?



Then again, PHP has never been know for it's security.



Both servers infected?



Maybe rebuild Apache and php and re-deploy.

 

[TRLRTRSH]

Hang in there Steve, Thanks for your work!

 

[bighammer]

You do have a Tripwire /AIDE report generated every so often don't you?

Not sure about that. I think he just uses toothpicks under his eyelids and stays on it 24/7.

 

[Grizzly]

Thanks Steve, You do good work.

 

[Steve St.Laurent]

Found it!!Oo. Oo. There was a vulnerability in a sitemap package for vBulletin that we actually weren't even using any more - it had been replaced with a new one when I installed vBSEO (the software that changed the url's to content driven on posts if you hadn't noticed). I had left it in place while the new pages were crawled. The vulnerability had been discovered back the end of May and I now have the pages where I found the vulnerability bookmarked to watch for others. Sendmail will be back up and operating momentarily. The good news is it didn't inject any code onto the server it just ran this code from their server on ours and it was an email form that allowed them to send out spam emails using our server. I'll be monitoring the site closely to make sure that the one I found was the actual problem but I'm 99. 99% sure it is.

-Steve St. Laurent
(a VERY relieved webmaster :-laf )

 

[Steve St.Laurent]

You do have a Tripwire /AIDE report generated every so often don't you?



Then again, PHP has never been know for it's security.



Both servers infected?



Maybe rebuild Apache and php and re-deploy.

Yes, in this case it actually wasn't even on our server. The vulnerable script was allowing code to be run off someone else's server. When I disabled sendmail that's what brought the server to it's knees because their script was replicating over and over loading it up.

 

[PRout]

Its amazing how many sickos are out there. . Thanks Steve. Nice Job

 

[JasonCzerak]

Its amazing how many sickos are out there. . Thanks Steve. Nice Job

More so, it's amazing how much bad software is out there. Have you tried to be productive with a windows desktop lately?

Tho, people putting that energy into more productive things like, I dunno, planting a tree or getting a second job wouldn't hurt either.