server is too busy

[illflem]

After quite awhile with no server is too busy, it is back with a vengeance tonight. What's up?

 

[The patriot]

Yea, I'm getting this all together too much too!!

I dont have all night to be on this thing, and if I get off, all the posts I havent read will be marked that I did.

It's a lose lose situation. #@$%! #@$%!

Maybe (if even it can be done?) can the powers that be, make the board not accept non members if it gets that busy?

I dunno, just thinking out loud?

Eric

 

[paccool]

I keep getting the same message.



Stan

 

[Steve St.Laurent]

We're working on it guys. I was on a four way conference call earlier tonight with Ken, Robin, myself, and an internet host provider. We asked Ken to run a file server log analysis to see where the loads are coming from on the site so that we can make an informed decision on where to go with the next update. Depending on where the loads are coming from on the site will drive what solution makes the most sense. Right now they are looking at solutions that cost up to $3000/month for the hosting and use up to 3 servers to provide that service. Right now the server is putting out close to 400GB/month which is a HUGE load - that's approximately six times the amount on the ford-diesel.com site for a comparison, btw. Just running detailed server logs to see where the usage is at will take several days because of the amount of traffic on the site. Running those logs will increase the load on the server and will possibly cause more "server busy" messages but it's what we need to make an informed decision on what solution to use to resolve the problems. Please have patience, we're working on it hard - I spent a couple of hours on the phone just today talking with Ken and Robin along with different providers on a solution. We just want to make sure that whatever route that's taken that it will actually solve the current issues as well as have a growth path in the future - the site's traffic level is growing continuously. That is great but it does cause problems - what will solve the issues today may not tomorrow. We want to be sure that whatever solution is decided on will be able to grow with the traffic level of the site for some time.



-Steve St. Laurent

Lead Moderator

 

[BigUgly]

Cool. Thanks Steve.

Josiah

 

[paccool]

Thanks for the explanation Steve. I'll keep quiet about "server busy" for awhile.



Stan

 

[WBerry]

its coming up busy again. bill

 

[Steve St.Laurent]

It's going to take a few days to run the logs, please be patient.



-Steve

 

[paccool]

Sunday evening here on the west coast. Took forever to get on the site as the "server was too busy".



Come on guys, a week ago you were running logs to see what the problem was. I said I wouldn't bug you for a few days about this. Tonight was as bad as I've seen it. I could get on the classifieds but not into the forums. There is less than 200 people on the site, much less than the record.



When is this going to be fixed? Is it time for a new server? I don't know anything about this set up but there are lots of web sites with many more visiters than this and they don't have the problems you folks are having.







Stan

 

[Steve St.Laurent]

From above:

We asked Ken to run a file server log analysis to see where the loads are coming from on the site so that we can make an informed decision on where to go with the next update. Depending on where the loads are coming from on the site will drive what solution makes the most sense. Right now they are looking at solutions that cost up to $3000/month for the hosting and use up to 3 servers to provide that service. Right now the server is putting out close to 400GB/month which is a HUGE load - that's approximately six times the amount on the ford-diesel.com site for a comparison, btw.

The log analysis was STILL running on friday and I've been out of town over the weekend. Regardless of that, server busy messages will still come up occasionally until we make a move from what those log file results are. Running the logs simply made the problem come up even more often than would otherwise happen. The problem is two fold - bandwidth usage (400 Gigabytes per month and up to 2. 5 Megabytes per second is a LOT of data to move). The second issue (and the bigger one) is that there is a bug between MySQL and Linux that causes servers to crash on high usage sites. As I said above we are looking at options using up to 3 servers on two different OS's (database server on FreeBSD to fix the bug issue, 2 web servers on Linux for compatibility and to retain the current user file so EVERYONE doesn't have to re-sign up). I promise that we are working HARD on trying to find a solution for the problem, but we can't make a rash decision on this.



At the level the TDR is dealing with 99. 9% of the vendors want a 1 year contract. What if the TDR signed a 1 year contract with a vendor and after a month found they couldn't handle it? Also, at this level costs can vary greatly depending on the vendor (up to double). Some of the concerns are future growth possibilities (traffic doubled in the last six months, so in a year it could be 4 times what it is today), the level of redundancy at the vendor (multiple lines running from multiple directions as well as different line vendors). Costs are not easy to figure out either, some vendors are low on the server lease fees but expensive on bandwidth usage; others have high server lease fees but cheaper on bandwidth. We're also looking at co-locating hosts where the TDR will have to supply the hardware at their site and they will just supply the line.



I will be talking with Ken and Robin tomorrow regarding the results from the log analysis and a decision will be made this week as to where to go with the site - we needed the details of the log analysis to make that decision. It will then take 1-3 weeks to physically make the move, depending on the solution. I'll again ask for your patience please.



-Steve St. Laurent

Lead Moderator

 

[paccool]

Thanks for the reply Steve. I'll shutup and crawl back into the corner for a while!



Stan

 

[illflem]

Thanks for working on it folks. When you consider there are 12,000+ members out of the 300,000 engines Cummins sold Dodge (4%) it only makes sense to plan for the future.