Server load problems

[rjdrew1986]

I think this is a great idea. Antus, what money transfer account could we submit donation to help you?

[antus]

I'll send you guys a PM with my paypal. Whatever you donate I'll put towards hardware and hosting. If I get to much I'll use it toward subsequent monthly charges. If anyone else wants to chip in via paypal send me a PM. I don't want to post that in public. Thanks!

[immortality]

I'll chip in.

[AngelMarc]

Instead of upgrading the one, is a mirrored server in other countries a more reasonable option?

[AngelMarc]

I know you're tech savvy, but a lot of tech savvy people here that might have helpful suggestions.
https://forum.level1techs.com/
The owner of the site almost certainly understands more about AI than any of us do.
Side note, I like being able to ctrl+V an image into what I post. Just bam, inline attached image, done.

[antus]

Instead of upgrading the one, is a mirrored server in other countries a more reasonable option?

No. Its prohibitively expensive. The reason there are no adds, no trackers, and no bullshit on the site is because it has been so cheap for outgoing expenses and only my own time to maintain it for about the last 15 years. First public post still available: viewtopic.php?t=3

It's certainly technically possible but with that comes another level of complexity and another level of cost to scaling like that. Solutions can be bought (such as AWS RDS) for an easily scaleable DB but from my commercial experience with those they are very expensive to run, very quickly on loaded databases. Further, I enjoy looking after this site. My career is less technical than it used to be but this site keeps my skills sharp.

For anyone wondering the old - 1.7Gb RAM for OS, DB and Web, 2 CPU cores (older xeon).

[antus]

I made some temporary changes earlier tonight because the forums were in a bad state. So far, so good. Though the number of users is right down. Maybe a bunch of you Aussies gave up for the night while I was working on it. We'll see what happens at peak time tomorrow, and assuming all good look at the permanent fix early next week, which will probably look like 1 more short outage.

[AngelMarc]

No idea if this is actually helpful.

Capture.PNG

[antus]

Thanks. Cloudflare is reporting only 2% of traffic is coming from cache. The database is elsewhere now (temporary location, with a permanent one comeing later) and it has more cores and more RAM and its looking much healthier. No outages since that change, over 40,000 unique users in the last 24 hours, and DB load not passed 8% of a possible 800%. Where it lands on the new hardware will be different, but it seems we are in a good place, and my current theory is that we were actually memory limited which reduced the DB performance, so the forums responded too slow, meaning more connections built up waiting for the DB, which then compounded the RAM problem, causing the DB to run very inefficiently and run out of CPU. But a bit more headroom and its all smooth again.

Screenshot 2025-07-18 101501.png

I think the spike in cacheing is when the site had closed down under load, and it was just the same static page going to everyone. Now its running properly everything is dynamic so it can only cache images.

[vlad01]

I know you're tech savvy, but a lot of tech savvy people here that might have helpful suggestions.
https://forum.level1techs.com/
The owner of the site almost certainly understands more about AI than any of us do.
Side note, I like being able to ctrl+V an image into what I post. Just bam, inline attached image, done.

Ahh good old Wendel!

Although I never worked with enterprise stuff, I still watch his videos every now and then, very insightful into the poorly covered aspects of hardware these days.