Error on reporting a thread - And Other Site Issues

[2kaud]

I wanted to report a thread to ask the mods to move it but I get this error

Error 503 first byte timeout

first byte timeout
Error 54113

Details: cache-chi-kigq8000076-CHI 1708190750 2941714729

Varnish cache server

[Steve R Jones]

ALL of our eleven sites are experiencing "an issue."

Some thing like Edit Post - Reply with Quotes are not functioning well or not at all. There are also some issues with image loading... It's been reported but I don't thing anyone will look at it till Monday.

[Steve R Jones]

The saga continues. Each of the eleven sister sites have similar and or different issues....

Some site - Avatars are missing.
Buttons are missing but still might be functional.

[VanGoghGaming]

Just out of curiosity, what are those sister sites you mentioned?

[Elroy]

Yeah, here are the problems I'm having:

• When I attach a ZIP file and then download it, I get a file that's exactly the same size, but it's a VERY different file (when check with a binary file comparison).
• When I try and include a picture (Advanced Edit), the page goes "gray" and completely hangs.
• I typically use Chrome, but on either Edge or Firefox, all the editing toolbar buttons are completely missing. They're there on Chrome.
• Weird characters appearing in my signature blurb.
• All kinds of weirdness on the "attachment" popup window.

[Elroy]

I just added two more bullet points to my above post, so this is really a bump.

But, since all of this appeared at once, I'm thinking it's all got the same root cause. The fact that my signature blurb is fouled up makes me think it could be rooted in some kind of Unicode translation problem. To my way of thinking, that could certainly cause all these problems, especially if the primary source code of these forums is PHP, which it almost certainly is.

Also, I'm somewhat confused over why they just don't do some kind of roll-back to get things going. I'm feeling sort of "blinded" and more than a little "worried" about these problems. For me, this is the last of the forums that does a good job of allowing VB6 discussions, which I personally enjoy a great deal, and even cherish.

[Zvoni]

Well, for me it was the issue, that i wanted to post some SQL-Code,
and the Submit-Button just didn't work, if the SQL-Code contained the Keyword C ASE (all in Uppercase, i had to leave a space between C and A to even post it here. Haven't tested lowercase).

Took me some 10 minutes to figure it out

[SamOscarBrown]

Well, it's Tuesday....any update from the moderators about who is attempting to fix what? Progress?

Agree with Elroy....this is my daily go-to site...to get advice, give it, ask questions, etc. Probably will go mad if it doesn't get repaired!

[Steve R Jones]

Nope. No Updates. Will let everyone know something when I know something.

[fafalone]

IMO it's a ridiculous situation that administrators have no server access and those that do don't post here.

[Steve R Jones]

I did see one of the HQ guys on another site. So our issue have made their way to the Next in Line part.

[Episcopal]

I can't score anyone, including people I've never scored...

[Eduardo-]

IMO it's a ridiculous situation that administrators have no server access and those that do don't post here.

My worry is that the people that really can do something, maybe don't know that what happened is very serious, and that whatever value they have with these forums, all this code that is many years of code that people shared, is lost now.
Maybe they think that it is a minor issue, that it is like the bug in the attached images that we had for years, or the not receiving email notification issue that we had for more than a year, but this issue is different. All the forum is rendered useless.

I have a very basic question that is one of the first things someone should think about any system, and I would like to ask this question to a person that has technical information about the site. The question is: do you have a backup of the database?
We will need to know that before the backup is also overwritten.

[fafalone]

If the ads stopped showing up I bet it would be fixed in an hour.

[Eduardo-]

I mean, IDK if someone realizes that it is catastrophic, not just "an issue".
If there is value on this forum, it is because of the code that people shared for decades, and it is mostly inside the attachments.

We as users care because we need the code. They maybe don't care because they think that the advertisements keeps showing anyway, but... for how long? If the site is not useful anymore because the files are corrupted for how long the advertisers will pay for showing ads in a site ruined like that?

Again: are there backups? I can't believe that no one is able to figure that out, or at least to tell us if the files are intact at the source or not.

[fafalone]

I mean it's pretty clear at this point the admins here are vbulletin admins only and have zero access to the underlying hosting setup, so probably can't even answer without, it seems, just filing a support ticket or shooting off an email to be answered whenever, and the people with access to the server aren't active participants here. Probably not even 'people' plural, they've probably got one webmaster for all their sites.

[Eduardo-]

I mean it's pretty clear at this point the admins here are vbulletin admins only and have zero access to the underlying hosting setup, so probably can't even answer without, it seems, just filing a support ticket or shooting off an email to be answered whenever, and the people with access to the server aren't active participants here. Probably not even 'people' plural, they've probably got one webmaster for all their sites.

It is only a guess but... it seems to me that they are probably working with freelancers, or people that are not qualified for the tasks.
And the worst: nobody seems to care .

[TysonLPrice]

Was the reply with quote locks up reported?

[Schmidt]

The reason for all the recent problems was already pointed out by us...
It's the (serverside) prepending of *any* *.php-delivered content with a GTM-script...
That prepending-operation of the GTM-script-snippet is entirely in "the wrong place"

It does not belong:
- before all other file-content
- and it should not be prepended on "non-html-content"
- if at all, it should be inserted into the head of (only) html-content

So, the "bug-hunting" on the side of the admins should concentrate on,
who tf did place this GTM-snippet on top of everything - in some "master-php-script" (about 10 days ago)
as the "first thing any php-script should return on any mime-type-reply".

Olaf

[Steve R Jones]

Thank you Olaf. I've forwarded you post to the guys upstairs.

[Steve R Jones]

Again: are there backups? I can't believe that no one is able to figure that out, or at least to tell us if the files are intact at the source or not.

Eduardo - Just for you... I asked one of the IT Guys I was working with today about backups.

QUESTION: Is it safe to say that the vBulletin sites get backed up? Often???

If the server the sites are on were to BLOW UP ? would we be back up and running in a few days?


ANSWER: Very fair question. We do nightly backups so if the absolute WORST were to happen, with our set up we could go from totally down to brand new version with yesterday's data in probably an hour.

[Eduardo-]

Thanks for answering!

ANSWER: Very fair question. We do nightly backups so if the absolute WORST were to happen, with our set up we could go from totally down to brand new version with yesterday's data in probably an hour.

That's a huge problem, because suppose that now the forum data was mangled, irrecoverable (fortunately that wasn't the case this time).
This issue happened like 10 days ago, so the only backup would have been from yesterday and would have had bad data already.
In other words: the last good backup would have been overwritten the day after the issue.

I think they should have another backup, from time to time, like once in a month.
Or even one per year, but not losing everything in such a situation.
The backup should cover the database + all files on disk, that are basically the attachments and I think some images.

If you don't mind I can make a program to backup the most important parts of the site.
I mean the things that are public, that program wouldn't need any privileged access to do that.

[jdc2000]

As long as the do not just have ONE backup which is overwritten each night, but instead keep a week's or a month's worth of backups, that would be OK. If there is just one backup, the chances that it also has corrupted data once the issue is discovered is very high.

[Steve R Jones]

In twenty fives years of doing this, I don't think I've ever seen a restore from backup... If there was a restore - it was a long time ago.

Lets not make up things that may or may not happen.

Sites like this are probably more likely to shut down versus break down.

[jdc2000]

Steve,

That may be the case, however, the general rule is that the only time you ever need to restore from a backup is when you don't have one.

The database I work on has been the same, except for one time when some unknown genius ran a script that marked all users as Inactive, so that no one could log in. To correct this without restoring from a backup would have taken weeks, and if we had only one backup, we would have been out of luck. In this case, a restore fixed the issue within an hour.

Just thought I would mention this.