Jump to content
Hondo's Bar

Forum's current/recent status


Recommended Posts

  • Replies 131
  • Created
  • Last Reply

Top Posters In This Topic

Top Posters In This Topic

Popular Posts

Alright Everyone   So, I actually have a PC set up at the moment (ol' Compy 386) and a bit of free time......SO IMMA FINALLY DO THE FUCKING UPGRADE     Probably in the ne

Ok, here are my official findings on the issue. I'm not sure if this is what caused the downtime earlier this year, but I suspect this is the culprit. Also, I will not be naming any names or companies

by popular demand, Recent Topics have returned!  they should be seen on the main page above the hollabox for most groups here, lemme know if they're not showing for you.    the default is sh

Ok, here are my official findings on the issue. I'm not sure if this is what caused the downtime earlier this year, but I suspect this is the culprit. Also, I will not be naming any names or companies for reasons that will become clear momentarily.

 

 

The Problem:

 

Upon examining the logs, it appears that Hondo's is being overrun with bots. For the most part, these seem to be your run-of-the-mill web crawlers that do little more than index various pages in a site for use in search engines. There are several types of these:

  • The bots that index several pages per second and scurry off to other parts of the internet to do their thing. They return from time to time, but don't really use much as far as resources go.
  • The bots that index several thousand pages per second, take a break, and may or may not come back again. These use up a fair bit of resources but they don't hit things multiple times, generally speaking.
  • The bots that index hundreds of thousands of pages per second, use scripts, drain resources, and potentially make backups of everything. These are the strip miners of the internet.

Without writing a program to parse everything out, and because the logs are so long (the log for June, when unzipped, is 190 megabytes in size and contains 716,081 records), I was basically looking for records that stood out. I was able to identify approximately 7-9 bots that accounted for approximately 10.13% of all GET calls to the server (for those who don't know, basically your computer tells the server to GET a file for you, and the server sends it to you). These bots were of the first two varieties from the list above.

 

I reported this to Nick, and we began discussing what to do.

 

As we were discussing, I was absentmindedly scrolling through the records, just in case there were any more bots that I had yet to find. Lo and behold, I came across one that made a lot of calls (thousands) in a matter of seconds. Of the 716,081 records, this bot was responsible for 195,322 or 27.28% of all traffic. Upon further investigation, this web crawler may actually be part of a malicious botnet. I will not speculate on the motivations for targeting this site, when it may, in fact, be random.

 

Upon even further investigation, the company behind the crawler has pursued legal action against various groups for mentioning the malicious botnet, so I'm not bringing that up.

 

It should be noted that this bot acted different from other bots, which leads me to believe that this was the problem. This bot would actually use various scripts that are part of the code of the forum. For instance, the bot(s) would visit specific posts and then share them as a blog post several thousand times a second. As far as I know (and I might be wrong), this functionality is unused as part of Hondo's everyday operations. This bot, from what I was able to find online, targets IPB forums for various reasons.

 

The Current Solution

 

Theoretically, every site on the internet has a little file called robots.txt that has instructions for the bots that visit aforementioned page. This is generally set up by the webmaster, but sometimes the hosting company has one for all sites on their servers. The file can basically tell bots not to visit certain folders, or which bots can visit certain areas, etc.

 

From all appearances, this did not exist on the domain, nor did it appear that the hosting company has server-wide policies. This is not the norm, but it is far from being uncommon. It's easy to forget, or not even know about.

 

A robots.txt file has been created, and a blanket ban on all crawlers / bots has been implemented. Again, this is not an uncommon thing to do. From what I can tell, there's very little benefit (if any) for this site to be crawled by untold bots.

 

This being said, bots do not have to obey robots.txt, which is why other measures will be implemented if necessary.

 

 

The Future

 

In the event that the new robots.txt file fails to block a potentially malicious botnet / bot, I will curate a list of known bot IP addresses from the logs. This would likely be done automatically via a program that I'll write. It's parsing information, not a huge pain. This list would then be used to implement IP bans at the domain level.

 

In the event that this wasn't the problem to begin with, I'll likely start over and try to figure out what else might be going wrong. That being said, I'm pretty sure that 37% of all traffic being bots and 27% of all being "bad" bots probably did it.

 

I also currently recommend that the IPB software be upgraded to the latest version and kept at the latest version at all times. I understand that this may not be possible due to $$$, but it's usually a good idea. The older a piece of software gets, the more well-known any exploits will become. I'm not saying that another attack is more probable, but it is certainly more possible the further the board gets from the current version.

 

 

So that's my tear-down on what I think has been going on and what I've done to try to stop it. I may or may not be around, but if anyone needs me, Nick knows how to get to me. Thank you, and good night!

 

 

Edit: Also, here's the data from my tear-down. Names will be censored:

  • Total Records: 716,081
  • M*********: 25,852 records
  • B*********: 16,738 records
  • B*********: 10,661 records
  • Y*********: 1,323 records
  • O*********: 5,598 records
  • A*********: 252 records
  • N**********: 3 records
  • S***********: 830 records
  • S***********: 220 records
  • S*********: 9,669 records
  • M**********: 195,332 records

Edited by Stillbored
  • Upvote 4
Link to post
Share on other sites

This issue isn't because of me logging in from my job, is it? Web security is kind of insane here.

Nah, that shouldn't have anything to do with it. You'd likely be accessing the site via HTTPS in that case, but there were only a relative few requests. You're good.
Link to post
Share on other sites

it's not youse guys; short of refreshing at light speed, the regulars connecting/logging on & off/posting etc aren't putting those #'s on the board, that's why that day in particular was an anomaly. ya'll are good.

 

stilly - the same tech support that was on about editing our robots protocol (before you pointed out we lacked one, haha) said he found something in a scan of our site a while back, but ive run malwarebytes & MS defender scans over full backups of the board & found nothing, gonna send you the notes on that too just in case there's something to that.

Link to post
Share on other sites

+1 man. Your efforts are greatly appreciated.

 

And now that you're here, no need to rush off.

so_say_we_all1.gif

 

And I look at so much porn that there's no way bots aren't using my phone as the Skynet staging grounds for the uprising. So much porn.

unsure.png

 

tumblr_mcxbmmbTyy1r9pbwpo1_500.gif

 

  • Upvote 1
Link to post
Share on other sites
  • 2 weeks later...

updates:

 

we're paid up through fall of 2018, due to the generosity of ASC, Vagrant & a huge helping hand from Drifter - thanks so much you guys!

 

meanwhile, stillbored convinced me to update this place, which should keep us more secure both from external exploits/hacks and internal failures of our old database. that'd mean a new look around here - well, mostly likely it'd be a same-y look for all 16 or so of our forums until stillbored can help sort out skinning variations, that might be an ongoing project...i'm hoping we can nail down signatures once per page too, so they don't get too obnoxious for mobile users. he's found us replacements for the shoutbox & other board features.

 

if my work schedule holds, i plan on taking the board offline and (after a most thorough backup, i assure you) running the upgrade. it's our same software so hopefully nothing is lost in the process, i'm guessing we'll go back to having standard avatars as an option at that time though.

sure hope the reputation systems isn't affected :yup:

 

 

the only other change is that dude to IPB's current licensing system, we'd be adding $25 every 6 months (or $50/year) to our expenses. i'll cover the first 6 months, but any help offered is always appreciated. big changes coming!

Link to post
Share on other sites

Let me know when the next fund drive is, I'll see about sending more... I gots plans ya know; I'm gonna give Tony Stark a run for his money (Mentally-unsound, Marginally-well-to-do, Shut-in, Philanthropist!) I've decided that when my car goes it's going to NPR... you'll just have to settle for cash.

  • Upvote 1
Link to post
Share on other sites

Yeah, people are charging between $15-$30 per skin which is stupid if you've got anyone on the board that has the slightest inkling of what they're doing.

 

Should be able to have everything ported over "fairly quickly" (school and work schedule allowing, I'll still try to bash everything out quick-ish), but then I'll begin trying to somewhat modernize a few of them.

 

I assure you, this won't be HondosInc all over again. Totally promise that.

 

Edit: Totally. For reals.

Edited by Stillbored
  • Upvote 2
Link to post
Share on other sites
  • 3 weeks later...

so, IPB is saying they can do this one last upgrade for us, all i need to do on my end is complete another full site backup & grab the plugins we'd need (signature limitations, chat etc). i'm not likely to be at a post with a computer again for a while, so i might schedule this for the next time i have off to mess with things & get it properly going before tossing back a barebones mess online.

 

i need to coordinate with stilly on this for help with images/skins here too!

Link to post
Share on other sites

I'm ready to get started on my end. I'll make a backup of the current skins and such over the weekend, and start working on it a bit here and there. Progress will come quicker once I see exactly what I'm working with as far as the update is concerned!

Link to post
Share on other sites

×
×
  • Create New...