Why I like running my own site

Turns out I learned something today…something I had not expected to learn.  Running your own website may not give you “god-like” status over the Interwebs but it does give you that status for your own site.  It’s a rather…powerful feeling to be able to control what you want displayed on your site.  This is the reason I am not cut out for a position of power in our society.  That is all.

Awstats MySQL access_log Export

Alright, so I thought I would start posting some of my code here since this blog gets a fair bit of traffic. Anyway, I thought I would kick things off with a small script I wrote just last night and “perfected” this morning. I was a bit disappointed that Awstats doesn’t have a way to pull logs from a MySQL database. I have been logging my Apache access_log information to a MySQL database for quite some time now and last night it hit me that I could actually just export that data into a flat file for Awstats.  The script, though quite simple in design, does that job and does it well.  Simply set a few configuration options at the top and you are good to go.

You can also have the script run the awstats.pl file to generate the stats after the data has been exported.  This is definable in the configuration section at the top of the script.  Everything should be pretty straightforward really, just define the username/password/database/table at the top, set the location you are exporting to, and you are good to go.  Just make sure the user you run it as has permissions to write to wherever you are exporting the log.  It can be called from command line using “php /path/to/generate_awstats_log.php” or as a cron job similar to this “0 0  * * *  root php /path/to/generate_awstats_log.php > /dev/null 2>&1” which would run every night at midnight.

It’s also very quick, it can export tens of thousands of records in seconds, you will actually notice that it takes Awstats longer to generate your stats then it does for this script to create the file.  If you have any questions or problems you can post them in the comments below or in the forums over at SourceForge.  I’m not going to post the script here as I don’t want to keep it updated in two places but you can grab a copy over at SourceForge.  Enjoy!!

Clean The Fan

So I have a much greater appreciation for this video today after taking apart this exact laptop model.  One of the ladies at work came to us with the laptop and said it was shutting down every 10 minutes.  The 3 of us suspected that it was just overheating so we decided to tear it apart.  Almost 2 hours later we finally had it torn down and cleaned out.  We did however opt not to removed the thermal gel and instead used canned air and some small screw drivers to get the massive dust bunnies out of the fan.  So without farther ado, I present Clean The Fan:

And no, this is my my work, the creator is here: http://www.youtube.com/user/jaymegutierrez

The First Global Cyber-War

For those of you hiding under a rock or otherwise out of the loop the past 2 weeks a quick Google News search for Wikileaks should help fill you in.  The well-known whistle-blower site has hit global headlines the past few weeks as it releases some 250,000 classified US documents.  The site has come under heavy fire from US officials, some of which want to classify it as a terrorist organization.  Amazon, one of the hosts of Wikileaks pulled the plug on their hosting, EveryDNS stopped providing DNS service to Wikileaks, and Visa/Mastercard/PayPay have refused to process donations to the site.

In retaliation of all these actions, a group of supporters (that have no affiliation with Wikileaks whatsoever) known as ‘Anonymous’ launched a number of DDoS attacks against many of these sites.  Visa and Mastercard had their sites taken offline for a number of hours, PayPal’s blog and API site (a site used by maybe third-parties to process payments) were also hit and taken down/slowed down for a number of hours.  Other causalities of the attacks include Swedish prosecutors, PostFinance, and many others.

But this is not a one-sided battle.  The ‘Anonymous’ site has been hit several times on various fronts bring parts of its websites and chat rooms down.  Meanwhile Wikileaks was taken offline from a combination of DDoS attacks and Amazon/EveryDNS shutting down their accounts.

Also, Earlier in the year we saw attacks against the RIAA and MPAA by a group known as ‘4chan’ for attempting to take down a number of file sharing sites with DDoS attacks of their own.  This same group also targeted Tumblr for allegedly copying other people’s work and not giving them credit for it.

Data breaches of companies large and small are happening at an alarming rate too.  McDonald’s, University of Wisconsin, and Gawker are some of the latest victims.

Searches on Google News and may other sites for terms such as “DDoS Attack“, “4chan Attack“, and “data breach” yield thousands and thousands of results.  Reports of new attacks are coming daily and sometimes hourly at a rate that doesn’t appear to be slowing.  While most Internet users do not, and most likely will not, see the ramifications of these attacks, many individuals are starting to see the writing on the wall.

I myself believe we are on the verge of a global cyber-war.  We seem to be moving away from organized armies with rifles to vigilantes armed only with a laptop and an Internet connection.  Their targets can be a block down the road or 5000 miles away on the other side of the globe.  There is no age, race, religious, or any other barriers to doing such work.  Attacks from these ‘armies’ can be organized to strike anywhere in the world within hours.  Becoming part of the attack is so simple most children could do it.  Simply download a piece of software and give it the address of the control server.  That’s it, your PC is now a part of the attacks.

According to the download site, over 80 THOUSAND copies of the program have been downloaded in the last week.  What’s also clear from the graph is that downloads have trickled off since the 9th.  However, some statistics I managed to uncover indicate that there are almost 2 BILLION Internet users worldwide.  Assuming one quarter of those own a computer that’s roughly 500 MILLION Internet connected computers worldwide.  I’m sure this number is VERY low as a lot of these people have more than one computer but for the sake of argument let’s just go with it.  Let’s then say 1% of those Internet connected computers belong to a group of people who believe very strongly one way and want to do something to show it.  This brings us to about 1MILLION pissed off Internet users with computers out to prove a point.  Finally let’s take our original 80 THOUSAND downloads of that software and double it to 160 THOUSAND.  If 160 THOUSAND computers managed to take down the web sites of Visa and Mastercard (ranked 326 and 411 on the fortune 500 list of 2010) imagine what 1 MILLION could do?

But it doesn’t stop there; new technologies are being leveraged to make attacking even easier.  A small Javascript page makes attacking even easier.  All you have to do is visit a specific website and click a button.  No software to download, no computer knowledge required, and you could even participate from other Internet connected devices such as smart phones and tablet PCs.

We are starting to see what the creators of SETI@home discovered many years ago….one personal computer trying to analyze a decade’s worth of scientific data is all but worthless.  However, hundred of thousands, even millions, of computers all focused on the same task can really make a HUGE impact.  The shared CPU, RAM, disk space, and bandwidth of all these computers focused on one specific goal can rival, even exceed, the capabilities of the worlds great supercomputers.

Attacks such as those see the past two weeks or so show us that dedicated individuals, those that believe strongly in a cause, have the ability to take down not only your Facebook and Twitter pages but your bank, favorite on-line merchant, even government sites.  They can organize at a moments notice and vanish just as quickly.  And no, it’s not just the ‘bad guys’ that have this power, the US government, Chinese government, and I’m sure many other government organizations possess this ability.

If individuals can cause the amount of damage we have see these past two weeks imagine what would happen if two governments started attacking each-other through DDoS attacks?  Imagine the Axis and Allies attacking not with heavy bombers and armored tanks but with nothing except small electric pulses sent zooming around the globe at the speed of light?  Your snipers are highly experienced hackers, your Generals become CIO’sFirewalls and IPS devices become your shields and walls.  Your heavy artillery is worms and Trojans.  Wars could be decided in minutes…in seconds.

We sit at a crossroad and are watching history unfold before our very eyes.  The true potential, the true power, of millions of Internet connected devices is just starting to come into light.  Nobody knows what the future holds, and I would never claim such things.  The Internet is, and has always been, about the open flow of information.  What started as a small lab experiment has turned into a tool that most of us cannot do without in our day-to-day lives.  As cars, home appliances, and many other devices connect to the Internet the number of possible ‘attack devices’ is only going to grow.  I, for one, look to the future with much anticipation and wonder, contemplating what happens next….

Farther reading:

CPU Stress Test Scripts

As I sit here stress-testing one of my Linux boxes I figured I would share with you guys the two scripts I have written to accomplish this. These two scripts simply peg one CPU/core. You will have to execute 1 instance of the script for each CPU/core on your machine to truly put it through it’s paces.


Alright, now that we have that out of the way, here ya go:

Windows VBScript:
1) Copy and paste this into notepad:
while 1<2
i = i + 1
2) Save as ‘max_cpu.vbs’ in a location that’s convenient for you.
3) OPEN TASK MANAGER NOW (If you don’t do it now you will never get it open)!!!!!
4) Run the script (by double clicking) for each CPU/core you have on your machine. In task manager you should see a wscript.exe for each time you double click it.
5) When you have completed the testing kill all the wscript.exe processes you see in task manager.

Linux bash script:
1) Copy and paste this into gedit:
let count=0
while :
echo “Count is: $count”
2) Save as ‘max_cpu.sh’ in a location that’s convenient for you.
3) Make the script executable (‘chmod a+x max_cpu.sh’ usually works).
4) Run the script, ‘./max_cpu.sh’ will do it. Repeat this for every CPU/core on your machine. You will have to have a separate terminal for each one.
5) When you have completed your testing do a ‘CTRL + C’ in each of the terminal windows that you executed the script in to kill the process.

While not really required, it’s strongly advised that you reboot after these ‘tests’ to make sure things get cleaned up. These don’t make any change to the OS but sometimes (especially with Windows) it will not fully release the resources used after the process(es) is(are) terminated.