MyPersonalVPS.com

Server Admins (me included) the world over woke up today to find that a major upgrade to the cPanel/WHM control panel had been rolled out. Within the release are a multitude of upgrades to features within cPanel along with some major optimization to increase the performance speed and reduce overheads.

Sounds great so far? Yeah, well - This upgrade hasn’t exactly gone too smoothly for some folks.

Having put the first coffee on to brew I turned on the screens in my office to do the first scan of emails received overnight from my servers, and noticed one repeating theme from a particular box telling me (at regular 5 minute intervals) that FTP had failed. Hmmm, OK, a quick visit to WHM for that box, scroll to the bottom of the menu (Restart Services), hit “FTP server” and “OK”, and I’ll be heading back to the kitchen for that all-important first cup of coffee in no time, and then back to investigate the logs in a few minutes … Gotta get that coffee first!

WRONG!

For some inexplicable reason WHM isn’t playing ball. I know that I’ve entered the right password (It’s just a paste from my password manager) so why am I not seeing WHM load up? Instead I’m getting a 404 error, with the message:

The server was not able to find the document (./scripts3/initial_setup_wizard1) you requested.
Please check the url and try again. You might also want to report this error to your webhost.

Huh?

I glance across to another screen that’s showing my live monitoring and FTP is definitely showing green for that server. WTF? I happen to have a login to an account on that server saved in my FTP clent so I hit it quick and, behold, FTP is fine on the server - I logged in without a problem.

OK, decision time - I decide to go get that coffee (and a cigarette) and come straight back to this.

Safely armed with injections to hand of caffeine and nicotine I glance at some other emails and notice the one from cPanel announcing the upgrade release. “Cool” I think “I’ll check that out in a bit after I find out what’s wrong with this dang server”.

Glancing left to another screen I notice the amber flashing item in the toolbar which tells me there’s been new messages in the IRC channel we run between ourselves (my support guys and me).  I open that up to find a few items about failed webmail logins, customers cannot access their cPanel logins, etc., etc.

Mostly, my over-night guys have fixed all the issues and our customers are all fairly content, but there still remains this issue of FTP and WHM on one server. In my book 2 + 2 generally equals 4, so I follow the link in the cPanel announcement email to see what’s changed.

To cut a long story short, the update roll-out hasn’t gone entirely like it should, and a number of issues have surfaced. Rather than post a blow-by-blow account of how the rest of today unfolded, I’ll paste below what we’ve found and how we fixed it, in the hope that it will be of use to someone else out there …..

1) The cPanel update servers have taken a battering so, if you have a box that’s slow, or is only getting a slow connection to the cPanel update servers, TailWatch has a habit of restarting processes the update stopped (before it’s ready for them to restart).

This causes the update to stall, so stop TailWatch first with killall -TERM tailwatchd, then run killall upcp to stop the failed update, then /scripts/upcp –force to fire it off again.

2) There’s LOTS of changes in this release, so it’s worth spending some time running through all the options in WHM to make sure everything is as you want it …. and secure!

For example, I just took a look at the new FTP configuration options to find 2 settings set to YES by default:

Allow Anonymous Uploads This is set to YES and right next to it is a warning about how it affects server security … doh!

Allow Logins with Root Password This is also set to YES! So, someone guesses/sniffs/cracks a root password and then can merrily FTP into any account on the box and upload malicious scripts, even if you’ve secured SSH, changed the port, etc, etc. NOT NICE!

3) Message just in from cPanel:

Please ensure to read the following message carefully as there is important information about MySQL and Automated process killers in relation to the cPanel 11.24 upgrade. MySQL Root Password ——————————— With cPanel 11.24 now available, it is imperative to ensure that you have set your MySQL root password before upgrading to the latest version of cPanel. If you have previously skipped this step, you can set it now using the “MySQL Root Password” function in WebHost Manager. Failure to set a MySQL root password may cause database corruption on systems running MySQL 4.1 with InnoDB tables. Please set your MySQL root password as soon as possible to avoid any issues. Machines without a MySQL root password set allow access to any database by any user so it is imperative that a password is set as soon as possible. If you have moved /root —————————— If you have set root’s home directory to something other than /root, you will need to copy .my.cnf from root’s home directory to /root/my.cnf. In the future, this process will be automated. Automated process killers ———————————— If you have a system in place that automatically kills processes taking up a lot of CPU time, you should disable them before running the cPanel update as the size of the update has triggered many of these systems. These process killers will kill updates or related process in the middle of the process and cause issues. If you have a failed update, you can force a re-update by running /scripts/upcp –force
If you have already updated to cPanel 11.24 and experienced this problem please open a ticket at https://tickets.cpanel.net/submit/

Thank You,

Eric Gregory

cPanel, Inc.

Fortunately, that one didn’t get us.

4) DNS Clusters: If you have more than one VPS/server and are running a DNS cluster in WHM, then check it after this update.

Normally, when you access Configure Cluster from WHM you would see a list of hostnames that are configured within the cluster. If WHM is now showing just the IP addresses instead (in the left hand column), then the trust relationship between the servers is possibly only working in one direction. This can screw up the parity of DNS records on all servers in the cluster very quickly.

A quick fix we’ve discovered is to renew the trust relationship in BOTH directions - In other words, install the remote access key for one server on the other, and then repeat it from the other server back to the first (previously the trust relationship would be established both ways by just installing a key from one server to the other).

Once you have the trust relationship installed both ways, WHM will then display the hostname rather than the IP and all should be well.

If you have users/customers who are likely to have created/deleted domains or sub-domains to their accounts since the cPanel update was applied, you might also want to run “Synchronize all zones to this server only” on all servers (one at a time) to ensure that all your records match.

So far so good - All my boxes are running and we don’t have any major disasters (unlike some), but we HAVE disabled automatic updates from cPanel on all boxes apart from one test box that’s allowed to break … Once bitten, twice shy!

And today’s Phrase of the Day, as seen in numerous PuTTy windows during the course of today … “Performing sanity check“

I’ve been quiet for a couple of weeks, so I had better explain why. September 25th my dad passed away at the good age of 92. He had been ill for a few weeks and his illness had come to the point where if it had gone on much longer he would really have started to suffer. We had just managed to get him out of hospital and into a private nursing home, so our last day with dad was a good one, and a day that will be remembered fondly. That night he passed away quietly, without pain, in his sleep.

Yesterday was the funeral and cremation, and we were pleasantly overwhelmed by the huge turn out of family and friends, almost filling our (large) Church to capacity.

I think dad would have enjoyed (and probably did enjoy) yesterday. The arrangements all went without a hitch, the music was wonderful, and Chris and Beryl (our local clergy) handled the proceedings with just the right level of dignity and respect that was needed, without making it too somber an occasion. Dad would have said “Don’t make a fuss” and we didn’t, but I’m pretty sure that he’ll be sitting in a comfy chair in Heaven right now, puffing on the new pipe I sent with him (with a spare packof St. Bruno baccy) and remarking to anyone that will listen what a good turnout he had for his send-off, and how nice it was that so many of his old friends were there to see him off.

Thank you to everyone for your messages of sympathy. You know dad would have said “now, get on with it.” so we are, and we will, knowing that he’ll always be there keeping an eye on us.

R.I.P. Dad, and thank you. Gone to a better place now, but never forgotten.

Thanks to Gordon - Yet another piece of weekend humor we all need to lighten up our days

BARACK OBAMA : The chicken crossed the road because it was time for change !    The chicken wanted change!

JOHN MC CAIN: My friends, that chicken crossed the road because he recognized the need to engage in cooperation and dialogue with all the chickens on the other side of the road.

HILLARY CLINTON : When I was First Lady, I personally helped that little chicken to cross the road. This experience makes me uniquely qualified to ensure right from Day One that every chicken in this country gets the chance it deserves to cross the road.   But then, this really isn’t about me.

SARAH PALIN:  As a Mayor and Governor of Alaska I have fought against and stopped the good old boy chickens attempts to cross the road against God’s will.  It appears I have not fully succeeded. Where’s my gun?

DICK CHENEY : Where’s my gun?

GEORGE W. BUSH : We don’t really care why the chicken crossed the road. We just want to know if the chicken is on our side of the road, or not. The chicken is either against us, or for us. There is no middle ground here.

COLIN POWELL : Now to the left of the screen, you can clearly see the satellite image of the chicken crossing the road with what is certainly weapons of mass destruction, perhaps nuclear.  We must bomb the chicken before it attacks us and destroys our American way of life!

BILL CLINTON: I did not cross the road with that chicken. What is your definition of chicken?

AL GORE: I invented the chicken.

JOHN KERRY: Although I voted to let the chicken cross the road, I am now against it!  It was the wrong road to cross, and I was misled about the chicken’s intentions.    I am not for it now, and will remain against it.

AL SHARPTON: Why are all the chickens white? We need some black chickens.

DR. PHIL: The problem we have here is that this chicken won’t realize that he must first deal with the problem on this side of the road before it goes after the problem on the other side of the road. What we need to do is help him realize how stupid he’s acting by not taking on his current problems before adding new problems.

OPRAH: Well, I understand that the chicken is having problems, which is why he wants to cross this road so bad. So instead of having the chicken learn from his mistakes and take falls, which is a part of life, I’m going to give this chicken a car so that he can just drive across the road and not live his life like the rest of the chickens.

ANDERSON COOPER, CNN: We have reason to believe a chicken crossed the road, but we have not yet been allowed to have access to the other side of the road to verify the crossing.

BILL O’REILLY, FOX NEWS: Another left-wing pinko chicken has crossed the road, probably looking for another government relocation handout. Get over it buddy, as far as I’m concerned, you’re on your own.

NANCY GRACE: That chicken crossed the road because he’s guilty ! You can see it in his eyes and the way he walks.

PAT BUCHANAN: To steal the job of a decent, hardworking American.

MARTHA STEWART: No one called me to warn me which way that chicken was going.  I had a standing order at the Farmer’s Market to sell my eggs when the price dropped to a certain level. No little bird gave me any insider information.

DR SEUSS: Did the chicken cross the road? Did he cross it with a toad? Yes, the chicken crossed the road, but why it crossed I’ve not been told.

ERNEST HEMINGWAY: To die in the rain, alone.

GRANDPA: In my day we didn’t ask why the chicken crossed the road. Somebody told us the chicken crossed the road, and that was good enough.

BARBARA WALTERS: Isn’t that interesting? In a few moments, we will be listening to the chicken tell, for the first time, the heart warming story of how it experienced a serious case of molting, and went on to accomplish its lifelong dream of crossing the road.

ARISTOTLE: It is the nature of chickens to cross the road.

JOHN LENNON: Imagine all the chickens in the world crossing roads together, in peace.

BILL GATES: I have just released eChicken2009, which will not only cross roads, but will lay eggs, file your important documents, and balance your checkbook. Internet Explorer is an integral part of eChicken2009.  This new platform is much more stable and will never need to reboot.

ALBERT EINSTEIN: Did the chicken really cross the road, or did the road move beneath the chicken?

COLONEL SANDERS: Did I miss one?

Or, You named your domain name WHAT???!!!

Thanks a bunch to Jose for these!!!

All of these are legitimate companies that didn’t spend quite enough time to consider how their online name might appear!

These are not made up. Check them out yourself!

1. “Who Represents” is where you can find the name of the agent that represents any celebrity. Their Web site is:www.whorepresents.com

2 “Experts Exchange” is a knowledge base where programmers can exchange advice and views at:

www.expertsexchange.com

3. Looking for a great pen? Look no further than

” Pen Island “. It can be found at: www.penisland.net

4. Need a therapist? Try “Therapist Finder” at:www.therapistfinder.com

5. Then there’s the “Italian Power Generator” company. Check it out at: www.powergenitalia.com

6. “IP computer” software, there’s always:www.ip_anywhere.com

8. “The First Cumming Methodist Church” Web site is: www.cummingfirst.com

9. And the designers at “Speed of Art” await you at their wacky Web site: www.speedofart.com

Have a fun day! Just be careful what you name YOUR new web site

It’s an odd feeling writing this and remembering exactly where I was and what I was doing on this date exactly 7 years ago today. But worse, I still cringe now when I think back to that day, and the events that have imprinted today’s date so firmly upon my mind.

It was 9/11. That’s the 11th of September in English money. It was just after 2pm (us Brits are 5 hours or more ahead of the folks in the USA). I was in a shopping mall in the suburbs of Manchester, UK, and strolling past a TV shop where all the TVs in the window were haphazardly tuned to various different TV stations we have available here in the UK. So why were they all showing the same picture? Because a skyscraper in the USA was on fire. It was news. A few of us gathered outside the window of the TV shop and watched. Then, out of the corners of all our eyes we watched in amazement as a plane flew into the picture and hit the skyscraper right next to the one on fire.

This can’t be right. It’s just a film, right?  Within a few seconds the guy in the TV shop opened the doors and beckoned us inside. The volume was turned up and we started to hear (but not understand) the commentary behind the news story.

And then there were the gasps from all of us as we each began to understand what we were watching.

My cellphone rang. It was a pal of mine in the USA who was near the Pentagon. He told me to turn on my TV. I told him where I was and what I was watching. He made some silly remark about where he was would not make the news then, if there was something bigger going on elsewhere. And then it all became clear.

7 years later I can still recount every second of every minute of that day. And I was just a watcher.  I got a phone call telling me that all American airspace had been closed to all flights - All planes were heading for Canada. I can still remember saying “No way!”. Silly really, but the sheer massiveness of what had happened was still sinking in.

That evening we watched the news round-up of the day. We saw again the plane hit the second tower. We saw the firemen and cops covered in dust, and weeping. We saw the convoy of ambulances. We saw the smoke.

Only later did the full realization that a handful short of 3000 people died sink in.

To the people of America, and elsewhere, that lost loved ones, or know others that lost loved ones, God bless you. You must still remember this day, even more vividly than I can.

To the people behind this, or of a like mind, I have only one message. Idiots! If you thought you were “fighting for peace” or fighting some sort of holy war, you’re idiots. Go f!!k yourselves. You have brought the might of the USA and the World against you and, with those odds, you don’t stand a chance. Beware, because you have brought upon yourselves the absolute certainty that we’ll come find you, and deal with you, possibly more humanely but certainly no less decisively. As sure as those 3000 people are dead, you can be sure that you’re heading the same way.

RIP all the innocent victims of 9/11.

It’s an odd feeling, today.

The guys over at Wordpress have rolled out an urgent upgrade up to version 2.6.2 to protect against a recent vulnerability which, although difficult to execute, does pose a threat.

Stefan Esser recently warned developers of the dangers of SQL Column Truncation and the weakness of mt_rand().  With his help we worked around these problems and are now releasing WordPress 2.6.2.  If you allow open registration on your blog, you should definitely upgrade.

Full details are in this post on their blog.

Google have let the cat out of their bag (apparently a little earlier than planned) and announced the up-coming release of their own browser, called Chrome. On the official Google blog, they say:

All of us at Google spend much of our time working inside a browser. We search, chat, email and collaborate in a browser. And in our spare time, we shop, bank, read news and keep in touch with friends — all using a browser. Because we spend so much time online, we began seriously thinking about what kind of browser could exist if we started from scratch and built on the best elements out there. We realized that the web had evolved from mainly simple text pages to rich, interactive applications and that we needed to completely rethink the browser. What we really needed was not just a browser, but also a modern platform for web pages and applications, and that’s what we set out to build.

So with Internet Explorer, Firefox (sponsored by Google), Opera, Safari, and several others already in use, do we really need another browser? Well Google seem to think so, and they promise to make it open-source and available for multiple platforms. They’ve even produced a comic strip to explain what they’re doing.

I’m trying to promise myself that I won’t be drawn in to installing yet another browser on my development machine here but, of course, the geeky side of me will yearn to give it a test drive, reasoning that if I develop web sites I have to test their performance within all the available popular browsers that my site visitors might use.

Oh well - I guess it’s time to order yet another bigger disk drive for my dev. machine.

OK, so it’s not quite the Presidential Elections currently taking the USA by storm, but over on HotBloggerCalendar they’re running an election of their own to find the top 12 male and female bloggers out there, the prize being a photoshoot and glamorous (infamous?) page in one of their calendars due out later this year.

Being a hot-blooded male of this planet my interest in any of the guys winning is, well, diddly squat.

However! In amongst all the female candidates vying for your vote is one Charnell Pugsley - my favourite  customer service angel, and so-called Community Evangelist over at PEER1 and ServerBeach (who just happen to provide me with some great servers).

So, unashamedly, I want you to vote for Charnell. Nobody else. Just Charnell. Why? Well, because I say so, and, if that isn’t good enough reason, without this post you wouldn’t have thought of voting for her (or anyone else) anyway, would you? So just do as I bid, and go cast a vote for her. This is my humble little experiment in viral voting and there’s around 48 hours before the voting closes so you need to go vote RIGHT NOW!

HOW TO VOTE:

It’s simple.

Go visit http://hotbloggercalendar.com/vote-hottest-female/

Scroll almost half-way down the list and find the name Charnell Pugsley

Click the radio button to the left of her name to cast your vote, and your done!

Easy huh?

As time is short I’ve not been able to get any bumper stickers, posters, badges, etc, made up, so I’ve gone out and blown the entire ‘Vote for Charnell’ campaign budget on one, single promotional video presentation. As luck would have it I happened to bump into a staffer on the outgoing “Hilary Clinton for President” publicity team and they’ve very kindly cobbled together a short video presentation from unused clips they had, but bear in mind there’s one or two errors in the scripting here due to the original purpose of the clips, which they’ve not had time to alter.

So, without further ado, watch the video! …….

And when your done, remember …….

Go visit http://hotbloggercalendar.com/vote-hottest-female/

Scroll almost half-way down the list and find the name Charnell Pugsley

Click the radio button to the left of her name to cast your vote, and your done!

THANK YOU!

Back in November last year (how time flies!) I posted about a Wordpress plug-in (Wordpress Automatic Upgrades) which saves so much time and energy when updating to the latest and greatest Wordpress. Well, I updated to the new 2.6.1 today using that utility, and noticed that the VPS is beginning to have a hard time keeping up with all the jobs I set it to do.

So I guess tomorrow it’s time to upgrade a few things on this little box, with some more RAM and file-space, etc. I actually did a bit of housekeeping today, trimming down some log files, clearing out some unnecessary files, restarting a few services, etc., but she’s struggling a bit now because of all the background processes I run.

So! If the site vanishes before your eyes for a short while tomorrow, please don’t get upset. It simply means that I’m treating the box to some extra goodies, and she’ll be back on line within the hour

I thought these villains had been quiet for too long and then, last night, I happened to notice a support ticket in our queue from a customer claiming that we had hijacked his site, or the server it was on. The ticket had already been pushed up the line to one of our senior techs so I called him up and asked what he had discovered.

“It’s a new MPACK trick” he told me - Now it seems that instead of inserting iframes and javascript into every page they can find on sites they manage to invade, they’re going for HTACCESS files. Basically they target referrers (typically the big 3 search engines Google, Yahoo! and MSN) and if your site gets a hit via a link on the search engine the HTACCESS file then redirects your visitor away to a malware site. And for n00bs who don’t really know (or care?) what an HTACCESS file should look like or contain, they’re inserting 30 to 40 blank lines at the top of the file in order to convince you that it’s actually empty.

Fortunately this hack is easier to fix than its predecessor as it usually only involves one file. We now have a stock of standard HTACCESS files for popular scripts like Wordpress and Joomla! that we can just drop straight in and over-write the malicious file if anyone else reports this issue.

<sigh> I wonder what they’ll think of next? </sigh>

EDIT: After writing this I noticed an article on TheRegister about what is possibly the result of one of these hacks. It’s worth a read, and thanks to the author, Jesper M. Johansson, for the time he obviously spent researching this.