Un-denying a Denied Host

A great guy named Nicholas put out an easy to use script to unban an ip address that’s been banned by denyhosts. Here’s me gushing with praise since I could be lazy and not have to write it, along with the script so you can benefit from his contribution as well.

A big thanks to Nicholas Avenell, who solved a problem I’ve been having for a while: How to remove a host that denyhosts has banned. Yes, the FAQ for denyhosts has the manual steps listed, but Nicholas scripted it. And it works great.

Copy Nicholas’s script into a file, give it execute permissions, and run it with root privileges passing in the IP address you’d like to unban:

#/bin/sh
REMOVE=$1
/etc/init.d/denyhosts stop
cd /var/lib/denyhosts
for THISFILE in hosts hosts-restricted hosts-root hosts-valid users-hosts;
do
mv $THISFILE /tmp/;
cat /tmp/$THISFILE | grep -v $REMOVE > $THISFILE;
rm /tmp/$THISFILE;
done;
mv /etc/hosts.deny /tmp/
cat /tmp/hosts.deny | grep -v $REMOVE > /etc/hosts.deny;
rm /tmp/hosts.deny
/etc/init.d/denyhosts start

Nicholas, you rule. Thanks.

WordPress FTPS is not SFTP

Is WordPress giving you server errors when you try to update or upload themes? Are your only two options FTP and FTPS (which is not SFTP)? Here’s how to fix that, make WordPress work with SSH, giving you an SSH2 option for updating. And it works.

A major shout of thanks goes to Jon for his WordPress post entitled Enabling SSH/SFTP Updates.

I’d been struggling with making WordPress updates work properly for the longest time, setting up file ownerships and permissions. Things always seemed to be mostly right. If I tried to do an update of any form, I’d see this:

Error: There was an error connecting to the server

WordPress offers FTP and FTPS (SSL).

I’d try the secure option and things just would not work. Then I’d go examine my SFTP connections and my SSH, and it’d all be working fine.

It took bdawson‘s post on FTPS vs. SFTP to make me realize I was so falling into the habit of typing STFP that I didn’t realize that wasn’t what WordPress was offering!

With Jon’s post, enabling SFTP in WordPress becomes trivial.

  1. # apt-get install libssh2-php
  2. # /etc/init.d/apache2 restart

The SSH2 option then presents itself, and all is right in the world. Thanks guys!

Virtual Server Problems

I recently switched operating system vendors, and to my surprise when I went to port over the web content from one system to another, things didn’t go as smoothly as I had hoped. What used to be /etc/httpd was now /etc/apache2, and inside this directory files were organized differently that I was used to, and so forth. Still, I would have hoped moving from Apache2 on RedHat to Apache2 on Ubuntu would have been easier.

Empirical evidence was suggesting that all of my user’s virtual sites were working, but all of mine, no matter where I had them on the system, were reporting the error: You don’t have permission to access / on this server.

Here’s what was going on, primarily recounted so if I ever do this to myself in the future, I’ll know what to look for.

The virtual host files, which now appear in /etc/apache2/sites-available as *.conf files, had a slight difference. Some of them had this in their Directory directive:

    Order allow,deny
    allow from all

Mine did not. But, then again, some other websites did not as well. Turns out those directives were placed inside their .htaccess files.

Now, not all sites had the .htaccess file, and things had been working before without the explicit directive in each virtual host .conf file.

Turns out I had some how tromped on the default file, which contains a directive that looks like this:

    <Directory />

      Options FollowSymLinks
      AllowOverride None

    </Directory>

If it is not present, then all virtual hosts must explicitly allow access (via .htaccess or their .conf file).

This directive allows Apache to serve up any file the URL asks for… which one may not want to do. It seems the secure way is to edit the virtual host .conf files, and not rely on some default magic.

But, because that was in my old configuration long ago, and not in the new one, my virtual host .conf files didn’t have it, but my more modern ones for my users did. Depending on which template I used to base new sites off was how some sites worked and some didn’t.

After fixing this, I ran into a new problem. Some sites weren’t coming up still, but this time with permission errors.

When I migrated over the web content, it preserved user and group ownerships from the other system. These did not match the new Apache2 user and group on the new system.

However, I got lucky. There was no user/group mapping on the new system, which meant I could execute a find command to find and fix them. It looked something like this:

    find /home -nogroup -print
    find /home -nouser -print
    find /home -group 49 -exec chgrp www-data ‘{}’ \;

FIX: undefined symbol: apr_ldap_ssl_init

Did an update to Ubuntu Jaunty and Apache stopped working with the message “undefined symbol: apr_ldap_ssl_init”. This post is how I fixed it.

This is a geek entry for resolving the problem:

* Restarting web server apache2
/usr/sbin/apache2: symbol lookup error: /usr/sbin/apache2: undefined symbol: apr_ldap_ssl_init [fail]

Non-geeks will want to move along…
Continue reading “FIX: undefined symbol: apr_ldap_ssl_init”

LIBLDAP2 Not Installable

Unable to find a solution related apt-get failing on Ubuntu while trying to upgrade packages depending on the libldap2 (>= 2.1.17-1) package, I figured out what was causing the problem… additional repositories in my /etc/apt/sources.list file. Here’s what I did to finally be able to upgrade cleanly.

Warning this is a very geeky entry aimed at apt-get users of Ubuntu, readers seeking humorous content should skip this post. Remember, this is a technical blog.

If you’re still with me, then I suspect you’ve just been plagued by the message:

Depends: libldap2 (>= 2.1.17-1) but it is not installable

I’m using Ubuntu 8.04 LTS Server Hardy Heron, specifically on a 64-bit AMD system.

Normally, when I do an $ sudo apt-get update things go very smoothly, but not today. Here’s what I got.

The following packages have been kept back:
alpine dovecot-common dovecot-imapd dovecot-pop3d libpq5 postgresql-8.3 texlive-base-bin trac

The following packages have unmet dependencies:
alpine: Depends: libldap2 (>= 2.1.17-1) but it is not installable
dovecot-common: Depends: libldap2 (>= 2.1.17-1) but it is not installable
Depends: libpq4 (>= 8.1.4) but it is not installable
libpq5: Depends: libldap2 (>= 2.1.17-1) but it is not installable
postgresql-8.3: Depends: libldap2 (>= 2.1.17-1) but it is not installable
texlive-base-bin: Depends: libpoppler0c2 (>= 0.4.2) but it is not installable
trac: Depends: python-genshi (>= 0.5) but it is not going to be installed
E: Broken packages

Unfortunately, where ever I went, I didn’t find a solution. [1] [2] [3]

The ‘recommended’ solution is: $ sudo apt-get -f install
This did not work for me, nor others.

Neither did: $ sudo apt-get dist-upgrade

At this point, I went on an apt-get remove and apt-get autoremove binge. This didn’t help either.

This got me into a horrible loop, where packages sysvinit-utils, sysvinit, and initscripts needed to be installed, but could not because:
Unpacking sysvinit-utils (from …/sysvinit-utils_2.86.ds1-47~bpo40+1_amd64.deb) …

dpkg: error processing /var/cache/apt/archives/sysvinit-utils_2.86.ds1-47~bpo40+1_amd64.deb (–unpack):
trying to overwrite `/usr/share/man/man1/mesg.1.gz’, which is also in package sysvutils

I even tried manually installing packages one at a time. Didn’t work. I was even so desperate as to move the file mesg.1.gz elsewhere. That didn’t work.

Then I tried the following and things got a little better:

$ sudo apt-get clean
$ sudo apt-get autoclean
$ sudo apt-get check
$ sudo apt-get purge
$ apt-get -f upgrade

But I now had a problem where packages, specifically alpine, depended on on libdlap2, and it was telling me that it couldn’t install it, so upgrading wasn’t possible.

I made the mistake of $ sudo apt-get remove alpine, which would not let me undo that mistake by reinstalling.

My hunt brought me to libldap2-dev, but while this installed, it didn’t help alpine’s dependencies.

Even with the super-duper do-everything command, nothing helped:

$ sudo apt-get update && sudo apt-get upgrade -y && sudo apt-get dist-upgrade -y

Then it dawned on me, perhaps some of the repositories that I added to /etc/apt/sources.list were giving conflicting dependencies. Luckily, I annotated heavily what I had ever added to this file.

There were only two things: Subversion, and Mono. Here they are. You want to comment out these lines:

## Subversion obtained from https://edge.launchpad.net/~clazzes.org/+archive
deb http://ppa.launchpad.net/clazzes.org/ubuntu hardy main
deb-src http://ppa.launchpad.net/clazzes.org/ubuntu hardy main

## Mono added by request of FogBugz installation
## http://www.fogcreek.com/FogBugz/docs/60/topics/setup/UnixGettingYourServerRead.html#deb
deb http://www.backports.org/debian etch-backports main contrib non-free

Then, I did a $ sudo apt-get updatee, followed by a $ sudo apt-get dist-upgrade, then a $ sudo apt-get dist-upgrade.

All of my problems were solved. No package dependency problems what-so-ever, and I was able to install alpine, and all the others, bringing me up to the latest and greatest.

Finally, I uncommented my sources.list file back to the way it was and tried the upgrade again. No errors. Everything was fine.

The solution was that something, and I don’t know which one, was causing conflicts. Reverting back to the virgin sources.list file state was enough to get Ubuntu happy to do the upgrades.

Unfortunately, since re-commenting the lines didn’t reintroduce the problem, I’m unable to tell you which repository caused the problem in the first place.

Ubuntu and Parallels Rescue Broken XP

How I recovered a badly damaged XP box with severe Windows Update problems using Ubuntu and Parallels for OS X.

Normally, I don’t provide XP support, however, because I was the one who recommended the owner perform a Windows Update that precipitated the total incapacitation of the machine, I felt a slight guilty streak of obligation.

Because of the horrible reputation of Windows Genuine Advantage disabling legal installations, the owner of the box disabled all Windows Updates for fear his system would become disabled and he’d lose his data. As such, when I recommended keeping the system patched, there were well over 60+ patches to start with.

Frozen XP DesktopProblem was, one of those patches was for the NVIDIA GeForce Ti 4200 graphics card, and during the installation process, when the Microsoft Version was applied, the machine froze, requiring a manual reboot via the reset switch.

Naturally, after a forced shutdown one should invoke a check disk. However something insidious occurred. Explorer, and I don’t mean Internet Explorer, no I mean Explorer – the GUI shell, would lock up shortly after login. The start menu would go dead, icons didn’t function, start/run couldn’t invoke programs, applications invoked from the command line wouldn’t work, Internet Explorer wouldn’t even start, and Windows Update did nothing. Even Ctrl-Alt-Del wouldn’t work, as the Task Manager couldn’t start. Nor could the user logout or shutdown the machine. Things were bad. It was like the desktop was there, but the underlying services that made it function were dead.

I’ve had easier recoveries from the blue screen of death. If you can get past that, usually you got yourself a working system. In this case, the system would boot, and even allow a login, but once there, the interface wouldn’t function.

Of course you’d think booting and reverting to the last known good configuration would help. It didn’t. Safe mode was equally hosed. Anything past the login prompt rendered the machine in a frozen state, popping up a message about a Windows General Services failing, with an option to report the problem to Microsoft.

That’s the state of the machine as I received it prior to repair.

Here’s how I fixed it.

The detail message reported that the offending file as WUAUENG.DLL. A quick Google search showed this was the Windows Update module. It seems between going from Windows Update to Microsoft Update, the DLL got corrupted. As Windows booted after login, it accessed the DLL, and the system froze.

My goal was to replace at least this file from a working system. Problem was, I was in a catch-22. I couldn’t access the broken system, and if it was possible, the files would be in use by the operating system anyhow.

I downloaded Ubuntu and burned it to a CD using OS X. I then booted off the live CD on the broken machine, however while it could see the NTFS volume, it couldn’t write to it.

So, I enabled all the repositories by going to System / Software Sources, making sure Universe and Multi-verse were included. Then I opened up the terminal and entered sudo apt-get install ntfs-config, and installed the package that allowed writing to NTFS drives.

I added root to the fuse group, and then went to Applications / System Tools / NTFS Configuration Tool. It was quick to tell me I needed to run ntfs /dev/hda1, which fixed the volume and set it to check the disk on boot.

I shutdown Ubuntu, booted Windows, which caused a check disk, and when I finally got to the login prompt, shutdown again without ever logging in.

I booted back off the Ubuntu CD, did the same trick as before with the repositories and installation of the NTFS driver, and this time was able to mount the drive as writable.

I went to the WINDOWS\System32 directory, and found the following files, to which I renamed them, appending .old to their extension for the purposes of a backup: wuaueng.dll, wuaueng.dll.mui, and wuaueng1.dll.

Then I booted Parallels on OS X, brought up a copy of XP, went to its C:\WINDOWS\System32 directory, and copied those three files to a USB stick. I unmounted the USB stick and shutdown Parallels.

With Ubuntu still running on the broken machine, I plugged in the USB stick, which instantly appeared on the desktop, and copied over three files to the broken machine’s system32 directory.

I then shutdown Ubuntu, removed the USB stick and CD, and booted into Windows. The error message was gone, but it was obvious things were still fragile.

Back on OS X, I downloaded Windows XP Service Pack 2, burned it to CD, and stuck it in the broken machine, executing it. A bit later, it finished and I rebooted.

I was suddenly able to run Windows Update again, and that downloaded 40+ updates, effectively jump starting the process by grabbing only the critical updates. In a rise-lather-repeat cycle, I did this until all the critical updates were obtained. Then I did the same with the optional software.

Each time I came in from a mandatory reboot, I made a system restore checkpoint.

Just to confirm it was the NVIDIA driver, I downloaded just that option from Microsoft, and the machine locked up. Which, to get out of I had to hit the reset button, screwing up the disk again. No problem though, I booted, holding down F8, and booted to the last known good configuration. When it came up, I right clicked properties on the C: drive, and forced a check disk, rebooting. The machine came up fine.

Going over to NVIDIA’s site, it was a trivial matter to download the latest driver for the GeForce 4200 card, and unsurprisingly, it worked without incident.

Ubuntu saved the day for being able to repair and manipulate the NTFS volume, while Parallels made it possible to see what needed fixing, where it went, and a working copy without having to have a second dedicated Windows box.

A recovery solution wouldn’t have been possible with an disc of an OEM version of XP alone. Honestly, I don’t know why users put up with this, or how Microsoft can sleep at night.

The recover process, non-stop, took from 10am – 7pm straight. No breaks. No food. No stalling. That’s nine hours of my life I’m never getting back.