Ramblings of Narc

a.k.a. “Inter-dimensional vortexes? In my database?”

Yah, so I screwed up. Back when I moved the Web server from one computer to another, I thought I might’ve forgotten something important, but couldn’t quite figure out what it was. Then I forgot about that, too, and everything was just fine until yesterday morning…

Being that I was pissed off by the web server’s inability to hold an erection function properly without periodically stopping and starting udev (seriously, I still have no idea why that was happening!), I decided a dist-upgrade to Jaunty might help. I’d already done a couple of them, so I wasn’t totally going into the unknown… And it seemed to work, too! It asked for a reboot, I gave it one. It lost its nvidia driver, but asked for another reboot, so I gave it that one, too… and then things got heavily reminiscent of the Windows days.

You see, after that last reboot, the system would come up, all the way through GDM and showing the nice xfce desktop… and then reboot. Out of the blue. Lather, rinse, repeat.

At this point, I realized I was more or less fuckt… so I dug up a fresh Jaunty iso, burned it to about four CD-RWs before one finally worked, and found I was supposed to plug in a monitor because the install GUI wouldn’t come up on the TV, and… sort of backed up the interesting bits. You know, the old /home, and the old /etc, and also the /opt/www dir.

Did you notice me forgetting anything at this point? If you said “THE DATABASES!!!1oneone!”, you’d be right. Yes, I forgot to back up the databases. The reason I forgot is that I remembered there being 4x daily dumps to a directory inside /opt. Which there are. On the old web server. The one that’s not running anymore.

So, long story short, the databases are back to where they were before I finally disabled apache on the old web server. I’m going to recover the old posts from Google cache (come to think of it, I probably also still have them in the Atom feed) and repost them, but the 4 comments that were posted in the mean time are gone. So sorry.

And after I do that, I’ll turn on that damned automated backup…

I don’t believe I’ve mentioned this here on the blog before, but I work what might be defined as the night shift. More accurately, I work eight hour “days” in the time zone of my (de facto) employer, who happens to be in Toronto, Ontario, Canada. What this translates to is a seven hour time difference (with the exception of daylight savings, where for one week we’re only SIX hours apart… but that’s another blog post), and it effectively means I work from 8 PM to 4 AM every weekday.

Why, yes, that does mean I work Saturday mornings. It also means I don’t work Monday mornings. No big difference, if you ask me.

Anyway, the major problem with doing all this is, as pretty much anyone who’s ever worked night shift will tell you: other people. More specifically, having to function in a world where pretty much everybody (that matters) works on a different clock than you do, which means they’re not going to think twice about calling you at 10 AM to ask you if the online order you made at 2 AM the previous morning was still valid, and if so, to tell you how long it’s going to take until delivery can be made. And they shouldn’t have to think about it, really, since you’re the edge case here and they have to optimize for the common cases.

But, what this also translates to is a situation where it’s very possible to not be able to sleep (or at least, not be able to sleep long enough), especially if, like me, you have trouble falling asleep again after having been woken up. Which leads me nicely into a very short story: last week.

Last week, the winter holiday period ended, and a lot of people went back to work. This effectively meant that the two online orders I’d made over the holidays finally got some human attention, and those humans were very interested in getting the backlog of orders cleared up. As a result, I got an average of six point something hours of uninterrupted sleep last week. Now, I know people generally tend to vary a lot in how much sleep they need, and I suspect I’m off to the side of the bell curve here, but I generally tend to need about nine hours of sleep per day, or else I get… slow. And being slow makes me cranky, because my brain isn’t processing information as fast as it should, and it pisses me off. To all the people who’ve ever had to deal with me in this state: I sincerely apologize for taking it out on you. I promise, if you weren’t sure, it was never anything personal. I like you guys, and I appreciate you all.

But that’s just one of the problems I’m facing, and if it were the only one, I could deal with it very simply (as I usually do, actually): by making fewer purchases, and lumping things together as much as possible. Unfortunately, the really big problem, as highlighted very nicely just now, are the neighbors.

Let me make a little aside here and explain my living circumstances: I live in a two bedroom apartment I share with my mother (for two simple reasons: 1 – she’d be basically alone now that my Dad is dead, and 2 – I don’t have to pay rent), in an eight-story building of over 100 such apartments (some smaller, most the same size), facing the back of the building, which is shared with three other very similar buildings (which creates an almost square enclosure — come to think of it: my house, let me show you it). We also have a high-school on the other side of the square (that C-shaped thing with partially shiny roof in the picture (it’s all shiny now, if you’re wondering)).

What all this adds up to is an enormous opportunity for noise. At night, when all the kids are home and (probably) asleep, and all the people in their respective homes are (almost definitely) asleep, it’s so quiet you could hear a pin drop eight stories below. During the day, on the other hand, it’s madness. Now, I’m happy to enjoy some pretty damn good windows (the physical kind, not the Microsoft kind), but even with them, if I haven’t made it to sleep by 6:30 to 7 AM, I’m going to have a hard time falling asleep with all the noise outside.

And that fails to mention the (very many) times the neighbors decide to do some home improvement. For the most part, once I’ve managed to fall asleep, it takes a lot to wake me up again. Unfortunately, most home improvement nowadays seems to involve some very loud power tools, especially drills, which not only make an incredible amount of noise when you’re around them, but which also make noise that tends to travel through the walls, because that’s what the drill bit is being applied to. The end result of this has been innumerable times that I’ve been woken up by the horrible noise of drilling, accompanied many times by a desire to use that power drill on the neighbor’s head. Unfortunately, this being Romania, that kind of activity is not as rare as someone from a civilized country might think it is. Instead, it’s almost a daily occurrence during the summer, and at least weekly during winter (WTF?). Sometimes I wonder how long it’s going to be until the building itself, having been drilled so full of holes, is going to crumble under its own weight. But I digress.

The conclusion of all of the above is that going against the “rules” of society, be they the simplest ones like what time it is, or the really complex pseudo-religious wars of abortion and gay rights and all that other stuff, is not for the faint of heart. It’s tough, in that it’s very difficult, and you may often be pissed off at the world around you just for being the way it is. But if you think it’s working, keep doing it. Just, don’t forget to vent once in a while.

---

This has been another post in the hopefully long series of “personal experience” posts, wherein I detail the things I’ve learned and how that learning came about.

The previous post was Blogging For The Perennial Lurker, a somewhat meta expression of my experience as a habitual information consumer rather than producer.

The first post in the series is Recognizing Failures, explaining how to handle a potential failed project, and what to do about it.

I’m part of what’s believed to be the majority of Internet users — a consumer of information, for the most part, rather than a producer. Obviously, this blog is an attempt to shift my own personal ratio further towards the producer side, but I still most often find myself reading a lot rather than writing.

I would estimate that slightly more than 50% of my online reading is sourced from blogs, forums, or some derivations thereof, all of which tend to offer a relatively low barrier of entry to the process of commenting on the original article. And pretty much all blogs I’ve read that have said anything on the subject have stated that comments are a supremely important part of any such system (comment spam notwithstanding), because they allow the process of owning a blog and creating content for it to become a two-way interaction with its readers. I agree with this view, but I have some caveats to add:

• Firstly, I have often come across interesting articles whose discussions I would’ve liked to take part in, or about which I had some insights, or wanted some clarification. Almost invariably, skimming through the comments to such an article tended to show other readers had been there already, and already asked the same questions I was going to ask. What this implies is that for any given article, there are generally a limited number of potential interesting, on-topic comments that can be added. Beyond that point, the resulting communication, if any, is likely to be extremely repetitive, and a potentially misunderstood repetition of the previously well thought-out information at that.
• Secondly, as the logical extension of the above, when reading an older article from even a moderately successful blog owner, the likelihood of making a useful addition to the discussion tends to drop proportionally with the age of the article. Caveat to the caveat, though: if older posts have few (or no) comments, they are likely to contain plenty of hidden gems of discussion waiting to be uncovered.
• Thirdly, there are a lot more interesting people than you think there are out there, who think the same way as you do, whoever you are.

These are possibly common-sense observations, and potentially I’m repeating what a lot of people have already said, but that follows from my first point above. And the most interesting implication of all this is that, to be a blogger, one does not strive to write things that have never been written before — because that’s impossible. Instead, one acknowledges that there are other people who have expressed the same things they are expressing, and instead one tries to write successfully in the battle for popularity.

What this means, in this context, is that if you’re considering blogging as a hobby, with a goal of popularity — being read and written about — you should be looking towards:

1. Maximizing the number of potential interesting comments, thus giving your readers the chance to comment — giving them a stake in the success of your blog.
2. Writing a lot. I don’t want to add any quantifiers here, because “a lot” means different things for different people, and depends on too many factors to enumerate, but I will attempt to explain why: because every new post you write opens up a new series of potential comments, and reduces the average age of posts on your blog. Be forewarned, though: this is a diminishing returns function — the older your blog, the more new posts are required to reduce the average age. On the other hand, if your older posts are poorly commented on, new posts (and hopefully new readers that go with them) increase the older posts’ chances of coming out into the limelight and having their moment of glory. Which also has diminishing returns, of course.
3. I’m not sure what to do with the third point, honestly. Perhaps the rule here should be: be confident that, however poor your popularity might be now, there are people out there willing to read your posts and think about what you’re saying. At the same time, don’t forget that there are likely to be even more people who disagree; and more people still who don’t want to think about it. Don’t be discouraged by this! Accept it, and have your delete button/link handy for the inevitable flame-bait and trolling.

---

This is the third of hopefully many “personal experience” posts, where I detail the things I’ve learned and how I came to learn them.

The previous post was One Is The Most Important Number, about not repeating yourself in software development.

The next is Living In A World Out Of Synch, a long ramble about working what is, effectively, “the night shift”.

I’m sure any half-decent software developer has had this drilled into their minds many times over, but perhaps it’s worth repeating (otherwise, feel free to skip this post): don’t repeat yourself!

Now, I’m probably a statistical outlier in that most of my programming education (and computer education in general) has been of an informal nature: I taught myself. So when trying to get points across to others I tend to refer to my experiences for examples of what can go wrong. So here’s how I ended up learning the “don’t repeat yourself” rule:

Way back in 2005, I was trying to figure out how I could standardize my PHP applications’ layouts on disk. I’d started in early 2004 with a single firm idea: that includes went into a separate subdirectory, from whence they could be referenced by any script the application was comprised of. From there, I got the idea of doing the same for images, stylesheets, and so on. By early 2005, I’d set up a simple template system I still use today, consisting simply of text files with placeholders for variable data. For example, this would be a valid template text:

Hello, {{UserName}}, and welcome to {{ProgName}}! Where do you want to foo today?

The bits between curly brackets are the variables, and they can get filled in with basically any text you want. For instance, set “UserName” to “Narc”, and “ProgName” to “CMBlog”, and you get:

Hello, Narc, and welcome to CMBlog! Where do you want to foo today?

Simple, clean, and surprisingly fast, at least in my use case.

I liked this so much that I decided to use it in all my personal projects. And that’s where I started running into trouble.

You see, what I ended up doing was copying the code from one project to the next, but then I’d find a bug in it and have to update all the copies. Can you say, oops? But wait, it got worse: I had some ideas for improvements, extra features that would be handy, like automagically including other templates into the current one by using a special version of the curly brackets placeholder. But whenever I thought of it, I either added the improvement only to one version of the code (whichever the latest project I’d been working on had been), or I didn’t add it at all because of this duplication tax.

Now, clearly, the tax is not that high, after all, I only needed to copy some files from one place to the next. Maybe I’m just too lazy for my own good (though there is some belief that laziness is a good feature for programmers, but that’s a topic for another post), but clearly the multiple copy model wasn’t working for me.

So I decided, instead of making these multiple copies, I would keep all the interesting, reusable bits of code in one place, and just include them as needed. But I also knew that reading in all those bits of code would be a performance hit if they weren’t needed, so they had to be split up into segments of functionality. Like the template system I just described above, or a set of functions that make database interactions less painful.

What I ended up creating was a very interesting (to me) system of includes driven by a bunch of configuration variables. You could set a variable (a key of a well-defined and standardized array) and the bits of code you’d need would be included, and you could use the functions you needed without including any of the dozens of other functions (the current count is 176, if you’re curious), and thus without wasting CPU time waiting for unnecessary resources to load from disk. That system is now called NeoFW and I still love it very much. It lets me make changes in one place, and have them show up instantly in all the places it’s used.

Let me restate that, in case I haven’t bashed you over the head with it enough: I make one change in one place, and it shows up everywhere!

On top of that, this code provides an interface for user-handling functions, meaning I can have a user log in only one time in one application, and have him logged into all of them at the same time. This is a wonderful change from the previous implementation where I had to be careful to use the same session variables across all applications, and the same session storage method, and heck, the same code, for that matter. Which is why it was perfect for this system.

Generalizing from this example, we find that it’s a very good idea not to duplicate code when you can help it. Don’t repeat yourself. When you repeat yourself, one of two things (or a combination thereof) is bound to happen: either you’ll end up with synchronization issues (some versions of your code work differently than others), or you won’t want to make any more changes to your code because it’s so much of a hassle to keep track of where else that same code exists.

If I’d started with a Linux server, I’d probably have ended up using a lot of symlinks, and then I wouldn’t have made NeoFW, which would be a shame, because I’ve learned many useful things while putting it together. But I couldn’t have failed to learn this lesson: that the best number of repetitions of your code is one. Not two, not four, but one. And, of course, five is right out, as Monty Python taught us.

---

This is the second of hopefully many “personal experience” posts, where I detail the things I’ve learned and how I came to learn them. The previous post was Recognizing Failures, about not biting off more than you can chew. The next is Blogging For The Perennial Lurker, a slightly meta expression of my experience as a habitual information consumer, as applied to the inherently productive medium of blogging.

(or, just how much can you chew?)

Way back in late 2005 (a lifetime ago, as far as I’m concerned), a friend of mine came for a visit one day. He was coming to me on behalf of a (non-mutual) friend who needed a programmer because he had an application in mind. He tried to explain the application to me, and on the face of it, I didn’t think it was a particularly difficult one. I still don’t, actually, but that’s a purely academic distinction at this point.

So because I didn’t think it was going to be that difficult, I went ahead and told him to have his friend contact me; he did, and he did, and we met, and I got to sketch out a reasonable idea for the program both in my mind, and hopefully in the client’s mind as well.

And then things got bad.

I strongly suspect I was right, that the project was not particularly difficult. I also suspect that, just as I couldn’t finish (or even start) it then, I would be completely unable to finish it now. It’s just beyond my capabilities.

The first hint of this “too big to chew” property of the project was when I tried to start writing some kind of database abstraction for it and kept coming up with flaws in that abstraction. I had gotten a number of details on what the final application would have to deal with, and even though I had decided (together with the client) that the initial version would be limited to some much simpler-to-model data, it became more and more clear that this initial iteration would have to be thrown away almost entirely after its implementation.

And here’s where I went wrong: I could have created the initial, very limited application. It would’ve had to have been (at least partially) thrown away after its launch, and the next iteration would probably have been more complex, but it would have been something tangible, more than just a concept in our heads, but something that could be interacted with, could be tested, could be considered for the recycle bin or for the improvements treadmill. Instead, I froze. Almost literally. I couldn’t write the code. I couldn’t make it happen. All I could think of was how complicated an architecture would be required, and how it was so completely beyond me because man, I’ve never written something like this before and I’m so fucked; I told this man I’d deliver something and now I can’t deliver, I can’t do it, and I’d rather die than see this man again and dash his hopes after I made it sound so good…

Now, if that sounds like I’m maybe hiding the fact that I couldn’t bear the shame of not being able to write something, and I’m lying to myself (hello, armchair psychologists out there on the Interwebs!), the fact is I’m not. Yes, I was ashamed, but not that I couldn’t do it — I was ashamed that I’d ended up lying to my client about being able to do it. And, to be honest, I don’t think I even ever said I would be able to do it, but I’d made it sound so good, like it was just a few steps away from being real… which, of course, it never had been — except in my head. Before I’d really thought about it, that is.

So, the lesson I took from this was: don’t promise you can do something you’ve never done before, and don’t make it sound like you can. Because if you end up biting off more than you can chew, you’ll make a liar out of yourself, and your client will not only not want to hire you again, but they will also tell their friends that you’re unreliable, that you make promises you can’t deliver on, and that will fuck you more than anything. Obviously, this is just after-the-fact reasoning, the real reason for me was that it made me feel like shit — but it also happens to be true, and my feeling made me look at that, analyze it, and see it for myself. And now I can share it with you, anonymous reader.

---

This is the first of hopefully many “personal experience” posts, where I will detail the things I’ve learned — some good, some not so good. Merry Xmas, Internet.

The next post in the series is One Is The Most Important Number, and refers to the age-old programming concept: “Don’t repeat yourself”.