On RSS geekery, and LiveJournal. Yes, I still use that–sort of.

One down side to my having moved away from LiveJournal and back to hosting my own blog is I have even less reason to go back and check various LJ pages. Lately, it’s included the friends page. Sort of sucks, since there are still several LJ users I want to keep up with, in spite of the fact I’m drifting away from using it myself.

A couple months back, in my quest for an RSS reader I could use from anywhere, I came across this software, which runs on a self-hosted environment. So far, it’s working out a lot better than I’d figured it would–so well, in fact, that I’ve actually set it up for Jessica to use. Now, if we can just get her using it. I’ve recently started adding the RSS feeds of various LJ people I follow, just so I don’t have to remember to pull up the friends page–I go into my RSS reader on a daily basis anyway, so they’ll be right there with everything else I read. So far, I have it set up to access all the public entries. But, thing about being on the friends list of some of these folks is that I actually like to read, you know, actual content. No such luck so far.

I tried to implement a form of basic access authentication, passing the username and password I need to give the LJ server as part of the RSS feed’s URL (example: http://me:password@you.livejournal.com). either LJ or the feed reader, though, doesn’t like that–I still only get the public entries. Trying to massage LJ’s version of digest-based authentication, which I don’t think the RSS reader likes either, gets me no farther.

LJ does some funky things with its digest-based authentication, which doesn’t lend itself to much in the way of strict URL manipulation. So I can’t, for example, do something relatively simplistic like, say, http://me:password@you.livejournal.com/rss?auth=digest and expect it to work. I mean, it should, but it doesn’t.

So, for the moment, I have some partially working LJ RSS feeds, and plenty of time to exercise my google skillz. In the meantime, so far what I’m using seems to be working fairly well. If I can just crack this digest auth issue, or find some other way to do it that doesn’t seem to be horribly broken, this thing’ll run like a dream. fortunately I have all kinds of time to do it. And I’ve been looking for an excuse to mess around with http stuff anyway. Probably just be a quick and dirty hack, but quick and dirty’s good, if it works. And this had better bloody well work.

Update. Well. It works. Kind of. Except not really, and not quite the way I was hoping. Messing around with trying to get my RSS reader to play nice with digest auth just wasn’t happening. Probably a good thing–I have too many people to be trying that on, and it’d get really freaking tedius after a while. Say, after about 5 minutes. So I did the second best thing. I cheated.

It’s quick, it’s dirty, and it may not amount to being a whole lot more than a temporary hack, but it gets the job done. Well, when it wants to. It involved snagging a copy of LJProxy, which is supposed to be able to make use of their login implementations, thus cutting my headache in half right there. It takes the LJ username/password I’m more than willing to provide it, makes use of LJ’s own client/server API–the same API any LJ client uses to do things like post entries, and grabs a list of friend groups and the friends you’ve listed in the said groups. Then, it makes easy use of their broken implementation of digest auth to nab the RSS feeds belonging to those friends. Finally, it compiles them into a single RSS feed, stored locally, that I can then plug into my feed reader–making about ten times less work for one lazy geek. Then I just slam that one feed into my reader, and away she goes.

Now, if it starts working on a regular basis, I’ll have what you might call my ideal package. There’s only one small problem. It doesn’t quite work that way yet. The instalation process was easy enough, and it does run–just not all the way through. It gets stopped before it can complete its pulling in of content and build the agrigated feed, possibly due to resource issues but even more likely due to problems staying connected to the LJ service. Currently, I’ve got 30 people in the group I’m testing it–the documentation says for best performance, keep it under 50. It’s only gotten all the way through the 30 maybe… twice. The highest I’ve personally seen it go, and stop, is 20. right now, it’s stopped at 8. How do I know it’s not a resource issue? During its second to last run so far, I pulled this from the server.

04:49:04 up 6 days, 20:36, 2 users, load average: 0.00, 0.02, 0.00

So, during the running of the collection routine, the system’s barely being touched. That rules out being killed on grounds of causing issues with something else. So I’m leaning towards timeout issues with LJ. When I get motivated, I’ll poke around and see what I can do about maybe correcting that. But, it’s entirely possible I’m going to have to wait for the connection to improve on its own. Or, failing that, go back to the drawing board. Fortunately, on this, I’m flexible. And, in the meantime, my dream rss solution is near. And hey, it’s an excuse to break something. I can deal with that.

2 comments
  1. Ah damn, so that means anything friends-only won’t show up?

    1. Not yet, no. I’m working on it, though. Now here’s hoping I don’t bust the intertubes while doing it.

Leave a Reply

Your email address will not be published. Required fields are marked *