atkins

Friend's Interests

So far, my friend's interests program is coming along nicely, but there are still a few bugs to work out. Excluding communities, syndicated feeds, and a couple of dropped connections (argh!), my top friend's interests work out like this:



Note that this list excludes interests that only one person lists. This is because I would be adding an extra three thousand interests if I did.

dancing 21
movies 19
reading 18
sex 17
portland 16
music 14
travel 14
books 14
politics 13
poetry 13
photography 13
coffee 12
art 12
history 12
sushi 12
cats 12
the cure 12
tattoos 12
industrial 11
writing 11
geeks 11
perl 10
philosophy 10
candles 9
wine 9
friends 9
nature 8
neil gaiman 8
love 8
rain 8
computers 8
depeche mode 8
corsets 8
learning 7
cooking 7
boots 7
piercings 7
hiking 7
beer 7
dead can dance 7
skinny puppy 7
goth 7
perlmonks 7
beauty 6
yoga 6
tea 6
jazz 6
anime 6
black 6
swimming 6
conversation 6
npr 6
honesty 6
chocolate 6
road trips 6
unix 6
erotica 6
comics 6
salvador dali 5
opera 5
astrology 5
halloween 5
oregon 5
san francisco 5
vnv nation 5
creativity 5
david lynch 5
simpsons 5
mythology 5
mountains 5
classical music 5
science 5
sarcasm 5
science fiction 5
johnny cash 5
radiohead 5
seattle 5
haujobb 5
darkwave 5
programming 5
synthpop 5
portishead 5
wumpscut 5
germany 5
ebm 5
words 5
kissing 5
museums 4
new wave 4
beaches 4
passion 4
horror movies 4
massive attack 4
die form 4
singing 4
nightmare before christmas 4
noam chomsky 4
bjork 4
independent films 4
family 4
used book stores 4
meat beat manifesto 4
thai food 4
edward gorey 4
buddhism 4
invader zim 4
electroclash 4
pacific northwest 4
neal stephenson 4
water 4
intelligence 4
drawing 4
japan 4
william gibson 4
gin 4
velvet 4
antiques 4
terry pratchett 4
chaos theory 4
sonar 4
vodka 4
gardening 4
food 4
laughing 4
converter 4
star trek 4
literature 4
douglas adams 4
concrete blonde 4
freaks 4
cake 4
tolkien 4
bettie page 4
privacy 4
fetish 4
cheese 4
cocktails 4
industrial music 4
camping 4
massage 4
linux 4
the smiths 4
french 4
biking 4
lust 4
film noir 4
glasses 4
shakespeare 4
snow 4
noise 4
cocteau twins 3
beborn beton 3
nudity 3
astronomy 3
capitalism 3
ani difranco 3
men 3
david bowie 3
michael moore 3
akira 3
bookstores 3
rocky horror 3
london after midnight 3
gothic 3
red 3
flowers 3
romance 3
cowboy bebop 3
adventure 3
darkness 3
hope 3
red wine 3
feminism 3
dante 3
foreign films 3
buffy the vampire slayer 3
star wars 3
autumn 3
drum and bass 3
bauhaus 3
death in june 3
front 242 3
livejournal 3
german 3
twilight 3
civil rights 3
sewing 3
autechre 3
c++ 3
snog 3
noisex 3
get fuzzy 3
art nouveau 3
coil 3
ocean 3
spirituality 3
old books 3
living 3
thinking 3
bisexuality 3
powell's 3
ghost in the shell 3
thrift stores 3
irony 3
hockey 3
punk 3
xhtml 3
paris 3
web design 3
vegetarianism 3
winter 3
lord of the rings 3
cranes 3
socialism 3
forests 3
de/vision 3
incense 3
robots 3
nerds 3
vampires 3
blade runner 3
truth 3
liberal 3
chaos 3
women 3
physics 3
imagination 3
soil %26 eclipse 3
shoes 3
meditation 3
six feet under 3
surrealism 3
nag champa 3
art history 3
joy division 3
tim burton 3
walking 3
joy 3
fight club 3
plato 3
lain 3
aeon flux 3
elvis costello 3
change 3
revolution 3
icon of coil 3
black and white photography 3
kompressor 3
herbs 3
hello kitty 3
liberals 3
theology 3
animals 3
baking 3
open source 3
pink 3
cyberpunk 3
sociology 3
wolfsheim 3
ant-zen 3
bdsm 3
peace 3
the sims 3
caffeine 3
archaeology 3
covenant 3
nin 3
conservation 3
football 3
london 2
sylvia plath 2
magic 8-ball 2
asche 2
liberalism 2
theatre 2
pink floyd 2
flesh 2
soul coughing 2
animal rights 2
∞ 2
acoustic guitar 2
crafts 2
people 2
deep thoughts 2
education 2
the cars 2
china 2
djing 2
tiaras 2
sci-fi 2
martinis 2
motorhead 2
smiling 2
sting 2
nine inch nails 2
lycia 2
drugs 2
mp3 2
panacea 2
digital photography 2
butterfly messiah 2
melotron 2
geek girls 2
eminem 2
swing dancing 2
green 2
sumerland 2
board games 2
logic 2
:wumpscut: 2
talking heads 2
bats 2
parties 2
sleep 2
dragonball z 2
tetsuo the iron man 2
drag queens 2
hedonism 2
drinking 2
jello biafra 2
eating 2
mr. bungle 2
helpdesk 2
dar williams 2
gnosticism 2
thunderstorms 2
1984 2
atheism 2
tori amos 2
thine eyes 2
boats 2
the ocean 2
html 2
edges 2
goth music 2
the onion 2
ministry 2
kerouac 2
monty python 2
law & order 2
powerpuff girls 2
new order 2
informatik 2
akira kurosawa 2
debian 2
missy elliott 2
kilts 2
soccer 2
socrates 2
glis 2
manhattan 2
war 2
honey 2
rambling 2
conversations 2
smallville 2
project pitchfork 2
kandinsky 2
sonic youth 2
smoking 2
xml 2
csi 2
red hair 2
alternative 2
ireland 2
bookbinding 2
life 2
clubbing 2
talking 2
h.p. lovecraft 2
pugs 2
waterfalls 2
death rock 2
japanese seizure robots 2
masochism 2
eyes 2
rottweilers 2
perversity 2
christian death 2
empathy 2
big boots 2
cigarettes 2
folk music 2
candy 2
metropolis 2
shopping 2
vintage 2
cruxshadows 2
the occult 2
security 2
ayn rand 2
games 2
purple 2
h.r. giger 2
16 volt 2
billy bragg 2
taoism 2
the daily show 2
volcanoes 2
kids in the hall 2
system administration 2
hacking 2
punk rock 2
albert einstein 2
baroque 2
modern art 2
military history 2
dr. seuss 2
etta james 2
independence 2
ayria 2
bella morte 2
foetus 2
technology 2
texture 2
iron chef 2
karma 2
anne rice 2
democracy 2
norma waterson 2
parenting 2
democrats 2
autumn leaves 2
sensuality 2
costumes 2
india 2
bruce campbell 2
family guy 2
microbrews 2
clear vision 2
tarot 2
cultures 2
political philosophy 2
power noise 2
kid 606 2
funker vogt 2
tom waits 2
brian froud 2
london underground 2
silence 2
urban decay 2
exploring 2
strawberries 2
it 2
pez 2
clive barker 2
cartoons 2
economics 2
dragonflies 2
current 93 2
traveling 2
seinfeld 2
javascript 2
bicycling 2
new jersey 2
hair removal 2
black eyeliner 2
linguistics 2
samurai 2
crop circles 2
monkeys 2
light 2
pulp fiction 2
radical politics 2
chai 2
storms 2
neruda 2
the moon 2
theater 2
kurt vonnegut 2
moving on 2
blues 2
blood 2
decadence 2
oral sex 2
clerks 2
x-files 2
vegetarian 2
labyrinth 2
body modification 2
fate 2
whiskey 2
london.pm 2
tricky 2
god module 2
swans 2
snatch 2
leather 2
evolution 2
biopsy 2
martin carthy 2
snuggling 2
tool 2
bubble baths 2
corpus delicti 2
liquor 2
terrorfakt 2
activism 2
kevin spacey 2
waiting for guffman 2
bikini kill 2
meeting new people 2
extreme programming 2
firearms 2
walks 2
mulletsgalore 2
projekt 2
aromatherapy 2
oceans 2
pittsburgh 2
greg palast 2
howard dean 2
dark ambient 2
and one 2
hocico 2
oneiroid psychosis 2
lipstick 2
dave matthews band 2
voltaire 2
hypnoskull 2
happiness 2
rugby 2
dive 2
pigface 2
boys 2
al franken 2
dogs 2
stephen king 2
touch 2
big stompy boots 2
questioning 2
graphic design 2
culture 2
curve 2
painting 2
dreams 2
city lights 2
chemistry 2
pointy boots 2
transmetropolitan 2
free speech 2
tank girl 2
hugs 2
pool 2
pacific ocean 2
the lone gunmen 2
apathy 2
alan moore 2
david sylvian 2
rasputina 2
the clash 2
madonna 2
henry rollins 2
agnosticism 2
ancient history 2
challenge 2
fashion 2
depression 2
classic rock 2
the pixies 2
smashing pumpkins 2
clan of xymox 2
applied common sense 2
debate 2
jesus 2
dark amber 2
graveyards 2
bondage 2
ginger 2
manga 2
john cusack 2
eliza carthy 2
margaret cho 2
nietzsche 2
tom tykwer 2
andy warhol 2
gothic music 2
adventures 2
ren & stimpy 2
metrosexual 2
c 2
baseball 2
franka potente 2
black clothes 2
jack kerouac 2
guinness 2
europe 2
aldous huxley 2
twin peaks 2
gaming 2
libraries 2
architecture 2
school 2
suicide commando 2
potatoes 2
ice cream 2
abstract art 2
seabound 2
futurama 2
gas masks 2
photographs 2
fog 2
fun 2
johnny knoxville 2
dune 2
erotic fiction 2
bob dylan 2
modest mouse 2
banter 2
java 2
psytrance 2
holding hands 2
office space 2
24 2
tones on tail 2
schoolhouse rock 2
david foster wallace 2
john waters 2
scrabble 2
gargoyles 2
conspiracy theories 2
underworld 2
flirting 2
dashboard confessional 2
silliness 2
harry potter 2
vinyl 2
people watching 2
questioning authority 2
the breeders 2
leonard cohen 2
nursing 2
s.i.n.a 2
vampire hunter d 2
toys 2
ballet 2
mac's 2
public sex 2
clouds 2
akbar & jeff 2
quotes 2
live music 2
dorothy parker 2
socializing 2
scars 2
apoptygma berzerk 2
puns 2
the cruxshadows 2
powells 2
touching 2
curiosity 2
morbid curiosity 2
rocky horror picture show 2
lace 2
bruce lee 2
  • Current Mood: cheerful cheerful
So . . . you're going to make this program available on the Web, right?

(preferably with a banner for @political on the page)
I don't know how familiar you are with Perl, but if you are, you'll see a quick hack that I threw together and listed in the lj_clients community with a request for some help. It requires a couple of non-standard modules, so if you don't know how to install them, you may be out of luck.

One problem is that I can't make a single fetch, get the info, and summarize it. I have to make a separate request for every friend, grab the data, and summarize it. That can eat up bandwidth pretty quickly.

I've been looking into the LiveJournal protocol for dealing with RSS and XML-RPC, but the protocol is primarily limited to things you might want to do if you were using the LiveJournal Web interface. Trying to do data mining is not an option.
Suggestions
Without looking at the code, may I suggest sorting ASCIIbetically. Also - it may interesting once the data has been collected to use something like String::Approx to see if perhaps some of those 3000+ unique interests aren't so unique afterall. I'm assuming the biggest issue is having to fetch each page individually and not the actual parsing of the data, but it may be worth while to locally store an MD5 hash sum to avoid work on subsequent runs.

L~R
Re: Suggestions
Yes, a better sorting algorithm would be good. I didn't think about that, but it's trivial to implement.

String::Approx? Hmm ... another good idea. I'll give it a try. You are correct, though, that fetching the pages is the bulk of the work. Running this over dialup is sloooow.