October 15, 2004

Recommendations from/for Amazon

Posted by Arcane Gazebo at October 15, 2004 4:46 PM

In recent days I have been amusing myself by attempting to optimize my recommendations page on Amazon. The biggest difficulty is that Amazon is offering a system to recommend creations [books, albums, films] but I want recommendations of creators [authors, bands, directors]. I don't need Amazon to tell me that if I liked Quicksilver and Cryptonomicon and Snow Crash I should read the rest of Neal Stephenson's bibliography. Unfortunately my ability to teach it not to do this is pretty limited. Here's what's on my recommendations page at the moment:

1. [Book] Assassin's Apprentice, Robin Hobb
2. [Book] The Golden Age, John C. Wright
3. [Album] Franz Ferdinand, Franz Ferdinand
4. [Film] Lost in Translation
5. [Book] Royal Assassin, Robin Hobb
6. [Graphic Novel] The Hedge Knight, George R. R. Martin and Ben Avery
7. [Book] Assassin's Quest, Robin Hobb
8. [Book] The Diamond Age, Neal Stephenson
9. [Book] Fool's Errand, Robin Hobb
10. [Album] You Forgot It in People, Broken Social Scene
11. [Book] Pattern Recognition, William Gibson
12. [Book] Zodiac, Neal Stephenson
13. [Film] Underworld
14. [Book] Golden Fool, Robin Hobb
15. [Book] The Phoenix Exultant, John C. Wright

The first four items are actually pretty good suggestions: I haven't read either Hobb or Wright, and while I already knew about the Franz Ferdinand album and Lost in Translation, they are the sort of items I'd like to have recommended. (I've actually heard the album and liked it, though I don't own it. I haven't seen the film yet, but enough people have recommended it that I think I would probably like it as well.)

After that it goes downhill. Having decided I would like Robin Hobb, Amazon goes on to recommend four more of her books, along with another by Wright and four books by authors I'm familiar with. Can Amazon really only come up with two writers I might like? On the other hand, You Forgot It in People is an appropriate recommendation (although I haven't listened to it). Underworld is an interesting case: in terms of style and genre it's a good recommendation for me, but I've heard that it's terrible, and I'm inclined to say that Amazon shouldn't recommend bad movies. On the other hand, its customer rating is 3.5, so Amazon had no way of knowing that it's bad. (And maybe it's not. I should rent it because, hey, Kate Beckinsale.)

My recommendations list is heavily weighted towards books, away from music and films. This is because Amazon has information about many more books that I own than CDs or DVDs. The logical conclusion is that I buy more books than other media, and am therefore more interested in book recommendations. This isn't actually true: it's easy for me to find books that look interesting, but harder to find music that I think I'll like. Fortunately I can get a list of pure music recommendations. Amazon seems to be better at finding music for me than it is at finding books, in terms of suggesting new bands rather than the entire catalogs of artists I already own. But something mysterious is going on: the recommendations lists in particular departments aren't sublists of the main set of recommendations. For example, Franz Ferdinand disappears when I look at the music list by itself, new items appear, and others get reordered. Does it recalculate the list from a different set of preferences?

Anyway, I understand that getting a computer to make good media recommendations is an extraordinarily difficult problem. However, it doesn't seem like it would be so hard to include an option that would give a list of only the top results from each author, band, etc.


My 2 cents on the recommendations... Robin Hobb is very hit or miss. I've read some of the ship books (Ship of Magic, Mad Ship, etc), but they were mostly eh. I've heard both good things and bad things about the assassin series too, though i've been pondering trying the first one anyways.

Franz Ferdinand, if indeed the same person who has a song in the soundtrack for Burnout 3 is indeed very intriguing. I liked that track a lot.

Underworld is another I can speak about, as Lem and I saw it a while back, rented. I thought it was pretty good, considering my low expectations. Very stylistic and reminded me of Vampire a lot. Good mood movie before say, playing Vampire, etc. Lorian says she thought it was pretty good too, as far as action movie with no intelligence required goes. :) (which is to say it was good, I think)

Anyways... I just finished Storm of Swords... i'm gonna go rant about it on the open thread, I think.

Posted by: Zifnab | October 15, 2004 11:04 PM

Uh oh, you said Robin Hobb....

The Assassin's trilogy, and the following Tawny Man trilogy (they're really all together, I do not know why she split them except for a minor character focus shift, not even), is a million times better than the Ships series.
That's the series I fall back on when I just need to read something excellent, regardless of how many times I've read it. It's rich enough that you pick up different dynamics you didn't catch in the first read, and she hooks you into the story so well.
I think I would take her to Mass. and have a Civil Union with her, if I could.

If you decide on Lost in Translation, try not to see it in company. If you do, make sure they shut the hell up- It's a great film, but it's one of those that you need to focus on.
Was HedgeKnight the one he did as a comic?

I also suggest Garden State. It was excellent, and even the soundtrack got my attention. I think you'd like it. Besides....Natalie Portman.

"Anyway, I understand that getting a computer to make good media recommendations is an extraordinarily difficult problem."
We actually worked a bit on this last semester. Collaborative filtering, or some such method. One of the rare times I actually enjoyed the Information Retrieval class.
It's fascinating as hell and has terrific possibilities, but it really needs to be perfected. Like everything else =p Amazon, of course, was the prof's top example.
B&N's version is full of crap- It has recs for books that haven't even been released yet. I don't think they've bridged into recs for music and movies yet either.

Longest Vanessa comment ever... See what you did?

Posted by: Vanessa | October 16, 2004 7:53 AM

Thanks for the tips; I think I'll add this first Assassin book to the pile. (Or at least the virtual pile; I use the Amazon wishlist as a reminder of "items that looked interesting".)

Zifnab: I discover by Google that the track you're talking about from Burnout 3 is "This Fire". If you liked that one you'd probably like the rest of the album (it's fairly representative).

I eagerly anticipate your Storm of Swords commentary. As you may have guessed, Jaime became my favorite character by the end of the book (in fact, by halfway through).

Vanessa: I think there's also a short-story version of the Hedge Knight that's in one of the Legends compilations, but Amazon is recommending the comic version.

Judging from the bands on the Garden State soundtrack, I should at least hear the CD, if not actually see the film. I remember seeing a trailer and not having the slightest clue what it was about. But, Natalie Portman!

Posted by: Arcane Gazebo | October 16, 2004 3:40 PM

Underworld is an alright movie, but for an alright movie it is very enjoyable. Kate Beckinsale == Yum! She was kinda cute when she played Hero in _Much Ado_, but didn't draw my attention away from the rest of the movie. In Underworld, she makes the movie work. (Of course, she also goes from being an improtant supporting role to the lead, so eh... she also goes from "kinda cute" to "smoking hot")

Vanessa is dead right about Lost in Translation--it's a damn fine movie, but you wouldn't want to watch it with, say, me for example.

Franz Ferdinand--I've listened to samples on iTunes, and found it decent. Then, while playing Burnout 3, that one track ("Fire", you say?) has really grown on me, and yes it is representative of the rest, at least as far as I remember. I plan on buying the album.

There are lots of simple, accurant, and trivial to implement ways to tackle the problem of statistically estimating preferences. The easy ones (well, the really easy ones) are all n^2 or worse, however. There are, I believe, ways to do this elegantly in reasonable time, though I'm not sure. I'm tempted to look into this a bit. I could hazard a guess or two as to just how they implement their system, and I imagine it's something like what Everything2 does. The system itself doesn't know anything about the nodes, just that there is a 1-1 mapping between nodes and things in the store, and that nodes get links to other nodes reinforced when someone browses both items within some span of time. First of all, this system would obviously have exactly that problem of clumping things by a single author. Second, the algorithm (heuristic?) is so simple that adding in the ability to recognize the notion of an "author" would be a signifigant addition (relative to the size of the code, I mean).

Posted by: Lemming | October 17, 2004 2:35 AM

I think i'm likely to try the first Assassin book also. :) I wasn't sure how good a representation the ship series was, and since that has a lot of promise, hopefully the assassin books would be better. Actually, i've a bunch still on my reading list. Notably _System of the World_ by Neal Stephenson, _Jonathan Strange & Mr Norrell_ by Susanna Clark, and _Newton's Wake_ by Ken MacLeod.

Yep, "This Fire" was the song, and I really wanted to hear the full version. If that's a representative sample, that album's on my list to find. :)

Still a bit tired to write the commentary on storm of swords - Lem had a birthday dinner tonight, after which we went to his place and played a ton of DDR. We even got Lorian and Gary to play some. And I played for the first time in many years and hadn't lost as much as i'd thought I would.

Oh, and Jaime definately got a lot cooler. I know exactly the point, too: "A dozen quips came to mind, each crueler than the one before, but Jaime only shrugged. 'I dreamed of you,' he said." I still like Jon and Tyrion better (of viewpoints remaining ; ; ), though. If I keep writing i'm going to start writing the commentary right here and i'm really tired... more later. ^^

Posted by: Zifnab | October 17, 2004 2:38 AM

Hah, I see Lem has beat me to the comments. Only one thing to add though:

If you do get out something that has no notion of author, why can't you do a simple after-pass filter that clumps the works by author and does some sort of munging of the 'utility' values to get a sample by author? Like if Amazon just collapsed all the recommendations for Travis's original list that were by the same author, and gave the 10 ones with distinct authors? That shouldn't add to the complexity of the heuristic, as it's just a filter on the results. Probably a linear time computation in the result size, or so.

Posted by: Zifnab | October 17, 2004 2:43 AM

Yeah, Zif, I thought of that pretty soon after I submitted the comment. Everything is, at worst, a log(n) computation, and I'm pretty sure it could easily be constant time (per association and per lookup).

Posted by: Lemming | October 17, 2004 1:46 PM

_System of the World_ and _Jonathan Strange & Mr Norrell_ are both on my list as well.

Somehow Jaime grew on me even when he was still a mean bastard, so that I was thinking "What's wrong with me that I enjoy reading about Jaime so much?" But then he started getting less evil and I felt more justified.

Posted by: Arcane Gazebo | October 18, 2004 6:12 PM

Wired had an awesome quote from the book Pattern Recognition (which I haven't read, by the way, but seeing the book title reminded me the quote was from that book) about the extra Tokyo pieces one would need to build Tokyo out of lego. (I have the exact quote on the quotes page of my website, but I'm feeling too tired and lazy to look it up right now.)

Writing grant proposals is fun... I meant to start this 7 weeks ago and have 8-9 weeks to work on it instead of 1.5. Stupid me.

Posted by: Mason | October 18, 2004 9:53 PM
Post a comment