Photo cross-references. In progress. Every photo may depict multiple people doing multiple activities, may be taken in multiple places, may show a variety of special things, and so forth. Furthermore, people are organized into a loose categorization scheme; some people are family, some are friends, some are friends from a particular job, and so forth. Ditto places. Ditto events.
I will soon be adding cross-references on each individual photo page. So if you’re looking at a photo of me and Dora and my father in my house in front of the Christmas tree at Christmas, there would be links to see other pictures of me, other pictures of Dora, other pictures of my father, other pictures taken in our house, other pictures of Christmas trees, and other pictures taken around Christmas.
Now.
I do not know how to express all of this in RDF. I am aware that I have maligned RDF in the past, and not without reason, and I will malign it a bit more in short order. It’s complicated, and it’s not for everything. However, this seems to me to be the perfect application for it.
Thus.
I have been looking at what Norman Walsh has done, setting up a taxonomy
of topics, and then associating each log entry and photo with topics in that taxonomy. It’s quite impressive. I believe that’s what I want to do here.
So.
First, a taxonomy:
[Reception] [is part of] [Wedding]
[Wedding] [is a type of] [Event]
[Lara] [is a] [friend-who-works-at-Prophet]
[Friend-who-works-at-Prophet] [is a type of] [Friend]
[Friend] [is a type of] [Person]
Second, information about each photograph:
[Photo X] [was taken on] [a specific date]
[Photo X] [was taken at] [a specific place]
[Photo X] [depicts] [a certain person]
[Photo X] [depicts] [another person]
[Photo X] [shows people doing] [a specific activity]
That pseudocode looks suspiciously like RDF triples, but I don’t know how to make it into actual RDF. My grasp of the RDF model is still shaky, my cumulative experience with the rdflib library can be measured in hours, and as almost everyone in the world will admit, the syntax of the RDF/XML seralization is opaque. All of which is simply to say that RDF has a steep learning curve, and I am still on the bottom of it, and this is what that feels like.
Taking the advice of those who say RDF is easier with the right tools, I have downloaded the latest development release of rdflib, an RDF library for Python. I have read through all the documentation and included examples, and parsed a few FOAF files with it, but I can’t quite make the leap into getting it to express what I want, probably because I’m not entirely sure what I want to express.
Any constructive advice would be greatly appreciated.
Scattershot thoughts:
Is there a reason you’re not using FOAF to express your friend-related triples? (Not a rhetorical question, nor intended to be insulting. You may well have a perfectly good reason. I don’t know FOAF that well.)
How much of this would XFML handle?
Comment by Dorothea Salo — Tuesday, June 17, 2003 @ 5:00 pm
Yup, might use FOAF. 2 things:
1. don’t know how to express sub-types (”friend from Prophet”, “friend from Labcorp”)
2. don’t know how to express different relationship types (”friend”, “family”, “random person I met once”)
Comment by Mark — Tuesday, June 17, 2003 @ 5:08 pm
I’m happy to answer more specific requests. Here’s a translation of your psuedo-triples into N3.
Wedding a Event .
Wedding contains Reception .
Lara a Friend .
Lara employer Prophet .
Friend rdfs:subClassOf Person .
PhotoX dc:created “2003-02-11″;
location Place ;
foaf:depicts Person, OtherPerson;
foaf:shows Kissing .
Comment by Aaron Swartz — Tuesday, June 17, 2003 @ 5:09 pm
Oh, I see I got the Friend thing wrong. You meant:
Lara a FriendFromProphet .
FriendFromProphet rdfs:subClassOf Friend .
Comment by Aaron Swartz — Tuesday, June 17, 2003 @ 5:11 pm
Honestly though, why do you want to use RDF? This is very cool already. Are you looking just to export the data?
Comment by Aaron Swartz — Tuesday, June 17, 2003 @ 5:30 pm
Aaron: thanks, that helps a bit. Subclasses are expressed with the predicate rdfs:subClassOf. FriendFromProphet is simply a URI to a node within my own taxonomy. Maybe Friend is too.
I think it’s the predicates that are throwing me. dc:created makes sense for dates. foaf:depicts makes sense for people, and probably things (like Christmas trees). I don’t see foaf:shows in the FOAF docs. ( http://xmlns.com/foaf/0.1/ ) What about the other predicates? Do I just make them up?
Comment by Mark — Tuesday, June 17, 2003 @ 5:31 pm
Re: why RDF. Um, because I want to learn more about RDF, and this is a good amount of real data to work with, and it seems like a good fit, and Norman Walsh did it, and Norman Walsh is cool.
Comment by Mark — Tuesday, June 17, 2003 @ 5:33 pm
You might want to check out Protege (http://protege.stanford.edu/) .
Comment by Greg Schueler — Tuesday, June 17, 2003 @ 5:34 pm
Protege looks very cool. Will experiment.
Comment by Mark — Tuesday, June 17, 2003 @ 6:04 pm
“and Norman Walsh did it, and Norman Walsh is cool.”
If Norman Walsh jumped off a cliff would you do it?
Comment by sparticus — Tuesday, June 17, 2003 @ 6:18 pm
I would just use a WikiName approach. Fuck the ontologies.
You might be interested in FotoWiki, which combines this point with the idea of embedding areas/identities within JPEG files themselves.
http://www.fotowiki.net/
http://fotonotes.net/index.php?action=DisplayPage&page=about
Comment by BillSeitz — Tuesday, June 17, 2003 @ 6:25 pm
Oooh I feel sorry for us living in this era.
Manual content annotation is laborious, and hence costly. An alternative is (semi) automatic feature extraction, on which a lot of research has been done, but unfortunatily there is little hope for this to work well for arbritrary media formats in the near future.
I do find it very cool what you are doing here Mark, it has to take you quite some time. Your long descriptions were a Good Thing ™, but this sort of thing goes way beyond. And I also think images.google.com can enjoy this sort of work :)
Comment by Martijn — Tuesday, June 17, 2003 @ 6:32 pm
Just a quick comment on your current hierarchy (as seen on the photos/xref page) : there’s no way of viewing all of the cat pictures! Considering your search engine statistics I see this as a pretty major limitation ;) I suppose cats ends up being a sub-thingy of pets.
Comment by Simon Willison — Tuesday, June 17, 2003 @ 6:32 pm
Would the FOAF relationship module be useful for some o’ this guff?
Comment by Marcus — Tuesday, June 17, 2003 @ 6:53 pm
I think there may be a foaf:Friend. I guess I misremembered foaf:shows. Feel free to make up your own predicates. If they’re useful, make a namespace for them (and a little page describing them).
Comment by Aaron Swartz — Tuesday, June 17, 2003 @ 7:19 pm
At Simon’s suggestion, I have subclassed Pets into Dogs and Cats, to make it easier to find all the cat pictures.
http://diveintomark.org/photos/xref/pets/cats/
Comment by Mark — Tuesday, June 17, 2003 @ 7:38 pm
Shouldn’t it be:
LongEvent isa Event
Wedding isa LongEvent
WeddingReception isa Event
WeddingCeremony isa Event
WeddingReception follows WeddingCeremony
and so on.
RDF can drive one crazy like an onion with multiple personality disorder.
Comment by Don Park — Tuesday, June 17, 2003 @ 8:47 pm
Have you seen this? “A Semantic Web shoebox - annotating photos with RSS and RDF”
http://www.hackdiary.com/archives/000020.html
Just coincidence that I stumbled across that page and your post on the same day… Lots of other RDF stuff on hackdiary too.
Comment by Joe Germuska — Tuesday, June 17, 2003 @ 11:54 pm
more random thoughts:
When making up predicates, note that they too can be derived from other predicates, using rdfs:subPropertyOf
foaf:depicts can be used with most any object: a person, an activity, a thing.
An interesting use of Wordnet, http://xmlns.com/2001/08/wordnet/ and http://www.openhealth.org/RDDL/wnbrowse
I find it easier to visualize RDF using ER-style, node-centric views, like this one I did for RSS/RDF http://bitsko.slc.ut.us/2003/06/dc_diag.png
The entity relationships in RDF are dynamic though, http://www.w3.org/DesignIssues/RDFnot.html#ER plus the ability to use sub-properties.
Comment by Ken MacLeod — Tuesday, June 17, 2003 @ 11:58 pm
This is one great cross-referencing. I am myself thinking about the possibilities to put photos on my server for all to admire, and xref is something I’d like to have but have yet to grasp how to achieve it.
Also : a dog named Xavier ?!? ;)
http://diveintomark.org/photos/xref/pets/dogs/xavier/
Comment by Xavier — Wednesday, June 18, 2003 @ 4:29 am
While not quite as elaborate as you’ve suggested, I’ve tried some of this with a few rolls of film back in february: http://www.wasab.dk/morten/2003/02/photos/1/
Some choices I’d perhaps make differently now, but the basics are there (inspired by Matt Biddulph’s work).
Comment by Morten Frederiksen — Wednesday, June 18, 2003 @ 4:41 am
The stuff I describe in http://www.hackdiary.com/archives/000034.html can be seen at http://www.picdiary.com
A picture collection like http://www.picdiary.com/londoneye/ has an associated RDF file at http://www.picdiary.com/rss/londoneye.rss with people markup from FOAF and keyword markup from wordnet (eg http://xmlns.com/wordnet/1.6/Sunset-2 ). Interesting vocabularies for geographic annotation are being developed by Jo Walsh at http://space.frot.org/
The RDF is linked from the HTML via a link tag in the head for autodiscovery.
Comment by Matt Biddulph — Wednesday, June 18, 2003 @ 5:42 am
Matt, thank you, that is freaking awesome. Exactly the kind of concrete example I need to get my head around this. You totally rock.
Comment by Mark — Wednesday, June 18, 2003 @ 12:29 pm
Mark, pretty cool and certainly near and dear to my heart. After recently getting married myself and exceeding the 2000 mark for my digital photo archive, some sort of serious organization was in order. I’ve taken a different approach and evaluated the major “thick-client” offerings like Adobe Photoshop Album, Picasa, etc. I finally settled on Photoshop Album for its maturity with tagging over the rest. It offers a mature way of defining a taxonomy and tagging photos, and it keeps all this in an MS Access Databasee format (their .psa files) that is easily accessible.
Out of curiosity, how do you currently “tag” your photos?
Comment by Brian Theodore — Wednesday, June 18, 2003 @ 2:14 pm
Surprised that nobody has mentioned http://www.w3.org/TR/photo-rdf/
Comment by Dylan Moreland — Wednesday, June 18, 2003 @ 6:29 pm
XFML would handle this perfectly except for the verbs: it doesn’t do [is part of], [is a type of] and so on. Then again, for your purposes, you don’t seem to need the verbs.
Comment by PeterV — Thursday, June 19, 2003 @ 9:54 am
Dylan, I found your comment when I was about to do it. I remember it from the Jigsaw java web server:
http://www.w3.org/2002/Talks/0920-yl-qh/Overview.html
There was a working demo somewhere.
Comment by Santiago Gala — Thursday, June 19, 2003 @ 7:56 pm
Santiago: http://jigsaw.w3.org/Yves/Australia/1998/04/
But it’s fairly boring — he does nothing with people representation. Not a great example.
Comment by Dylan Moreland — Friday, June 20, 2003 @ 12:23 pm
For the record, I have no plans to jump of any cliffs.
But I have to say, the suggestion that I might really, really makes me want to find some way to photograph just that event. A cliff over water, presumably. Or maybe I need to try parasailing or hang-gliding or something. Nah, that’d reduce the drama. I think it has to be a free-fall.
Comment by Norman Walsh — Saturday, June 21, 2003 @ 1:37 pm
Norm, I recommend skydiving.
http://diveintomark.org/photos/xref/places/north_carolina/midair/
Comment by Mark — Saturday, June 21, 2003 @ 3:39 pm
Hey Mark, I’ve been thinking about this and I have a question for you (or Norman, or whoever). Every predicate naturally has an associated predicate that expresses the statement in the opposite direction. (Image1 depicts Mark, Mark isDepictedBy Image1.)
The question is this: Which do you store? Do you store both? If you only store Image1 depicts Mark, then to find all pictures of Mark you have to look at every image and see if it contains Mark. This seems inefficient. Then again, recording two statements for everything you wish to express also seems inefficient.
Let me know if you have any thoughts on the matter.
Comment by Michael — Friday, June 27, 2003 @ 8:49 am