Welcome to the first annual “dive into mark” show! It’s just like reading my blog, except it takes forever to download, requires an unwieldy array of third-party software, and it’s not accessible to blind people, deaf people, or search engines.
Also, it involves shaving. I hate shaving.
In the “news I don’t care about” department, a company I’ll never work for has announced that it will not be shipping a new filesystem I’d never trust in an operating system I’ll never use. The so-called “WinFS” filesystem was supposed to feature rich metadata and schemas to help you organize your ever-growing porn collection.
Joe Gregorio, seen here preaching the Gospel of Atom, predicted the non-shipping-ness of WinFS in 2003, saying “WinFS is the file system formerly called Cairo and has repeatedly not shipped since 1995. If it ever did ship it would be a complete failure because it does not solve a problem that anyone actually has.”
Ouch.
And now, here’s a handy phrase I’ve taught my two-year-old to say: “Mama, bite me.”
Regular readers of my blog are painfully aware that I recently switched to Ubuntu after 22 years of servitude to Apple. Ubuntu is an ancient African word meaning “can’t install Debian”. One thing I neglected to mention about my switch is EasyUbuntu, an application designed to take all the pain out of violating patents, breaking laws, and compromising the very principles that led you to Linux in the first place, in exchange for being able to watch a recreation of the latest box office hit in 30 seconds with bunnies.
Bunnies!
In personal news, my book Dive Into Python, which hit bookstores almost two years ago, has finally earned out its advance. “Earned out” is a technical term that means that enough suckers have paid money for a book they could have downloaded from free that I’m finally starting to get royalty checks. My first check was for $247. Here’s a tip for aspiring young authors: don’t quit your day job.
§
Thank god for suckers. I guess that’s why you never went the full time author route.
— Darryl 
Do I detect a hint of Ze Frank?
— Glen C.
Glen: He’s shaving, so I don’t have to.
— Jesper
Zefrank was obviously my inspiration to try out video blogging. Of course, Dave Winer was my inspiration to try out text blogging, and we all know how well that turned out. Here’s hoping.
— Mark
I don’t see a place to comment on the original article, so I just wanted to point out that ALL the metadata used in OS X, from iCal to iTunes to Mail.app to Finder is stored in SQLite (an open source platform). If you’re so hard core about digging into the bowels of your OS… you should have been able to tap your corrupted iPhoto db.
And what corrupted your db anyway? I administer hoards of Macs and have never, ever seen a corrupt iPhoto or iTunes database. Was the problem Apple’s fault, or were you mucking around where they suggest you don’t?
I’ll grant you that OS X isn’t perfection, but for those of us who don’t want to spend our days compiling software and trying to figure out how to get pre-release, undocumented, hacked-together open source apps to even run let along do fancy stuff… OS X is several kajillion times better than anything else out there.
And most of it is open source anyway. Sure the front-end stuff is all proprietary, as it is Apple’s core business, but much of the rest is simply open source made easy (as in absolutely, pathetically easy).
— Toe
@Toe: peering through the haze of your superiority and self-righteousness, I see the faint outline of a very useful tip. If indeed my iTunes database is a SQLite database (something I can’t verify until I get home, but I will verify), then I should have no trouble extracting my remaining metadata.
In the meantime, I would like to note that you have simultaneously (a) accused me of causing the catastrophic corruption of my own data, (b) taunted me for not knowing how to recover that data when Apple’s tools could no longer read it, and (c) claimed that everything Apple makes is “absolutely, pathetically easy”.
Paging Joe Clark: when I call people “fanboys”, this is the sort of thing I’m talking about.
— Mark
Needs more duckies.
Come on Mark. OSX is *only* “several kajillion times better” How is that a fanboy comment?
More on point, how long did it take you to create the bunny show?
— Darryl
I hate shaving too.
Darryl: about 4 hours, but most of that time was spent writing the script beforehand and fiddling with video encoding afterwards. If I can get it down to an hour, I’ll probably do it more often. I rarely have 4 hours to devote to anything.
— Mark
BUNNIES!
And thanks for the Easy Ubuntu link. ;)
— nikkiana
It’s GREAT to have you back online!
(Of course I’ve been meaning to say so since, well, your first piece in the new weblog, but this is as a good a time as any, I guess. Except this is better, because this video is great. I was kinda disappointed that it was so short!)
— Manuzhai
And after reading the comments: I think John Gruber also pointed out in one of his pieces reacting to your switch that the data stores for iWhatever are SQLite files, but I’m not sure where or when exactly.
— Manuzhai
Man, that was so great. I was laughing through it all.
— Tom Corbin
First?
…I had to do it.
— Adam
Hmmm. Downloaded the larger version, watched it on VLC under OSX. I see about a second of action, then the picture freezes while the sound continues. If I pause then restart, the picture catches up, pixellates, then freezes while the sound continues. You didn’t need to shave for me to hear you.
Either you recorded it in a mode VLC doesn’t like, or VLC totally sucks at displaying this format. I suspect the latter.
I’m not sure how much of it is supplanted by EasyUbuntu, but I found the Ubuntu Guide (Now updated for Dapper) extremely helpful when I was setting up Ubuntu a year ago.
By the way, I’d like to echo the comment above about how good it is to see you blogging again. Your expressive ability, combined with interesting ideas and awesome insights make so much of your stuff good reading.
> I see about a second of action, then the picture freezes while the sound continues.
Works for me on the old Powerbook G4. I heard similar reports on older computers running Linux, so I doubt it’s a Mac-specific issue. Most likely cause is that your computer is too slow to decode the hi-res video at full speed. Have you tried the smaller version?
(Bear with me; I’m really new at this.)
— Mark
Have you thought about using Flash to deliver the video via a progressive download? It should start quicker without the need for VLC.
You’ll be needing a Flash Video player.
Convert the MPEG file, like so:
find /home/hendry/videos/ -name “maglev2.avi” | xargs -ixxx ffmpeg -i xxx -ab 56 -ar 22050 -b 500 -r 15 xxx.flv
Then you need to update the meta info of the FLV for some reason with this Ruby tool: http://www.inlet-media.de/flvtool2
Here is a demo of it in action. Or you could just visit Youtube or Google video. They use the same technology.
Wow, now we know what your voice sounds like.
Shaving is sooooo “Cosmopolitan”, take the bunny flintstones road. Get rough!
— karl
I’m reasonably sure the iTunes database has nothing to do with SQLite (Core Data came after iTunes). However, there’s some documentation here that you may find of use if you’re interested in attempting to parse it for recovery purposes.
— Dan Ridley
Okay, I feel totally gypped. I downloaded your video, watched it, and at no point during the video did I actually see bunnies. I had to come back here just to gaze upon the bunny pics again.
— Dan Ridley
> Okay, I feel totally gypped.
http://flickr.com/photos/foshie/133423377/
— Mark
another vote against shaving.
also, kajillion isn’t even a real word – use ‘muchos grande’ instead.
— gizo
About video bloging? What Maciej said about audioblogging : http://www.idlewords.com/2004/08/an_audioblogging_manifesto.htm.
I do enjoy your writing, and like the irony of you restarting a few months after I met you at ETECH and you told me that you were quitting because it fed the wrong part of your personality.
Lots of us are benefiting from your writing. Thanks. And videoblog if it amuses you.
— Tim
VLC is the only video player on my computer with an obvious option for “Subtitles Track.” It remains grayed-out for this video.
You’ve done the transcription for your show, confessing right away that it wasn’t entirely accessible. Was there a technical obstacle that prevented you from giving it subtitles?
I want to know only because I don’t believe a media player can do anything until I see it firsthand.
Technically, I “did” the transcription by posting the script I used to create the video in the first place. Obviously that’s suboptimal, and embedded subtitles would be better. I am an accessibility expert after all (at least that’s what they tell me). There is no technical reason I can’t do that; the OGM container format defines a method for embedding subtitles. I just don’t know how to do it yet.
— Mark
The answer appears to be a tool called ogmmerge:
http://axljab.homelinux.org/Mencoder_DVD_to_OGM#Creating_your_OGM_file
— Mark
Since no one detailed it: SQLite is one of three options (the other two are XML and a binary format, and neither are documented per se) of storage for the Core Data framework which is new and only works in Cocoa. iTunes is old, Carbon and cross-platform, and only stores its data in one XML file and another binary file (“.itl”).
Let’s recap: a portion of the new or remade OS X applications which use Core Data for storage *may* use SQLite for storage (like Aperture). Naturally, another portion may use SQLite anyway regardless of Core Data (like NNW). Toe assumes that *every* OS X application uses SQLite storage. This assumption is probably based on selective buzzword memory and Toe pulling things out of his ass; in any case, it’s not true.
— Jesper
I think you need to watch some more Ze Frank. The screen doesn’t flash, it cuts to a different point of view. Also, instead of music he talks in different voices.
But really, it was the soundtrack that made it all worth it.
Congratulations on coming over to the often masochistic but slightly brighter side of the Force.
— Sam
> I think you need to watch some more Ze Frank.
Which part of “inspired by” didn’t you understand?
— Mark
New version is up. It should play more smoothly, without the video blackout reported by Tim or stuttering reported by Paul. It also has subtitles. VLC is still your best bet for playing it on any platform.
— Mark
Yay for bunnies. Boo for making my brain hurt trying to remember which amiga game/demo that music module you have playing in the background is from.
— XIII
I don’t know either. The original file is called “Echoing”, which you can find on ModArchive. The first place I was exposed to it was on the Apple IIgs, in FTA’s Delta Demo. I actually ran Delta Demo in KEGS to find out who programmed it, found his email address on FTA’s website, and asked him for permission to reuse the music in my own videos. He replied that it wasn’t his music to begin with, that they had reused it from an earlier demo (probably on the Amiga, but he didn’t know). If you know where it originated, please do let me know.
But now I’ve had an email discussion with one of the FTA programmers. How cool is that?
— Mark
FWIW it played smooth as silk using vlc on *gasp* xp (even before you “fixed” it).
— Darryl
I think I remember it from an old Red Sector demo, or even from their demo maker. Anyway, thanks to the power of search engines; http://www.neoscientists.org/~cmuehlan/mods/
That seems to be the one.
— XIII
Amazing. I love the Internet.
OK, I’ve sent the author an email asking for permission to use it. I hope he says yes. (I hope he even still gets email at that address; I had to fish it out of Google’s cache because his home page seems to be gone.) If not, I’ll re-post the video YET AGAIN, with some properly-licensed music (or none at all).
— Mark
Hey Mark, I’m also a recent OSX — > Ubuntu switcher and came across your site from a related smouldering thread elsewhere.
I had no idea you wrote ‘Dive into Python’. Just like to say from this non-programmer’s perspective (I’m a designer) I found it super accessible and a real eye opener.
Thank you..
— Ord
Big cheers for ‘Dive into Python’ – now reccommended reading in the classes I give. I had no idea you wrote it..
I second the Easy Ubuntu plug also; so easy to have support for every popular proprietary media format it’s almost boring!
/me stares wistfully into times past, nights whose long heady nights of codec installation and driver compilation brought such a sense of personal accomplishment.
— Julian
I’ve added your rss feed (http://diveintomark.org/feed/) into democracy player, since you’ll be showing your posts in video(will you?), and you mentioned using it in your previous post as well.
(I’m still giving it a try. About this, and other podcast programs: Is it that special being able to automatically download podcast episodes? Isn’t this too much like “regular tv”? Let me sit at my sofa, and see what “the internet” has for me today…)
I now have a Dive into Mark channel! But it doesn’t show any “episodes” :( ! Will you eventually add “enclosures”? (What this is exactly, I ignore).
— jobezone
Mark, in case you never get a response from the original author. Ask these guys nicely. They deserve every plug they can get, for being nice and non-anal about licensing. From one of their download pages:
All songs are free to download, free to listen to, free to spread, free to burn, free to play as DJ on clubs. You cannot however use this material in ANY commercial situations. Just contact us for permission. We’re really nice to work with.
You decide whether you’re commercial or not.
— Arve
i love it. really nice to see you in person.
— ze
> Will you eventually add “enclosures”?
Yeah, I figured out how to do this once for http://howto.diveintomark.org/ . I’ll definitely go back and figure out how to do it in my main feed now.
Funny story: After I set up the howto feed, I tried it out an earlier version of Democracy Player and was disappointed to discover that it did not support Atom feed enclosures. In fact, it didn’t seem to support Atom 1.0 at all. Since it was open source, I dug into it a bit and discovered that they were using my own feed parser to parse the feeds! That was the instigator to get me off my ass and update my feed parser to support Atom 1.0, and I made sure that it supported Atom-style enclosures as well. And here we are, six months later, with a brand new version of Democracy Player. I checked the latest version, and sure enough, they updated to the latest version of my feed parser! Woohoo!
In college philosophy classes, we called this a “virtuous circle.”
— Mark
Those of you who hate shaving might want to give Corey Greenberg’s The Perfect Shave a read. It’s a paradigm shift in men’s personal hygiene and shows you that shaving can, in fact, be the high point of your day, provided you use the right tools. Which don’t include cartridge razors and shaving cream that comes in a can.
I actually already use a real razor, a badger hair brush, and Geo F. Trumpers shaving cream. (You can even buy it online!) I still hate it.
— Mark
From reading your other posts, I’m glad to see you miss Growl.
Oh, in iTunes, there’s an export id3 thing in the advanced menu. I haven’t read every post, but I caught part of you having problems with that.
Ping me via email if you have questions, cheers.
Well, you are no Zefrank, yet. But cudos for daring to take the plunge.
— Tomas
Wow, it’s kind of like karaoke for blogs, and similarly embarassing :)
To clarify, I only said that metadata is stored in SQLite. I did then go on to associate that with the database, which was my bad. However, as far as I can tell, the primary database for iTunes is the file system. You can toss all the tags associated with iTunes and still get back the important stuff (ie your music) by just rescanning your music folder.
And yes, Mac users are snobs. In a different way from the way Linux users are snobs. You started out by saying how horrible it is that Apple doesn’t make their OS ultimately hackable. My point was that if your goal is to hack away at your computer, then by all means, go compile your own OS. If instead you don’t want to deal with any of that crap… that’s what Macs are for (and what Windows miserably fails to emulate). Your criticisms of the Mac all depend on its being something other than what it is supposed to be.
Apple has taken a bunch of hard core BSD (etc.) tools and made them point-and-click in a fantastic way. It ain’t perfect, but I haven’t seen anyone else even come close to doing better. You can launch full-blown Apache by clicking “Enable Web Sharing.” You can listen to music, sort photos, build web pages, etc. all with an interface that even highly computer-illiterate people can figure out in seconds.
There are better tools than iChat, Safari, Terminal, etc. But there are few if any that are so imminently accessible to anyone. And if you don’t like the tools that Apple gives you, they do nothing to prevent you from using any other tool (unlike Microsoft… and in case you didn’t know, Apple is working hard on building OS-level virtualization into their next OS).
And if you don’t like Apple’s OS, then go ahead and install your own… Macs will run just about anything now. So you paid a little more and don’t get the great OS and all the great software they sent you, but you still have a wonderfully built, beautiful machine.
So they make a minor attempt to keep you from putting their intellectual property someone else’s hardware (even though people get it to work no problem). That’s ’cause they don’t want 50 million people calling them asking about arcane drivers from whiteboxes-are-us. They had to make choices somewhere, and they always choose ease over most anything else.
Did you know that Toyota deliberately designs their tail lights so that you can’t easily put a Ford tail light on your Corolla? I think I’ll bad-mouth Toyota all over the place for their draconian attempts to limit my freedom.
— Toe
Toe, you said “ALL the metadata used in OS X, from iCal to iTunes to Mail.app to Finder is stored in SQLite,” which is utter and complete poppycock. Claiming you didn’t say something isn’t terribly effective when the source material is a couple of PgUps away.
WRT to your Ford/Toyota comparison, I think a key issue about FairPlay is the fact that Apple has changed the terms, and could do so again. If Ford had the power to change their mind and make your tail lights only light for a maximum 5 hours a night (instead of 10 as previously), then they’d deserve all the badmouthing you can give them. (Obviously most people don’t drive all night and wouldn’t be adversely affected by the change…)
It’s a weak analogy, but I’m using it to counter a weaker one, so meh.
— Dan Ridley
Metadata and data are not the same thing. The metadata stored in SQLite is the information tapped by Spotlight to do searching on content and a huge host of other keys (you can search on photo exposure, music artist… all kinds of killer stuff). That is different from data, which in the case of iTunes is your music, playlists, ratings, etc.. As I already said… my bad for confusing the two. The data may be in SQLite, but I do not know.
And being mad at software publishers for making their software proprietary is… well fruitless. I look forward to the day when OSS becomes the norm, but that’s not today. I’m happy to pay $130 to get millions of man-hours of development work on a great OS plus a *ton* of really handy apps. Though usually I get all that for free when I just hold out until I buy a new Mac.
— Toe
Great article, you sound alot like John Hodgman (and have a similar sense of humor)
— Matt
Toe, Apple’s developer docs about Spotlight don’t talk about SQLite, but do talk about “a totally new hand-tuned database.” Apple’s done a great job integrating SQLite into Core Data, and they have used it in some other places in Tiger (Safari stores syndication data in an SQLite database, for example). They also seem to be favoring SQLite and Core Data in new applications (Aperture is built with Core Data, for example).
But Spotlight doesn’t use it, and applications that predate Tiger generally don’t either. (Mail is an exception; in Tiger its message index is in SQLite, although as Mark has noted the actual data — messages — got more proprietary in Tiger, so it’s a net loss.)
I’m still with Apple (not even close to unswitching), but the issues Mark raises are real. Throwing around the SQLite buzzword without knowing how it’s actually used doesn’t help your cause or his.
— Dan Ridley
I am no longer accepting public comments on this post, but you can use this form to contact me privately. (Your message will not be published.)
§
© 2001–9 Mark Pilgrim