Editing Scrivener ‘Sync’ files

Do you use Scrivener? If so, and you’d like to edit your Scrivener files on another plat­form such as your iPad, I may have some­thing that will help. I’ve made some ‘bun­dles’ and themes for the Text­Mate edi­tor and the Tex­tas­tic edi­tor on the iPad that dis­play inline anno­ta­tions, com­ments and block-quotes in the text. I’ve included some mark-up, too, for some com­mon gram­mar problems.

I’m writ­ing a cen­ten­nial his­tory of the Inter­na­tional Cham­ber of Com­merce in Scrivener. It’s a mod­er­ately large non-fiction book (about 115k words, at present… too long, already). There are some 90 man­u­script files in the Scrivener data­base: called the ‘Binder’ in the Scrivener inter­face. There are as many files, also in the Binder, that are out­side the man­u­script. These are notes, rejects, research etc.

I’m using Scrivener’s native RTF for­mat for writ­ing because my edi­tors will want an MS Word file to work on. Scrivener’s main export rou­tines order, for­mat and ‘com­pile’ a Word doc­u­ment from my man­u­script files. I back-up the Binder every time I close the pro­gram. But I’m also sync­ing (on close or save) the files in plain-text for­mat to a folder on my Drop­Box account. This is a sort of extra backup. These text files are ‘synced’ because Scrivener reads them every time it opens (or when you tell it to) and incor­po­rates any changes into its database.

The synced text files make it pos­si­ble to edit your Scrivener man­u­script using another com­puter and any text edi­tor you like. When Scrivener is open, it’s look­ing at its own data­base not at the text-format sync files. So you can change them when­ever you want with­out cor­rupt­ing the Scrivener database.

Lots of edi­tors can edit text files from your Drop­Box account. I’ve been using BBE­dit (on my Mac) for years. No prob­lems. But, for all its power, BBE­dit is clunky in some spe­cific ways. So I’ve moved over to Text­Mate 2 (alpha). I’m still uncer­tain about the switch because Text­Mate has had a trou­bled history.

Aside #1: The cre­ator of Text­Mate tried to com­pletely re-write his very suc­cess­ful first ver­sion. But he could not do it quickly or com­pletely to his own sat­is­fac­tion. The pro­gram lost a lot of market/attention-share. It seems to be back on track now as a public-domain pro­gram. But edi­tors are the devil to write damned hard to main­tain. No public-domain edi­tor that I know of is first-class… except maybe the EMACS and Vim dinosaurs (and life is too short for more than a nod­ding acquain­tance with them). I’m hop­ing that Text­Mate 2 will keep grow­ing and will some­how sur­vive and pros­per as a public-domain pro­gram. But its too early to be confident.

Aside #2: My main gripe about BBE­dit is that it has a very lim­ited capac­ity for lan­guage gram­mars. In other words, it doesn’t offer much flex­i­bil­ity in dis­play­ing syn­tax markup in the edi­tor. Sure, it comes packed with a very big bunch of syn­tax mark-up files. That’s fine if you use BBE­dit as a code edi­tor for a pop­u­lar lan­guage. But if, like me, you use text-editors to write marked-up prose or if you want to write a gram­mar for an unsup­ported lan­guage, be pre­pared for agony, puz­zle­ment, end­less dicking-around and piles of frus­tra­tion. I’ve tried. I failed. My other gripe about BBE­dit is its carbon-looking win­dow hard­ware and its gen­er­ally clunky set of pan­els and dialogs. Improved recently, but …

Now Text­Mate excels at incor­po­rat­ing lan­guage gram­mars and stylesheets (‘themes’) that dis­play selected syn­tax of the gram­mar. They’re sort-of-easy to com­pose (if you know reg­u­lar expres­sion gram­mar and you are com­fort­able with basic shell script­ing and maybe with adapting/hacking ruby or python or perl scripts). There’s a built-in edi­tor for the ‘bun­dles’ that con­tain the gram­mar files and the ‘themes’ that are used to colour the gram­mar. If you have a lit­tle CSS expe­ri­ence, the theme files that tie the gram­mar and the appear­ance together are not a big challenge.

How does this help with edit­ing Scrivener files? Well, Scrivener uses a light­weight set of con­ven­tions to rep­re­sent markup in its text-based sync files. For exam­ple, I’m using a con­ven­tional for­mat in Scrivener for foot­notes (actu­ally, end­notes in my case) that puts them ‘inline’ in the text. They appear in the Scrivener win­dow as grey-background ‘bub­bles’ inserted in the text where the footnote/endnote ref­er­ence marker will appear in the final RTF (or Word) doc­u­ment. This turns out to be a great way to write footnotes/endnotes because they appear to the author as a sort of paren­the­sis in the text (which is what they are in my book: end­notes are where the jokes and snarky stuff is located… as well as the ref­er­ence cita­tions, of course). In the synced text files these ‘inline’ footnotes/endnotes are rep­re­sented by text enclosed in dou­ble braces; like this {{…}}. Inline com­ments or anno­ta­tions are enclosed in dou­ble paren­the­ses, like this ((.…)).

I also have my own ‘markup’ in Scrivener. I have an RTF style for block-quotes whose indented, jus­ti­fied for­mat­ting is ‘pre­served’ by Scrivener when it out­puts its RTF files to Word. But there’s no spe­cial markup for your own styling in the synced text files that Scrivener cre­ates. (I wish there were.) The only thing that dis­tin­guishes my block-quotes in the text files is the quote marks (“) that begin them on a new line and end them at the end of the line or, some­times, just before the attri­bu­tion infor­ma­tion I include at the end of each block­quote. I enclose the attri­bu­tion in brack­ets like this […].

So, all of my synced text files have some very sim­ple markup of footnotes/endnotes, com­ments (that I use for glos­sary entries… let’s leave that for another time) and my ‘pre­served’ block­quotes. Oh… and most footnotes/endnotes also con­tain a cita­tion key for my bib­li­og­ra­phy soft­ware (Pages 2, at present. I’m ditch­ing it as soon as I can. Prob­a­bly for Book­ends). A cita­tion key looks like this: {Gallagher:xvr09}.

What I would like to do is to edit my Scrivener files in a text edi­tor on com­put­ers where I’m not run­ning Scrivener. I have Scrivener on my lap­top, of course. But since I leave Scrivener run­ning most of the time on my desk­top com­puter (my main edi­to­r­ial sta­tion), I never open it on the lap­top to edit the same project. That would con­fuse things. Scrivener (god bless it!) com­plains. So I edit the synced text-files on Drop­box, instead. BBE­dit was fine. But one aspect annoyed me: the light­weight markup in the Scrivener text files was all-but-invisible in the text files. With­out any visual clues, foot­notes, com­ments and block-quotes looked pretty-much like the rest of the text.

It was the same on the iPad. Some­times I’d like to spend a few otherwise-wasted half-hours (on trains, at air­ports, or stuck in dread­ful com­mit­tee meet­ings) edit­ing the synced files on my iPad. It cer­tainly has the stuff to do so. Drop­box edi­tors are legion on the app-store. Many can han­dle the sim­ple text of the synced files with­out a prob­lem. But, as with BBE­dit on my lap­top, most of them don’t per­mit flex­i­ble markup of lan­guage grammars.

I did two things. First, I (re)started using Text­Mate on my lap­top. I wrote my own (very light­weight) gram­mar file for the Scrivener text-file for­mat. I also cre­ated a ‘theme’ file to dis­play the syn­tax of these gram­mars. For exam­ple, the text of an inline footnote/endnote (defined as text between {{ and }}) is keyed via its name (‘meta.inline.footnote.text’) to a spe­cific back­ground colour (a sort of light grey) and font style (‘italic’) in a theme file asso­ci­ated with the gram­mar. The theme file I cre­ated is called “Nuage”. The gram­mar is embed­ded in a Text­Mate ‘bun­dle’ that I call “Scrivener-text”. When I choose to apply that bun­dle to a text file, I can also choose to apply the Nuage theme. Text­Mate remem­bers the kind of file and its loca­tion and re-applies the same bun­dle and theme to sim­i­lar files there­after. Now, every time I open one of the Scrivener text files from the Scrivener direc­tory on Drop­Box, Text­Mate applies the right lan­guage gram­mar and theme file. I see all my foot­notes dis­tin­guished in the text in an italic (acu­tally, slanted) font and with a light-grey back­ground. I have dif­fer­ent markup for com­ments and for block-quotes.

As for the IPad… I use the very capa­ble and flex­i­ble Tex­tas­tic. This is a mature text edi­tor com­pa­ra­ble in scope — on the iPad plat­form — to BBE­dit on the OSX plat­form. (There is also an OSX ver­sion, but I’ve never used it). Best of all, it will import Text­Mate lan­guage gram­mars (in TM ‘bun­dles’) and theme files. So I can use the same markup on my Mac lap­top and on my iPad. Tex­tas­tic also syncs its files to Drop­Box. So I can edit the marked up text using col­ored syn­tax and it will auto­mat­i­cally save (with­out the col­or­ing) as a text file to Drop­Box in the form that Scrivener expects.

But WAIT! “That’s not all!” ™ I like to keep my writ­ing clear. I want nouns and verbs to do the work. I don’t want to rely on weasely adverbs to fix-up weak verbs. I don’t want to write in pas­sion­less pas­sive voice with no sub­jects to take respon­si­bil­ity for the action. Nor do I want to make sim­ple edit­ing errors like repeat­ing one of the the words in a phrase acci­den­tally when I move things around with cut-and-paste.

But I’m prone to do all of those things. With­out even noticing.

So I have added a few tricks to my gram­mar file for Scrivener that, in com­bi­na­tion with the Nuage theme file, high­light the most com­mon mis­takes I make. I stole the imple­men­ta­tion of this idea from the “Eng­lish” bun­dle which is itself a repli­ca­tion of three nifty lit­tle Shell scripts by Matt Might. Now my pas­sives pong in red; my weasel words writhe in greeny-yellow; my double-ups dis­play in orange. Hoorah!

A screen shot of Textastic (iPad) editing a Scriv-text file

A screen shot of Tex­tas­tic (iPad) edit­ing a Scriv-text file

So… if you use Scrivener, Text­mate and Tex­tas­tic… allow me to offer you (with­out any guar­an­tees, warranty-against-damage or after-gift-support) my cur­rent ver­sion of the ‘Scrivener-text’ bun­dle for Text­Mate 2 (haven’t tried it with TM1.x) and the “Nuage” tmTheme file (for use in Text­Tas­tic) and the ‘Nuage’ bun­dle (that will install the theme in Text­Mate 2).

Installing the Text­Mate bun­dles is easy. Click on the down­loaded files. They should install them­selves. If not, copy them to Users/[your-user-name]/Library/Application Support/Avian/Bundles. The instal­la­tion of the bun­dles and theme file in Tex­tas­tic is also very easy. Down­load the files to the spe­cial #Tex­tas­tic folder that you cre­ate in the side-bar of the Tex­tas­tic edi­tor. Restart the pro­gram. Then point the style and language-grammar files at the file you’re edit­ing. (I’ll leave you to con­sult the Tex­tas­tic help file for the details of how to do that.) The result? You should be able to edit syntax-coloured, marked-up Scrivener sync files anywhere.

Down­load: After all that, here’s the link to a zip file of my Scrivener-text bun­dle, the Nuage bun­dle (for Text­Mate) and the theme file (for Tex­tas­tic on iOS).


The longevity of plain TeX

Microsoft Word was first released for MS DOS in Octo­ber, 1983. Some five months ear­lier, Don­ald Knuth, a pro­fes­sor of com­put­ing at Stan­ford Uni­ver­sity, pub­lished “The TeX Book.” This was a jokey man­ual for, and a deft demon­stra­tion of, his TeX pub­lish­ing pro­gram whose sta­ble ver­sion first appeared in 1982. Each of these con­tem­po­rary works was […]


The physical science of climate change

I’m late find­ing this Non­govern­men­tal Inter­na­tional Panel on Cli­mate Change (NIPCC) report. I include the sum­mary of their find­ings on the phys­i­cal sci­ence here because they accord with my present assess­ment. I would have added only one thing to the recita­tion below. I do not know why the NIPCC omit­ted it. That is: the IPCC’s […]


If you need a word …

A com­bined Key­board Mae­stro and OSX Automa­tor action to access Princeton’s “Word­Net” data­base. Word­Net is a remark­able seman­tic net­work of words: not exactly a the­saurus but nonethe­less great for hunt­ing down the word you need. Select a word in any Mac appli­ca­tion, hit the key­board trig­ger you select and the macros pop-up a win­dow with the […]


The multilateral trade mess — 5 years later

In August 2008, shortly after the col­lapse of the WTO Doha Round’s agri­cul­tural nego­ti­a­tions in Geneva — which I rec­og­nized, then, as the ‘death-rattle’ of the Round — I wrote the attached paper; part of a project spon­sored by the Rural Indus­try Research & Devel­op­ment Cor­po­ra­tion to find a bet­ter way to nego­ti­ate in WTO. […]


They never talk about trade

The trade and invest­ment poli­cies of the two main polit­i­cal par­ties in Aus­tralia have flip-flopped in cho­rus over the years so they hardly ever fig­ure in elec­tion debates. As if they didn’t mat­ter enough to fight about. Yet there’s lit­tle that mat­ters more to the future pros­per­ity of a small, open, still-somewhat-remote econ­omy than build­ing its […]


Statistics for (non-mathematical) trade analysts

I teach a ‘sum­mer inten­sive’ grad­u­ate course in Trade Research Meth­ods at the Uni­ver­sity of Ade­laide. In five days we cover every­thing from the role of evi­dence in pub­lic pol­icy to how to write a report that your boss will read (and even under­stand). Most of my stu­dents have no for­mal train­ing in sta­tis­tics (or […]