New file format in Sketch 43

ale

Sketch 43 will introduce a new file format, designed to allow better integration with third party services and external tools. It will allow developers to access document information in an easier way, without having to use SketchTool. This opens new possibilities for inspecting Sketch documents in non-macOS platforms.

In case you're wondering: SketchTool is not going anywhere. We're committed to it, and have some new features in the pipeline. Also, you'll still need it if you want to extract assets from a Sketch document.

In this post we’ll explain how this new file format is structured, and some of the things you can do with it…

A zipped folder ?

New Sketch documents are basically zipped folders, containing JSON files that describe the document data, plus a number of binary assets (bitmap images, document preview…)

We'll take a look at a sample document, Fitness App. This is a Sketch 43 document, so don’t try to open it with a prior version of Sketch. We just released a public beta for v43 in case you want to play with it.

To explore its contents, we'll rename it to Fitness App.zip and double click it to expand it. We'll see it contains a folder structure similar to this:

- images
    - 0c2421c7228cf5e9d8133b6f598461a7cf439fc5.png
    - 00f051d3a27fec14ba725121eb2bffb3ac1c3e02.png
    - etc…
- pages
    - 9B2D6A75-D2C0-40F3-88AB-AE4BD625E520.json
    - 866BD861-A247-4826-A32A-C8DE4F109578.json
- previews
    - preview.png
- document.json
- meta.json
- user.json

Let’s take a look at those files and folders:

meta.json

Contains metadata about the document itself (a list of pages and artboards, Sketch version used to save the file, fonts used…). It's equivalent to the output you'd get from running sketchtool metadata on the file.

document.json

Contains common data for all pages of a document, like shared styles, and a link to the JSON files in the pages folder.

pages folder

Contains a JSON file per page of the document. Each file describes the contents of a page, in a format similar to what you'd get by running sketchtool dump on the file.

images folder

The images folder contains all the bitmaps that are used in the document, at their original scales (so if the user imported a 1000x1000px image and then resized it to 200x200px, the full 1000x1000px file will be stored).

user.json

Contains user metadata for the file, like the canvas viewport & zoom level for each page, UI metadata for the app (panel dimensions, etc…) and whether the document has been uploaded to Sketch Cloud.

previews folder

Contains a preview image of the last page edited by the user. If the page's size is less than 2048x2048 it will be stored at full size, otherwise it'll be scaled to fit a 2048x2048 square.

A file format for hackers 🛠

You can do some funky stuff with the contents of a Sketch document, once you expand the ZIP.

If you edit any asset or JSON, and then put them back into the ZIP file, Sketch will read those changes back.

Keep in mind though, that the new file format its great for reading stuff back out. Making changes is less trivial as there may be unexpected properties hanging together, like a text layer’s frame and its string contents causing unexpected wrapping…

Here are some ideas for things you can try to do with the documents:

editing page and layer names by editing the JSON
running bitmaps through ImageOptim to compress them
run text diffs using version control systems
run visual diffs using previews + version control systems
integrating previews for .sketch documents on your server-side apps
use server-side tools to generate a dynamic .sketch document
…

With great power comes great responsibility ?

Please note that for efficiency, we’re doing minimal error-correction when reading the document back, so if you make an incompatible change it’s quite possible to end up with an invalid document. Always work on backups, and treat your user’s documents with care.

Also, since the JSON files are machine-generated, the structure will definitely change in the future as we keep evolving the file format. We will migrate older structures, of course, but keep it in mind when developing your integrations.

Hello, future ?

We look forward to seeing what you can do with this new file format, and can’t wait to hear your feedback. If there’s anything you’re missing, something you’d like us to implement, or you need help with some incredible integration that will awe the world, send us a note to <developers@sketchapp.com> and we’ll be happy to chat with you.

Have a nice day!

yetone

ale previews folder
Contains a preview image of the last page edited by the user. If the page's size is less than 2048x2048 it will be stored at full size, otherwise it'll be scaled to fit a 2048x2048 square.

plz remove the size limit. otherwise will has a bad experience in retina display!

jasim_ab

This is truly a surprise. Would be an understatement to say how huge this would be!

shaneal

I've played around with the new format a bit and it's really awesome! It really opens up sketch in an entirely new way. Here is a small node script that reads a sketch file, changes a layer name on the page, and saves back to that file:

const fs = require('fs');

// lib to read and write to the zip
const JSZip = require('jszip');

fs.readFile('Untitled3.sketch', function(err, data) {
  if(err) throw err;
  JSZip.loadAsync(data).then(function(zip) {
    // zip contains a json file that describes all the directory & files in the zip file

    // read the top level page
    // hardcoding page because im lazy ;)
    const pagePath = Object.keys(zip.files)[1];

    zip.file(pagePath)
      .async('string')
      .then(function(str) {
        const json = JSON.parse(str);

        // grab the first layer which in my case is a text layer
        const text = json.layers[0];
        text.name = 'Changing the layer name';

        // write the page json back to the file in the zip
        zip.file(pagePath, JSON.stringify(json))

        // override the original file
        zip
        .generateNodeStream({ type:'nodebuffer', streamFiles:true })
        .pipe(fs.createWriteStream('Untitled3.sketch'))
        .on('finish', () => {
          console.log('yay saved file. Open me in sketch to see the changes');
        });
      });
  });
});

The Untitled sketch file is a blank document that and I created one text layer and saved it.

Couple Observations:

If you change the file while sketch is open, and the user saves the file, you'll get a alert that says something to the effect of: The document could not be saved. The file has been changed by another application
Some text information (stringValue, paragraph style, font) is encoded and cannot be read.

To get around (1) I did this (not sure if its correct, but after a bit of trial and error works for me):

var document = context.document;
var fileURL = document.fileURL();

// not sure about the 2nd arg
document.readDocumentFromURL_ofType_error(fileURL, "sketch", null);
document.revertToContentsOfURL_ofType_error(fileURL, "sketch", null);

For (2) If you have sketch open you can look at the respective class and there should be some helper method. For example To get the string value of a textLayer, you can look at MSAttributedString.decodeAttributedString. If you don't have sketch open or want to do some processing outside of sketch you can use coscript REPL:

var archive = NSString.stringWithString(ARCHIVED_STRING);
var data = NSData.alloc().initWithBase64EncodedString_options(archive, 0);
var res = NSKeyedUnarchiver.unarchiveObjectWithData(data);

// string value
log(res.string());

Hopefully this helps anyone just getting started. Can't wait to hack on this some more ?

icabanas

shaneal Hi, do you know how can I trigger an update in an opened sketch file if someone overrides the file externally?

mathieudutour

That's so good! Thanks!

mathieudutour

@ale out of curiosity, is there a binary that we can use to get a preview for a sketch file without opening sketch? maybe /Content/Library/QuickLook/SketchQuicklook.qlgenerator?

ale

mathieudutour you can use either sketchtool export or qlmanage

mathieudutour

Oh yes thanks!

I noticed a foreignSymbols field in the document.json. Is it for an upcoming feature where a document would be dependant on symbols defined elsewhere? That would be awesome

ale

mathieudutour who knows…

mathieudutour

Haha fair

jongold

This is awesome. One of our projects at Airbnb last year was a Sketch file indexer + tagged visual file search for 1000s of files.

By far the most difficult part of the process was indexing our files; we have a crazy setup of several Mac Pros & Minis chugging through them with a wrapper we wrote ontop of Sketch Tool…this gets rid of all of that pain.

Thanks / wish we'd had it 6 months ago!

jongold

er, also we're working on a React-to-Sketch renderer that currently uses private headers, but might be way easier if I just translate React ASTs into Sketch 43 ASTs…

v-vodyanov

Will sketchtool be able to convert old Sketch documents to new format? I guess it would be very useful for the first time)

ale

v-vodyanov good question, I'll ask the team!

ale

v-vodyanov jonlee108 asked the team: at the moment it's not possible to use sketchtool to migrate old documents. We could do it, but it's not a trivial change, and at this point it's easier for everyone if you just use Sketch itself to open and save the files.

We have added a feature to sketchtool that allows it to tell Sketch to run plugins (I should probably document this on a new thread). Try running sketchtool run with SketchTool 43 : )

jonlee108

ale I'm also really interested in whether there's a way to programmatically upgrade old sketch files to the new format. Please do let us know! ?

andrewpouliot

I was looking into how best to insert some layers from Sketch v43 JSON. I had to do some mucking about w/ private headers & hopper but I came up with this:

var page = {/*json from decompressed sketch file */};
var decoded = [[[MSJSONDictionaryUnarchiver alloc] initForReadingFromDictionary:page] decodeRoot];
var msPage = [[MSPage alloc] initWithImmutableModelObject:decoded];
context.document.currentPage().addLayers(msPage.layers());

It would be really nice if there were a more supported / correct way to do this as part of the API.

But huge props for taking this to the finish line, I know you've been working hard on it for a while ?

jdsimcoe

Have you guys considered handling Sketch files like .app bundles where it is a macOS package, essentially a folder format? This would enable version control like Git to handle Sketch files without having to resort to things like git-lfs. Any thoughts?

andrewpouliot

jdsimcoe On Origami we had this as our initial format, but it really didn't work out and we ended up switching to a zipped folder. The main issue is that you can't revert the whole folder on dropbox & it tracks changes to individual files separately so you can end up with corruption of the bundle as a whole. Yes, it's better for git the other way, so we left that in as an option temporarily.

ale

jdsimcoe fun fact: Sketch documents were bundles in the past, and it was a world of pain. Andrew explained some of the issues, and I'll add:

you can't attach a bundle to an email on most mail clients
you can't upload a bundle to a web page using a form
you can't drag and drop a bundle to some file-sharing apps without them going crazy
and many more I've luckily forgotten about

I'm really glad we moved away from that, and don't look forward to dealing with those issues again : )

andrewpouliot

I discovered you can dump JSON from the current selection this way (probably not optimal or complete):

const selection = context.selection;
const artboard = selection[0];
const artboardImm = artboard.immutableModelObject();
const string = [MSJSONDataArchiver archiveStringWithRootObject:artboardImm error:nil];
log(string);

ale

berk JSON serialisation is definitely on our radar. Meanwhile, see andrewpouliot for some ideas.

berk

\m/

This will be pretty useful for Zeplin!

ale

WonderCsabo that's very unlikely, for the reasons I outlined in ale, but never say never…