Using pandoc with a custom new syntax in a C program with live preview

[smnto]

Hello,

I would like to implement live preview for a custom syntax (very similar to djot) in a C (GTK) program.

1. Is it possible to embed the pandoc library in a C program and if yes - how? I saw this c binding but it seems to be abandoned for 7 years.
2. What is better - try to copy and tune the existing implementation of djot in Haskell or try to use Lua to add my new syntax? What are pros/cons of the two approaches?
3. Is it possible to implement recursive file inclusion for that new syntax in Lua or Haskell for pandoc? Is one of the existing formats supports file inclusion already to see how it's done?
4. Is pandoc fast enough to implement live preview for very long documents (as long as books e.g. War and Piece)?

Thank you!

[jgm]

re 1. Easiest approach would be to run pandoc server in a separate process and pass JSON back and forth. In principle it should be possible to use Haskell's FFI to expose a C library, but this would take some work.

re 2. The Haskell parser does not run in IO, so the parser can't directly include a file. However, you could use a later transformation stage on the AST, matching an AST element and replacing it with the result of parsing another file. In pandoc the RST reader, for example, implements inclusion.

re 4. Pandoc is not particularly fast, so I don't think it would be good for that purpose. The Haskell djot parser is probably fast enough (used by itself -- pandoc brings in other things that slow down conversion). Even faster is the Rust djot parser.

[smnto]

@jgm - thank you very much for the fast and informative response, for the great tool - pandoc, and for the new carefully designed markup syntax - djot! (I'd suggest two small changes to the latter - (a) link syntax: [link text]<url> which is more consistent with the <short_link_form> and (b) disallow alternative syntax (e.g. choose only one of possible list bullets - '*', '-' or '+') because texts do not look consistent if edited collaboratively by different people with different preferences).

2. Is recursive file inclusion possible to implement in the current djot.js. I.e. first include all the files in the input stream and then process the input stream as a whole. Or the only way to achieve this is to use a later transformation stage on the AST (as you've described)?
3. Is current javascript implementation of djot fast enough to allow live preview of very long djot documents/books in an embedded browser (like WebkitGTK)?
4. Is there going to be a C implementation of djot in the near future?
5. Is one of the current implementations of djot (like in Rust) can be embedded in C?

[jgm]

2 - yes it should be. 3 - probably yes, it's quite fast. 4 - no plans for that. 5 - certainly the Lua one, and the Rust one can probably be linked to a C program.