Skip to content

Commit

Permalink
Implement renderer test suite
Browse files Browse the repository at this point in the history
This implements a Markdown / Gemtext suite, testing the entire renderer
at complex Markdown documents, containing the entirety of current
Markdown features accessible with gmnhg. The test suite can be expanded
by adding a pair of .md/.gmi files.

Minor bug fixes in JSON/Org metadata parsing where bugs were detected
with the test suite are also included in this patch.

Fixes #13.
  • Loading branch information
tdemin committed Sep 30, 2021
1 parent c80b726 commit fc76187
Show file tree
Hide file tree
Showing 22 changed files with 653 additions and 5 deletions.
3 changes: 3 additions & 0 deletions README.md
Expand Up @@ -40,6 +40,9 @@ The renderer will also treat lists of links and paragraphs consisting of
links only the special way: it will render only the links block for
them.

To get a better idea of how source Markdown looks like after the
conversion to Gemtext, see [testdata](testdata) directory.

[gomarkdown]: https://github.com/gomarkdown/markdown

## gmnhg
Expand Down
3 changes: 3 additions & 0 deletions go.mod
Expand Up @@ -7,14 +7,17 @@ require (
github.com/Masterminds/sprig/v3 v3.2.2
github.com/gomarkdown/markdown v0.0.0-20210915032930-fe0e174ee09a
github.com/google/uuid v1.3.0 // indirect
github.com/hexops/gotextdiff v1.0.3
github.com/huandu/xstrings v1.3.2 // indirect
github.com/imdario/mergo v0.3.12 // indirect
github.com/kr/pretty v0.1.0 // indirect
github.com/mattn/go-runewidth v0.0.13 // indirect
github.com/mitchellh/copystructure v1.2.0 // indirect
github.com/niklasfasching/go-org v1.5.0
github.com/olekukonko/tablewriter v0.0.5
github.com/spf13/cast v1.4.1 // indirect
golang.org/x/crypto v0.0.0-20210915214749-c084706c2272 // indirect
golang.org/x/net v0.0.0-20210917163549-3c21e5b27794 // indirect
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 // indirect
gopkg.in/yaml.v2 v2.4.0
)
10 changes: 9 additions & 1 deletion go.sum
Expand Up @@ -21,12 +21,19 @@ github.com/gomarkdown/markdown v0.0.0-20210915032930-fe0e174ee09a/go.mod h1:JDGc
github.com/google/uuid v1.1.1/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/google/uuid v1.3.0 h1:t6JiXgmwXMjEs8VusXIJk2BXHsn+wx8BZdTaoZ5fu7I=
github.com/google/uuid v1.3.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/hexops/gotextdiff v1.0.3 h1:gitA9+qJrrTCsiCl7+kh75nPqQt1cx4ZkudSTLoUqJM=
github.com/hexops/gotextdiff v1.0.3/go.mod h1:pSWU5MAI3yDq+fZBTazCSJysOMbxWL1BSow5/V2vxeg=
github.com/huandu/xstrings v1.3.1/go.mod h1:y5/lhBue+AyNmUVz9RLU9xbLR0o4KIIExikq4ovT0aE=
github.com/huandu/xstrings v1.3.2 h1:L18LIDzqlW6xN2rEkpdV8+oL/IXWJ1APd+vsdYy4Wdw=
github.com/huandu/xstrings v1.3.2/go.mod h1:y5/lhBue+AyNmUVz9RLU9xbLR0o4KIIExikq4ovT0aE=
github.com/imdario/mergo v0.3.11/go.mod h1:jmQim1M+e3UYxmgPu/WyfjB3N3VflVyUjjjwH0dnCYA=
github.com/imdario/mergo v0.3.12 h1:b6R2BslTbIEToALKP7LxUvijTsNI9TAe80pLWN2g/HU=
github.com/imdario/mergo v0.3.12/go.mod h1:jmQim1M+e3UYxmgPu/WyfjB3N3VflVyUjjjwH0dnCYA=
github.com/kr/pretty v0.1.0 h1:L/CwN0zerZDmRFUapSPitk6f+Q3+0za1rQkzVuMiMFI=
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/text v0.1.0 h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/mattn/go-colorable v0.1.6/go.mod h1:u6P/XSegPjTcexA+o6vUJrdnUu04hMope9wVRipJSqc=
github.com/mattn/go-isatty v0.0.12/go.mod h1:cbi8OIDigv2wuxKPP5vlRcQ1OAZbq2CE4Kysco4FUpU=
github.com/mattn/go-runewidth v0.0.9/go.mod h1:H031xJmbD/WCDINGzjvQ9THkh0rPKHF+m2gUSrubnMI=
Expand Down Expand Up @@ -81,8 +88,9 @@ golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 h1:YR8cESwS4TdDjEe65xsg0ogRM/Nc3DYOhEAlW+xobZo=
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.3.0/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY=
Expand Down
7 changes: 5 additions & 2 deletions internal/gmnhg/org.go
Expand Up @@ -17,6 +17,7 @@ package gmnhg

import (
"bytes"
"errors"
"fmt"
"reflect"
"regexp"
Expand Down Expand Up @@ -48,6 +49,8 @@ func parseValue(value interface{}) interface{} {
return value
}

var errKeyNotFound = errors.New("cannot find tagged key in struct")

// for key "key" will set either map key "key" or struct field tagged
// `tag:"key"` with value; expects a pointer
func reflectSetKey(mapOrStruct interface{}, tag, key string, value interface{}) (err error) {
Expand Down Expand Up @@ -75,7 +78,7 @@ func reflectSetKey(mapOrStruct interface{}, tag, key string, value interface{})
fieldName = field.Name
}
if fieldName == "" {
return fmt.Errorf("cannot find tag %v with key %v in struct", tag, key)
return fmt.Errorf("%v: %v %w", tag, key, errKeyNotFound)
}
v.FieldByName(fieldName).Set(reflect.ValueOf(parseValue(value)))
default:
Expand Down Expand Up @@ -103,7 +106,7 @@ func unmarshalORG(data []byte, p interface{}) (err error) {
} else if k == "date" {
value = parseORGDate(v)
}
if err := reflectSetKey(p, "org", strings.ToLower(key), value); err != nil {
if err := reflectSetKey(p, "org", strings.ToLower(key), value); err != nil && !errors.Is(err, errKeyNotFound) {
return err
}
}
Expand Down
4 changes: 2 additions & 2 deletions internal/gmnhg/post.go
Expand Up @@ -62,7 +62,7 @@ var (
yamlDelimiter = []byte("---\n")
tomlDelimiter = []byte("+++\n")
jsonObjectRegex = regexp.MustCompile(`\A(\{[\s\S]*\})\n\n`)
orgModeRegex = regexp.MustCompile(`\A((?:#\+\w+: ?\S*\n)*)`)
orgModeRegex = regexp.MustCompile(`\A((?:#\+\w+\[?\]?: ?[^\n\r]*\n)+)`)
)

// ParseMetadata extracts TOML/JSON/YAML/org-mode format front matter
Expand Down Expand Up @@ -109,7 +109,7 @@ func ParseMetadata(source []byte) (markdown []byte, metadata Metadata) {
if err := json.Unmarshal(metadataContent, &metadata); err != nil {
return
}
markdown = source[blockEnd+1:] // JSON end + \n\n - 1
markdown = source[blockEnd:]
} else if match := orgModeRegex.FindIndex(source); match != nil {
blockEnd = match[1]
metadataContent = source[:blockEnd]
Expand Down
77 changes: 77 additions & 0 deletions render_test.go
@@ -0,0 +1,77 @@
// This file is part of gmnhg.

// gmnhg is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.

// gmnhg is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.

// You should have received a copy of the GNU General Public License
// along with gmnhg. If not, see <https://www.gnu.org/licenses/>.

package gemini

import (
"bytes"
"io/ioutil"
"os"
"path"
"regexp"
"testing"

"github.com/hexops/gotextdiff"
"github.com/hexops/gotextdiff/myers"
"github.com/hexops/gotextdiff/span"
"github.com/tdemin/gmnhg/internal/gmnhg"
)

var fileList []string

var (
mdFilenameRegex = regexp.MustCompile(`^(.+)\.md$`)
)

func TestMain(m *testing.M) {
// go test implicitly sets cwd to tested package directory; sadly,
// this fact is undocumented
files, err := ioutil.ReadDir("testdata")
if err != nil {
panic(err)
}
for _, fileInfo := range files {
if match := mdFilenameRegex.FindStringSubmatch(fileInfo.Name()); !fileInfo.IsDir() && match != nil {
fileList = append(fileList, match[1])
}
}
os.Exit(m.Run())
}

func TestRenderer(t *testing.T) {
for _, testName := range fileList {
t.Logf("testing %s", testName)
mdContents, err := ioutil.ReadFile(path.Join("testdata", testName+".md"))
if err != nil {
t.Fatalf("failed to open Markdown test %s: %v", testName, err)
}
gmiContents, err := ioutil.ReadFile(path.Join("testdata", testName+".gmi"))
if err != nil {
t.Logf("%s: cannot open Gemtext file, skipping: %v", testName, err)
continue
}
content, _ := gmnhg.ParseMetadata(mdContents)
geminiContent, err := RenderMarkdown(content, Defaults)
if err != nil {
t.Errorf("failed to convert %s Markdown to Gemtext: %v", testName, err)
}
if !bytes.Equal(geminiContent, gmiContents) {
diff := myers.ComputeEdits(span.URIFromPath("a.gmi"),
string(geminiContent), string(gmiContents))
t.Errorf("content mismatch on %s, diff:\n%s", testName,
gotextdiff.ToUnified("a.gmi", "b.gmi", string(geminiContent), diff))
}
}
}
1 change: 1 addition & 0 deletions testdata/front_matter_json.gmi
@@ -0,0 +1 @@
gmnhg test suite should parse the metadata above but disregard it.
6 changes: 6 additions & 0 deletions testdata/front_matter_json.md
@@ -0,0 +1,6 @@
{
"title": "JSON front matter test",
"draft": true
}

gmnhg test suite should parse the metadata above but disregard it.
1 change: 1 addition & 0 deletions testdata/front_matter_org.gmi
@@ -0,0 +1 @@
gmnhg test suite should parse the metadata above but disregard it.
5 changes: 5 additions & 0 deletions testdata/front_matter_org.md
@@ -0,0 +1,5 @@
#+title: "ORG front matter test"
#+draft: true
#+tags[]: org,gmnhg,emacs

gmnhg test suite should parse the metadata above but disregard it.
1 change: 1 addition & 0 deletions testdata/front_matter_toml.gmi
@@ -0,0 +1 @@
gmnhg test suite should parse the metadata above but disregard it.
6 changes: 6 additions & 0 deletions testdata/front_matter_toml.md
@@ -0,0 +1,6 @@
+++
title = "TOML front matter test"
draft = true
+++

gmnhg test suite should parse the metadata above but disregard it.
1 change: 1 addition & 0 deletions testdata/front_matter_yaml.gmi
@@ -0,0 +1 @@
gmnhg test suite should parse the metadata above but disregard it.
6 changes: 6 additions & 0 deletions testdata/front_matter_yaml.md
@@ -0,0 +1,6 @@
---
title: "YAML front matter test"
draft: true
---

gmnhg test suite should parse the metadata above but disregard it.
89 changes: 89 additions & 0 deletions testdata/general_text.gmi
@@ -0,0 +1,89 @@
# General text

Paragraphs are printed verbatim in gmnhg.

Single newlines (like in this multi-line paragraph) will get replaced by a space, as Gemini specification p. 5.4.1 recommends this for soft-wrapping text by clients.

Inline formatting bits (like this **bold** text, *emphasized* text, ~~strikethrough~~ text, `preformatted text`) are kept to make sure Gemini readers still have the stylistic context of your text.

## Blockquotes

Newlines in blockquote paragraphs, unlike usual paragraphs, aren't replaced with a space. This facilitates appending authorship information to the quote, or using blockquotes to write poems.

> "Never trouble another for what you can do yourself"
> — Thomas Jefferson, 3rd president of the US

> "Wow, writing comprehensive test suites is hard!"
> — Timur Demin, while writing this very test file

> "Somehow I know these two paragraphs will be broken into two separate
> blockquotes by gmnhg. I think my knowledge of that comes from being
> the author of this program."

> — also Timur Demin, in the process of writing this test file

## Code

gmnhg will use Gemtext preformatted blocks for that. Markdown alt-text for preformatted blocks is supported, and is used to render alt-text as specified by Gemini spec p. 5.4.3.

```go
package main
func main() {
println("gmnhg is awesome!")
}
```

Preformatted Markdown of course isn't rendered:

```
# I am a test Markdown document
I contain text in **bold**.
```

## Links

gmnhg supports links, images, and footnotes. Links are a very interesting topic on itself; see a separate document for those.

=> links.md separate document

## Lists

Definition lists, numbered and ordered lists are all supported in gmnhg. There's also a separate document displaying those.

=> lists.md separate document

## Tables

Markdown tables are supported in gmnhg, and are better displayed by a separate document.

=> tables.md separate document

## Headings

Gemini specification allows up to three heading levels, with an optional space after the last heading symbol, `#`. With Markdown, you get 6; gmnhg will simply print the relevant number of #-s, making the client up to parse more heading levels and keeping context of the source document.

Since clients like Lagrange treat the fourth and the rest of #-s as heading content, it's best to avoid using H4-H6 in Gemini-aware Markdown entirely. Headings from H3 to H6 are provided below so you can test how your client handles that.

### Heading 3

#### Heading 4

##### Heading 5

###### Heading 6

## Misc

Inline HTML is currently stripped, but HTML contents remain on-screen. This may change in the future.

> There's currently a bug in gmnhg which prevents it from
> stripping HTML in certain scenarios. HTML is noticeably still present
> inside <span>blockquotes</span>.

=> https://github.com/tdemin/gmnhg/issues/6 bug in gmnhg

---

The Markdown horizontal line above is rendered as triple dashes.

0 comments on commit fc76187

Please sign in to comment.