Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Strip HTML tags (but keep any text content) when rendering text #33

Merged
merged 16 commits into from Oct 2, 2021
Merged

Strip HTML tags (but keep any text content) when rendering text #33

Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
7834c27
Render blockquote children recursively
mntn-xyz Sep 12, 2021
34cd4c6
Remove fmt.Println
mntn-xyz Sep 12, 2021
5df2399
Merge branch 'master' into blockquote
mntn-xyz Sep 19, 2021
9e78c95
Integrate blockquote fix with textWithNewlineReplacement
mntn-xyz Sep 19, 2021
b4a9bea
Render HTML block content
mntn-xyz Sep 19, 2021
8fdc8c0
Remove nonewline change, does not apply
mntn-xyz Sep 19, 2021
25aec5f
Merge branch 'master' into blockquote
mntn-xyz Sep 20, 2021
a13aa93
Implement tag stripping for HTML blocks
mntn-xyz Sep 20, 2021
9b4b07c
Improve handling of HTMLBlock vs HTMLSpan
mntn-xyz Sep 22, 2021
1f902d1
Fix hardbreak
mntn-xyz Sep 22, 2021
6dc81d8
Merge from master
mntn-xyz Oct 1, 2021
280c969
Add test for hard break
mntn-xyz Oct 1, 2021
53c388e
Add more HTML tests
mntn-xyz Oct 1, 2021
1f6bb3a
Unescape HTML escapes in HTML blocks, fix for hard break, more tests
mntn-xyz Oct 1, 2021
443e1c5
Add unescaping of HTML entities
mntn-xyz Oct 1, 2021
247c5c7
Fix HTML block in blockquote, organize HTML tests
mntn-xyz Oct 1, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions go.mod
View file
Open in desktop
Expand Up @@ -7,6 +7,7 @@ require (
github.com/Masterminds/sprig/v3 v3.2.2
github.com/gomarkdown/markdown v0.0.0-20210915032930-fe0e174ee09a
github.com/google/uuid v1.3.0 // indirect
github.com/grokify/html-strip-tags-go v0.0.1
github.com/huandu/xstrings v1.3.2 // indirect
github.com/imdario/mergo v0.3.12 // indirect
github.com/mattn/go-runewidth v0.0.13 // indirect
Expand Down
2 changes: 2 additions & 0 deletions go.sum
View file
Open in desktop
Expand Up @@ -21,6 +21,8 @@ github.com/gomarkdown/markdown v0.0.0-20210915032930-fe0e174ee09a/go.mod h1:JDGc
github.com/google/uuid v1.1.1/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/google/uuid v1.3.0 h1:t6JiXgmwXMjEs8VusXIJk2BXHsn+wx8BZdTaoZ5fu7I=
github.com/google/uuid v1.3.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/grokify/html-strip-tags-go v0.0.1 h1:0fThFwLbW7P/kOiTBs03FsJSV9RM2M/Q/MOnCQxKMo0=
github.com/grokify/html-strip-tags-go v0.0.1/go.mod h1:2Su6romC5/1VXOQMaWL2yb618ARB8iVo6/DR99A6d78=
github.com/huandu/xstrings v1.3.1/go.mod h1:y5/lhBue+AyNmUVz9RLU9xbLR0o4KIIExikq4ovT0aE=
github.com/huandu/xstrings v1.3.2 h1:L18LIDzqlW6xN2rEkpdV8+oL/IXWJ1APd+vsdYy4Wdw=
github.com/huandu/xstrings v1.3.2/go.mod h1:y5/lhBue+AyNmUVz9RLU9xbLR0o4KIIExikq4ovT0aE=
Expand Down
30 changes: 28 additions & 2 deletions internal/renderer/renderer.go
View file
Open in desktop
Expand Up @@ -24,6 +24,7 @@ import (
"regexp"
"strings"

"github.com/grokify/html-strip-tags-go"
"github.com/gomarkdown/markdown/ast"
"github.com/olekukonko/tablewriter"
)
Expand Down Expand Up @@ -346,13 +347,20 @@ func textWithNewlineReplacement(node ast.Node, replacement []byte) []byte {
if node, ok := node.(*ast.Link); ok && node.Footnote != nil {
fmt.Fprintf(&buf, "[^%d]", node.NoteID)
}
if node := node.AsLeaf(); node != nil {
if leaf := node.AsLeaf(); leaf != nil {
// replace all newlines in text with preferred symbols; this may
// be spaces for general text, allowing for soft wrapping, which
// is recommended as per Gemini spec p. 5.4.1, or line breaks
// with a blockquote symbols for blockquotes, or just nothing
buf.Write(delimiter)
buf.Write(lineBreakCharacters.ReplaceAll(node.Literal, replacement))
switch node.(type) {
case *ast.HTMLSpan:
buf.Write(lineBreakCharacters.ReplaceAll(leaf.Content, replacement))
case *ast.HTMLBlock:
buf.Write(lineBreakCharacters.ReplaceAll([]byte(strip.StripTags(string(leaf.Literal))), replacement))
default:
buf.Write(lineBreakCharacters.ReplaceAll(leaf.Literal, replacement))
}
buf.Write(delimiter)
}
if node := node.AsContainer(); node != nil {
Expand Down Expand Up @@ -440,6 +448,20 @@ func (r Renderer) table(w io.Writer, node *ast.Table, entering bool) {
}
}

func (r Renderer) htmlSpan(w io.Writer, node *ast.HTMLSpan, entering bool) {
if entering {
r.text(w, node)
}
}

func (r Renderer) htmlBlock(w io.Writer, node *ast.HTMLBlock, entering bool) {
if entering {
r.text(w, node)
w.Write(lineBreak)
w.Write(lineBreak)
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Subroutines called from RenderNode generally do the final linebreak that splits Gemtext paragraphs with noNewLine. Not that it matters much; it's just a consistency thing (entering and double AST passthrough was initially intended for that job in gomarkdown, but block elements in HTML and Gemtext are different, so it doesn't get much use for that).

}
}

// RenderNode implements Renderer.RenderNode().
func (r Renderer) RenderNode(w io.Writer, node ast.Node, entering bool) ast.WalkStatus {
// entering in gomarkdown was made to have elements of type switch
Expand Down Expand Up @@ -487,6 +509,10 @@ func (r Renderer) RenderNode(w io.Writer, node ast.Node, entering bool) ast.Walk
r.table(w, node, entering)
noNewLine = false
fetchLinks = true
case *ast.HTMLBlock:
mntn-xyz marked this conversation as resolved.
Show resolved Hide resolved
r.htmlBlock(w, node, entering)
case *ast.HTMLSpan:
r.htmlSpan(w, node, entering)
}
if !noNewLine && !entering {
w.Write(lineBreak)
Expand Down