gitea/modules/templates/helper.go

213 lines
6.8 KiB
Go
Raw Normal View History

// Copyright 2018 The Gitea Authors. All rights reserved.
2014-04-10 20:20:58 +02:00
// Copyright 2014 The Gogs Authors. All rights reserved.
// SPDX-License-Identifier: MIT
2014-04-10 20:20:58 +02:00
package templates
2014-04-10 20:20:58 +02:00
import (
"context"
2014-04-10 20:20:58 +02:00
"fmt"
"html"
2014-04-10 20:20:58 +02:00
"html/template"
"net/url"
2014-04-10 20:20:58 +02:00
"strings"
"time"
2014-05-26 02:11:25 +02:00
system_model "code.gitea.io/gitea/models/system"
"code.gitea.io/gitea/modules/base"
Support unicode emojis and remove emojify.js (#11032) * Support unicode emojis and remove emojify.js This PR replaces all use of emojify.js and adds unicode emoji support to various areas of gitea. This works in a few ways: First it adds emoji parsing support into gitea itself. This allows us to * Render emojis from valid alias (:smile:) * Detect unicode emojis and let us put them in their own class with proper aria-labels and styling * Easily allow for custom "emoji" * Support all emoji rendering and features without javascript * Uses plain unicode and lets the system render in appropriate emoji font * Doesn't leave us relying on external sources for updates/fixes/features That same list of emoji is also used to create a json file which replaces the part of emojify.js that populates the emoji search tribute. This file is about 35KB with GZIP turned on and I've set it to load after the page renders to not hinder page load time (and this removes loading emojify.js also) For custom "emoji" it uses a pretty simple scheme of just looking for /emojis/img/name.png where name is something a user has put in the "allowed reactions" setting we already have. The gitea reaction that was previously hard coded into a forked copy of emojify.js is included and works as a custom reaction under this method. The emoji data sourced here is from https://github.com/github/gemoji which is the gem library Github uses for their emoji rendering (and a data source for other sites). So we should be able to easily render any emoji and :alias: that Github can, removing any errors from migrated content. They also update it as well, so we can sync when there are new unicode emoji lists released. I've included a slimmed down and slightly modified forked copy of https://github.com/knq/emoji to make up our own emoji module. The code is pretty straight forward and again allows us to have a lot of flexibility in what happens. I had seen a few comments about performance in some of the other threads if we render this ourselves, but there doesn't seem to be any issue here. In a test it can parse, convert, and render 1,000 emojis inside of a large markdown table in about 100ms on my laptop (which is many more emojis than will ever be in any normal issue). This also prevents any flickering and other weirdness from using javascript to render some things while using go for others. Not included here are image fall back URLS. I don't really think they are necessary for anything new being written in 2020. However, managing the emoji ourselves would allow us to add these as a feature later on if it seems necessary. Fixes: https://github.com/go-gitea/gitea/issues/9182 Fixes: https://github.com/go-gitea/gitea/issues/8974 Fixes: https://github.com/go-gitea/gitea/issues/8953 Fixes: https://github.com/go-gitea/gitea/issues/6628 Fixes: https://github.com/go-gitea/gitea/issues/5130 * add new shared function emojiHTML * don't increase emoji size in issue title * Update templates/repo/issue/view_content/add_reaction.tmpl Co-Authored-By: 6543 <6543@obermui.de> * Support for emoji rendering in various templates * Render code and review comments as they should be * Better way to handle mail subjects * insert unicode from tribute selection * Add template helper for plain text when needed * Use existing replace function I forgot about * Don't include emoji greater than Unicode Version 12 Only include emoji and aliases in JSON * Update build/generate-emoji.go * Tweak regex slightly to really match everything including random invisible characters. Run tests for every emoji we have * final updates * code review * code review * hard code gitea custom emoji to match previous behavior * Update .eslintrc Co-Authored-By: silverwind <me@silverwind.io> * disable preempt Co-authored-by: silverwind <me@silverwind.io> Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: Lauris BH <lauris@nix.lv> Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
2020-04-28 20:05:39 +02:00
"code.gitea.io/gitea/modules/emoji"
"code.gitea.io/gitea/modules/markup"
"code.gitea.io/gitea/modules/setting"
"code.gitea.io/gitea/modules/svg"
Use a general Eval function for expressions in templates. (#23927) One of the proposals in #23328 This PR introduces a simple expression calculator (templates/eval/eval.go), it can do basic expression calculations. Many untested template helper functions like `Mul` `Add` can be replaced by this new approach. Then these `Add` / `Mul` / `percentage` / `Subtract` / `DiffStatsWidth` could all use this `Eval`. And it provides enhancements for Golang templates, and improves readability. Some examples: ---- * Before: `{{Add (Mul $glyph.Row 12) 12}}` * After: `{{Eval $glyph.Row "*" 12 "+" 12}}` ---- * Before: `{{if lt (Add $i 1) (len $.Topics)}}` * After: `{{if Eval $i "+" 1 "<" (len $.Topics)}}` ## FAQ ### Why not use an existing expression package? We need a highly customized expression engine: * do the calculation on the fly, without pre-compiling * deal with int/int64/float64 types, to make the result could be used in Golang template. * make the syntax could be used in the Golang template directly * do not introduce too much complex or strange syntax, we just need a simple calculator. * it needs to strictly follow Golang template's behavior, for example, Golang template treats all non-zero values as truth, but many 3rd packages don't do so. ### What's the benefit? * Developers don't need to add more `Add`/`Mul`/`Sub`-like functions, they were getting more and more. Now, only one `Eval` is enough for all cases. * The new code reads better than old `{{Add (Mul $glyph.Row 12) 12}}`, the old one isn't familiar to most procedural programming developers (eg, the Golang expression syntax). * The `Eval` is fully covered by tests, many old `Add`/`Mul`-like functions were never tested. ### The performance? It doesn't use `reflect`, it doesn't need to parse or compile when used in Golang template, the performance is as fast as native Go template. ### Is it too complex? Could it be unstable? The expression calculator program is a common homework for computer science students, and it's widely used as a teaching and practicing purpose for developers. The algorithm is pretty well-known. The behavior can be clearly defined, it is stable.
2023-04-07 15:25:49 +02:00
"code.gitea.io/gitea/modules/templates/eval"
"code.gitea.io/gitea/modules/timeutil"
"code.gitea.io/gitea/modules/util"
"code.gitea.io/gitea/services/gitdiff"
2014-04-10 20:20:58 +02:00
)
2016-11-25 07:23:48 +01:00
// NewFuncMap returns functions for injecting to templates
func NewFuncMap() template.FuncMap {
return map[string]interface{}{
"DumpVar": dumpVar,
// -----------------------------------------------------------------
// html/template related functions
"dict": dict, // it's lowercase because this name has been widely used. Our other functions should have uppercase names.
"Eval": Eval,
"Safe": Safe,
"Escape": html.EscapeString,
"QueryEscape": url.QueryEscape,
"JSEscape": template.JSEscapeString,
"Str2html": Str2html, // TODO: rename it to SanitizeHTML
"URLJoin": util.URLJoin,
"DotEscape": DotEscape,
"PathEscape": url.PathEscape,
"PathEscapeSegments": util.PathEscapeSegments,
// utils
"StringUtils": NewStringUtils,
"SliceUtils": NewSliceUtils,
"JsonUtils": NewJsonUtils,
// -----------------------------------------------------------------
// svg / avatar / icon
"svg": svg.RenderHTML,
"avatar": Avatar,
"avatarHTML": AvatarHTML,
"avatarByAction": AvatarByAction,
"avatarByEmail": AvatarByEmail,
"repoAvatar": RepoAvatar,
"EntryIcon": base.EntryIcon,
"MigrationIcon": MigrationIcon,
"ActionIcon": ActionIcon,
"SortArrow": SortArrow,
// -----------------------------------------------------------------
// time / number / format
"FileSize": base.FileSize,
"CountFmt": base.FormatNumberSI,
"TimeSince": timeutil.TimeSince,
"TimeSinceUnix": timeutil.TimeSinceUnix,
"DateTime": timeutil.DateTime,
"Sec2Time": util.SecToTime,
"LoadTimes": func(startTime time.Time) string {
return fmt.Sprint(time.Since(startTime).Nanoseconds()/1e6) + "ms"
},
// -----------------------------------------------------------------
// setting
2016-03-06 22:40:04 +01:00
"AppName": func() string {
return setting.AppName
},
"AppSubUrl": func() string {
return setting.AppSubURL
2016-03-06 22:40:04 +01:00
},
"AssetUrlPrefix": func() string {
return setting.StaticURLPrefix + "/assets"
},
2016-03-06 22:40:04 +01:00
"AppUrl": func() string {
// The usage of AppUrl should be avoided as much as possible,
// because the AppURL(ROOT_URL) may not match user's visiting site and the ROOT_URL in app.ini may be incorrect.
// And it's difficult for Gitea to guess absolute URL correctly with zero configuration,
// because Gitea doesn't know whether the scheme is HTTP or HTTPS unless the reverse proxy could tell Gitea.
return setting.AppURL
2016-03-06 22:40:04 +01:00
},
"AppVer": func() string {
return setting.AppVer
},
"AppDomain": func() string { // documented in mail-templates.md
2016-03-06 22:40:04 +01:00
return setting.Domain
},
"AssetVersion": func() string {
return setting.AssetVersion
},
Add context cache as a request level cache (#22294) To avoid duplicated load of the same data in an HTTP request, we can set a context cache to do that. i.e. Some pages may load a user from a database with the same id in different areas on the same page. But the code is hidden in two different deep logic. How should we share the user? As a result of this PR, now if both entry functions accept `context.Context` as the first parameter and we just need to refactor `GetUserByID` to reuse the user from the context cache. Then it will not be loaded twice on an HTTP request. But of course, sometimes we would like to reload an object from the database, that's why `RemoveContextData` is also exposed. The core context cache is here. It defines a new context ```go type cacheContext struct { ctx context.Context data map[any]map[any]any lock sync.RWMutex } var cacheContextKey = struct{}{} func WithCacheContext(ctx context.Context) context.Context { return context.WithValue(ctx, cacheContextKey, &cacheContext{ ctx: ctx, data: make(map[any]map[any]any), }) } ``` Then you can use the below 4 methods to read/write/del the data within the same context. ```go func GetContextData(ctx context.Context, tp, key any) any func SetContextData(ctx context.Context, tp, key, value any) func RemoveContextData(ctx context.Context, tp, key any) func GetWithContextCache[T any](ctx context.Context, cacheGroupKey string, cacheTargetID any, f func() (T, error)) (T, error) ``` Then let's take a look at how `system.GetString` implement it. ```go func GetSetting(ctx context.Context, key string) (string, error) { return cache.GetWithContextCache(ctx, contextCacheKey, key, func() (string, error) { return cache.GetString(genSettingCacheKey(key), func() (string, error) { res, err := GetSettingNoCache(ctx, key) if err != nil { return "", err } return res.SettingValue, nil }) }) } ``` First, it will check if context data include the setting object with the key. If not, it will query from the global cache which may be memory or a Redis cache. If not, it will get the object from the database. In the end, if the object gets from the global cache or database, it will be set into the context cache. An object stored in the context cache will only be destroyed after the context disappeared.
2023-02-15 14:37:34 +01:00
"DisableGravatar": func(ctx context.Context) bool {
return system_model.GetSettingWithCacheBool(ctx, system_model.KeyPictureDisableGravatar)
2016-03-06 22:40:04 +01:00
},
"DefaultShowFullName": func() bool {
return setting.UI.DefaultShowFullName
},
"ShowFooterTemplateLoadTime": func() bool {
return setting.Other.ShowFooterTemplateLoadTime
},
"AllowedReactions": func() []string {
return setting.UI.Reactions
},
2021-06-29 16:28:38 +02:00
"CustomEmojis": func() map[string]string {
return setting.UI.CustomEmojisMap
},
"MetaAuthor": func() string {
return setting.UI.Meta.Author
},
"MetaDescription": func() string {
return setting.UI.Meta.Description
},
"MetaKeywords": func() string {
return setting.UI.Meta.Keywords
},
"EnableTimetracking": func() bool {
return setting.Service.EnableTimetracking
},
"DisableGitHooks": func() bool {
return setting.DisableGitHooks
},
"DisableWebhooks": func() bool {
return setting.DisableWebhooks
},
"DisableImportLocal": func() bool {
return !setting.ImportLocalPaths
},
"DefaultTheme": func() string {
return setting.UI.DefaultTheme
},
"NotificationSettings": func() map[string]interface{} {
return map[string]interface{}{
"MinTimeout": int(setting.UI.Notification.MinTimeout / time.Millisecond),
"TimeoutStep": int(setting.UI.Notification.TimeoutStep / time.Millisecond),
"MaxTimeout": int(setting.UI.Notification.MaxTimeout / time.Millisecond),
"EventSourceUpdateTime": int(setting.UI.Notification.EventSourceUpdateTime / time.Millisecond),
Use AJAX for notifications table (#10961) * Use AJAX for notifications table Signed-off-by: Andrew Thornton <art27@cantab.net> * move to separate js Signed-off-by: Andrew Thornton <art27@cantab.net> * placate golangci-lint Signed-off-by: Andrew Thornton <art27@cantab.net> * Add autoupdating notification count Signed-off-by: Andrew Thornton <art27@cantab.net> * Fix wipeall Signed-off-by: Andrew Thornton <art27@cantab.net> * placate tests Signed-off-by: Andrew Thornton <art27@cantab.net> * Try hidden Signed-off-by: Andrew Thornton <art27@cantab.net> * Try hide and hidden Signed-off-by: Andrew Thornton <art27@cantab.net> * More auto-update improvements Only run checker on pages that have a count Change starting checker to 10s with a back-off to 60s if there is no change Signed-off-by: Andrew Thornton <art27@cantab.net> * string comparison! Signed-off-by: Andrew Thornton <art27@cantab.net> * as per @silverwind Signed-off-by: Andrew Thornton <art27@cantab.net> * add configurability as per @6543 Signed-off-by: Andrew Thornton <art27@cantab.net> * Add documentation as per @6543 Signed-off-by: Andrew Thornton <art27@cantab.net> * Use CSRF header not query Signed-off-by: Andrew Thornton <art27@cantab.net> * Further JS improvements Fix @etzelia update notification table request Fix @silverwind comments Co-Authored-By: silverwind <me@silverwind.io> Signed-off-by: Andrew Thornton <art27@cantab.net> * Simplify the notification count fns Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: silverwind <me@silverwind.io>
2020-04-24 05:57:38 +02:00
}
},
"MermaidMaxSourceCharacters": func() int {
return setting.MermaidMaxSourceCharacters
},
Multiple GitGraph improvements: Exclude PR heads, Add branch/PR links, Show only certain branches, (#12766) * Multiple GitGraph improvements. Add backend support for excluding PRs, selecting branches and files. Fix #10327 Signed-off-by: Andrew Thornton <art27@cantab.net> * as per @silverwind Signed-off-by: Andrew Thornton <art27@cantab.net> * as per @silverwind Signed-off-by: Andrew Thornton <art27@cantab.net> * Only show refs in dropdown we display on the graph Signed-off-by: Andrew Thornton <art27@cantab.net> * as per @silverwind Signed-off-by: Andrew Thornton <art27@cantab.net> * use flexbox for ui header Signed-off-by: Andrew Thornton <art27@cantab.net> * Move Hide Pull Request button to the dropdown Signed-off-by: Andrew Thornton <art27@cantab.net> * Add SHA and user pictures Signed-off-by: Andrew Thornton <art27@cantab.net> * fix test Signed-off-by: Andrew Thornton <art27@cantab.net> * fix test 2 Signed-off-by: Andrew Thornton <art27@cantab.net> * fixes * async * more tweaks * use tabs in tmpl Signed-off-by: Andrew Thornton <art27@cantab.net> * remove commented thing Signed-off-by: Andrew Thornton <art27@cantab.net> * fix linting Signed-off-by: Andrew Thornton <art27@cantab.net> * Update web_src/js/features/gitgraph.js Co-authored-by: silverwind <me@silverwind.io> * graph tweaks * more tweaks * add title Signed-off-by: Andrew Thornton <art27@cantab.net> * fix loading indicator z-index and position Co-authored-by: silverwind <me@silverwind.io> Co-authored-by: techknowlogick <techknowlogick@gitea.io> Co-authored-by: Lauris BH <lauris@nix.lv>
2020-11-08 18:21:54 +01:00
// -----------------------------------------------------------------
// render
"RenderCommitMessage": RenderCommitMessage,
"RenderCommitMessageLinkSubject": RenderCommitMessageLinkSubject,
Multiple GitGraph improvements: Exclude PR heads, Add branch/PR links, Show only certain branches, (#12766) * Multiple GitGraph improvements. Add backend support for excluding PRs, selecting branches and files. Fix #10327 Signed-off-by: Andrew Thornton <art27@cantab.net> * as per @silverwind Signed-off-by: Andrew Thornton <art27@cantab.net> * as per @silverwind Signed-off-by: Andrew Thornton <art27@cantab.net> * Only show refs in dropdown we display on the graph Signed-off-by: Andrew Thornton <art27@cantab.net> * as per @silverwind Signed-off-by: Andrew Thornton <art27@cantab.net> * use flexbox for ui header Signed-off-by: Andrew Thornton <art27@cantab.net> * Move Hide Pull Request button to the dropdown Signed-off-by: Andrew Thornton <art27@cantab.net> * Add SHA and user pictures Signed-off-by: Andrew Thornton <art27@cantab.net> * fix test Signed-off-by: Andrew Thornton <art27@cantab.net> * fix test 2 Signed-off-by: Andrew Thornton <art27@cantab.net> * fixes * async * more tweaks * use tabs in tmpl Signed-off-by: Andrew Thornton <art27@cantab.net> * remove commented thing Signed-off-by: Andrew Thornton <art27@cantab.net> * fix linting Signed-off-by: Andrew Thornton <art27@cantab.net> * Update web_src/js/features/gitgraph.js Co-authored-by: silverwind <me@silverwind.io> * graph tweaks * more tweaks * add title Signed-off-by: Andrew Thornton <art27@cantab.net> * fix loading indicator z-index and position Co-authored-by: silverwind <me@silverwind.io> Co-authored-by: techknowlogick <techknowlogick@gitea.io> Co-authored-by: Lauris BH <lauris@nix.lv>
2020-11-08 18:21:54 +01:00
"RenderCommitBody": RenderCommitBody,
"RenderCodeBlock": RenderCodeBlock,
"RenderIssueTitle": RenderIssueTitle,
"RenderEmoji": RenderEmoji,
"RenderEmojiPlain": emoji.ReplaceAliases,
"ReactionToEmoji": ReactionToEmoji,
"RenderNote": RenderNote,
"RenderMarkdownToHtml": RenderMarkdownToHtml,
"RenderLabel": RenderLabel,
"RenderLabels": RenderLabels,
// -----------------------------------------------------------------
// misc
"ShortSha": base.ShortSha,
"ActionContent2Commits": ActionContent2Commits,
"IsMultilineCommitMessage": IsMultilineCommitMessage,
"CommentMustAsDiff": gitdiff.CommentMustAsDiff,
"MirrorRemoteAddress": mirrorRemoteAddress,
"FilenameIsImage": FilenameIsImage,
"TabSizeClass": TabSizeClass,
}
Use templates for issue e-mail subject and body (#8329) * Add template capability for issue mail subject * Remove test string * Fix trim subject length * Add comment to template and run make fmt * Add information for the template * Rename defaultMailSubject() to fallbackMailSubject() * General rewrite of the mail template code * Fix .Doer name * Use text/template for subject instead of html * Fix subject Re: prefix * Fix mail tests * Fix static templates * [skip ci] Updated translations via Crowdin * Expose db.SetMaxOpenConns and allow non MySQL dbs to set conn pool params (#8528) * Expose db.SetMaxOpenConns and allow other dbs to set their connection params * Add note about port exhaustion Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com> * Prevent .code-view from overriding font on icon fonts (#8614) * Correct some outdated statements in the contributing guidelines (#8612) * More information for drone-cli in CONTRIBUTING.md * Increases the version of drone-cli to 1.2.0 * Adds a note for the Docker Toolbox on Windows Signed-off-by: LukBukkit <luk.bukkit@gmail.com> * Fix the url for the blog repository (now on gitea.com) Signed-off-by: LukBukkit <luk.bukkit@gmail.com> * Remove TrN due to lack of lang context * Redo templates to match previous code * Fix extra character in template * Unify PR & Issue tempaltes, fix format * Remove default subject * Add template tests * Fix template * Remove replaced function * Provide User as models.User for better consistency * Add docs * Fix doc inaccuracies, improve examples * Change mail footer to math AppName * Add test for mail subject/body template separation * Add support for code review comments * Update docs/content/doc/advanced/mail-templates-us.md Co-Authored-By: 6543 <24977596+6543@users.noreply.github.com>
2019-11-07 14:34:28 +01:00
}
2016-11-25 07:23:48 +01:00
// Safe render raw as HTML
2015-08-08 11:10:34 +02:00
func Safe(raw string) template.HTML {
return template.HTML(raw)
}
2016-11-25 07:23:48 +01:00
// Str2html render Markdown text to HTML
2014-04-10 20:20:58 +02:00
func Str2html(raw string) template.HTML {
return template.HTML(markup.Sanitize(raw))
2014-04-10 20:20:58 +02:00
}
// DotEscape wraps a dots in names with ZWJ [U+200D] in order to prevent autolinkers from detecting these as urls
func DotEscape(raw string) string {
return strings.ReplaceAll(raw, ".", "\u200d.\u200d")
}
Use a general Eval function for expressions in templates. (#23927) One of the proposals in #23328 This PR introduces a simple expression calculator (templates/eval/eval.go), it can do basic expression calculations. Many untested template helper functions like `Mul` `Add` can be replaced by this new approach. Then these `Add` / `Mul` / `percentage` / `Subtract` / `DiffStatsWidth` could all use this `Eval`. And it provides enhancements for Golang templates, and improves readability. Some examples: ---- * Before: `{{Add (Mul $glyph.Row 12) 12}}` * After: `{{Eval $glyph.Row "*" 12 "+" 12}}` ---- * Before: `{{if lt (Add $i 1) (len $.Topics)}}` * After: `{{if Eval $i "+" 1 "<" (len $.Topics)}}` ## FAQ ### Why not use an existing expression package? We need a highly customized expression engine: * do the calculation on the fly, without pre-compiling * deal with int/int64/float64 types, to make the result could be used in Golang template. * make the syntax could be used in the Golang template directly * do not introduce too much complex or strange syntax, we just need a simple calculator. * it needs to strictly follow Golang template's behavior, for example, Golang template treats all non-zero values as truth, but many 3rd packages don't do so. ### What's the benefit? * Developers don't need to add more `Add`/`Mul`/`Sub`-like functions, they were getting more and more. Now, only one `Eval` is enough for all cases. * The new code reads better than old `{{Add (Mul $glyph.Row 12) 12}}`, the old one isn't familiar to most procedural programming developers (eg, the Golang expression syntax). * The `Eval` is fully covered by tests, many old `Add`/`Mul`-like functions were never tested. ### The performance? It doesn't use `reflect`, it doesn't need to parse or compile when used in Golang template, the performance is as fast as native Go template. ### Is it too complex? Could it be unstable? The expression calculator program is a common homework for computer science students, and it's widely used as a teaching and practicing purpose for developers. The algorithm is pretty well-known. The behavior can be clearly defined, it is stable.
2023-04-07 15:25:49 +02:00
// Eval the expression and return the result, see the comment of eval.Expr for details.
// To use this helper function in templates, pass each token as a separate parameter.
//
// {{ $int64 := Eval $var "+" 1 }}
// {{ $float64 := Eval $var "+" 1.0 }}
//
// Golang's template supports comparable int types, so the int64 result can be used in later statements like {{if lt $int64 10}}
func Eval(tokens ...any) (any, error) {
n, err := eval.Expr(tokens...)
return n.Value, err
}