Commit Graph

2852 Commits

Author SHA1 Message Date
6543 f8503b5fbf
Fix bug of migrated repository not index (#16991) (#16995)
Fix #16986, #16152

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-09-08 22:32:54 +01:00
6543 a66ff8a210
switch to maintained lib (#16535)
Co-authored-by: Matti R <matti@mdranta.net>
2021-07-24 11:13:56 -04:00
zeripath 3299f044d3
Make cancel from CatFileBatch and CatFileBatchCheck wait for the command to end (#16479) (#16481)
* Make cancel from CatFileBatch and CatFileBatchCheck wait for the command to end

Fix #16427 (again!)

* handle sharing violation error code

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-07-21 18:34:53 +02:00
zeripath e6c222511d
Retry rename on lock induced failures (re-fix) (#16461) (#16463)
Backport #16461

Unfortunately #16435 asserts the wrong error and should use
os.LinkError not os.PathError.

Fix #16439

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-07-16 17:48:39 +01:00
zeripath ca55e49cc0
Retry rename on lock induced failures (#16435) (#16439)
Backport #16435

Due to external locking on Windows it is possible for an
os.Rename to fail if the files or directories are being
used elsewhere.

This PR simply suggests retrying the rename again similar
to how we handle the os.Remove problems.

Fix #16427

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-07-15 22:57:51 +03:00
6543 d98694e6ca
Update bluemonday to v1.0.15 (#16379) (#16380)
* Update bluemonday to v1.0.15 (#16379)

* Fix TESTS
2021-07-09 02:47:27 +02:00
6543 ac0f452b30
Redirect on bad CSRF instead of presenting bad page (#14937) (#16378)
The current CSRF handler is a bit harsh with bad CSRF tokens on webpages
I think we can be a little kinder and redirect to base page with a flash error

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: zeripath <art27@cantab.net>
2021-07-08 20:12:20 +02:00
zeripath c65e49d72f
Fix relative links in postprocessed images (#16334) (#16340)
* Fix relative links in postprocessed images (#16334)

If a pre-post-processed file contains relative img tags these need to be updated
and joined correctly with the prefix. Finally, the node attributes need to be updated.

Fix #16308

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
2021-07-04 16:28:29 +02:00
6543 c7db7438b7
Fix API to return author for author on commits(#16276) (#16277) 2021-06-27 21:37:42 -04:00
zeripath b935472cdf
Improve efficiency in FindRenderizableReferenceNumeric and getReference (#16251) (#16255)
* Improve efficiency in FindRenderizableReferenceNumeric and getReferences (#16251)

* The Fuzzer is running on a non-repo urlprefix which is incorrect for RenderRaw
* Make FindRenderizableReferenceNumeric and getReferences more efficient

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>

* as per comment on original pr

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-06-26 08:31:03 +01:00
6543 8ac48584ec
Use html.Parse rather than html.ParseFragment (#16223) (#16225)
* Use html.Parse rather than html.ParseFragment
  There have been a few issues with html.ParseFragment - just use html.Parse instead.

* Skip document node

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: zeripath <art27@cantab.net>
2021-06-22 09:46:39 +08:00
6543 f832e8eeea
Fix some API bugs (#16184) (#16190)
* Fix some API bugs (#16184)

* Repository object only count releases as releases (fix #16144)

* EditOrg respect RepoAdminChangeTeamAccess option (fix #16013)

* adjut to v1.14
2021-06-18 19:47:34 +08:00
zeripath 544ef7d394
Encrypt migration credentials at rest (#15895) (#16187)
Backport #15895

Storing these credentials is a liability.

* Encrypt credentials with SECRET_KEY before persisting to task queue table (they need to be persisted due to the nature of the task queue)
  - security in depth: helps when attacker has access to DB only, but not app.ini
* Delete all credentials (even encrypted) from the task table, once the migration is done, for safety
  - security in depth: minimizes leaked data if attacker gains access to snapshot of both DB and app.ini
2021-06-17 22:59:28 +02:00
zeripath 5ff807acde
Run processors on whole of text (#16155) (#16185)
Backport #16155

There is an inefficiency in the design of our processors which means that Emoji
and other processors run in order n^2 time.

This PR forces the processors to process the entirety of text node before passing
back up. The fundamental inefficiency remains but it should be significantly
ameliorated.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-06-17 20:01:33 +02:00
zeripath 849d316d8d
issue-keyword class is being incorrectly stripped off spans (#16163) (#16172)
Backport #16163

Bluemonday sanitizer regexp rules are not additive, so the addition of the icons,
emojis and chroma syntax policy has led to this being stripped.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-06-16 06:35:54 -04:00
zeripath f034804e5d
Set self-adjusting deadline for connection writing (#16068) (#16123)
In #16055 it appears that the simple 5s deadline doesn't work for large
file writes. Now we can't - or at least shouldn't just set no deadline
as go will happily let these connections block indefinitely. However,
what seems reasonable is to set some minimum rate we expect for writing.

This PR suggests the following algorithm:

* Every write has a minimum timeout of 5s (adjustable at compile time.)
* If there has been a previous write - then consider its previous
deadline, add half of the minimum timeout + 2s per kb about to written.
* If that new deadline is after the minimum timeout use that.

Fix #16055

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: 6543 <6543@obermui.de>
2021-06-11 00:26:32 +03:00
6543 ac84bb7183
Fix data URI scramble (#16098) (#16118)
* Fix data URI scramble (#16098)

* Removed unused method.

* No prefix for data uris.

* Added test to prevent regressions.

Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
2021-06-09 16:31:40 +02:00
6543 2265058c31
git migration: don't prompt interactively for clone credentials (#15902) (#16082)
* don't prompt interactively for clone credentials

* apply GIT_TERMINAL_PROMPT=0 to all git cmds

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>

Co-authored-by: Norwin <noerw@users.noreply.github.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-06-06 14:02:34 +08:00
Jimmy Praet 8007602b40
Don't manipulate input params in email notification (#16011) (#16033)
Backport #16011
2021-05-31 02:17:34 -04:00
techknowlogick 3a79f1190f
Fix setting of SameSite on cookies (#15989) (#15991)
Fix #15972

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>

Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-05-27 10:02:39 -04:00
zeripath 159bc8842a
Restore PAM user autocreation functionality (#15825) (#15867)
Backport #15825

* Restore PAM user autocreation functionality

PAM autoregistration of users currently fails due to email invalidity.
This PR adds a new setting to PAM to allow an email domain to be set
or just sets the email to the noreply address and if that fails falls
back to uuid@localhost

Fix #15702

Signed-off-by: Andrew Thornton <art27@cantab.net>

* As per KN4CKER

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: 6543 <6543@obermui.de>
2021-05-19 10:42:36 -04:00
6543 8c4bf4c3b4
GitHub: migrate draft releases too (#15884) (#15888)
* GitHub: migrate draft releases too

* refactor
2021-05-16 09:24:28 +02:00
6543 3bcf2e5c18
Close the gitrepo when deleting the repository (#15876) (#15887)
Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: zeripath <art27@cantab.net>
2021-05-16 00:45:17 +03:00
zeripath 41422f0df0
Add timeout to writing to responses (#15831) (#15872)
Backport #15831

In #15826 it has become apparent that there are a few occasions when a response can
hang during writing, and because there is no timeout go will happily just block
interminably. This PR adds a fixed 5 second timeout to all writes to a connection.

Fix #15826

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-05-14 17:42:27 +01:00
KN4CK3R f773733252
Fix LFS commit finder not working (#15856) (#15874)
* Create a copy of the sha bytes.

Co-authored-by: Andrew Thornton <art27@cantab.net>
2021-05-14 16:39:59 +01:00
zeripath cbaf8e8785
Stop calling WriteHeader in Write (#15862) (#15873)
Backport #15862

Fixes http: superfluous response.WriteHeader call from code.gitea.io/gitea/modules/context.(*Response).WriteHeader (response.go:67)

* Looking again we don't need this writeHeader as all of our downstream
implementations will always do it for us

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-05-14 15:38:35 +01:00
zeripath 387a1bc472
fix truncate utf8 string (#15828) (#15854)
Backport #15828

* fix truncate utf8 string.

* revoke truncated user info.

Co-authored-by: yan <sxty32@gmail.com>
2021-05-13 16:10:29 +02:00
zeripath a83cde2f3f
Tagger can be empty, as can Commit and Author - tolerate this (#15835) (#15839)
Backport #15835

Unfortunately some old repositories can have tags with empty Tagger, Commit
or Author. Go-Git variants will always have empty values for these whereas
the native git variant leaves them at nil. The simplest solution is just to
always have these set to empty Signatures.

v156 migration also makes the incorrect assumption that these cannot be empty.
Therefore add some handling to this and add logging and adjust broken
logging elsewhere in this migration.

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: 6543 <6543@obermui.de>
2021-05-12 20:09:16 +01:00
zeripath 332eb2f6d2
Queue manager FlushAll can loop rapidly - add delay (#15733) (#15840)
Backport #15733

* Queue manager FlushAll can loop rapidly - add delay

Add delay within FlushAll to prevent rapid loop when workers are busy

Signed-off-by: Andrew Thornton <art27@cantab.net>

* as per lunny

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-05-12 18:48:11 +01:00
John Olheiser d054c4e7f3
Add err to log (#15813) (#15824)
Signed-off-by: jolheiser <john.olheiser@gmail.com>
2021-05-10 16:38:37 -04:00
Lunny Xiao 5e562e9b30
Move restore repo to internal router and invoke from command to avoid open the same db file or queues files (#15790) (#15816)
* Move restore repo to internal router and invoke from command to avoid open the same db file or queues files

* Follow @zeripath's review

* set no timeout for resotre repo private request

* make restore repo cancelable
2021-05-10 21:14:59 +08:00
6543 c57e908f36
Tests should use test files (#15801) (#15806) 2021-05-10 01:39:14 +08:00
6543 d644709b22
Exponential Backoff for ByteFIFO (#15724) (#15793)
This PR is another in the vein of queue improvements. It suggests an
exponential backoff for bytefifo queues to reduce the load from queue
polling. This will mostly be useful for redis queues.

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Lauris BH <lauris@nix.lv>

Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lauris BH <lauris@nix.lv>
2021-05-08 14:27:00 -04:00
6543 22d700edfd
Set GIT_DIR correctly if it is not set (#15751) (#15769)
* Set GIT_DIR correctly if it is not set

* Expand out templates

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: zeripath <art27@cantab.net>
2021-05-07 20:01:25 +02:00
zeripath 6782a64a4a
Defer closing the gitrepo until the end of the wrapped context functions (#15653) (#15746)
* Defer closing the gitrepo until the end of the wrapped context functions (#15653)

Backport #15653

There was a mistake in #15372 where deferral of gitrepo close occurs before it should.

This PR fixes this.
2021-05-07 18:28:02 +02:00
zeripath 1ec11ac87e
Drop back to use IsAnInteractiveSession for SVC (#15749) (#15762)
Backport #15749

* Drop back to use IsAnInteractiveSession for SVC

There is an apparent permission change problem when using
IsWindowsService to determine if the SVC manager should be
used.

This PR simply drops back to using IsAnInteractiveSession as
this does not change behaviour.

Fix #15454

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Yes staticcheck I know this is deprecated

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Just leave me alone lint

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: 6543 <6543@obermui.de>
2021-05-07 17:44:35 +02:00
6543 2c2a30d6bb
Fix bug where repositories appear unadopted (#15757) (#15767)
Fix bug where repositories with capital letters in their names appear unadopted.

Fix #15755

Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-05-07 17:07:39 +02:00
a1012112796 a83fb3a83a
fix some ui bug about draft release (#15137) (#15745)
* fix some ui bug about draft release

- should not show draft release in tag list because
  it will't create real tag
- still show draft release without tag and commit message
  for draft release instead of 404 error
- remove tag load for attachement links because it's useless

Signed-off-by: a1012112796 <1012112796@qq.com>

* add test code

* fix test

That's because has added a new release in relaese test database.

* fix dropdown link for draft release
2021-05-06 21:23:26 +02:00
zeripath 462c6fdee2
Fix setting redis db path (#15698) (#15708)
Backport #15698

There is a bug setting the redis db in the common nosql manager whereby the db path
always fails.

This PR fixes this.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-05-03 22:30:30 +01:00
zeripath 4fa2804238
Performance improvement for last commit cache and show-ref (#15455) (#15701)
Backport #15455

* Improve performance when there are multiple commits in the last commit cache

* read refs directly if we can

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-05-03 16:51:41 +02:00
6543 a725d31496
Delete references if repository gets deleted (#15681) (#15684)
* Remove DeletedBranch and LFSLocks.

* Sort beans.

Co-authored-by: KN4CK3R <KN4CK3R@users.noreply.github.com>
Co-authored-by: zeripath <art27@cantab.net>
2021-05-01 00:09:58 +02:00
KN4CK3R 54263ff123
Delete protected branch if repository gets removed (#15658) (#15676)
* Added missing error parameters.

* Delete protected branch if repository gets removed.

* Added doctor fix.
2021-04-30 19:59:42 +01:00
6543 3bde297121
[API] pull notification subject status: add "merged" (#15344) (#15654)
Current subject status can be "", "open" and "closed". This add "merged" to it.
2021-04-28 20:24:56 +01:00
Lunny Xiao cae46216e4
fix webhook timeout bug (#15613) (#15621)
* Also fix the potential problem in httplib
2021-04-26 14:42:12 +02:00
zeripath 6e2dacfef6
If the default branch is not present do not report error on stats indexing (follow-up of #15546) (#15583) (#15594)
Backport #15546
Backport #15583

 #15546 doesn't completely fix this problem because the error returned is an ObjectNotExist
error not a BranchNotExist error.

Add test for ErrObjectNotExist too

Fix #15257

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-04-22 22:30:18 +02:00
Lunny Xiao c0869c295a
Fix lfs management find (#15537) (#15578)
* Fix lfs management find (#15537)

Fix #15236

* Do not do 40byte conversion within ParseTreeLine
* Missed a to40ByteSHA

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Andrew Thornton <art27@cantab.net>

* Remove space

Co-authored-by: Andrew Thornton <art27@cantab.net>
2021-04-22 20:32:48 +02:00
zeripath 248b67af6f
Fix NPE on view commit with notes (#15561) (#15573)
Backport #15561

Fix #15558

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-04-21 15:11:43 -04:00
KN4CK3R 5da024a019
Add ETag header (#15370) (#15552)
* Add ETag header.

* Comply with RFC 7232.

* Moved logic into httpcache.go

* Changed name.

* Lint

* Implemented If-None-Match list.

* Fixed missing header on *

* Removed weak etag support.

* Removed * support.

* Added unit test.

* Lint

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-04-20 12:01:58 -04:00
Lunny Xiao eff2499be7
Fix bug on commit graph (#15517) (#15530) 2021-04-17 14:46:30 +02:00
6543 729fa06468
migration: github: if rate limit is not enabled, ignore it (#15490) (#15495)
Co-authored-by: Lauris BH <lauris@nix.lv>
2021-04-15 18:24:01 +03:00