gitea/integrations
Lunny Xiao 62e6c9bc6c
Add a storage layer for attachments (#11387)
* Add a storage layer for attachments

* Fix some bug

* fix test

* Fix copyright head and lint

* Fix bug

* Add setting for minio and flags for migrate-storage

* Add documents

* fix lint

* Add test for minio store type on attachments

* fix test

* fix test

* Apply suggestions from code review

Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>

* Add warning when storage migrated successfully

* Fix drone

* fix test

* rebase

* Fix test

* display the error on console

* Move minio test to amd64 since minio docker don't support arm64

* refactor the codes

* add trace

* Fix test

* remove log on xorm

* Fi download bug

* Add a storage layer for attachments

* Add setting for minio and flags for migrate-storage

* fix lint

* Add test for minio store type on attachments

* Apply suggestions from code review

Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>

* Fix drone

* fix test

* Fix test

* display the error on console

* Move minio test to amd64 since minio docker don't support arm64

* refactor the codes

* add trace

* Fix test

* Add URL function to serve attachments directly from S3/Minio

* Add ability to enable/disable redirection in attachment configuration

* Fix typo

* Add a storage layer for attachments

* Add setting for minio and flags for migrate-storage

* fix lint

* Add test for minio store type on attachments

* Apply suggestions from code review

Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>

* Fix drone

* fix test

* Fix test

* display the error on console

* Move minio test to amd64 since minio docker don't support arm64

* don't change unrelated files

* Fix lint

* Fix build

* update go.mod and go.sum

* Use github.com/minio/minio-go/v6

* Remove unused function

* Upgrade minio to v7 and some other improvements

* fix lint

* Fix go mod

Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
Co-authored-by: Tyler <tystuyfzand@gmail.com>
2020-08-18 12:23:45 +08:00
..
gitea-repositories-meta Kanban board (#8346) 2020-08-16 23:07:38 -04:00
migration-test Re-attempt to delete temporary upload if the file is locked by another process (#12447) 2020-08-11 21:05:34 +01:00
README.md log slow tests (#11487) 2020-06-01 21:39:44 -04:00
README_ZH.md Improve integration tests (#8276) 2019-09-25 14:13:18 +02:00
api_admin_org_test.go Fix "data race" in testlogger (#9159) 2019-11-26 01:21:37 +02:00
api_admin_test.go Fix "data race" in testlogger (#9159) 2019-11-26 01:21:37 +02:00
api_branch_test.go Add API Endpoint for Branch Creation (#11607) 2020-05-29 19:16:20 +01:00
api_comment_test.go [API] Add "before" query to ListIssueComments and ListRepoIssue… (#9685) 2020-01-13 17:02:24 +01:00
api_fork_test.go Fix "data race" in testlogger (#9159) 2019-11-26 01:21:37 +02:00
api_gpg_keys_test.go Handle expected errors in AddGPGkey API (#11644) 2020-05-28 17:25:54 -04:00
api_helper_for_declarative_test.go Correctly set the organization num repos (#11339) 2020-05-12 01:04:08 +03:00
api_issue_label_test.go Add Organization Wide Labels (#10814) 2020-04-01 01:14:46 -03:00
api_issue_milestone_test.go Add name filter to API for GetMilestoneList (#12336) 2020-07-28 12:30:40 +01:00
api_issue_reaction_test.go API: Move AllowedReactions endpoint into GetGenneralUI endpoint + creat new swagger section settings (#11854) 2020-06-22 14:21:31 -04:00
api_issue_stopwatch_test.go [API] extend StopWatch (#9196) 2019-12-11 23:23:05 -05:00
api_issue_subscription_test.go Return issue subscription status from API subscribe (#10966) 2020-04-30 23:55:24 +01:00
api_issue_test.go [API] ListIssues add filter for milestones (#10148) 2020-04-30 01:15:39 -03:00
api_issue_tracked_time_test.go Fix tracked time issues (#11349) 2020-05-09 11:18:44 -03:00
api_keys_test.go Fix "data race" in testlogger (#9159) 2019-11-26 01:21:37 +02:00
api_notification_test.go Extend Notifications API and return pinned notifications by default (#12164) 2020-07-12 00:46:01 +03:00
api_oauth2_apps_test.go Add Get/Update for api/v1/user/applications/oauth2 (#11008) 2020-04-09 19:37:31 -05:00
api_org_test.go [API] add GET /orgs endpoint (#9560) 2020-01-12 23:43:44 +08:00
api_pull_review_test.go Fix commenting on non-utf8 encoded files (#11916) 2020-06-18 10:07:09 -04:00
api_pull_test.go Add option to API to update PullRequest base branch (#11666) 2020-06-07 20:13:40 +01:00
api_releases_test.go Fix "data race" in testlogger (#9159) 2019-11-26 01:21:37 +02:00
api_repo_edit_test.go Api: advanced settings for repository (external wiki, issue tracker etc.) (#7756) 2019-10-02 17:30:41 +08:00
api_repo_file_create_test.go Handle expected errors in FileCreate & FileUpdate API (#11643) 2020-05-31 21:59:34 +01:00
api_repo_file_delete_test.go Contents API should return 404 on not exist (#10323) 2020-04-15 02:18:51 -03:00
api_repo_file_helpers.go Move sdk structs to modules/structs (#6905) 2019-05-11 18:21:34 +08:00
api_repo_file_update_test.go Handle expected errors in FileCreate & FileUpdate API (#11643) 2020-05-31 21:59:34 +01:00
api_repo_get_contents_list_test.go Contents API should return 404 on not exist (#10323) 2020-04-15 02:18:51 -03:00
api_repo_get_contents_test.go Contents API should return 404 on not exist (#10323) 2020-04-15 02:18:51 -03:00
api_repo_git_blobs_test.go Fix "data race" in testlogger (#9159) 2019-11-26 01:21:37 +02:00
api_repo_git_commits_test.go Consolidate API for getting single commit (#11368) 2020-05-11 12:44:36 +03:00
api_repo_git_hook_test.go Fix "data race" in testlogger (#9159) 2019-11-26 01:21:37 +02:00
api_repo_git_ref_test.go Fix "data race" in testlogger (#9159) 2019-11-26 01:21:37 +02:00
api_repo_git_tags_test.go Fix "data race" in testlogger (#9159) 2019-11-26 01:21:37 +02:00
api_repo_git_trees_test.go Fix "data race" in testlogger (#9159) 2019-11-26 01:21:37 +02:00
api_repo_languages_test.go give gitea time to calculate language stats (#11812) 2020-06-08 20:15:45 -04:00
api_repo_lfs_locks_test.go Fix "data race" in testlogger (#9159) 2019-11-26 01:21:37 +02:00
api_repo_raw_test.go Fix "data race" in testlogger (#9159) 2019-11-26 01:21:37 +02:00
api_repo_tags_test.go Fix "data race" in testlogger (#9159) 2019-11-26 01:21:37 +02:00
api_repo_test.go Re-attempt to delete temporary upload if the file is locked by another process (#12447) 2020-08-11 21:05:34 +01:00
api_repo_topic_test.go Fix "data race" in testlogger (#9159) 2019-11-26 01:21:37 +02:00
api_team_test.go [API] orgEditTeam make Fields optional (#9556) 2020-01-09 13:15:14 +00:00
api_team_user_test.go Fix "data race" in testlogger (#9159) 2019-11-26 01:21:37 +02:00
api_token_test.go Fix "data race" in testlogger (#9159) 2019-11-26 01:21:37 +02:00
api_user_heatmap_test.go Fix "data race" in testlogger (#9159) 2019-11-26 01:21:37 +02:00
api_user_orgs_test.go Fix "data race" in testlogger (#9159) 2019-11-26 01:21:37 +02:00
api_user_search_test.go Fix "data race" in testlogger (#9159) 2019-11-26 01:21:37 +02:00
attachment_test.go Add a storage layer for attachments (#11387) 2020-08-18 12:23:45 +08:00
auth_ldap_test.go Refactor Cron and merge dashboard tasks (#10745) 2020-05-16 19:31:38 -04:00
benchmarks_test.go Missed defer prepareTestEnv (#9285) 2019-12-08 01:14:50 +00:00
branches_test.go Handle more pathological branch and tag names (#11843) 2020-06-11 19:49:47 -04:00
change_default_branch_test.go Fix "data race" in testlogger (#9159) 2019-11-26 01:21:37 +02:00
cmd_keys_test.go In authorized_keys use double-quote for windows compatibility (#10841) 2020-03-27 09:49:39 +02:00
cors_test.go Fix "data race" in testlogger (#9159) 2019-11-26 01:21:37 +02:00
create_no_session_test.go Re-attempt to delete temporary upload if the file is locked by another process (#12447) 2020-08-11 21:05:34 +01:00
delete_user_test.go Fix "data race" in testlogger (#9159) 2019-11-26 01:21:37 +02:00
download_test.go Fix "data race" in testlogger (#9159) 2019-11-26 01:21:37 +02:00
editor_test.go Add golangci (#6418) 2019-06-12 15:41:28 -04:00
empty_repo_test.go Fix "data race" in testlogger (#9159) 2019-11-26 01:21:37 +02:00
eventsource_test.go Extend Notifications API and return pinned notifications by default (#12164) 2020-07-12 00:46:01 +03:00
explore_repos_test.go Fix "data race" in testlogger (#9159) 2019-11-26 01:21:37 +02:00
git_helper_for_declarative_test.go Re-attempt to delete temporary upload if the file is locked by another process (#12447) 2020-08-11 21:05:34 +01:00
git_test.go Re-attempt to delete temporary upload if the file is locked by another process (#12447) 2020-08-11 21:05:34 +01:00
gpg_git_test.go Re-attempt to delete temporary upload if the file is locked by another process (#12447) 2020-08-11 21:05:34 +01:00
html_helper.go integration tests: Use t.Helper() (#7654) 2019-07-29 12:15:18 +08:00
integration_test.go Re-attempt to delete temporary upload if the file is locked by another process (#12447) 2020-08-11 21:05:34 +01:00
issue_test.go Prettify Timeline (#10972) 2020-04-11 01:01:41 +03:00
lfs_getobject_test.go Support Range header end in lfs (#11314) 2020-05-11 11:37:59 +03:00
links_test.go Kanban board (#8346) 2020-08-16 23:07:38 -04:00
mssql.ini.tmpl Don't put integration test data outside of test folder (#11746) 2020-06-03 15:07:02 -04:00
mysql.ini.tmpl Add a storage layer for attachments (#11387) 2020-08-18 12:23:45 +08:00
mysql8.ini.tmpl Use /tmp for test repositories (#11126) 2020-04-19 14:17:02 +03:00
nonascii_branches_test.go Fix "data race" in testlogger (#9159) 2019-11-26 01:21:37 +02:00
oauth_test.go Fix "data race" in testlogger (#9159) 2019-11-26 01:21:37 +02:00
org_count_test.go Correctly set the organization num repos (#11339) 2020-05-12 01:04:08 +03:00
org_test.go Fix "data race" in testlogger (#9159) 2019-11-26 01:21:37 +02:00
pgsql.ini.tmpl Don't put integration test data outside of test folder (#11746) 2020-06-03 15:07:02 -04:00
private-testing.key Fix verification of subkeys of default gpg key (#11713) 2020-06-03 11:36:41 -04:00
privateactivity_test.go Add hide activity option (#11353) 2020-06-05 16:01:53 -04:00
pull_compare_test.go Fix "data race" in testlogger (#9159) 2019-11-26 01:21:37 +02:00
pull_create_test.go Prettify Timeline (#10972) 2020-04-11 01:01:41 +03:00
pull_merge_test.go Fix styling for PR merge section when no checks (#11609) 2020-05-25 20:08:06 +01:00
pull_review_test.go Fix "data race" in testlogger (#9159) 2019-11-26 01:21:37 +02:00
pull_status_test.go Fix wrong hint when status checking is running on pull request view (#9886) 2020-01-22 11:46:04 +08:00
pull_update_test.go Add API to update pr headBranch (#12419) 2020-08-04 21:55:22 +01:00
release_test.go Honor DEFAULT_PAGING_NUM for API (#11805) 2020-06-08 22:46:51 -04:00
repo_activity_test.go Fix activity count in TestRepoActivity (#9959) 2020-01-24 13:06:40 +00:00
repo_branch_test.go Fix "data race" in testlogger (#9159) 2019-11-26 01:21:37 +02:00
repo_commits_search_test.go Fix "data race" in testlogger (#9159) 2019-11-26 01:21:37 +02:00
repo_commits_test.go Fix "data race" in testlogger (#9159) 2019-11-26 01:21:37 +02:00
repo_fork_test.go Fix "data race" in testlogger (#9159) 2019-11-26 01:21:37 +02:00
repo_generate_test.go Fix "data race" in testlogger (#9159) 2019-11-26 01:21:37 +02:00
repo_migrate_test.go Use a small size repo for migrate test (#12300) 2020-07-24 12:46:38 +08:00
repo_search_test.go Move code indexer related code to a new package (#9191) 2019-12-08 14:15:35 -05:00
repo_test.go Update Octicons to v10 (#12240) 2020-07-17 18:15:12 +03:00
repo_watch_test.go Auto-subscribe user to repository when they commit/tag to it (#7657) 2019-11-10 09:22:19 +00:00
repofiles_delete_test.go prefer NoError/Error over Nil/NotNil (#12271) 2020-07-19 12:53:40 +03:00
repofiles_update_test.go prefer NoError/Error over Nil/NotNil (#12271) 2020-07-19 12:53:40 +03:00
setting_test.go Add a /user/login landing page option (#9622) 2020-01-06 17:50:44 +01:00
signin_test.go Fix "data race" in testlogger (#9159) 2019-11-26 01:21:37 +02:00
signout_test.go Logout POST action (#10582) 2020-03-02 22:50:31 -06:00
signup_test.go Fix "data race" in testlogger (#9159) 2019-11-26 01:21:37 +02:00
sqlite.ini.tmpl Don't put integration test data outside of test folder (#11746) 2020-06-03 15:07:02 -04:00
ssh_key_test.go Re-attempt to delete temporary upload if the file is locked by another process (#12447) 2020-08-11 21:05:34 +01:00
testlogger.go Pause, Resume, Release&Reopen, Add and Remove Logging from command line (#11777) 2020-07-05 20:07:07 -04:00
timetracking_test.go Fix "data race" in testlogger (#9159) 2019-11-26 01:21:37 +02:00
user_test.go Fix "data race" in testlogger (#9159) 2019-11-26 01:21:37 +02:00
version_test.go Fix "data race" in testlogger (#9159) 2019-11-26 01:21:37 +02:00
xss_test.go Fix "data race" in testlogger (#9159) 2019-11-26 01:21:37 +02:00

README.md

Integrations tests

Integration tests can be run with make commands for the appropriate backends, namely:

make test-mysql
make test-pgsql
make test-sqlite

Make sure to perform a clean build before running tests:

make clean build

Run all tests via local drone

drone exec --local --build-event "pull_request"

Run sqlite integrations tests

Start tests

make test-sqlite

Run mysql integrations tests

Setup a mysql database inside docker

docker run -e "MYSQL_DATABASE=test" -e "MYSQL_ALLOW_EMPTY_PASSWORD=yes" -p 3306:3306 --rm --name mysql mysql:latest #(just ctrl-c to stop db and clean the container)
docker run -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" --rm --name elasticsearch elasticsearch:7.6.0 #(in a secound terminal, just ctrl-c to stop db and clean the container)

Start tests based on the database container

TEST_MYSQL_HOST=localhost:3306 TEST_MYSQL_DBNAME=test TEST_MYSQL_USERNAME=root TEST_MYSQL_PASSWORD='' make test-mysql

Run pgsql integrations tests

Setup a pgsql database inside docker

docker run -e "POSTGRES_DB=test" -p 5432:5432 --rm --name pgsql postgres:latest #(just ctrl-c to stop db and clean the container)

Start tests based on the database container

TEST_PGSQL_HOST=localhost:5432 TEST_PGSQL_DBNAME=test TEST_PGSQL_USERNAME=postgres TEST_PGSQL_PASSWORD=postgres make test-pgsql

Run mssql integrations tests

Setup a mssql database inside docker

docker run -e "ACCEPT_EULA=Y" -e "MSSQL_PID=Standard" -e "SA_PASSWORD=MwantsaSecurePassword1" -p 1433:1433 --rm --name mssql microsoft/mssql-server-linux:latest #(just ctrl-c to stop db and clean the container)

Start tests based on the database container

TEST_MSSQL_HOST=localhost:1433 TEST_MSSQL_DBNAME=gitea_test TEST_MSSQL_USERNAME=sa TEST_MSSQL_PASSWORD=MwantsaSecurePassword1 make test-mssql

Running individual tests

Example command to run GPG test:

For sqlite:

make test-sqlite#GPG

For other databases(replace MSSQL to MYSQL, MYSQL8, PGSQL):

TEST_MSSQL_HOST=localhost:1433 TEST_MSSQL_DBNAME=test TEST_MSSQL_USERNAME=sa TEST_MSSQL_PASSWORD=MwantsaSecurePassword1 make test-mssql#GPG

Setting timeouts for declaring long-tests and long-flushes

We appreciate that some testing machines may not be very powerful and the default timeouts for declaring a slow test or a slow clean-up flush may not be appropriate.

You can either:

  • Within the test ini file set the following section:
[integration-tests]
SLOW_TEST = 10s ; 10s is the default value
SLOW_FLUSH = 5S ; 5s is the default value
  • Set the following environment variables:
GITEA_SLOW_TEST_TIME="10s" GITEA_SLOW_FLUSH_TIME="5s" make test-sqlite