本文完整阅读约需 18 分钟,如时间较长请考虑收藏后慢慢阅读~

最近因为服务器迁移,我将原先的GitLab仓库备份后还原到了新的服务器上,却发现在打开仓库的『集成』页面时出现500错误。

0x01

该错误覆盖了所有仓库,并且无法通过简单的重启&reconfigure方法进行恢复,错误如图所示:

补充,后面发现除了该页面会报错500,在其他的杂项页面也会随机出现500的错误,如下图:

0x02

我尝试通过查看错误日志来获取相关信息:

$ tail -f /var/log/gitlab/gitlab-rails/production.log

Completed 500 Internal Server Error in 1463ms (ActiveRecord: 13.4ms)
Sending event a3554660aac844bdb01ce8b5ef6da000 to Sentry
Unable to record event with remote Sentry server (Raven::Error - Net::OpenTimeout):
/opt/gitlab/embedded/lib/ruby/gems/2.5.0/gems/sentry-raven-2.7.4/lib/raven/transports/http.rb:34:in `rescue in send_event'
/opt/gitlab/embedded/lib/ruby/gems/2.5.0/gems/sentry-raven-2.7.4/lib/raven/transports/http.rb:16:in `send_event'
/opt/gitlab/embedded/lib/ruby/gems/2.5.0/gems/sentry-raven-2.7.4/lib/raven/client.rb:37:in `send_event'
/opt/gitlab/embedded/lib/ruby/gems/2.5.0/gems/sentry-raven-2.7.4/lib/raven/instance.rb:81:in `send_event'
/opt/gitlab/embedded/lib/ruby/gems/2.5.0/gems/sentry-raven-2.7.4/lib/raven/instance.rb:126:in `capture_type'
/opt/gitlab/embedded/lib/ruby/gems/2.5.0/gems/sentry-raven-2.7.4/lib/raven/integrations/rack.rb:28:in `capture_type'
/opt/gitlab/embedded/lib/ruby/gems/2.5.0/gems/sentry-raven-2.7.4/lib/raven/integrations/rails/overrides/debug_exceptions_catcher.rb:8:in `render_exception'
/opt/gitlab/embedded/lib/ruby/gems/2.5.0/gems/actionpack-5.0.7.1/lib/action_dispatch/middleware/debug_exceptions.rb:59:in `rescue in call'
/opt/gitlab/embedded/lib/ruby/gems/2.5.0/gems/actionpack-5.0.7.1/lib/action_dispatch/middleware/debug_exceptions.rb:47:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.5.0/gems/actionpack-5.0.7.1/lib/action_dispatch/middleware/show_exceptions.rb:31:in `call'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/middleware/basic_health_check.rb:25:in `call'
Failed to submit event: ActionView::Template::Error: 

ActionView::Template::Error ():
    1: %li
    2:   .row
    3:     .col-md-8.col-lg-7
    4:       %strong.light-header= hook.url
    5:       %div
    6:         - ProjectHook.triggers.each_value do |event|
    7:           - if hook.public_send(event)

app/views/projects/settings/integrations/_project_hook.html.haml:4:in `_app_views_projects_settings_integrations__project_hook_html_haml__2108412151178301037_70069868225340'
app/views/projects/hooks/_index.html.haml:20:in `block in _app_views_projects_hooks__index_html_haml__1067223084402946681_70069865180740'
app/views/projects/hooks/_index.html.haml:19:in `_app_views_projects_hooks__index_html_haml__1067223084402946681_70069865180740'
app/views/projects/settings/integrations/show.html.haml:4:in `_app_views_projects_settings_integrations_show_html_haml___1944333012056993997_70069864179020'
app/controllers/application_controller.rb:113:in `render'
lib/gitlab/i18n.rb:55:in `with_locale'
lib/gitlab/i18n.rb:61:in `with_user_locale'
app/controllers/application_controller.rb:423:in `set_locale'
lib/gitlab/middleware/rails_queue_duration.rb:24:in `call'
lib/gitlab/metrics/rack_middleware.rb:17:in `block in call'
lib/gitlab/metrics/transaction.rb:55:in `run'
lib/gitlab/metrics/rack_middleware.rb:17:in `call'
lib/gitlab/middleware/multipart.rb:103:in `call'
lib/gitlab/request_profiler/middleware.rb:16:in `call'
lib/gitlab/middleware/go.rb:20:in `call'
lib/gitlab/etag_caching/middleware.rb:13:in `call'
lib/gitlab/middleware/correlation_id.rb:16:in `block in call'
lib/gitlab/correlation_id.rb:15:in `use_id'
lib/gitlab/middleware/correlation_id.rb:15:in `call'
lib/gitlab/middleware/read_only/controller.rb:40:in `call'
lib/gitlab/middleware/read_only.rb:18:in `call'
lib/gitlab/middleware/basic_health_check.rb:25:in `call'
lib/gitlab/request_context.rb:20:in `call'
lib/gitlab/metrics/requests_rack_middleware.rb:29:in `call'
lib/gitlab/middleware/release_env.rb:13:in `call'

从错误日志中,只能得到关于错误本身的日志,没有太多辅助信息,除了ActionView::Template::Error ():。于是我尝试在GitLab的GitLab仓库(有一点点点点绕)中搜索关于该报错的Issue,在偌大的Issue列表(不得不说GitLab的BUG太多)找到了这一条:#53465,其中给了如下的解决方法:

Stan Hu @stanhu · 4 months ago Owner
In your backup, did you restore /etc/gitlab/gitlab-secrets.json as mentioned in https://docs.gitlab.com/ee/raketasks/backup_restore.html?

0x03

按照Issue中的方法,我将/etc/gitlab/gitlab-secrets.json恢复到了新的服务器上,执行gitlab-ctl reconfigure && gitlab-ctl restart后再打开之前会出现500错误的页面,不再出现500,功能可以正常使用。