"Plugin failed to ServeHTTP" Errors with v3.1.1 #158

moschlar · 2021-06-22T08:56:53Z

Hi there,

I just installed v3.1.1 and now I'm seeing lots of

2021-06-22_08:55:47.03798 {"level":"error","ts":1624352147.0378995,"caller":"mlog/log.go:232","msg":"Plugin failed to ServeHTTP, RPC call failed","plugin_id":"bigbluebutton","error":"connection is shut down"}
2021-06-22_08:55:52.03819 {"level":"error","ts":1624352152.0381186,"caller":"mlog/log.go:232","msg":"Plugin failed to ServeHTTP, muxBroker couldn't Accept request body connection","plugin_id":"bigbluebutton","error":"timeout waiting for accept"}
2021-06-22_08:55:52.03826 {"level":"error","ts":1624352152.0381165,"caller":"mlog/log.go:232","msg":"Plugin failed to ServeHTTP, muxBroker couldn't accept connection","plugin_id":"bigbluebutton","serve_http_stream_id":2,"error":"timeout waiting for accept"}

What can cause this?

Regards,
Moritz

The text was updated successfully, but these errors were encountered:

harshilsharma63 · 2021-06-24T04:18:25Z

Hi @moschlar ,

seems like the plugin is crashing for you for some reason. Can you restart the plugin and monitor the logs to see why the plugin crashed in the first place?

moschlar · 2021-06-24T20:13:04Z

There's no error before the first line of that error...

Just the usual many lines of

2021-06-24_20:11:48.17026 {"level":"info","ts":1624565508.1701915,"caller":"mlog/sugar.go:21","msg":"Successfully got recordings info","plugin_id":"bigbluebutton"}

🤔

moschlar · 2021-06-24T20:17:43Z

Ah wait, loglevel was not DEBUG...

Now I got:

2021-06-24_20:15:30.21101 2021-06-24T22:15:30.210+0200	info	mlog/sugar.go:21	Successfully got recordings info	{"plugin_id": "bigbluebutton"}
2021-06-24_20:15:30.22238 2021-06-24T22:15:30.220+0200	debug	plugin/hclog_adapter.go:54	panic: runtime error: invalid memory address or nil pointer dereference	{"plugin_id": "bigbluebutton"}
2021-06-24_20:15:30.22241 2021-06-24T22:15:30.221+0200	debug	plugin/hclog_adapter.go:54	[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xf8b18c]	{"plugin_id": "bigbluebutton"}
2021-06-24_20:15:30.22242 2021-06-24T22:15:30.221+0200	debug	plugin/hclog_adapter.go:54		{"plugin_id": "bigbluebutton"}
2021-06-24_20:15:30.22242 2021-06-24T22:15:30.221+0200	debug	plugin/hclog_adapter.go:54	goroutine 21 [running]:	{"plugin_id": "bigbluebutton"}
2021-06-24_20:15:30.22242 2021-06-24T22:15:30.221+0200	debug	plugin/hclog_adapter.go:54	main.(*Plugin).Loopthroughrecordings(0xc000297590)	{"plugin_id": "bigbluebutton"}
2021-06-24_20:15:30.22244 2021-06-24T22:15:30.221+0200	debug	plugin/hclog_adapter.go:54		/Users/harshilsharma/codebase/go/src/github.com/blindsightnetworks/mattermost-plugin-bigbluebutton/server/responsehandlers.go:79 +0x2ac	{"plugin_id": "bigbluebutton"}
2021-06-24_20:15:30.22245 2021-06-24T22:15:30.221+0200	debug	plugin/hclog_adapter.go:54	github.com/mattermost/mattermost-plugin-api/cluster.(*Job).run.func1(0xc0004325a0, 0xc00043df00)	{"plugin_id": "bigbluebutton"}
2021-06-24_20:15:30.22246 2021-06-24T22:15:30.221+0200	debug	plugin/hclog_adapter.go:54		/Users/harshilsharma/codebase/go/src/github.com/blindsightnetworks/mattermost-plugin-bigbluebutton/vendor/github.com/mattermost/mattermost-plugin-api/cluster/job.go:204 +0x4af	{"plugin_id": "bigbluebutton"}
2021-06-24_20:15:30.22246 2021-06-24T22:15:30.221+0200	debug	plugin/hclog_adapter.go:54	github.com/mattermost/mattermost-plugin-api/cluster.(*Job).run(0xc0004325a0)	{"plugin_id": "bigbluebutton"}
2021-06-24_20:15:30.22247 2021-06-24T22:15:30.221+0200	debug	plugin/hclog_adapter.go:54		/Users/harshilsharma/codebase/go/src/github.com/blindsightnetworks/mattermost-plugin-bigbluebutton/vendor/github.com/mattermost/mattermost-plugin-api/cluster/job.go:214 +0x185	{"plugin_id": "bigbluebutton"}
2021-06-24_20:15:30.22247 2021-06-24T22:15:30.221+0200	debug	plugin/hclog_adapter.go:54	created by github.com/mattermost/mattermost-plugin-api/cluster.Schedule	{"plugin_id": "bigbluebutton"}
2021-06-24_20:15:30.22248 2021-06-24T22:15:30.221+0200	debug	plugin/hclog_adapter.go:54		/Users/harshilsharma/codebase/go/src/github.com/blindsightnetworks/mattermost-plugin-bigbluebutton/vendor/github.com/mattermost/mattermost-plugin-api/cluster/job.go:130 +0x3e5	{"plugin_id": "bigbluebutton"}
2021-06-24_20:15:30.22256 2021-06-24T22:15:30.222+0200	debug	plugin/hclog_adapter.go:52	plugin process exited	{"plugin_id": "bigbluebutton", "wrapped_extras": "path/var/opt/gitlab/mattermost/plugins/bigbluebutton/server/plugin.exepid11776errorexit status 2"}

moschlar · 2021-06-29T09:42:47Z

@harshilsharma63 are there any more steps I can perform to help you debug the problem?

moschlar · 2021-07-03T12:19:31Z

I deleted all rows for the plugin from the key value store in the database and for some minutes now it seems to be running fine...

moschlar · 2021-07-03T18:19:42Z

And again - something with the recordings goes terribly wrong after some time:

2021-07-03_12:36:00.08548 2021-07-03T14:36:00.085+0200	info	mlog/sugar.go:21	[{"RecordID":"87909aba78e7c49c7ee4d9a5996868e1459a8d54-1625314293996","MeetingID":"1unvi7EshpE0aYiVbAH9622mifF","Name":"Holz-Café","Published":"true","State":"published","StartTime":"1625314293996","EndTime":"1625314311097","Participants":"1","MetaData":{"Title":"","Subject":"","Description":"","Creator":"","Contributor":"","Language":""},"Playback":{"Format":[{"Type":"presentation","Url":"https://scalelite..../playback/presentation/2.3/87909aba78e7c49c7ee4d9a5996868e1459a8d54-1625314293996","Length":"0","Images":["https://scalelite..../presentation/87909aba78e7c49c7ee4d9a5996868e1459a8d54-1625314293996/presentation/d2d9a672040fbde2a47a10bf6c37b6a4b5ae187f-1625314294016/thumbnails/thumb-3.png","https://scalelite..../presentation/87909aba78e7c49c7ee4d9a5996868e1459a8d54-1625314293996/presentation/d2d9a672040fbde2a47a10bf6c37b6a4b5ae187f-1625314294016/thumbnails/thumb-2.png","https://scalelite..../presentation/87909aba78e7c49c7ee4d9a5996868e1459a8d54-1625314293996/presentation/d2d9a672040fbde2a47a10bf6c37b6a4b5ae187f-1625314294016/thumbnails/thumb-1.png"]}]}}]	{"plugin_id": "bigbluebutton"}
2021-07-03_12:36:00.17244 2021-07-03T14:36:00.172+0200	debug	plugin/hclog_adapter.go:54	panic: runtime error: invalid memory address or nil pointer dereference	{"plugin_id": "bigbluebutton"}
2021-07-03_12:36:00.17247 2021-07-03T14:36:00.172+0200	debug	plugin/hclog_adapter.go:54	[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xf8b18c]	{"plugin_id": "bigbluebutton"}
2021-07-03_12:36:00.17247 2021-07-03T14:36:00.172+0200	debug	plugin/hclog_adapter.go:54		{"plugin_id": "bigbluebutton"}
2021-07-03_12:36:00.17247 2021-07-03T14:36:00.172+0200	debug	plugin/hclog_adapter.go:54	goroutine 45 [running]:	{"plugin_id": "bigbluebutton"}
2021-07-03_12:36:00.17248 2021-07-03T14:36:00.172+0200	debug	plugin/hclog_adapter.go:54	main.(*Plugin).Loopthroughrecordings(0xc000297590)	{"plugin_id": "bigbluebutton"}
2021-07-03_12:36:00.17248 2021-07-03T14:36:00.172+0200	debug	plugin/hclog_adapter.go:54		/Users/harshilsharma/codebase/go/src/github.com/blindsightnetworks/mattermost-plugin-bigbluebutton/server/responsehandlers.go:79 +0x2ac	{"plugin_id": "bigbluebutton"}
2021-07-03_12:36:00.17251 2021-07-03T14:36:00.172+0200	debug	plugin/hclog_adapter.go:54	github.com/mattermost/mattermost-plugin-api/cluster.(*Job).run.func1(0xc00049f8c0, 0xc0005b7f00)	{"plugin_id": "bigbluebutton"}
2021-07-03_12:36:00.17252 2021-07-03T14:36:00.172+0200	debug	plugin/hclog_adapter.go:54		/Users/harshilsharma/codebase/go/src/github.com/blindsightnetworks/mattermost-plugin-bigbluebutton/vendor/github.com/mattermost/mattermost-plugin-api/cluster/job.go:204 +0x4af	{"plugin_id": "bigbluebutton"}
2021-07-03_12:36:00.17252 2021-07-03T14:36:00.172+0200	debug	plugin/hclog_adapter.go:54	github.com/mattermost/mattermost-plugin-api/cluster.(*Job).run(0xc00049f8c0)	{"plugin_id": "bigbluebutton"}
2021-07-03_12:36:00.17253 2021-07-03T14:36:00.172+0200	debug	plugin/hclog_adapter.go:54		/Users/harshilsharma/codebase/go/src/github.com/blindsightnetworks/mattermost-plugin-bigbluebutton/vendor/github.com/mattermost/mattermost-plugin-api/cluster/job.go:214 +0x185	{"plugin_id": "bigbluebutton"}
2021-07-03_12:36:00.17253 2021-07-03T14:36:00.172+0200	debug	plugin/hclog_adapter.go:54	created by github.com/mattermost/mattermost-plugin-api/cluster.Schedule	{"plugin_id": "bigbluebutton"}
2021-07-03_12:36:00.17253 2021-07-03T14:36:00.172+0200	debug	plugin/hclog_adapter.go:54		/Users/harshilsharma/codebase/go/src/github.com/blindsightnetworks/mattermost-plugin-bigbluebutton/vendor/github.com/mattermost/mattermost-plugin-api/cluster/job.go:130 +0x3e5	{"plugin_id": "bigbluebutton"}
2021-07-03_12:36:00.17451 2021-07-03T14:36:00.174+0200	debug	plugin/hclog_adapter.go:52	plugin process exited	{"plugin_id": "bigbluebutton", "wrapped_extras": "path/var/opt/gitlab/mattermost/plugins/bigbluebutton/server/plugin.exepid136221errorexit status 2"}

harshilsharma63 · 2021-07-05T01:51:14Z

@moschlar when you deleted the rows from the database, did you also restart the plugin?

Let's schedule a quick call to debug this. I've sent you an email for the same. Please check it out.

moschlar · 2021-07-05T08:17:00Z

@harshilsharma63

@moschlar when you deleted the rows from the database, did you also restart the plugin?

Yes, I disabled it beforehand, restarted Mattermost, deleted the rows, and then enable it again.

harshilsharma63 · 2021-07-07T03:57:28Z

@moschlar please use the latest v3.2.0 release until we fix the cluster issue.

moschlar · 2021-07-07T04:50:59Z

I just installed it - let’s see how it goes!

moschlar · 2021-07-07T09:42:46Z

Looking good so far!

harshilsharma63 · 2021-07-19T01:19:32Z

@moschlar what version of Scalite are you running?

moschlar · 2021-07-20T18:59:17Z

@harshilsharma63 I think that's 1.1.5.1 but maybe @christmart can clarify on that

harshilsharma63 · 2021-07-23T23:51:04Z

@christmart any idea?

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

"Plugin failed to ServeHTTP" Errors with v3.1.1 #158

"Plugin failed to ServeHTTP" Errors with v3.1.1 #158

moschlar commented Jun 22, 2021

harshilsharma63 commented Jun 24, 2021

moschlar commented Jun 24, 2021

moschlar commented Jun 24, 2021

moschlar commented Jun 29, 2021

moschlar commented Jul 3, 2021

moschlar commented Jul 3, 2021

harshilsharma63 commented Jul 5, 2021

moschlar commented Jul 5, 2021

harshilsharma63 commented Jul 7, 2021

moschlar commented Jul 7, 2021

moschlar commented Jul 7, 2021

harshilsharma63 commented Jul 19, 2021

moschlar commented Jul 20, 2021

harshilsharma63 commented Jul 23, 2021

"Plugin failed to ServeHTTP" Errors with v3.1.1 #158

"Plugin failed to ServeHTTP" Errors with v3.1.1 #158

Comments

moschlar commented Jun 22, 2021

harshilsharma63 commented Jun 24, 2021

moschlar commented Jun 24, 2021

moschlar commented Jun 24, 2021

moschlar commented Jun 29, 2021

moschlar commented Jul 3, 2021

moschlar commented Jul 3, 2021

harshilsharma63 commented Jul 5, 2021

moschlar commented Jul 5, 2021

harshilsharma63 commented Jul 7, 2021

moschlar commented Jul 7, 2021

moschlar commented Jul 7, 2021

harshilsharma63 commented Jul 19, 2021

moschlar commented Jul 20, 2021

harshilsharma63 commented Jul 23, 2021