Added config option to set size of transcode cache and cadence to enf… #520
Conversation
…orce that sizing via ejection.
|
Resolves #501 |
|
@sentriz Would love to get your feedback on this when you get a minute. Thanks. |
|
hiya Brian sorry for the delay . will hopefully take a look shortly |
|
@brian-doherty Could you please also update contrib/config accordingly with this new option? |
Done. Lint and test are running now but will pass. |
|
@sentriz I know you're busy but I would love you to have a look at this PR when you get a sec. |
transcode/transcoder_caching.go
Outdated
| @@ -64,6 +68,41 @@ func (t *CachingTranscoder) Transcode(ctx context.Context, profile Profile, in s | |||
| return nil | |||
| } | |||
|
|
|||
| func (t *CachingTranscoder) CacheEject() { | |||
There was a problem hiding this comment.
this could return an error, which we can just log in main.go
transcode/transcoder_caching.go
Outdated
| var files []file | ||
| var total int64 = 0 | ||
|
|
||
| _ = filepath.Walk(t.cachePath, func(path string, info os.FileInfo, err error) error { |
There was a problem hiding this comment.
can we use the new (and faster i belive) filepath.WalkDir here?
and return an error too
transcode/transcoder_caching.go
Outdated
| curFile := files[0] | ||
| files = files[1:] | ||
| total -= curFile.info.Size() | ||
| unlock := t.locks.Lock(curFile.path) |
There was a problem hiding this comment.
the t.locks is a keyedMutex but the keys are profile strings (ffmpeg commands with args and paths) not paths. so this has little effect i think
maybe we should have a global clean lock which the normal transcode thing respects too?
something like
type CachingTranscoder struct {
...
locks keyedMutex
cleanLock sync.RWMutex
}
...
func (t *CachingTranscoder) Transcode(...) error {
...
t.cleanLock.RLock()
defer t.cleanLock.RUnlock()
...
key := cacheKey(name, args)
unlock := t.locks.Lock(key)
defer unlock()
}
func (t *CachingTranscoder) CacheEject() error {
t.cleanLock.Lock()
defer t.cleanLock.Unlock()
...
}
that way the cleaning process locks the entire cache, while multiple goroutines looking to read the cache work fine, just not while we're cleaning
wdyt?
There was a problem hiding this comment.
Good call and my apologies. Done.
transcode/transcoder_caching.go
Outdated
| locks keyedMutex | ||
| } | ||
|
|
||
| var _ Transcoder = (*CachingTranscoder)(nil) | ||
|
|
||
| func NewCachingTranscoder(t Transcoder, cachePath string) *CachingTranscoder { | ||
| return &CachingTranscoder{transcoder: t, cachePath: cachePath} | ||
| func NewCachingTranscoder(t Transcoder, cachePath string, limitMb int) *CachingTranscoder { |
There was a problem hiding this comment.
looks megabits to me. how about a big B for bytes
| func NewCachingTranscoder(t Transcoder, cachePath string, limitMb int) *CachingTranscoder { | |
| func NewCachingTranscoder(t Transcoder, cachePath string, limitMB int) *CachingTranscoder { |
|
@sentriz All comments addressed. See what you think and thanks! |
cmd/gonic/gonic
Outdated
There was a problem hiding this comment.
looks like this was committed accidentally
There was a problem hiding this comment.
My bad. Fixed.
|
looks great, thanks! sorry for the delay |
…orce that sizing via ejection.