Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Asynchronous M3U parsing directly from URL #6

Merged
merged 33 commits into from
Mar 2, 2024
Merged
Changes from 1 commit
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
4234d82
add context for graceful shutdown
sonroyaalmerol Mar 1, 2024
f55227c
add basic m3u tests
sonroyaalmerol Mar 1, 2024
efbf65d
remove strings import
sonroyaalmerol Mar 1, 2024
7be1087
fix expected stream url
sonroyaalmerol Mar 1, 2024
56e273d
import fmt
sonroyaalmerol Mar 1, 2024
7400a50
remove newline from expected content
sonroyaalmerol Mar 1, 2024
38fcec5
individualize sql query functions
sonroyaalmerol Mar 2, 2024
bc35489
clean up m3u package and directly connect to database
sonroyaalmerol Mar 2, 2024
c21f25a
comment out to be fixed first
sonroyaalmerol Mar 2, 2024
9df85c2
do not insert channels without titles
sonroyaalmerol Mar 2, 2024
424dde6
fix mp4_handler.go
sonroyaalmerol Mar 2, 2024
8dcfd27
fix database_test
sonroyaalmerol Mar 2, 2024
c261741
add title to insertstream error log
sonroyaalmerol Mar 2, 2024
5394e58
separate insert stream and insert url
sonroyaalmerol Mar 2, 2024
599c990
add rows next
sonroyaalmerol Mar 2, 2024
1f8c874
add more log info test
sonroyaalmerol Mar 2, 2024
6276f5d
add more log info test
sonroyaalmerol Mar 2, 2024
4002bc4
fix test error
sonroyaalmerol Mar 2, 2024
eb38c01
switch to directly accessing url
sonroyaalmerol Mar 2, 2024
2dcd36b
switch to directly accessing url
sonroyaalmerol Mar 2, 2024
db38a94
clean up downloader.go
sonroyaalmerol Mar 2, 2024
ef963ea
add USER_AGENT env var
sonroyaalmerol Mar 2, 2024
3e22d1b
make m3u parsing asynchronous
sonroyaalmerol Mar 2, 2024
f2fc16a
improve logging messages
sonroyaalmerol Mar 2, 2024
5a9a265
fix formatting
sonroyaalmerol Mar 2, 2024
8db9d85
add hadolint and golanci-lint
sonroyaalmerol Mar 2, 2024
0658fbe
fix dockerfile lint issues
sonroyaalmerol Mar 2, 2024
8c5aff9
fix golinting issues
sonroyaalmerol Mar 2, 2024
dbe8f6c
fix linting issues
sonroyaalmerol Mar 2, 2024
e233cd5
add line number to golangci-lint
sonroyaalmerol Mar 2, 2024
b5a2b3b
fix golint issue
sonroyaalmerol Mar 2, 2024
37d161d
only rollback on err
sonroyaalmerol Mar 2, 2024
d1f7a9c
fix #EXTVLCOPT
sonroyaalmerol Mar 2, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
clean up downloader.go
  • Loading branch information
sonroyaalmerol committed Mar 2, 2024

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
commit db38a940b6dec94d06abc4cd6aca965942934b9c
127 changes: 0 additions & 127 deletions m3u/downloader.go

This file was deleted.

2 changes: 1 addition & 1 deletion m3u/m3u_test.go
Original file line number Diff line number Diff line change
@@ -101,7 +101,7 @@ http://example.com/fox
defer os.Remove(sqliteDBPath) // Cleanup the database file after the test

// Test the parseM3UFromURL function with the mock server URL
err = parseM3UFromURL(mockServer.URL, 0)
err = ParseM3UFromURL(mockServer.URL, 0)
if err != nil {
t.Errorf("Error parsing M3U from URL: %v", err)
}
16 changes: 14 additions & 2 deletions m3u/parser.go
Original file line number Diff line number Diff line change
@@ -10,10 +10,22 @@ import (
"m3u-stream-merger/database"
)

func parseM3UFromURL(m3uURL string, m3uIndex int) (error) {
func ParseM3UFromURL(m3uURL string, m3uIndex int) (error) {
// Set the custom User-Agent header
userAgent := "IPTV Smarters/1.0.3 (iPad; iOS 16.6.1; Scale/2.00)"

// Create a new HTTP client with a custom User-Agent header
client := &http.Client{
CheckRedirect: func(req *http.Request, via []*http.Request) error {
// Follow redirects while preserving the custom User-Agent header
req.Header.Set("User-Agent", userAgent)
return nil
},
}

fmt.Printf("Parsing M3U from URL: %s\n", m3uURL)

resp, err := http.Get(m3uURL)
resp, err := client.Get(m3uURL)
if err != nil {
return fmt.Errorf("HTTP GET error: %v", err)
}
14 changes: 14 additions & 0 deletions main.go
Original file line number Diff line number Diff line change
@@ -38,6 +38,20 @@ func updateSource(ctx context.Context) {
}

func main() {
index := 1
for {
m3uUrl, m3uExists := os.LookupEnv(fmt.Sprintf("M3U_URL_%d", index))
if !m3uExists {
break
}

err := m3u.ParseM3UFromURL(m3uUrl, index)
if err != nil {

}

index++
}
// Context for graceful shutdown
ctx, cancel := context.WithCancel(context.Background())
defer cancel()