Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SPARK-44681] Fix issues when writing Go application code using Spark Connect Go client library #14

Closed
wants to merge 2 commits into from
Closed

[SPARK-44681] Fix issues when writing Go application code using Spark Connect Go client library #14

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
58e83dc
Commit internal/generated protobuf files into repo
hiboyang Aug 4, 2023
099bbca
Rename v_3_4 to v34
hiboyang Aug 4, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,6 @@
#

# All generated files
internal/generated
internal/generated.out

# Ignore Coverage Files
Expand Down
2 changes: 1 addition & 1 deletion client/channel/channel_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ import (
"strings"
"testing"

"github.com/apache/spark-connect-go/v_3_4/client/channel"
"github.com/apache/spark-connect-go/v34/client/channel"
"github.com/stretchr/testify/assert"
)

Expand Down
2 changes: 1 addition & 1 deletion client/sql/dataframe.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ import (
"github.com/apache/arrow/go/v12/arrow"
"github.com/apache/arrow/go/v12/arrow/array"
"github.com/apache/arrow/go/v12/arrow/ipc"
proto "github.com/apache/spark-connect-go/v_3_4/internal/generated"
proto "github.com/apache/spark-connect-go/v34/internal/generated"
"io"
)

Expand Down
2 changes: 1 addition & 1 deletion client/sql/dataframe_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ import (
"github.com/apache/arrow/go/v12/arrow/float16"
"github.com/apache/arrow/go/v12/arrow/ipc"
"github.com/apache/arrow/go/v12/arrow/memory"
proto "github.com/apache/spark-connect-go/v_3_4/internal/generated"
proto "github.com/apache/spark-connect-go/v34/internal/generated"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"testing"
Expand Down
2 changes: 1 addition & 1 deletion client/sql/dataframereader.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
package sql

import proto "github.com/apache/spark-connect-go/v_3_4/internal/generated"
import proto "github.com/apache/spark-connect-go/v34/internal/generated"

// DataFrameReader supports reading data from storage and returning a data frame.
// TODO needs to implement other methods like Option(), Schema(), and also "strong typed"
Expand Down
2 changes: 1 addition & 1 deletion client/sql/dataframewriter.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ package sql

import (
"fmt"
proto "github.com/apache/spark-connect-go/v_3_4/internal/generated"
proto "github.com/apache/spark-connect-go/v34/internal/generated"
"strings"
)

Expand Down
2 changes: 1 addition & 1 deletion client/sql/dataframewriter_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
package sql

import (
proto "github.com/apache/spark-connect-go/v_3_4/internal/generated"
proto "github.com/apache/spark-connect-go/v34/internal/generated"
"github.com/stretchr/testify/assert"
"testing"
)
Expand Down
4 changes: 2 additions & 2 deletions client/sql/sparksession.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,8 @@ import (
"errors"
"fmt"

"github.com/apache/spark-connect-go/v_3_4/client/channel"
proto "github.com/apache/spark-connect-go/v_3_4/internal/generated"
"github.com/apache/spark-connect-go/v34/client/channel"
proto "github.com/apache/spark-connect-go/v34/internal/generated"
"github.com/google/uuid"
"google.golang.org/grpc/metadata"
"io"
Expand Down
2 changes: 1 addition & 1 deletion cmd/spark-connect-example-raw-grpc-client/main.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ package main
import (
"context"
"flag"
proto "github.com/apache/spark-connect-go/v_3_4/internal/generated"
proto "github.com/apache/spark-connect-go/v34/internal/generated"
"github.com/google/uuid"
"google.golang.org/grpc"
"google.golang.org/grpc/credentials/insecure"
Expand Down
2 changes: 1 addition & 1 deletion cmd/spark-connect-example-spark-session/main.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ import (
"flag"
"log"

"github.com/apache/spark-connect-go/v_3_4/client/sql"
"github.com/apache/spark-connect-go/v34/client/sql"
)

var (
Expand Down
2 changes: 1 addition & 1 deletion go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.

module github.com/apache/spark-connect-go/v_3_4
module github.com/apache/spark-connect-go/v34
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's interesting that this hasn't caused issues before really. But I'm fine with the changes.

Copy link
Contributor Author

@hiboyang hiboyang Aug 10, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for reviewing! Yeah, previously we have code inside this repo referencing this same library , thus v_3_4 does not cause problem. When I wrote another Go application outside of this repo, referencing this repo as a library, then got issue.


go 1.19

Expand Down
Loading