Skip to content

Commit

Permalink
[SPARK-47842][CONNECT] Use v1 as spark connect go library starting ve…
Browse files Browse the repository at this point in the history
…rsion

### What changes were proposed in this pull request?

Use v1 for this Spark Connect Go Client library, instead of previously using v34.

Also regenerate protobuf files based on Spark latest 3.5 release (3.5.1).

### Why are the changes needed?

There is a recent discussion in Spark community for Spark Operator version naming convention. People like to use version independent of Spark versions. That applies to Spark Connect Go Client as well. Thus change the version here to start from v1.

### Does this PR introduce _any_ user-facing change?

Yes, Spark Connect Go Client users will use v1 instead of use v34.

### How was this patch tested?

Run unit test.

Closes #19 from hiboyang/bo-dev-07.

Authored-by: Bo (AIML) Yang <bo_yang6@apple.com>
Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
  • Loading branch information
hiboyang authored and HyukjinKwon committed Apr 26, 2024
1 parent f7ad518 commit 884ae1c
Show file tree
Hide file tree
Showing 18 changed files with 4,824 additions and 2,817 deletions.
2 changes: 1 addition & 1 deletion client/channel/channel_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ import (
"strings"
"testing"

"github.com/apache/spark-connect-go/v34/client/channel"
"github.com/apache/spark-connect-go/v1/client/channel"
"github.com/stretchr/testify/assert"
)

Expand Down
2 changes: 1 addition & 1 deletion client/sql/dataframe.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ import (
"github.com/apache/arrow/go/v12/arrow"
"github.com/apache/arrow/go/v12/arrow/array"
"github.com/apache/arrow/go/v12/arrow/ipc"
proto "github.com/apache/spark-connect-go/v34/internal/generated"
proto "github.com/apache/spark-connect-go/v1/internal/generated"
"io"
)

Expand Down
2 changes: 1 addition & 1 deletion client/sql/dataframe_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ import (
"github.com/apache/arrow/go/v12/arrow/float16"
"github.com/apache/arrow/go/v12/arrow/ipc"
"github.com/apache/arrow/go/v12/arrow/memory"
proto "github.com/apache/spark-connect-go/v34/internal/generated"
proto "github.com/apache/spark-connect-go/v1/internal/generated"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"testing"
Expand Down
2 changes: 1 addition & 1 deletion client/sql/dataframereader.go
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
package sql

import proto "github.com/apache/spark-connect-go/v34/internal/generated"
import proto "github.com/apache/spark-connect-go/v1/internal/generated"

// DataFrameReader supports reading data from storage and returning a data frame.
// TODO needs to implement other methods like Option(), Schema(), and also "strong typed"
Expand Down
2 changes: 1 addition & 1 deletion client/sql/dataframewriter.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ package sql

import (
"fmt"
proto "github.com/apache/spark-connect-go/v34/internal/generated"
proto "github.com/apache/spark-connect-go/v1/internal/generated"
"strings"
)

Expand Down
2 changes: 1 addition & 1 deletion client/sql/dataframewriter_test.go
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
package sql

import (
proto "github.com/apache/spark-connect-go/v34/internal/generated"
proto "github.com/apache/spark-connect-go/v1/internal/generated"
"github.com/stretchr/testify/assert"
"testing"
)
Expand Down
4 changes: 2 additions & 2 deletions client/sql/sparksession.go
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,8 @@ import (
"errors"
"fmt"

"github.com/apache/spark-connect-go/v34/client/channel"
proto "github.com/apache/spark-connect-go/v34/internal/generated"
"github.com/apache/spark-connect-go/v1/client/channel"
proto "github.com/apache/spark-connect-go/v1/internal/generated"
"github.com/google/uuid"
"google.golang.org/grpc/metadata"
"io"
Expand Down
2 changes: 1 addition & 1 deletion cmd/spark-connect-example-raw-grpc-client/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ package main
import (
"context"
"flag"
proto "github.com/apache/spark-connect-go/v34/internal/generated"
proto "github.com/apache/spark-connect-go/v1/internal/generated"
"github.com/google/uuid"
"google.golang.org/grpc"
"google.golang.org/grpc/credentials/insecure"
Expand Down
2 changes: 1 addition & 1 deletion cmd/spark-connect-example-spark-session/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ import (
"flag"
"log"

"github.com/apache/spark-connect-go/v34/client/sql"
"github.com/apache/spark-connect-go/v1/client/sql"
)

var (
Expand Down
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.

module github.com/apache/spark-connect-go/v34
module github.com/apache/spark-connect-go/v1

go 1.19

Expand Down
Loading

0 comments on commit 884ae1c

Please sign in to comment.