Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: getsolc binary to locate or download Solidity compiler #1244

Draft
wants to merge 1 commit into
base: arr4n/precompile-go-tests
Choose a base branch
from
Draft
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
154 changes: 154 additions & 0 deletions scripts/getsolc/main.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,154 @@
// The getsolc binary downloads the Solidity compiler from official sources. If
// a `solc` binary in the PATH is of the requested version then a symlink is
// created instead of downloading a new copy.
package main

import (
"bytes"
"context"
"encoding/json"
"flag"
"fmt"
"io"
"net/http"
"net/url"
"os"
"os/exec"
"path"
"runtime"
"strings"

"github.com/ethereum/go-ethereum/crypto"
)

func main() {
var c config

flag.StringVar(&c.version, "version", latestVersion, fmt.Sprintf("Version of solc; {major}.{minor}.{patch} or %q", latestVersion))
flag.StringVar(&c.outputFile, "out", "./solc", "Path to which the `solc` binary will be saved")
flag.BoolVar(&c.ignoreGOARCH, "ignore_goarch", false, "Download amd64 binary even if on another architecture")

flag.Parse()

if err := c.run(context.Background()); err != nil {
fmt.Fprintln(os.Stderr, err)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(No action required) Thoughts on using panic in a script to provide immediate feedback and a stack trace? I've heard it suggested that getting a stack trace in the context of a script can be preferential to passing errors around, but its use would probably have to be limited to main entrypoints or reusability of library functionality would be compromised.

os.Exit(1)
}
}

const latestVersion = "latest"

type config struct {
version, outputFile string
ignoreGOARCH bool
}

func (c *config) run(ctx context.Context) error {
goos := runtime.GOOS
switch goos {
case "darwin":
goos = "macosx"
case "linux":
default:
return fmt.Errorf("unsupported OS %q", goos)
}

// solc only provides amd64 binaries, but this can be ignored if there is a
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(No action required) I guess rosetta is fine, but what is up with automated release procedures not being updated to create arm64 binaries? I guess it's an indication that most people are using a package manager that does ship arm64.

// translator (e.g. Rosetta on MacOS)
if !c.ignoreGOARCH && runtime.GOARCH != "amd64" {
return fmt.Errorf("unsupported GOARCH %q", runtime.GOARCH)
}

jsonList, err := httpGetSolFile(ctx, goos, "list.json")
if err != nil {
return err
}
defer jsonList.Body.Close()
var list solcList
if err := json.NewDecoder(jsonList.Body).Decode(&list); err != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(No action required) Is there an advantage to using Decode vs Unmarshal where the data is in-memory (as I'm assuming is the case here)?

return err
}

if c.version == latestVersion {
c.version = list.LatestRelease
}
if p, ok := c.bestEffortFindInPATH(ctx); ok { // NOTE: this is not an error path
fmt.Fprintf(os.Stderr, "Creating symlink from %q to %q\n", p, c.outputFile)
return os.Link(p, c.outputFile)
}
fmt.Fprintln(os.Stderr, "Downloading solc...")

solc, err := httpGetSolFile(ctx, goos, list.Releases[c.version])
if err != nil {
return err
}
defer solc.Body.Close()

hash := crypto.NewKeccakState()
tee := io.TeeReader(solc.Body, hash)

out, err := os.OpenFile(c.outputFile, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0755)
if err != nil {
return fmt.Errorf("os.OpenFile: %v", err)
}
if _, err := io.Copy(out, tee); err != nil {
return fmt.Errorf("io.Copy: %v", err)
}
if err := out.Close(); err != nil {
return err
}

fmt.Fprintf(os.Stderr, "Keccak256 of download: %#x\n", hash.Sum(nil))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(No action required) Best practice for the use of stderr vs stdout? I've defaulted to thinkiing of stderr as for error conditions - which isn't the case here - but I recall that a more nuanced view is prevalent among experts.

return nil
}

// httpGetSolFile issues an HTTP(s) GET to download the specified file from the
// official binaries.soliditylang.org source. `goos` can be either linux or
// macosx, and file can be "list.json" or any of the paths in [solcList].
func httpGetSolFile(ctx context.Context, goos, file string) (*http.Response, error) {
u := url.URL{
Scheme: "https",
Host: "binaries.soliditylang.org",
Path: path.Join(fmt.Sprintf("%s-amd64", goos), file),
}

req, err := http.NewRequestWithContext(ctx, http.MethodGet, u.String(), nil)
if err != nil {
return nil, err
}
resp, err := http.DefaultClient.Do(req)
if err != nil || resp.StatusCode != 200 {
return nil, fmt.Errorf("HTTP GET %q; Status = %d; err = %v", u.String(), resp.StatusCode, err)
}
return resp, nil
}

// solcList mirrors the JSON lists published by the solc team; e.g:
// https://binaries.soliditylang.org/linux-amd64/list.json
type solcList struct {
Releases map[string]string `json:"releases"`
LatestRelease string `json:"latestRelease"`
Builds []struct {
Path, Version, Build, Keccak256 string
LongVersion string `json:"longVersion"`
SHA256 string `json:"sha256"`
} `json:"builds"`
}

// bestEffortFindInPATH attempts to find `solc` in the PATH and, if it has the
// required version, the path to said binary is returned. The boolean indicates
// succesful location of a matching binary.

Check failure on line 140 in scripts/getsolc/main.go

View workflow job for this annotation

GitHub Actions / Lint

`succesful` is a misspelling of `successful` (misspell)
func (c *config) bestEffortFindInPATH(ctx context.Context) (string, bool) {
solc := exec.CommandContext(ctx, "solc", "--version")
out, err := solc.CombinedOutput()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(No action required) For general-purpose scripting in golang, maybe we'll want a standard way to invoke commands so that we can replicate the behavior of set -x for the whole script? Or maybe I'm just scarred from too much bash and should get used to the idea of using a debugger instead... 😄

if err != nil || !bytes.Contains(out, []byte(c.version)) {
return "", false
}

which := exec.CommandContext(ctx, "which", "solc")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(No action required) Maybe prefer command as per https://www.shellcheck.net/wiki/SC2230? It is optional and a bit pedantic, but its been enforced on me before.

I guess a slight con of using golang over bash for scripting is that we don't benefit from a scripting-centric linter? Not a deal-breaker by any means, but we'll inevitably have shell interaction in golang scripts and in the absence of a linter we'll have to rely on review.

solcPath, err := which.CombinedOutput()
if err != nil {
return "", false
}
return strings.TrimSpace(string(solcPath)), true
}
Loading