Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

24.5 #49

Merged
merged 61 commits into from
Sep 13, 2024
Merged

24.5 #49

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
577c302
updating for nightly
thatstoasty Jul 9, 2024
87430e6
update writer trait to use span
thatstoasty Jul 9, 2024
05d9901
update for latest nightly
thatstoasty Aug 2, 2024
f14c578
checkin
thatstoasty Aug 13, 2024
f71b3be
update for nightly
thatstoasty Aug 15, 2024
3204a48
add scripts
thatstoasty Aug 15, 2024
a9df70c
nightly update
thatstoasty Aug 15, 2024
3eb65b3
parametrize bytes funs
thatstoasty Aug 19, 2024
ffe0686
update to use trivial type
thatstoasty Aug 19, 2024
6ad3019
Cleanup buffered io structs
thatstoasty Aug 28, 2024
0082a13
cleaned up buffer docs
thatstoasty Aug 28, 2024
6e07ab5
update bufio docs
thatstoasty Aug 28, 2024
fff4eea
fix broken filewrapper funcs
thatstoasty Aug 28, 2024
de8b662
add more benchmarks, but scanner needs to be fixed now
thatstoasty Aug 28, 2024
a2ed7f0
partial fix for scanner
thatstoasty Aug 28, 2024
502e020
fix scanner
thatstoasty Aug 28, 2024
7dfe559
fix scanner for input larger than internal buffer
thatstoasty Aug 28, 2024
defb377
fix reader from
thatstoasty Aug 29, 2024
df928dc
fix reader from
thatstoasty Aug 29, 2024
b340a2a
break up bufio
thatstoasty Aug 29, 2024
3d8958d
update scanner benchmark
thatstoasty Aug 29, 2024
22b7b66
reorganizing repo to move towards using mojopkg more
thatstoasty Aug 29, 2024
ffc62f8
cleaning up socket and making it work again
thatstoasty Aug 29, 2024
f0f27de
update tests to include the newline added to txt files by precommit
thatstoasty Aug 29, 2024
99f31da
rm init from benchmarks
thatstoasty Aug 29, 2024
9c2179c
updated rattler build recipe
thatstoasty Aug 30, 2024
e724031
add max dep
thatstoasty Sep 2, 2024
a4ab45e
update socket stuff
thatstoasty Sep 3, 2024
c0d5620
wip updating gojo ver
thatstoasty Sep 3, 2024
f65cff6
fix exclusivity err
thatstoasty Sep 5, 2024
7317e67
add run benchmarks command
thatstoasty Sep 5, 2024
9095c3e
prep for 24.5
thatstoasty Sep 6, 2024
cd1d33b
update github workflow
thatstoasty Sep 6, 2024
c915d11
fix workflow
thatstoasty Sep 6, 2024
0d7d616
fix workflow
thatstoasty Sep 6, 2024
5b796eb
fix workflow
thatstoasty Sep 6, 2024
287e182
add linux support to mojoproject
thatstoasty Sep 6, 2024
640fdc1
add linux support to mojoproject
thatstoasty Sep 6, 2024
1837112
remove mac runner for now due to hang on tests
thatstoasty Sep 6, 2024
6855f52
update installation instructions
thatstoasty Sep 6, 2024
240d5bf
update installation instructions
thatstoasty Sep 6, 2024
33a2f63
remove build script
thatstoasty Sep 6, 2024
298c518
add build workflow
thatstoasty Sep 6, 2024
d28039b
redeploy linux
thatstoasty Sep 6, 2024
1a9b7b1
revert condition
thatstoasty Sep 6, 2024
522fdc6
lifetime cleanup
thatstoasty Sep 9, 2024
27c67b7
fix up bufio writer
thatstoasty Sep 11, 2024
4c89dfc
fix up bufio writer
thatstoasty Sep 11, 2024
764c9ae
tighten up bytes
thatstoasty Sep 12, 2024
8c95d96
tighten up bytes
thatstoasty Sep 12, 2024
efa5498
tighten up bytes
thatstoasty Sep 12, 2024
8d47ecb
tighten up bytes
thatstoasty Sep 12, 2024
62576f6
tighten up bytes
thatstoasty Sep 12, 2024
8a93da5
up max version
thatstoasty Sep 12, 2024
5403b1e
up max version
thatstoasty Sep 12, 2024
33d8f51
up to 0.1.6.nightly
thatstoasty Sep 12, 2024
a40ba1c
add missing files
thatstoasty Sep 12, 2024
cacfb18
lean on string slice to prevent copies
thatstoasty Sep 12, 2024
6e9253c
version up to 0.1.8
thatstoasty Sep 13, 2024
5651a59
version up to 0.1.8
thatstoasty Sep 13, 2024
d3fc1f1
revert to build in main
thatstoasty Sep 13, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 54 additions & 0 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
name: Build all packages

on:
push:
branches:
- main
workflow_dispatch:
pull_request:
branches:
- main
# on: ["push"]

jobs:
build:
strategy:
matrix:
include:
- { target: linux-64, os: ubuntu-latest }
# - { target: osx-arm64, os: macos-14 } # mac runner crashes on mojo test
fail-fast: false

runs-on: ${{ matrix.os }}
timeout-minutes: 5

defaults:
run:
shell: bash

steps:
- name: Checkout repo
uses: actions/checkout@v4

- name: Install magic
run: |
curl -ssL https://magic.modular.com | bash

- name: Build package for target platform
env:
TARGET_PLATFORM: ${{ matrix.target }}
PREFIX_API_KEY: ${{ secrets.PREFIX_API_KEY }}
CONDA_BLD_PATH: ${{ runner.workspace }}/.rattler

run: |
source $HOME/.bash_profile

# Temporary method to fetch the rattler binary.
RATTLER_BINARY="rattler-build-aarch64-apple-darwin"
if [[ $TARGET_PLATFORM == "linux-64" ]]; then RATTLER_BINARY="rattler-build-x86_64-unknown-linux-musl"; fi
curl -SL --progress-bar https://github.com/prefix-dev/rattler-build/releases/latest/download/${RATTLER_BINARY} -o rattler-build
chmod +x rattler-build

# Build and push
magic run build --target-platform=$TARGET_PLATFORM
magic run publish
46 changes: 31 additions & 15 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -1,23 +1,39 @@
name: Run Tests

on: ["push"]
on:
pull_request:
types: [opened, synchronize, reopened, ready_for_review]

permissions:
contents: read
pull-requests: read

jobs:
test:
runs-on: ubuntu-latest
environment: basic
name: with ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
# os: ["ubuntu-latest", "macos-14"] # mac tests just hang for some reason?
os: ["ubuntu-latest"]

runs-on: ${{ matrix.os }}
timeout-minutes: 5

defaults:
run:
shell: bash

steps:
- name: Check out repository code
uses: actions/checkout@v2
- name: Install dependencies
- name: Checkout repo
uses: actions/checkout@v4

- name: Install magic
run: |
curl https://get.modular.com | MODULAR_AUTH=${{ secrets.MODULAR_AUTH }} sh -
modular auth ${{ secrets.MODULAR_AUTH }}
modular install mojo
pip install pytest
pip install git+https://github.com/guidorice/mojo-pytest.git
- name: Unit Tests
curl -ssL https://magic.modular.com | bash

- name: Run tests
run: |
export MODULAR_HOME="/home/runner/.modular"
export PATH="/home/runner/.modular/pkg/packages.modular.com_mojo/bin:$PATH"
pytest
source $HOME/.bash_profile
magic run tests
magic run benchmarks
16 changes: 16 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -161,3 +161,19 @@ cython_debug/
# and can be added to the global gitignore or merged into this file. For a more nuclear
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
#.idea/

# pixi environments
.pixi
*.egg-info

# magic environments
.magic

# Rattler
output

# Mojo
**/*.mojopkg

# VSCode
.vscode
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ repos:
hooks:
- id: mojo-format
name: mojo-format
entry: mojo format -l 120
entry: magic run mojo format -l 120
language: system
files: '\.(mojo|🔥)$'
stages: [commit]
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,10 @@ and this project adheres to [Semantic Versioning](http://semver.org/).

## [Unreleased] - yyyy-mm-dd

## [0.1.8] - 2024-09-13

- Lot's of changes since Mojo 24.5. Sorry, I don't have a more granualar changelog!

## [0.0.2] - 2024-06-19

### Added
Expand Down
232 changes: 7 additions & 225 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,12 @@

Experiments in porting over Golang stdlib into Mojo and extra goodies that make use of it. It will not always be a 1:1 port, it's more so code inspired by the Golang stdlib and the Mojo community's code. This is not intended to be a full port, but rather a learning exercise and a way to experiment with Mojo's capabilities. Please feel free to contribute or use this as a starting point for your own projects! The codebase will remain in flux and will evolve with Mojo as future releases are created.

## Installation

1. First, you'll need to configure your `mojoproject.toml` file to include my Conda channel. Add `"https://repo.prefix.dev/mojo-community"` to the list of channels.
2. Next, add `gojo` to your project's dependencies by running `magic add gojo`.
3. Finally, run `magic install` to install in `gojo`. You should see the `.mojopkg` files in `$CONDA_PREFIX/lib/mojo/`.

## Projects that use Gojo

### My projects
Expand Down Expand Up @@ -46,231 +52,7 @@ All of these packages are partially implemented and do not support unicode chara

## Usage

Some basic usage examples. These examples may fall out of sync, so please check out the tests for usage of the various packages!

You can copy over the modules you want to use from the `gojo` or `goodies` directories, or you can build the package by running:
For `gojo`: `mojo package gojo -I .`

`bufio.Scanner`

```mojo
from tests.wrapper import MojoTest
from gojo.bytes import buffer
from gojo.io import FileWrapper
from gojo.bufio import Reader, Scanner, scan_words, scan_bytes


fn test_scan_words() raises:
var test = MojoTest("Testing scan_words")

# Create a reader from a string buffer
var s: String = "Testing this string!"
var buf = buffer.new_buffer(s)
var r = Reader(buf^)

# Create a scanner from the reader
var scanner = Scanner(r^)
scanner.split = scan_words

var expected_results = List[String]()
expected_results.append("Testing")
expected_results.append("this")
expected_results.append("string!")
var i = 0

while scanner.scan():
test.assert_equal(scanner.current_token(), expected_results[i])
i += 1
```

`bufio.Reader`

```mojo
from tests.wrapper import MojoTest
from gojo.bytes import buffer
from gojo.builtins.bytes import to_string
from gojo.bufio import Reader, Scanner, scan_words, scan_bytes, Writer
from gojo.io import read_all, FileWrapper
from gojo.strings import StringBuilder


fn test_read():
var test = MojoTest("Testing bufio.Reader.read")

# Create a reader from a string buffer
var s: String = "Hello"
var buf = buffer.new_buffer(s)
var reader = Reader(buf^)

# Read the buffer into List[UInt8] and then add more to List[UInt8]
var dest = List[UInt8](capacity=256)
_ = reader.read(dest)
dest.extend(String(" World!").as_bytes())

test.assert_equal(to_string(dest), "Hello World!")
```

`bytes.Buffer`

```mojo
from tests.wrapper import MojoTest
from gojo.bytes import new_buffer
from gojo.bytes.buffer import Buffer


fn test_read() raises:
var test = MojoTest("Testing bytes.Buffer.read")
var s: String = "Hello World!"
var buf = new_buffer(s)
var dest = List[UInt8](capacity=16)
_ = buf.read(dest)
dest.append(0)
test.assert_equal(String(dest), s)


fn test_write() raises:
var test = MojoTest("Testing bytes.Buffer.write")
var b = List[UInt8](capacity=16)
var buf = new_buffer(b^)
_ = buf.write(String("Hello World!").as_bytes_slice())
test.assert_equal(str(buf), String("Hello World!"))
```

`bytes.Reader`

```mojo
from tests.wrapper import MojoTest
from gojo.bytes import reader, buffer
import gojo.io


fn test_read() raises:
var test = MojoTest("Testing bytes.Reader.read")
var reader = reader.new_reader("0123456789")
var dest = List[UInt8](capacity=16)
_ = reader.read(dest)
dest.append(0)
test.assert_equal(String(dest), "0123456789")

# Test negative seek
alias NEGATIVE_POSITION_ERROR = "bytes.Reader.seek: negative position"
var position: Int
var err: Error
position, err = reader.seek(-1, io.SEEK_START)

if not err:
raise Error("Expected error not raised while testing negative seek.")

if str(err) != NEGATIVE_POSITION_ERROR:
raise err

test.assert_equal(str(err), NEGATIVE_POSITION_ERROR)
```

`io.FileWrapper`

```mojo
from tests.wrapper import MojoTest
from gojo.io import read_all, FileWrapper


fn test_read() raises:
var test = MojoTest("Testing FileWrapper.read")
var file = FileWrapper("tests/data/test.txt", "r")
var dest = List[UInt8](capacity=16)
_ = file.read(dest)
dest.append(0)
test.assert_equal(String(dest), "12345")
```

`io.STDWriter`

```mojo
from tests.wrapper import MojoTest
from gojo.syscall import FD
from gojo.io import STDWriter


fn test_writer() raises:
var test = MojoTest("Testing STDWriter.write")
var writer = STDWriter[FD.STDOUT]()
_ = writer.write_string("")
```

`fmt.sprintf`

```mojo
from tests.wrapper import MojoTest
from gojo.fmt import sprintf, printf


fn test_sprintf() raises:
var test = MojoTest("Testing sprintf")
var s = sprintf(
"Hello, %s. I am %d years old. More precisely, I am %f years old. It is %t that I like Mojo!",
String("world"),
29,
Float64(29.5),
True,
)
test.assert_equal(
s,
"Hello, world. I am 29 years old. More precisely, I am 29.5 years old. It is True that I like Mojo!",
)

s = sprintf("This is a number: %d. In base 16: %x. In base 16 upper: %X.", 42, 42, 42)
test.assert_equal(s, "This is a number: 42. In base 16: 2a. In base 16 upper: 2A.")

s = sprintf("Hello %s", String("world").as_bytes())
test.assert_equal(s, "Hello world")
```

`strings.Reader`

```mojo
from tests.wrapper import MojoTest
from gojo.strings import StringBuilder, Reader, new_reader
import gojo.io


fn test_read() raises:
var test = MojoTest("Testing strings.Reader.read")
var example: String = "Hello, World!"
var reader = new_reader("Hello, World!")

# Test reading from the reader.
var buffer = List[UInt8](capacity=16)
var bytes_read = reader.read(buffer)
buffer.append(0)

test.assert_equal(bytes_read[0], len(example))
test.assert_equal(String(buffer), "Hello, World!")
```

`strings.StringBuilder`

```mojo
from tests.wrapper import MojoTest
from gojo.strings import StringBuilder

fn test_string_builder() raises:
var test = MojoTest("Testing strings.StringBuilder")

# Create a string from the builder by writing strings to it.
var builder = StringBuilder()

for i in range(3):
_ = builder.write_string("Lorem ipsum dolor sit amet ")

test.assert_equal(
str(builder),
(
"Lorem ipsum dolor sit amet Lorem ipsum dolor sit amet Lorem ipsum dolor"
" sit amet "
),
)

```
Please check out the `test`, `examples`, and `benchmarks` directories for usage of the various packages!

## Sharp Edges & Bugs

Expand Down
Loading