Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Big interpreter rewrite #5406

Draft
wants to merge 82 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
82 commits
Select commit Hold shift + click to select a range
9d1c413
Comment most things, towards interpreter rewrite
StachuDotNet Jul 20, 2024
ac475b3
Delete (unused) ProgramTypesAst.fs
StachuDotNet Jul 20, 2024
1295b47
Rewrite basics of new (bytecode) interpreter
StachuDotNet Jul 25, 2024
06236ce
A few more exprs
StachuDotNet Jul 31, 2024
545ee99
no-op commentary/thinking
StachuDotNet Jul 31, 2024
1b3c045
handle EString in new interpreter
StachuDotNet Aug 1, 2024
af8a95b
handle EDict
StachuDotNet Aug 2, 2024
37f8f58
implement EIf in new interpreter
StachuDotNet Aug 6, 2024
aae717a
tidy PT2RT
StachuDotNet Aug 6, 2024
30ce430
support tuples in new interpreter
StachuDotNet Aug 6, 2024
56c0d85
Pre-work towards package fn calls in new interpreter
StachuDotNet Aug 7, 2024
c2f3f6e
Handle LPTuple in new interpreter
StachuDotNet Aug 12, 2024
ed141e7
Handle match in new interpreter
StachuDotNet Aug 15, 2024
5532b19
Update new interpreter to be loop, not recursive
StachuDotNet Aug 15, 2024
4995d05
(no-op) tidy interpreter
StachuDotNet Aug 15, 2024
b29cdd1
(new interpreter) tidy list- and dict- creation
StachuDotNet Aug 20, 2024
ab40164
Interpreter rewrite: tests, RTEs, let exprs
StachuDotNet Aug 21, 2024
e6168e7
Work on RTEs, Enums, Records; uncomment many Builtins
StachuDotNet Aug 27, 2024
a0097e1
More interpreter rewrite
StachuDotNet Sep 9, 2024
4c12a96
goodbye symtable
StachuDotNet Sep 12, 2024
b5d1ca4
uncomment more, since symtable removal
StachuDotNet Sep 12, 2024
7c92a8b
match works again
StachuDotNet Sep 12, 2024
688bdd6
Starts to apply named fns; partial app works; pkgs barely started
StachuDotNet Sep 13, 2024
51be1a9
trying to call frames
StachuDotNet Sep 13, 2024
b1ea257
we can call functions now
OceanOak Sep 16, 2024
6fc96f2
added recursion tests
OceanOak Sep 16, 2024
0bfe82c
lambdas seem to work
StachuDotNet Sep 16, 2024
c31df32
we tested lambdas
StachuDotNet Sep 16, 2024
b31b39f
tidy
StachuDotNet Sep 16, 2024
0b656e3
record updates work (basic)
StachuDotNet Sep 16, 2024
3497553
EInfix works
StachuDotNet Sep 17, 2024
7358ad0
Uncomment many Builtins
OceanOak Sep 17, 2024
292c2e4
constants work
StachuDotNet Sep 17, 2024
4357388
pipes work
StachuDotNet Sep 17, 2024
32f97aa
uncomment some of LibParser
StachuDotNet Sep 17, 2024
38be0c9
uncomment PT2ST
StachuDotNet Sep 18, 2024
4d1e23d
Add more interpreter and PT2RT tests
OceanOak Sep 18, 2024
6c78a10
uncomment more builtins
StachuDotNet Sep 18, 2024
2d17efb
make test faster
StachuDotNet Sep 18, 2024
c210b5b
uncommented LibPackageManager
StachuDotNet Sep 18, 2024
cecacd1
PM Tests running
StachuDotNet Sep 18, 2024
b5e6622
some LibParser tests are uncommented and working
StachuDotNet Sep 18, 2024
ab7f303
localexec uncommented; many packages being loaded again
StachuDotNet Sep 18, 2024
4f54d37
uncomment many packages
OceanOak Sep 19, 2024
41991cb
rename something
StachuDotNet Sep 19, 2024
95d6e04
tidied RuntimeTypes
StachuDotNet Sep 19, 2024
ddc51fe
More RT tidying
StachuDotNet Sep 19, 2024
6a8a893
Build ProdExec (largely commented)
StachuDotNet Sep 19, 2024
2924561
QW is 'building'
StachuDotNet Sep 19, 2024
5db0234
all projects building
StachuDotNet Sep 20, 2024
0ca256d
Uncomment more of the codebase
StachuDotNet Sep 20, 2024
510f071
Move some Builtin functions to package space
OceanOak Sep 20, 2024
c590547
Get some of LibExe tests running; note one failing test
StachuDotNet Sep 20, 2024
9c24686
700 tests running+passing now
StachuDotNet Sep 21, 2024
a8f4996
1200/9000 tests passing
StachuDotNet Sep 21, 2024
e0808a3
1400/9000 tests passing
OceanOak Sep 23, 2024
86fef94
1800 tests passing
StachuDotNet Sep 23, 2024
aba7ee3
commit simplest failing test for fn issue
StachuDotNet Sep 26, 2024
4c11b8b
PT2RT: fix symbols in EMatch, Interpreter: remove some usage of Map.f…
OceanOak Oct 1, 2024
1276835
3000 tests passing
OceanOak Oct 2, 2024
ba3510b
Bring back some BuiltinDarkInternal fns
StachuDotNet Oct 28, 2024
aaa5d38
lots of work around RTEs, type-checking, Builtins, Dval-creation
StachuDotNet Oct 31, 2024
cd9360e
Fix impl of constant-eval
StachuDotNet Oct 31, 2024
e802b1a
kinda bring back a few .fs test files
StachuDotNet Oct 31, 2024
e2e1190
some minor tidying
StachuDotNet Oct 31, 2024
e4864b4
bring back altjson tests
StachuDotNet Oct 31, 2024
0f77da0
format
StachuDotNet Oct 31, 2024
3e20e26
fix type-checking of record updates
StachuDotNet Nov 4, 2024
e793bb4
misc tidyings to get PR closer to mergeable
StachuDotNet Nov 4, 2024
53ce353
Some work around type-checking, esp related to records and fn-calling
StachuDotNet Nov 5, 2024
50678db
fix some httpclient testing
StachuDotNet Nov 5, 2024
c4bd61f
slowly bringing more .fs tests back
StachuDotNet Nov 5, 2024
bef8433
misc tidyings to interpreter
StachuDotNet Nov 5, 2024
4f54ffb
DvalCreator result and option helpers now TC consistently
StachuDotNet Nov 5, 2024
6e5a067
bring back some user dbs, secrets, call stacks, bwdserver
StachuDotNet Nov 13, 2024
ed37375
bring back more tests
StachuDotNet Nov 13, 2024
9f767ab
bring back routing tests
StachuDotNet Nov 13, 2024
19d80bb
allow run-backend-server to run QW and CC again
StachuDotNet Nov 13, 2024
e54ea40
remove comment
StachuDotNet Nov 13, 2024
42ee977
bring back DvalReprDeveloper in a limited capacity
StachuDotNet Nov 13, 2024
280f32c
minor tidyings
StachuDotNet Nov 13, 2024
e21381a
wip
StachuDotNet Nov 14, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -44,4 +44,5 @@ deploy-lock-manual-deploy

clis/

.mono
.mono
.fake
16 changes: 8 additions & 8 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -335,14 +335,14 @@ ENV NUGET_SCRATCH=/tmp/NuGetScratch
# Emscripten,
# for compiling the tree-sitter parser to wasm
#############
RUN git clone https://github.com/emscripten-core/emsdk.git --depth 1 \
&& cd emsdk \
# TODO pin to a recent stable version (i.e. 3.1.37)
# we are using the latest version because Linux arm64 binaries aren't available in all releases.
# see: https://github.com/emscripten-core/emscripten/issues/19275
&& ./emsdk install latest \
&& ./emsdk activate latest
ENV PATH="$PATH:/home/dark/emsdk/upstream/emscripten"
# RUN git clone https://github.com/emscripten-core/emsdk.git --depth 1 \
# && cd emsdk \
# # TODO pin to a recent stable version (i.e. 3.1.37)
# # we are using the latest version because Linux arm64 binaries aren't available in all releases.
# # see: https://github.com/emscripten-core/emscripten/issues/19275
# && ./emsdk install latest \
# && ./emsdk activate latest
# ENV PATH="$PATH:/home/dark/emsdk/upstream/emscripten"


#############
Expand Down
298 changes: 61 additions & 237 deletions backend/fsdark.sln

Large diffs are not rendered by default.

Binary file modified backend/serialization/toplevels-binary-latest.bin
Binary file not shown.
48 changes: 48 additions & 0 deletions backend/serialization/toplevels-binary-pretty-latest.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

10 changes: 5 additions & 5 deletions backend/src/BuiltinCli/Libs/Directory.fs
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ open FSharp.Control.Tasks

open Prelude
open LibExecution.RuntimeTypes
module VT = ValueType
module VT = LibExecution.ValueType
module Dval = LibExecution.Dval
module Builtin = LibExecution.Builtin
open Builtin.Shortcuts
Expand All @@ -20,7 +20,7 @@ let fns : List<BuiltInFn> =
description = "Returns the current working directory"
fn =
(function
| _, _, [ DUnit ] ->
| _, _, _, [ DUnit ] ->
uply {
let contents = System.IO.Directory.GetCurrentDirectory()
return DString contents
Expand All @@ -41,7 +41,7 @@ let fns : List<BuiltInFn> =
let resultOk r = Dval.resultOk KTUnit KTString r |> Ply
let resultError r = Dval.resultError KTUnit KTString r |> Ply
(function
| _, _, [ DString path ] ->
| _, _, _, [ DString path ] ->
try
System.IO.Directory.CreateDirectory(path)
|> ignore<System.IO.DirectoryInfo>
Expand All @@ -64,7 +64,7 @@ let fns : List<BuiltInFn> =
let resultOk r = Dval.resultOk KTUnit KTString r |> Ply
let resultError r = Dval.resultError KTUnit KTString r |> Ply
(function
| _, _, [ DString path ] ->
| _, _, _, [ DString path ] ->
try
System.IO.Directory.Delete(path, false)
resultOk DUnit
Expand All @@ -83,7 +83,7 @@ let fns : List<BuiltInFn> =
description = "Returns the directory at <param path>"
fn =
(function
| _, _, [ DString path ] ->
| _, _, _, [ DString path ] ->
uply {
// TODO make async
let contents =
Expand Down
6 changes: 3 additions & 3 deletions backend/src/BuiltinCli/Libs/Environment.fs
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ open FSharp.Control.Tasks
open Prelude
open LibExecution.RuntimeTypes

module VT = ValueType
module VT = LibExecution.ValueType
module Dval = LibExecution.Dval
module Builtin = LibExecution.Builtin
open Builtin.Shortcuts
Expand All @@ -22,7 +22,7 @@ let fns : List<BuiltInFn> =
"Gets the value of the environment variable with the given <param varName> if it exists."
fn =
(function
| _, _, [ DString varName ] ->
| _, _, _, [ DString varName ] ->
let envValue = System.Environment.GetEnvironmentVariable(varName)

if isNull envValue then
Expand All @@ -43,7 +43,7 @@ let fns : List<BuiltInFn> =
"Returns a list of tuples containing all the environment variables and their values."
fn =
(function
| _, _, [ DUnit ] ->
| _, _, _, [ DUnit ] ->
let envVars = System.Environment.GetEnvironmentVariables()

let envMap =
Expand Down
9 changes: 5 additions & 4 deletions backend/src/BuiltinCli/Libs/Execution.fs
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ let fns : List<BuiltInFn> =
returnType = TCustomType(Ok executionOutcomeTypeName, [])
fn =
(function
| _, _, [ DString command ] ->
| _, _, _, [ DString command ] ->
let command =
command.Replace(
"$HOME",
Expand All @@ -36,15 +36,16 @@ let fns : List<BuiltInFn> =
let cmdName, cmdArgs =
if RuntimeInformation.IsOSPlatform OSPlatform.Windows then
"cmd.exe", $"/c {command}"
// TODO: run in whatever the default shell is -- not just bash.
else if
RuntimeInformation.IsOSPlatform OSPlatform.Linux
|| RuntimeInformation.IsOSPlatform OSPlatform.OSX
then
// TODO: run in whatever the default shell is -- not just bash.
"/bin/bash", $"-c \"{command}\""
else
"Executing CLI commands is not supported for your operating system (Linux, Windows, or Mac not detected)"
|> raiseUntargetedString
Exception.raiseInternal
"Executing CLI commands is not supported for your operating system (Linux, Windows, or Mac not detected)"
[]

let psi =
System.Diagnostics.ProcessStartInfo(
Expand Down
20 changes: 10 additions & 10 deletions backend/src/BuiltinCli/Libs/File.fs
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ let fns : List<BuiltInFn> =
let resultOk = Dval.resultOk (KTList(ValueType.Known KTUInt8)) KTString
let resultError = Dval.resultError (KTList(ValueType.Known KTUInt8)) KTString
(function
| _, _, [ DString path ] ->
| _, _, _, [ DString path ] ->
uply {
try
let path =
Expand Down Expand Up @@ -53,7 +53,7 @@ let fns : List<BuiltInFn> =
let resultOk = Dval.resultOk KTUnit KTString
let resultError = Dval.resultError KTUnit KTString
(function
| _, _, [ DList(_, contents); DString path ] ->
| _, _, _, [ DList(_, contents); DString path ] ->
uply {
try
let path =
Expand All @@ -65,7 +65,7 @@ let fns : List<BuiltInFn> =
do!
System.IO.File.WriteAllBytesAsync(
path,
Dval.DlistToByteArray contents
Dval.dlistToByteArray contents
)
return resultOk DUnit
with e ->
Expand All @@ -84,7 +84,7 @@ let fns : List<BuiltInFn> =
description = "Deletes the file specified by <param path>"
fn =
(function
| _, _, [ DString path ] ->
| _, _, _, [ DString path ] ->
uply {
try
System.IO.File.Delete path
Expand Down Expand Up @@ -112,7 +112,7 @@ let fns : List<BuiltInFn> =
let resultOk = Dval.resultOk KTUnit KTString
let resultError = Dval.resultError KTUnit KTString
(function
| _, _, [ DString path; DString content ] ->
| _, _, _, [ DString path; DString content ] ->
uply {
try
do! System.IO.File.AppendAllTextAsync(path, content)
Expand All @@ -136,7 +136,7 @@ let fns : List<BuiltInFn> =
let resultOk r = Dval.resultOk KTString KTString r |> Ply
let resultError r = Dval.resultError KTString KTString r |> Ply
(function
| _, _, [ DUnit ] ->
| _, _, _, [ DUnit ] ->
try
let tempPath = System.IO.Path.GetTempFileName()
resultOk (DString tempPath)
Expand All @@ -156,7 +156,7 @@ let fns : List<BuiltInFn> =
"Returns true if the file specified by <param path> is a directory, or false if it is a file or does not exist"
fn =
(function
| _, _, [ DString path ] ->
| _, _, _, [ DString path ] ->
uply {
try
let attrs = System.IO.File.GetAttributes(path)
Expand All @@ -179,7 +179,7 @@ let fns : List<BuiltInFn> =
"Returns true if the file specified by <param path> is a normal file (not a directory), or false if it is a directory or does not exist"
fn =
(function
| _, _, [ DString path ] ->
| _, _, _, [ DString path ] ->
uply {
try
let attrs = System.IO.File.GetAttributes(path)
Expand All @@ -204,7 +204,7 @@ let fns : List<BuiltInFn> =
"Returns true if a file or directory exists at the specified <param path>, or false otherwise"
fn =
(function
| _, _, [ DString path ] ->
| _, _, _, [ DString path ] ->
uply {
try
let exists =
Expand All @@ -229,7 +229,7 @@ let fns : List<BuiltInFn> =
let resultOk r = Dval.resultOk KTInt64 KTString r |> Ply
let resultError r = Dval.resultError KTInt64 KTString r |> Ply
(function
| _, _, [ DString path ] ->
| _, _, _, [ DString path ] ->
try
let fileInfo = System.IO.FileInfo(path)
resultOk (DInt64 fileInfo.Length)
Expand Down
2 changes: 1 addition & 1 deletion backend/src/BuiltinCli/Libs/LanguageServerProtocol.fs
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ let fns : List<BuiltInFn> =
description = "Reads a single incoming request from an LSP client, over stdin."
fn =
(function
| _, _, [ DUnit ] ->
| _, _, _, [ DUnit ] ->
let tryReadHeader (input : StreamReader) =
let mutable contentLength = None
let mutable complete = false
Expand Down
Loading