As mentioned in #539, I started playing around with deriving Getable/Pushable. The current implementation for Getable works quite well for normal enums, but does not work properly with lifetimes yet (which make a lot of sense for things like &str). I'm not sure how I can marshal generic parameters (looking at things like api::generic::* I assume there it's possible somehow).

Pushable should be a lot easier to implement because the data needs to be copied anyway, meaning there are no lifetimes getting in the way.

I'm using gluon to add some user functionality to a program, like this:

process_file {
  // do some built in processing
  // IF no script was specified at the command line, stop here.
  // create new gluon vm
  // create user library
  // compile and run gluon script => gluon script exports some callbacks
  // repeatedly call the gluon callbacks
  // vm should be freed at this point
}

When process_file is called for >100 files without a script, the memory usage stays at ~10MB all the time (measured using the time command). When a script is specified, the memory usage at the end of the program (according to valgrind) is >100MB. When the 100 files are specified multiple times at the command line, the memory usage rises accordingly. i.e., 4 times these 100 files, 400MB memory used at the end.

Where to look for the memory leak?

This is a somewhat working prototype of #653.

Still missing:

• [x] docs
• [ ] tests
• [x] fix broken tests
• [x] add breaking changes to commit messages
• [x] there's a bug in read_file_to_end I must have introduced in the last few commits
• [ ] I'm honestly not exactly sure what gluon's multithreading model is; right now, buffered IO definitely has a few race conditions

Closes #653

This is causing timeout woes with travis and makes things hard for users on stable. As discussed on https://github.com/Marwes/gluon/issues/33#issuecomment-233157725 and https://github.com/Marwes/gluon/pull/66#issuecomment-234803447 we could possibly consider switching to a parser generator like rustpeg of LALRPOP.

Hi folks!

I've been evaluating gluon and have a documentation question. In the features list there's this promise:

Simple embedding - Marshalling values to and from gluon requires next to no boiler plate allowing functions defined in Rust to be directly passed to gluon.

That sounds great! I've read the embedding API materials, have read through the API docs and have been poking around in the test code in this repository but can't figure out how you do this. In particular, I want to define a struct in rust, pass it to a function in gluon for modification and then continue on with my modified struct. I sort of see how part of that's done with with example:

fn factorial(x: i32) -> i32 {
    if x <= 1 { 1 } else { x * factorial(x - 1) }
}
let vm = new_vm();
vm.define_global("factorial", factorial as fn (_) -> _)
    .unwrap();
let (result, _) = Compiler::new()
    .run_expr::<i32>(&vm, "example", "factorial 5")
    .unwrap();
assert_eq!(result, 120);

in that .run_expr::<i32>(&vm, "example", "factorial 5") shows how a function with an argument gets called but if I wanted to pass a more complicated structure through like

struct Point {
  x: i64,
  y: i64,
  meta: HashMap<String, String>,
}

then I'm left at a loss. If I wanted to create a Point in rust and then pass that to a gluon function which would double both x and y and then add a pair "gluon": "very useful!" to the Point, how would I do that?

Thanks!

This PR implements row polymorphic records which should make records much more flexible at the expense of some performance.

Left to do:

• [x] Add comments
• [x] Handle polymorphic records in the vm
• [x] Fix pretty printing of polymorphic records
• [x] Add tests to test row kinds specifically
• [x] Improve row polymorphism related error messages

Fixes #92

@brendanzab mentioned putting the FnvMap alias elsewhere, the map appears to be most used in base/src though. I'm open to any suggestions.

All tests pass.

Fixes #95

Description

Adds a monad-transformer interface Transformer, a State monad transformer StateT, and a Lazy monad transformer LazyT to the standard library.

Related PRs

Lays groundwork for Streamlike and the new parser #687.

Status

Merged

Outstanding design questions

• [x] ~~Can and should StateT be replaced by the effects system?~~ (not for the time being, at least)
• [x] ~~Should Transformer require monad field?~~ (Basing monads on their transformers is the better option)
• [ ] Should StateT and LazyT (and other monad transformers) be merged with their non-transformer counterparts?

To Do

• [x] ~~Fix StateT's implicit parameter bugs~~ (#688 & #689)
• [x] ~~add tests~~
• [ ] add inline docs

Prior Art

• StateT
  • Haskell
  • Idris
  • Implementing Applicative (<*>) for StateT
• MonadTrans
  • Haskell
  • Idris

Will tackle #340 as well as some other related things with global variables (see e26c12803c4f691ae895d5176c356ec8cd4c08bd).

🚲 🏠

• What should string_prim, etc be called?
• Should we hide them away with more obviously internal names or lock them away with something like @NO-IMPLICIT-PRELUDE?
• array, reference, lazy are full modules meant to be used. Should we perhaps put make them loadable as import! "std.array" etc? (Probably)
• Since we have non-file modules now. Should we call import! as import! "std.bool" instead of import! "std/bool.glu"?
  • import! "std.bool" vs import! std.bool?

MaybeError is used to represent results for functions which may 'panic' (in the context of gluon not Rust). MaybeError is a really terrible name for this however so renaming it is a good idea.

Possible names:

enum Panicking<T, E> { }
enum Panic<T, E> { }
enum Diverging<T, E> { }
// More?

Let the bikeshedding comence!

This fixes the record-restructuring bug in the repl, where records are treated like tuples and let {foo} = {bar = “bar”, foo = “foo”} followed by foo displays ”bar”.

I couldn’t access any of the existing GcPtr<DataStruct>::get_field or Data::lookup_field methods from the repl crate, so I added one to RootedValue. It makes things consistent since there is already a get method that takes an index.

There are a few FIXME notes where I could use some advice on what to do.

let vm = gluon::new_vm();
vm.get_database_mut().set_implicit_prelude(true);
vm.run_io(true);

vm.run_expr::<()>("ex", r#"
  let x = 23
  ()
"#);

vm.run_expr::<i32>("ex", r#" x "); 

This gets me an error of "UndefinedVariable".

If I want do something "repl-like" where we define something in one command, then reference it in the next, is that possible in gluon ?

I tried reading https://github.com/gluon-lang/gluon/tree/master/repl/src but then got confused when it loads up https://github.com/gluon-lang/gluon/blob/master/repl/src/repl.glu

I'm trying to get a gluon REPL-ish to run on Chrome/wasm. Thus, I can't rely on the "io" stuff for now.

I found the function to convert a List to an Array (of), but it seems there is no function to do the reverse? It would be useful to have it in the std library, e.g.:

let list_to_arr l = 
    match l with
        | Nil -> [] 
        | Cons head tail -> append [head] (list_to_arr tail)

Or (with tail call & accumulator):

let list_to_array l = 
    let to_array acc l =
        match l with
            | Nil -> acc
            | Cons head tail -> to_array (append acc [head]) tail
    to_array [] l

Btw, does gluon do any tail call optimization? :)

I'm asking because I'm working with a lot of long arrays and I need to often sort them. The only sort function I found in std is for lists, but I need sort_by_key for arrays.

I tried doing

fn sort_by_key<T, K: std::cmp::Ord>(mut arr: Vec<T>, f: &dyn Fn(&T) -> K) -> Vec<T> {
	arr.sort_by_key(f);
	arr
}
// ...
ExternModule::new(vm, record! {
    sort_by_key => primitive!(2, sort_by_key),
    // ...
})

but I got all kinds of errors because of the generics and because the Fn can't cross the FFI boundary. Is there any way to make it work? :)

Then I tried to write sort_by_key for arrays in the repl, like this (based on sort for List):

let { append, Array } = import! std.array
let { List, of, ? } = import! std.list

let list_to_array l = 
    let to_array acc l =
        match l with
            | Nil -> acc
            | Cons head tail -> to_array (append acc [head]) tail
    to_array [] l

let sort_by_key arr key : forall a k. [Ord k] -> Array a -> (a -> k) -> Array a =
    let prelude @ { Ordering, ? } = import! std.prelude
    let { Semigroup, Monoid, Eq, Show } = prelude
    let { Functor, Applicative, Alternative, Monad } = prelude
    let { Foldable } = import! std.foldable
    let { Traversable } = import! std.traversable
    let { Bool } = import! std.bool
    let array @ { ? } = import! std.array
    let { (<>) } = import! std.semigroup
    let { compare } = import! std.cmp
    let { map } = import! std.functor
    let { (<*>), wrap } = import! std.applicative
    let { (<|>) } = import! std.alternative
    let { List, of, ? } = import! std.list
    rec let scan compare xs less equal greater : (a -> Ordering)
            -> List a
            -> List a
            -> List a
            -> List a
            -> (List a, List a, List a)
        =
        match xs with
        | Nil -> (less, equal, greater)
        | Cons y ys ->
            match compare y with
            | LT -> scan compare ys (Cons y less) equal greater
            | EQ -> scan compare ys less (Cons y equal) greater
            | GT -> scan compare ys less equal (Cons y greater)
    in
    rec
    let sort xs : /*[Ord a] ->*/ List a -> List a =
        match xs with
        | Nil -> Nil
        | Cons pivot ys ->
            let (less, equal, greater) = scan (\a -> compare (key a) (key pivot)) ys Nil (Cons pivot Nil) Nil
            sort less <> equal <> sort greater
    list_to_array (sort (of arr))

sort_by_key [{ t = 1., k = "a" }, {t = 0., k = "b" }] (\e -> e.t)

It works but it's quite slow. Ideally I'd want to use the FFI because I'm working with large arrays of thousands of items and the sorting should be fast if possible :)

When I try to use gluon on Wasm (in a web frontend using trunk serve), I get this runtime error:

Uncaught TypeError: Failed to resolve module specifier "env". Relative references must start with either "/", "./", or "../".

I'm using the latest version with these features:

gluon = { git = "https://github.com/gluon-lang/gluon", default-features = false, features = ["random"] }
gluon_codegen = { git = "https://github.com/gluon-lang/gluon" }

(When I don't call the function that uses gluon, the frontend works fine without errors.)

Any idea why it's not working? :)

In my use case, after evaluating a script, the final value can be one of several types that I want to support. E.g. I want to support 5 different custom types that the script can produce. The final value of the script should not be an enum though! So after evaluating a script, how can I check which gluon type the final value is, before I try to convert it to the equivalent Rust type? :) Or can I evaluate a script and then try converting the final value to a Rust type, to check if it succeeds?

Is it possible to disallow file/network access for sandboxing purposes? :)

The only way to interface with the outside world should be through the FFI that the host provides.

(Related to my use case https://github.com/gluon-lang/gluon/issues/936)

Bonus question: How can I capture stdout/stderr output so that I can send it over the network? (E.g. as two std channels that receive each line as it gets printed, or (if that's not possible) two Strings that contain the accumulated output at the end.)