I would appreciate it very much if anyone would like to design a logo for rlsl as I am not a very good artist myself.

I am also working on a GPU path tracer with a friend, maybe I can create some nice looking fractals in the future that could be used as a logo.

error: internal compiler error: librustc/infer/canonical.rs:695: failed to lift QueryResult { var_values: CanonicalVarValues { var_values: ['?0, '?1, '?2] }, region_constraints: [Binder(OutlivesPredicate('?0, '?0)), Binder(OutlivesPredicate('?0, '?0)), Binder(OutlivesPredicate('?0, '?0)), Binder(OutlivesPredicate('?0, '?0)), Binder(OutlivesPredicate(ReStatic, '?2))], certainty: Proven, value: NormalizationResult { normalized_ty: Input<'_> } }, canonicalized from QueryResult { var_values: CanonicalVarValues { var_values: ['_#0r, '_#1r, '_#2r] }, region_constraints: [Binder(OutlivesPredicate('_#0r, '_#3r)), Binder(OutlivesPredicate('_#0r, '_#4r)), Binder(OutlivesPredicate('_#3r, '_#0r)), Binder(OutlivesPredicate('_#4r, '_#0r)), Binder(OutlivesPredicate(ReStatic, '_#2r))], certainty: Proven, value: NormalizationResult { normalized_ty: Input<'_> } }

Currently glslang decorates a struct with BufferBlock. But BufferBlock has been deprecated.

BufferBlock Deprecated (use Block-decorated StorageBuffer Storage Class objects). Apply to a structure type to establish it is an SSBO-like shader-interface block.

The spec recommends to use the StorageBuffer storage class, but this requires the following extensions SPV_KHR_storage_buffer_storage_class, SPV_KHR_variable_pointers

Only ~30% of the gpus support those extensions pre 1.1, but they have been made core in 1.1.

Now what should rlsl target? BufferBlock or StorageBuffer or both?

SPIR-V only allowes to return values by copy. #[inline(always)] in Rust doesn't seem to force inlining.

This is a needed feature because a lot of things in Rust like ops::Index and ops::Deref return a reference.

A temporary workaround would be to implement an intrinsic for accessing elements in a storage buffer.

A possible implementation could be to transform functions from

fn index(&self, idx: usize) -> &Foo{..}

to

fn index(&self, idx: usize, output: &mut Foo){..}

Possibly those functions would need to be annotated with something like

#[spirv(transform_ref)]
fn index(&self, idx: usize) -> &Foo{..}

Another possibility would be to force inlining, by inserting those functions directly into the mir where they are used.

Hi @MaikKlein, this project looks super exciting, please keep up the great work!

I'm currently working on a creative coding framework and would love to allow users to use something like RLSL in the future for graphics and compute. I'd like to contribute at some point, but would appreciate it if you could clarify your plans for the licensing of RLSL first. FWIW, I'd love something close to the license used by Rust itself, but this is of course just my opinion!

Requirements

Standard Uniform Buffer Layout

Every member of an OpTypeStruct with storage class of Uniform and a decoration of Block (uniform buffers) must be laid out according to the following rules:

• The Offset decoration must be a multiple of its base alignment.

• Any ArrayStride or MatrixStride decoration must be an integer multiple of the base alignment of the array or matrix from above.

• The Offset decoration of a member must not place it between the end of a structure or an array and the next multiple of the base alignment of that structure or array.

• The numeric order of Offset decorations need not follow member declaration order.

reference

These also then follow the rules from std140. Also mentioned in 14.5.4.

Additionally from the SPIR-V spec

Composite objects in the UniformConstant, Uniform, and PushConstant Storage Classes must be explicitly laid out. The following apply to all the aggregate and matrix types describing such an object, recursively through their nested types:

• Each structure-type member must have an Offset Decoration.

• Each array type must have an ArrayStride Decoration.

• Each structure-type member that is a matrix or array-of-matrices must have be decorated with

  • a MatrixStride Decoration, and

  • one of the RowMajor or ColMajor Decorations.

  • The ArrayStride, MatrixStride, and Offset Decorations must be large enough to hold the size of the objects they affect (that is, specifying overlap is invalid). Each ArrayStride and MatrixStride must be greater than zero, and no two members of a given structure can be assigned to the same Offset.

Meaning

RLSL needs to explicit decorate members of a struct with the correct offset. That offset needs to follow the rules defined in std140, and this only applies to PushConstant and Uniform.

Implementation

Idea 1

RLSL can easily generate the correct offset from the rules from std140 but then the Rust and RLSL code would get out of sync.

One idea would be to only check if the struct is conformant with the rules from std140. Then RLSL would just extract the offset that was defined by the user.

Generally annotation of structs doesn't seem reasonable, for example you don't want your Vec3<f32> to be 16 bytes wide in the vertex input. Either accept the performance hit, or create separate types for the all the uniforms.

Idea 2

#[derive(Packed)]
pub struct SomeStruct{
    pub v: Vec4<f32>,
    pub f: f32
}

// generated
fn compute_packed(&self) -> Packed<SomeStruct>{..}

Compute the correct layout inside RLSL and implement Packed in normal Rust as a custom derive.

It would then know how to serialize the specified struct with the rules of std140. And Packed could look like this

struct Packed<T>{
    _m: PhantomData<T>,
    data: Vec<u8>
}

// Hopefully the max size can be computed statically
struct Packed<T: Uniform>{
    _m: PhantomData<T>,
    data: [u8; T::Size]
}

For example if you have

let uniforms: Vec<SomeStruct> = ..;

And if you want to send it to the shader you have to pack them correctly

for packed_struct in uniforms.iter().map(SomeStruct::compute_packed){
    // pseudo api
    some_shader.send(packed_struct);
    //draw
}

And inside RLSL you can just use SomeStruct directly.

I am unsure how exactly I could implement it because custom derive doesn't expose type information but I think there should be a way. I might have to use const_fn. Alternatively everything could be computed in a build script with the reflection api.

How do you currently handle alignment? What would be your preferred API?

Currently this is how PerVertex is implemented

#[spirv(vertex)]
fn vertex(vertex: &mut Vertex, pos: Vec4<f32>, color: Vec4<f32>) -> Vec4<f32> {
    vertex.position = pos;
    color
}

An alternative would be to output the PerVertex information

#[spirv(vertex)]
fn vertex(pos: Vec4<f32>, color: Vec4<f32>) -> (Vertex, Vec4<f32>) {
    let mut vertex = Vertex::new(..); 
    vertex.position = pos;
    (vertex, color)
}

Then there are other builtin variables

in int gl_VertexID;
in int gl_InstanceID;
in int gl_DrawID; // Requires GLSL 4.60 or ARB_shader_draw_parameters
in int gl_BaseVertex; // Requires GLSL 4.60 or ARB_shader_draw_parameters
in int gl_BaseInstance; // Requires GLSL 4.60 or ARB_shader_draw_parameters

#[spirv(VertexInput)]
pub struct VertexIn{
     instance_id: i32,
     ....
}

#[spirv(vertex)]
fn vertex(input: VertexIn, pos: Vec4<f32>, color: Vec4<f32>) -> (Vertex, Vec4<f32>) {
    let vertex = Vertex::new(..);
    vertex.position = pos;
    (vertex, color)
}

layout(location = 0) in vec2 a_Pos;

void test(inout vec2 v) {
    v = vec2(1,1);
}

void main() {
    vec2 v = a_Pos;
    test(v);
    gl_Position = vec4(v, 0.0, 1.0);
}

glslang always creates a new variable for a param, and then directly passes that variable.

        %v_0 = OpVariable %_ptr_Function_v2float Function
      %param = OpVariable %_ptr_Function_v2float Function
         %29 = OpLoad %v2float %a_Pos
               OpStore %v_0 %29
         %30 = OpLoad %v2float %v_0
               OpStore %param %30
         %31 = OpFunctionCall %void %test_vf2_ %param
         %32 = OpLoad %v2float %param
               OpStore %v_0 %32

This is because variables are always mutable, and changes inside a function would affect the value of the params.

I assume if one param was marked as inout glslang just stores the value of the param in the variable that was marked inout

         %32 = OpLoad %v2float %param
               OpStore %v_0 %32

In rlsl I don't pass variables directly, I only pass the load.

         %23 = OpVariable %_ptr_Function__ptr_Function_float Function
      %coord = OpVariable %_ptr_Function_v3float Function
         %26 = OpVariable %_ptr_Function_float Function
         %27 = OpVariable %_ptr_Function_v3float Function
         %30 = OpVariable %_ptr_Function__ZN4core6marker11PhantomData Function
               OpBranch %16
         %16 = OpLabel
               OpStore %f %float_1
               OpStore %23 %f
         %32 = OpLoad %_ptr_Function_float %23
         %33 = OpFunctionCall %void %_ZN6shader8test_mut %32

If something is marked &mut, I'll just pass the variable directly instead of passing the loaded value.

The current syntax is just a proof of concept but it is missing a lot of information. Two options come to mind

We use a macro. This macro then generates some code under the hood so that RLSL can retrieve the necessary information.

vertex!(
    input(location=0) pos_in: Vec4<f32>,
    input(location=1) color_in: Vec4<f32>,
    uniform(set = 0, binding = 0) foo: Foo,
    output(location=0, flat) mut color: Vec4<f32>,
    output(location=1) mut color: Vec4<f32>, // smooth default
    output(location=2, noperspective) mut color2: Vec4<f32> {
    }
)

Because macros are severally limited, so we pass everything explicitly with types. Alternatively I could generate many custom attributes with a proc_macro.

The more explicit alternative would be.

type VertexOut =(Output<Flat, 0, Vec4<f32>>,
                 Output<1, Vec4<f32>>,
                 Output<NoPersp, 2, Vec4<f32>>);

#[spirv(vertex)]
fn vertex(pos_in: Input<0, Vec4<f32>>,
          color_in: Input<1, Vec4<f32>>,
          foo: Descriptor<0, 0, Foo>) -> VertexOut {

}

I think under the hood, I need to pass types anyways. Also currently Rust doesn't have const generics so the code above would look like

type VertexOut =(Output<Flat, N0, Vec4<f32>>,
                 Output<N1, Vec4<f32>>,
                 Output<NoPersp, N2, Vec4<f32>>);

#[spirv(vertex)]
fn vertex(pos_in: Input<N0, Vec4<f32>>,
          color_in: Input<N1, Vec4<f32>>,
          foo: Descriptor<N0, N0, Foo>) -> VertexOut {

}

Where the constants will be defined like this

#[spirv(CONST0)]
pub struct N0;
#[spirv(CONST1)]
pub struct N1;
#[spirv(CONST2)]
pub struct N2;
...

I am currently in favor of the function like syntax with explicit types.

Note: I also want do redesign PerVertex which is why it doesn't appear in this issue.

What are your thoughts? Any feedback is greatly appreciated. Of course if you have a completely different idea, I would like to hear it.

rlsl currently still tries to link with the main

error: linking with `cc` failed: exit code: 1
  |
  = note: "cc" "-Wl,--as-needed" "-Wl,-z,noexecstack" "-m64" "-L" "/home/maik/.rlsl/compiler/lib/rustlib/x86_64-unknown-linux-gnu/lib" "/home/maik/projects/rlsl/rlsl-example/target/debug/deps/rlsl_example-0ed607f60771a702.1y16o1qfye96o7m0.rcgu.o" "-o" "/home/maik/projects/rlsl/rlsl-example/target/debug/deps/rlsl_example-0ed607f60771a702" "/home/maik/projects/rlsl/rlsl-example/target/debug/deps/rlsl_example-0ed607f60771a702.crate.allocator.rcgu.o" "-Wl,--gc-sections" "-pie" "-Wl,-z,relro,-z,now" "-nodefaultlibs" "-L" "/home/maik/projects/rlsl/rlsl-example/target/debug/deps" "-L" "/home/maik/.rlsl/compiler/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,-Bstatic" "/home/maik/projects/rlsl/rlsl-example/target/debug/deps/librlsl_math-b58264b8e9da687f.rlib" "/home/maik/.rlsl/compiler/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-57fdceb1c52b4db2.rlib" "/home/maik/.rlsl/compiler/lib/rustlib/x86_64-unknown-linux-gnu/lib/libpanic_unwind-4f85ba5d0e870e29.rlib" "/home/maik/.rlsl/compiler/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc_jemalloc-a655730befa35987.rlib" "/home/maik/.rlsl/compiler/lib/rustlib/x86_64-unknown-linux-gnu/lib/libunwind-c86c9565da689e14.rlib" "/home/maik/.rlsl/compiler/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc_system-655151fba596847e.rlib" "/home/maik/.rlsl/compiler/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-b8f9bb8294d9a014.rlib" "/home/maik/.rlsl/compiler/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-513d34708cb20443.rlib" "/home/maik/.rlsl/compiler/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd_unicode-5211f032242a5357.rlib" "/home/maik/.rlsl/compiler/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-e2f49b08d2bc06b5.rlib" "/home/maik/.rlsl/compiler/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-136e26942e0df602.rlib" "-Wl,-Bdynamic" "-l" "dl" "-l" "rt" "-l" "pthread" "-l" "pthread" "-l" "gcc_s" "-l" "c" "-l" "m" "-l" "rt" "-l" "pthread" "-l" "util" "-l" "util"
  = note: /usr/lib/gcc/x86_64-pc-linux-gnu/7.2.1/../../../../lib/Scrt1.o: In function `_start':
          (.text+0x20): undefined reference to `main'
          collect2: error: ld returned 1 exit status

rlsl depends on the nightly compiler and therefore requires a specific version of it. I don't want to update the nightly compiler every week or so just to make rlsl build.

The problem is targeting specific nightlies is a bit difficult with rustup. I am currently experimenting with a custom CI and an install script that downloads the artefacts for the user.

• spir-v differentiates between runtime arrays and static arrays

• Slices can't be supported in rlsl at the moment

• Things like len on a static array currently maps to Slice::len() in Rust, which can't be expressed.

• Const generics aren't a thing yet.

Because slices are not supported, we need to abstract over arrays with generics

fn foo(arr: &impl Array<f32>) { .. }

Have you given any thought on how the binding model from the CPU part of rust could look like? I have been thinking about this and wanted to ask what would you think to have something along the lines of this on the GPU side:

#[spirv(DescriptorLayout)]
struct InputData
{
	#[spirv(ShaderVisible_PS ...)]
	diffuse_textures: &[Texture],  //dynamic (un)bounded array slice
	
	#[spirv(ShaderVisible_ALL ...)]
	some_constant: u32,
	
	...
}

#[spirv(Interpolants)]
struct Interpolants
{
	color : vec3,
	uv: vec2,
	#[spirv(NoInterpolation)]
	index: u32,
}

#[spirv(vertex)]
fn vertex_shader(..., input: InputData) -> Interpolants
{ ... }

#[spirv(OutputMerger)]
struct OutputData
{
	#[spirv(Blendmode_Additive, ...)]
	color : vec3
}

#[spirv(fragment)]
fn fragment_shader(... , interp: Interpolants, input: InputData) -> OutputData
{
 	let color = input.diffuse_textures[interp.index].Sample(interp.uv);
 	...
}

#[spirv(ShaderPipeline)]
struct VSPSPipeline
{
	descriptor: InputData
	blendmode: OutputData
	vertex: typeof(vertex_shader)
	pixel: typeof(pixel_shader)
}

and some code similar to this on the CPU to create and launch a draw:

fn main()
{	
	let some_textures : Vec<Texture> = get_textures();
	let some_constant = 42;
	let input_data = InputData::new(some_textures , some_constant);

	let rendertarget: RenderTarget<vec3> = get_rendertarget()
	let output_data = OutputData::new(rendertarget);
	
	let pipeline = VSPSPipeline::new(input_data, output_data, vertex_shader, fragment_shader);
	
	cmd_buffer.set_pipeline(pipeline);
	cmd_buffer.draw_indexed(...);
}

One reason behind this is that I would like to have a typesafe interface between CPU and GPU maybe even go further down the road of having units of measure and types like NormalMapTexture to bind against. And the other reason would be that I am really really sick of writing all that CPU binding code which is super super boring. I am not attached to any particular syntax maybe you have another proposal?

Right now everything inside a bin file is one shader module. I think this is relatively convenient but there may be better ways, especially since we need to add configuration for each shader module. See also https://github.com/MaikKlein/rlsl/issues/67

What is the best way to tell rlsl which extensions should be enabled? We still need to stay compatible with rustc/cargo.

Also extensions should be enabled per shader module, and not the whole project. That means we also need a way to make it easy to share the configuration between different shader modules.

Most extensions should be trivial to support. All features should be enabled by default and a simple pass can check which features are currently used in a shader module, and outputs an error if the extension hasn't been enabled, but it used.