I am new to Rust, using this project to learn more.

Basic questions:

• Do you take the computational graph and then create one kernel on the fly?
• How do you create a simple 2D (or 3D) array with some data?

Hi, I've been trying to extend descent to be able to create a U-Net (as in https://arxiv.org/abs/1505.04597) with it.

A U-Net ist basically a bunch of convolutional layers that get concatenated with previous versions of their input.

I implemented the following two new operations in array.rs:

Upsampling: https://github.com/apexys/descent/blob/61415476ffaeda734b841e11b533092691c569b4/src/array.rs#L833-L848

pub fn upsample(self, x_grow_factor: usize, y_grow_factor: usize) -> Self{
    let (a, da) = self.into_inner();
    let input_shape = a.shape();
    assert_eq!(input_shape.len(), 4);
    assert_eq!(a.shape(), da.shape());
    let a_reshaped = a.reshape([input_shape[0], input_shape[1], 1, input_shape[2], 1, input_shape[3]]);
    let da_reshaped = da.reshape([input_shape[0], input_shape[1], 1, input_shape[2], 1, input_shape[3]]);
    let a_broadcasted = a_reshaped.broadcast([input_shape[0], input_shape[1], y_grow_factor, input_shape[2], x_grow_factor, input_shape[3]]);
    let da_broadcasted = da_reshaped.broadcast([input_shape[0], input_shape[1], y_grow_factor, input_shape[2], x_grow_factor, input_shape[3]]);
    let mut output_shape = input_shape;
    output_shape[2] *= y_grow_factor;
    output_shape[1] *= x_grow_factor;
    let a_backshaped = a_broadcasted.reshape(output_shape);
    let da_backshaped = da_broadcasted.reshape(output_shape);
    (a_backshaped, da_backshaped).into()
}

Cropping: https://github.com/apexys/descent/blob/61415476ffaeda734b841e11b533092691c569b4/src/array.rs#L850-L874

pub fn crop(self, left: usize, top: usize, right: usize, bottom: usize) -> Self{
    let (a, da) = self.into_inner();

    let input_shape = a.shape();
    assert_eq!(input_shape.len(), 4);

    let mut input_offsets: TinyVec<[isize; MAX_DIM]> = std::iter::repeat(0).take(input_shape.len()).collect();
    input_offsets[1] = top as isize;
    input_offsets[2] = left as isize;

    let mut output_shape = input_shape;
    output_shape[1] -= top + bottom;
    output_shape[2] -= left + right;

    let view = View{
        input_shape: a.shape(),
        input_offsets,
        output_mapping: (0..input_shape.len()).map(|i| input_shape.identity_mapping(Axis::from_index(i))).collect(),
        output_shape
    };

    let b = a.view(view);
    let db = da.view(view);
    (b, db).into()
}

The U-Net is then created as a recursive struct, where each layer just applies two Conv-Operations and if it's not the innermost layer, also a cropped and upsampled version of the inner layer, which is then concatenated to the output and goes through another double convolution.

Here's just the execution part, full code is at https://github.com/apexys/descent_unet_example/blob/b828ce8c4a034d9288f316c108a72229b163f493/src/main.rs

fn eval<'s>(&self, input: DualArray<'s>, ctx: &EvalContext) -> DualArray<'s> {
    let x = input.apply(&self.conv1, ctx);
    let x = x.apply(&self.conv2, ctx);
    let x = if let Some((pool, inner)) = self.inner.as_ref() {
        let [_, h_outer, w_outer, _]: [usize; 4] = x.shape().try_into().unwrap();

        let x_inner = x.apply(pool, ctx);
        let x_inner = inner.eval(x_inner, ctx);
        let [_, h_inner, w_inner, _]: [usize; 4] = x_inner.shape().try_into().unwrap();
        let x_inner = x_inner.upsample(
            w_outer.div_ceil(w_inner),             
            h_outer.div_ceil(h_inner)
        );
        let [_, h_inner, w_inner, _]: [usize; 4] = x_inner.shape().try_into().unwrap();
        assert_eq!(h_inner, w_inner);
        let left = (w_inner - w_outer) / 2;
        let right = (w_inner - w_outer) - left;
        let top = (h_inner - h_outer) / 2;
        let bottom = (h_inner - h_outer) - top;
        let x_inner = x_inner.crop(
            left, top, right, bottom
        );
        x.concat(x_inner, -1)
    } else {
        x
    };
    let x = x.apply(&self.conv3, ctx);
    let x = x.apply(&self.conv4, ctx);
    x
}

When I create the graph as in I run into the following problem:

Somewhere in the creation of the graph, a Mov-Operation is created, but no inputs are connected to it.

I've added logging after every stage of the optimization process, but the error seems to be there from the start (see the upper right Node n295 in https://github.com/apexys/descent_unet_example/blob/main/svgs/after_build_clusters.svg).

I've also tried isolating the problem and it seems that my upsample and crop operations are "fine", at least I couldn't get the error to show up with just them, but they might still cause problems further down the line.

Do you have an idea of what might cause this?

Thank you for your help!