I could imagine some fun tools making use of vl-convert in a terminal context where having a general config file would be nice (with semantics like { config: { ...userConfig, ...specConfig } }). I would imagine that the config would just be a regular config.

Hey all, thanks for working on this! Just to let you know, there is similar development effort going on at okab. It started as a way to provide a better altair-saver method I think. I've asked the author if he's interested in somehow joining forces, see this thread.

The thread started with my idea of using QuickJs javascript engine, which might also be relevant here. See also quickjs-rs. QuickJS does run the minified vega and vega-lite code.

What are your thoughts?

Continuation of the discussion that started from this comment and onwards.

Vega 5 specifications can currently be rendered using Altair, with support of interactivity:

import altair.vega.v5 as alt
import json
vg_spec = r"""
{
  "$schema": "https://vega.github.io/schema/vega/v5.json",
  "description": "A basic bar chart example, with value labels shown upon mouse hover.",
  "width": 400,
  "height": 200,
  "padding": 5,
  "data": [
    {
      "name": "table",
      "values": [
        {"category": "A", "amount": 28},
        {"category": "B", "amount": 55},
        {"category": "C", "amount": 43},
        {"category": "D", "amount": 91},
        {"category": "E", "amount": 81},
        {"category": "F", "amount": 53},
        {"category": "G", "amount": 19},
        {"category": "H", "amount": 87}
      ]
    }
  ],
  "scales": [
    {
      "name": "xscale",
      "type": "band",
      "domain": {"data": "table", "field": "category"},
      "range": "width",
      "padding": 0.05,
      "round": true
    },
    {
      "name": "yscale",
      "domain": {"data": "table", "field": "amount"},
      "nice": true,
      "range": "height"
    }
  ],
  "axes": [
    {"orient": "bottom", "scale": "xscale"},
    {"orient": "left", "scale": "yscale"}
  ],
  "marks": [
    {
      "type": "rect",
      "from": {"data": "table"},
      "encode": {
        "enter": {
          "x": {"scale": "xscale", "field": "category"},
          "width": {"scale": "xscale", "band": 1},
          "y": {"scale": "yscale", "field": "amount"},
          "y2": {"scale": "yscale", "value": 0}
        },
        "update": {"fill": {"value": "steelblue"}},
        "hover": {"fill": {"value": "red"}}
      }
    }
  ]
}
"""
alt.vega(json.loads(vg_spec))

Now, one can also use vl-convert (pip/conda install vl-convert-python) to render these as png or svg in a notebook:

import vl_convert as vlc
from IPython.display import Image
from IPython.display import SVG, display

#vg_png = vlc.vega_to_png(vg_spec)  # Image(vg_png)
vg_svg = vlc.vega_to_svg(vg_spec)
display(SVG(vg_svg))

But as far as I can tell it does not support the interactivity (btw just like a svg-export from the Vega Editor with above example).

Introducing errors in the spec, propagate JavaScript warnings back to the notebook, in both cases: Changing eg: "from": {"data": "table"}, to "from": {"data": "FOO"}, Results with Altair variant:

Javascript Error: Undefined data set name: "FOO"

And with vl-convert variant:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[19], line 6
      3 from IPython.display import SVG, display
      5 #vg_png = vlc.vega_to_png(vg_spec)  # Image(vg_png)
----> 6 vg_svg = vlc.vega_to_svg(vg_spec)
      7 display(SVG(vg_svg))

ValueError: Vega to SVG conversion failed:
Error: Undefined data set name: "FOO"
    at b (https://cdn.skypack.dev/-/[email protected]/dist=es2020,mode=imports,min/optimized/vega-util.js:1:324)
    at ue.getData (https://cdn.skypack.dev/-/[email protected]/dist=es2020,mode=imports,min/optimized/vega-parser.js:1:45069)
    at At (https://cdn.skypack.dev/-/[email protected]/dist=es2020,mode=imports,min/optimized/vega-parser.js:1:21597)
    at Sa (https://cdn.skypack.dev/-/[email protected]/dist=es2020,mode=imports,min/optimized/vega-parser.js:1:21499)
    at Bt (https://cdn.skypack.dev/-/[email protected]/dist=es2020,mode=imports,min/optimized/vega-parser.js:1:24270)
    at https://cdn.skypack.dev/-/[email protected]/dist=es2020,mode=imports,min/optimized/vega-parser.js:1:39099
    at Array.forEach (<anonymous>)
    at hn (https://cdn.skypack.dev/-/[email protected]/dist=es2020,mode=imports,min/optimized/vega-parser.js:1:39088)
    at al (https://cdn.skypack.dev/-/[email protected]/dist=es2020,mode=imports,min/optimized/vega-parser.js:1:39762)
    at Module.sl (https://cdn.skypack.dev/-/[email protected]/dist=es2020,mode=imports,min/optimized/vega-parser.js:1:47716)

Since it is not the core of Altair, we can decide for maintenance purposes to remove the Vega (currently v5) wrappers and leave this to vl-convert if people like to make use of this within a notebook environment.

Altair had until this commit support in the main version on Github support for the following Vega-Lite versions: Altair Version | Vega-Lite Version | Vega Version -- | -- | -- 4.3.0.dev0 | v3.4.0, v4.17.0, v5.2.0 | v5.21.0

Three versions of Vega-Lite seems too many, and we are still discussing if we can reduce it to only a single version of Vega-Lite. But the v3 version is obsolete anyway. These packages/tests can safely be removed from main.

I just noticed that this was a functionality of Altair saver as mentioned here https://stackoverflow.com/a/63103365/2166823. Is this something that would fit into vl-convert as well? I'ts more like a small convenience, but curious if this would be an option. Some of this functionality might already exist here https://github.com/altair-viz/altair_viewer

This code:

from vl_convert import vegalite_to_svg
from urllib.request import urlopen
import json
prefix = "https://raw.githubusercontent.com/vega/vega-lite/next/examples/specs/"
f = "sample_scatterplot.vl.json"
spec = json.loads(urlopen(prefix + f).read())
with open("test.svg", "w") as out:
  out.write(vegalite_to_svg(spec))

Yields this output for me (I uploaded the actual SVG so view-source as needed):

I'm on MacOS 11.6 on Apple Silicon, Python 3.10 installed by mamba

Overview

This PR adds support for converting Vega-Lite charts using named themes and custom config objects.

It also closes #18 by loading a default config object from a user settings directory (CLI only for now)

Rust updates

The Rust VlConverter struct now supports passing theme strings and config JSON objects to Vega-Lite conversion functions.

In addition, a new get_themes() function was added that returns a JSON object containing all of the built-in themes. This makes it possible to inspect the config object corresponding to built-in themes.

Python updates

The Python changes mirror the Rust changes exactly. There is a new get_themes function that returns a Python dict containing all of the built-in theme config objects.

CLI

The vl2* subcommands now accept --theme and --config options, where --theme expects a theme name and --config expects a path to a file containing a JSON object.

If a file exists at ~/.config/vl-convert/config.json, The CLI will use this path as the default value of the --config flag across all subcommands.

In addition, two new subcommands were added to get info on built in themes: ls-themes and cat-theme.

ls-themes

$ vl-convert ls-themes --help

List available themes

Usage: vl-convert ls-themes

Options:
  -h, --help  Print help information

Here is an example of listing the names of all available built-in themes.

$ vl-convert ls-themes

dark
excel
fivethirtyeight
ggplot2
googlecharts
latimes
powerbi
quartz
urbaninstitute
vox

cat-theme

$ vl-convert cat-theme --help

Print the config JSON for a theme

Usage: vl-convert cat-theme <THEME>

Arguments:
  <THEME>  Name of a theme

Options:
  -h, --help  Print help information

For example, print the config JSON associated with the built-in dark theme

$ vl-convert cat-theme dark

{
  "background": "#333",
  "title": {
    "color": "#fff",
    "subtitleColor": "#fff"
  },
  "style": {
    "guide-label": {
      "fill": "#fff"
    },
    "guide-title": {
      "fill": "#fff"
    }
  },
  "axis": {
    "domainColor": "#fff",
    "gridColor": "#888",
    "tickColor": "#fff"
  }
}

This PR updates the VlConverter to check Vega-Lite specs for a named theme specified in usermeta.embedOptions.theme. This is what vega-embed does, and this is where Altair places it's named theme.

This will make named these work with Altair without any Altair changes.

See background in https://github.com/altair-viz/altair/issues/781

This is to handle the case where Intl.DateTimeFormat().resolvedOptions().timeZone is undefined in Deno.

I ran into this running VlConvert on Binder, where Intl.DateTimeFormat().resolvedOptions().timeZone is undefined and this caused get_local_tz to crash.

This updates deno to the latest version and resvg to 0.27.0. There is more to do to update to 0.28.0 (https://github.com/RazrFalcon/resvg/blob/master/CHANGELOG.md), so holding off on that for now.

Add a get_local_tz function to the Rust and Python API that returns the named timezone of the Deno runtime (which Vega is inherently using for timezone math).

When a font specification includes a font family quoted in double quotes, image export crashes the Python kernel.

import vl_convert as vlc
spec = {
  "$schema": "https://vega.github.io/schema/vega-lite/v5.json",
  "layer": [
    {
      "data": {
        "name": "df",
        "values": [{"a": 1, "b": 1}, {"a": 2, "b": 3}, {"a": 3, "b": 2}]
      },
      "layer": [
        {
          "transform": [],
          "layer": [
            {
              "mark": {"type": "line"},
            }
          ],
          "encoding": {
            "x": {
              "field": "a",
              "type": "quantitative"
            },
            "y": {
              "field": "b",
              "type": "quantitative"
            },
          }
        }
      ]
    }
  ],
  "config": {
    "style": {
      "guide-label": {
        "font": "Arial, sans-serif"
      },
      "guide-title": {
        "font": "Arial, sans-serif"
      }
    }
  }
}

# Works ok
svg = vlc.vegalite_to_svg(spec)

# Update font strings to quote Arial. Crashes kernel
spec["config"]["style"]["guide-label"]["font"] = "\"Arial\", sans-serif"
spec["config"]["style"]["guide-label"]["font"] = "\"Arial\", sans-serif"
svg = vlc.vegalite_to_svg(spec)

A little more info in the stack trace is available when running the example in Rust:

thread '<unnamed>' panicked at 'Failed to parse text SVG: ParsingFailed(ParserError(InvalidAttribute(InvalidSpace(73, TextPos { row: 3, col: 54 }), TextPos { row: 3, col: 54 })))', vl-convert-rs/src/text.rs:187:59
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
fatal runtime error: failed to initiate panic, error 5
Fatal Python error: Aborted

The first thing we should do is update error handling to not panic and crash the kernel on SVG parse failure. Then we should fix the bug 😄

Here's a little test case... the first one works and the other two don't:

from vl_convert import vegalite_to_svg
from urllib.request import urlopen
import json
prefix = "https://raw.githubusercontent.com/vega/vega-lite/next/examples/specs/"
files = ["arc_donut.vl.json", "point_binned_color.vl.json", "line_color_binned.vl.json"]
for f in files:
  spec = json.loads(urlopen(prefix + f).read())
  vegalite_to_svg(spec)

I'm on MacOS 11.6 on Apple Silicon, Python 3.10 installed by mamba

I noticed that enabling the data server transformer causes vl convert to hang indefinitely. Is this something that is fixable or is vl-convert meant to be used with vega-fusion instead? Since we are planning to have vl-convert take precedence over altair_saver, it would be great if it also worked with the data server transformer like altair_saver does to ensure full backwards compatibility.

Example that hangs for me on vl-convert 0.5.0 and altair 4.2.0, python 3.10.x

import pandas as pd
import altair as alt

alt.data_transformers.enable('data_server')
source = pd.DataFrame({
    'a': ['A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I'],
    'b': [28, 55, 43, 91, 81, 53, 19, 87, 52]
})
chart = alt.Chart(source).mark_bar().encode(
    x='a',
    y='b'
)


# Saving the chart
import vl_convert as vlc

png_data = vlc.vegalite_to_png(chart.to_dict())
with open("chart.png", "wb") as f:
    f.write(png_data)

Given this python snippet:

import vl_convert as vlc
from IPython.display import Image
vl_spec = '''
{
  "$schema": "https://vega.github.io/schema/vega-lite/v5.json",
  "data": {
    "values": [
      {"a": "J", "b": 28, "c": "I am more than 18 characters"},
      {"a": "B", "b": 55},
      {"a": "C", "b": 43},
      {"a": "D", "b": 91},
      {"a": "E", "b": 81},
      {"a": "F", "b": 53},
      {"a": "G", "b": 19},
      {"a": "H", "b": 87},
      {"a": "I", "b": 52}
    ]
  },
  "mark": "bar",
  "encoding": {
    "x": {"field": "a", "type": "nominal"},
    "y": {"field": "b", "type": "quantitative"},
    "color": {"field": "c"}
  }
}
'''
png_data = vlc.vegalite_to_png(vl_spec=vl_spec)
Image(png_data)

Will return the following truncated legend label :

Where the vega-editor render this as: And in editor: Open the Chart in the Vega Editor

The default labelLimit within legend is 160 pixels, but this seems not being respected and changing this value has no positive effect on the image.

Would it be possible to release this package as well on conda-forge? That would make it a bit more accessible for users:

conda/mamba install vl-convert-python