One of the most frequent requests I got when I first added tables to package ui was background color manipoulation

• Windows: entire row, custom draw
• GTK+: per-cell; not sure if there's an alternate way (TODO)
• OS X: per-row (cells have spacing and other stuff so doing that won't look good)

So what do we do here?

For NSTableView there is setIntercellSpacing:

That intercell spacing is not what I am referring to. I am referring to the necessary spacing required for alignment purposes.

Okay, GTK+ only has per-cell backgrounds. So for background colors I'll provide something like

void uiTableSetBackgroundRowModelColumn(uiTable *t, int colNumber);

and that will provide the row color for the background.

That still leaves open the question of whether I should make a tree as part of this or separate — which will block development, so I need a consensus in the next 24 hours — and whether I use the Windows API list view, which can wait.

Leaning toward just having uiTree like GTK+ does and treating the separation between treeview and tableview as an implementation detail. This will reduce the amount of code libui has in it, but makes the simpler table case slightly more complex to work with.

Is there any chance you can link to docs or some really basic sample code / screenshots to illustrate the differences? I've been excited for table views for a little bit now, so I think there is a solid chance I will have an opinion, but I do not feel I have a solid enough grasp of the trade-off here to offer anything helpful.

Sorry, which trade off do you refer to?

The trade-offs between treating trees and tables separately or treating them as the same thing (the choice you are asking people to weigh in on).

I am not clear on what the benefits and negatives of each option are.

Facts: trees and tables have the same basic UI: you have multiple columns and rows of data. The only difference is that a tree can have expanding data

Pros: keeps libui smaller, gets rid of code reuse in libui, gives us two birds with one stone, and a tree model is flexible enough to double as a table model

Cons: slightly more complex to work with in the case of non-tree tables; might need flags to treat tables specially visually(? windows only?), will REQUIRE a custom control or complicated custom draw on Windows

In that case, I think it probably makes more sense, at least for now, to keep libui simpler and let the extra needed complexity be handled when it's necessary.

So you vote for having a combined widget? Or not?

Combined widget; sorry that wasn't clear.

Another issue that just came up is that I can't seem to find a way to ensure that a NSTableView is fully distinguishable from a NSOutlineView. Other than that unifying might work.

We'll see how I implement this. I'll start now. Thanks!

For the sake of accessibility I really favor separate controls and using native controls on each platform. On Windows making a treeview/listview hybrid accessible would require some form of UIA.
Not sure if you've already started the implementation here, sorry I missed this when it was open.

Since this is already closed, perhaps I'm too late with this feedback, but still, as a developer who particularly cares about accessibility, my suggestions are as follows:

1. Keep tables and trees separate, since these are distinct on Windows.
2. Please don't implement custom controls if you can help it, since that significantly increases the work you need to do to make them accessible. On Windows, the appropriate common controls are SysListView32 and SysTreeView32.

Staying with the comctl32 listview would limit listviews on all platforms somewhat: one checkbox, have to load images into an image list before they can be used.

Staying with the comctl32 treeview would limit treeviews on all platforms sorely: one column, one checkbox, one image, one text, have to load images into an image list before they can be used.

Do people want these limitations?

UI Automation is not an issue as libui requires Vista anyway. In fact, if I didn't change how table columns were prepared, I would only have needed to add accessibility to my wintable to use it as-is.

If you have a Windows table control that's close to working, and you're willing to implement a UI Automation provider for it, then it's fine with me if you do it that way. I just assumed using the SysListView32 control would be less work, particularly with regard to accessibility. But I didn't realize it had such serious limitations.

Do people want these limitations?

I think it makes sense to have basic functionality and go full Qt in a separate library.

I'm curious to know what the state of the table code is. Does the 'table' branch represent the latest work on it so far? What kind of state is it in? (I've just got a test program running on linux, and the basics seem solid enough so far... will try it on windows and mac tomorrow)

What are the next steps, and is there anything I can do to help?

The status is it kinda mostly works on both GTK+ and OS X and is completely unimplemented on Windows. I still don't know what to do about Windows, and I've been working on reworking uiDrawText() and friends for now.

Ahh yes, I'd forgotten how idiosyncratic the comctl controls were on windows. Sigh.

In the longer term, I don't think people will put up with the windows comctl limitations when the other platforms are so much more flexible. So I feel some form of custom control on windows is inevitable - presumably your wintable control.
Can you give us a quick high-level view of the current state of wintable? What needs to be done to get it ready to use in libui and is there anything I can help with?

Not sure this is helpful, but you may want to explore these grid implementations, if only to lo take a look at the way they were designed.

This is a pretty decent, powerful, and lightweight table/grid control.
Unfortunately it's written in Object Pascal.
https://github.com/Steema/TeeGrid

If you could build that with Lazarus, and if you could integrate it with the C binding as an OCX control, that would save you possibly thousands of lines of code it would take a full re-implementation.
Windows system controls are sometimes exposed as MSCOMM32.OCX after all.

Alternatively, the Lazarus code for its TGrid control may also be of help.
http://wiki.freepascal.org/Grids_Reference_Page

These guys in the Delphi/Turbo Pascal tradition have been designing grids for 20 years now.

@htrob That is non-free software, so it can't be used in this project (which I believe is MIT licensed).

@DemiMarie : FreePascal and Lazarus are Free Sofware. Lazarus comes with a Grid component.
TeeGrid isn't Free in the FSF sense, but I was offering inspiration on component design, not necessarily to reuse code. Besides, sometimes all it takes is a gentle email to the authors for them to see that their work could be adopted and be made popular if they educated themselves a bit regarding licences.
There's vast component design knowledge in that community and it's a shame to see it go to waste.
Just my two cents.

@andlabs: I've been gearing up to do a windows implementation of uiTable using the standard CommonControls owner-data ListView.
Obviously, there'd be limitations - essentially, it'd be limited to text and numeric columns. But I think this somewhat lowered lowest-common-denominator would still be useful enough to be worthwhile (it certainly covers everything I need).
The idea would be that eventually it can be dropped in favour of something more fully-featured.

So, some questions:

1. is this something you'd actually want, or is the prospect just too awful? ;-)
2. is the table branch still the right place to start, or is it just too stale now?
3. any other advice?

OK, I've started on my win32 table support. You can track the work-in-progress here: https://github.com/bcampbell/libui/tree/table
I've added a new example at example/table (there's also an example of table usage in the test suite - page 16).

Oh, sorry for forgetting to answer the questions.

1. The biggest problem for any implementation is accessibility, and retrofitting accessibility onto the comctl32 listview is going to be trickier since the comctl32 listview has a very different paradigm; either way, I was hoping to use UI Automation instead of MSAA. Plus, I had written my own table control a few years ago (the wintable repository), but that one has an entirely different design from this one, and I was going to just rewrite it (reusing what code I could) to make it fit, assuming I don't just use your code instead.

2. Yes.

3. Not that I can remember at the moment, though I may be thinking of stuff for uiImage (assuming I wrote that in the first place).

That being said, I'll definitely take a look at what you wrote so far; thanks!

On selection, yes, libui doesn't have every necessary function for every control just yet, but =P I'll read that markdown file later. Thanks in the meantime!

Progress update: I've added a speculative API for accessing the selected item(s).

I also added a flags parameter to uiNewTable to pass in any creation-time style flags. Currently the only one defined is to turn on multi-selection, but there are a few others I'm looking at adding (hide the header row, allow the user to resize columns, allow the user to reorder columns, etc etc...)

Lastly, I added uiTableOnSelectionChanged, to allow the user to set up a callback to monitor selection alterations.

Everything seems to work OK across the three current platforms, but I've not yet implemented uiTableOnSelectionChanged on OSX (my cocoa knowledge is a little on the lean side).

Feedback most welcome!

Just another quick update - I've just added OSX support for onSelectionChangedon my table branch.

That pretty much covers the real basic core stuff I think uiTable should handle.
The big thing missing is an API change on uiTableModel to add/update large numbers of rows without slowing things to a crawl with an extra full redraw for every row touched... Either some sort of begin/commit batched update notification, or a "just assume everything has changed" call on the model. The latter sounds simpler.

After that, if everyone's happy with the general API I've implemented, I'll start getting it into shape for merging.

I was going to say this when I am able to respond to the .md file (assuming you are updating it too), but:

The problem with an "update everything" is that last time I checked GTK+ has no such facility.

Which platform(s) do the redraw the row immediately on every individual row change thing? It sounds odd that that would be done, especially since the norm is to request a redraw for the next iteration through the main loop, and if you're issuing multiple such queries in one place...

All right. @bcampbell, this is a response to the TABLE.md in commit de88440dcd1d990ad693c431d79e0d9a6e5c22ac.

win32 comctl listview TODOs

• support editing of cells

This will require model changes; I forget if I already planned that out in the model. As for editing text, we'll need to make an edit control much like Windows itself does, which I'll need to figure out...

• how to determine column width? random sample of data?

LVM_SETCOLUMNWIDTH has some autosizing constants, but I'm not sure how they integrate with owner-draw or owner-data.

Is the parts system the right abstraction?

I'm not sure either, but it's the best abstraction that I can think of that would still be flexible. If I were to limit myself to what Windows itself is capable of doing, I'd be limited to text columns with images in each column OR checkboxes in the first one and images in others (maybe), and only the first column would be editable. Both GTK+ and Cocoa have a parts-like thing, though in GTK+ the most common approach is to only have one part ("cell renderer") per column. Do you have any better suggestions?

The parts API gives some of the flexiblity, but would likely get in the way if libui went the whole hog on custom item view layouts.

I'd rather have tables remain for columnar data. I do want to have a general-layout ListBox or List control of sorts, but that'll need to be entirely custom — the ListView is probably simply not cut out for things that Explorer doesn't need... I'm not sure though.

API change: support bulk invalidation of model
Currently, the only way to notify the control that new data has been added is via the uiTableModelRow[Inserted|Changed|Deleted] functions. This inevitably causes redraws for each item, which can get really slow for large numbers of items.

Are you calling these functions one at a time per main loop iteration? A well-designed GUI framework would merely queue the item rect for redraw if it's visible. If you call these functions multiple times per main loop iteration, those item rects will add together and accumulate into a single redraw for the whole bunch.

That being said, OS X does provide -[NSTableView/NSOutlineView beginUpdates] and -[NSTableView/NSOutlineView endUpdates] (primarily for animation, but IIRC certain methods require these to be called regardless; I forget now). Not sure about Windows (tricks with WM_SETREDRAW, maybe?) or GTK+, though. (Looking at the implementation of GtkListStore and GtkTreeStore might give insights, assuming they even do things in bulk at the API level, which I forget now as well.)

• sorting (eg clicking on various column headings to change the sort field and order. The user-supplied model needs to handle the sorting, the table control can't help.

Sorting — and filtering too, while we're at it — are interesting cases. I'm not sure how they would work on Windows (though the list view does have a few messages for sorting, I'm not sure how they work with owner-data). GTK+ has GtkTreeModelFilter and GtkTreeModelSort that do most of the work for us. I forget how I did filtering on OS X with my github.com/andlabs/ohv project, and I think NSObjectController has helpers for sorting, but again, I'm not sure.

I think a single uiTableModelAllChanged() function would be enough. The underlying control needs to know that it should to redraw everything visible (and the number of items might have chnaged).

As mentioned above, GTK+ does not support this, or at least v3.10 seems not to. Windows does, kinda (LVM_SETITEMCOUNT), and OS X does (-[NSTableView/NSOutlineView reloadData]).

API change: support flags/styles

• single or multiselect

Sure. Need to figure out a good API for this.

• hide column headers?

Maybe?

• allow column reordering?

Maybe? Not sure how reordering works on GTK+ or OS X (or what I have to change on Windows to make sure things still work, since the API requires either the column number or the column order in different places, and I forget how or where). (The parts system might be useful here, because the parts are the ones that talk to the model, IIRC.)

• allow column resizing?

Did I not enable this already? I forget now... Either way, the ability to disallow it would also be needed.

What's your current state of the code? Thanks again in the meantime!

I'm pretty happy about the code as it stands right now. The biggest deficiency is that the win32 version ignores most of the parts functions and only supports text-only columns.

Stuff I've added:

• an OnSelectionChange event/callback
• functions for iterating through the selected items
• uiTableCreate takes a new parametry, for style flags. Currently only used to pick between multi or single select, All platforms seem to support simple flags for:
  • allow the the user to resize columns
  • allow the the user to reorder columns
  • hide column headers
  • allow the user to click on column headers (would require an extra event)
    So I'll probably add these in too. I originally avoided making this a creation-time thing, but windows (I think) doesn't let you switch between single and multi select once the control has been created.

Regarding sorting:

I don't think that the control is in any position to do any sorting of it's own. To do so, it'd need to access every item in the model, which could get out of hand pretty quickly. Imagine the case where the model is backed by a large, on-disk database, for example - the app/model just keeps track of the sort field and 'ascending/descending' order and leaves the heavy lifting to the database.
So I think the sorting needs to be left up to the model side. All the table needs to do is allow for the user clicking on columns to pick the sort field and ordering (and a way to be told that the view needs redrawing).

Regarding bulk row updates:

The code I'm using to add rows is in my noddy little table example app here:
https://github.com/bcampbell/libui/blob/0e566386595aaca9bd54ee93dedc2e029535283c/examples/table/main.cpp#L138

From my very cursory tests (adding 100000 rows), OSX and windows seem to hold things back for the next update, and run in OK (although not lightning-quick) time, while the Gtk+version seems to run very slowly, and I can see the scrollbar shrinking as it runs...

I'll have a look through the various APIs and see if I can spot any common functionality relevant to bulk updating.
(for background: the app I want to use all this stuff for is a database-backed app with upwards of a million rows, with lots of bulk operations for adding, tagging and deleting etc, so that's my motiviation :- )

I'd rather have tables remain for columnar data. I do want to have a general-layout ListBox or List control of sorts, but that'll need to be entirely custom — the ListView is probably simply not cut out for things that Explorer doesn't need... I'm not sure though.

Just to illustrate the kinds of layout I was meaning:

(I don't think this one is implemented using comctl ListView, but I could be wrong).
This is from the windows 10 file explorer search results, but I've seen this kind of thing across loads of different platforms.
Either way, I don't really have any plans (or use) for this, but I figured it was the logical extension of what you started with the parts API, so it was worth bearing in mind.

I think my first choice would be to hold back the parts API and just limit uiTable to textual/numeric data only - in line with the lowest-common-denominator win32 implementation. But that's just me being selfish :- )
The alternatives are to a) keep the whole uiTable back (ugh) or b) document that the parts API doesn't work on windows (I'd be fine with this too).

(out of interest, here's a QML code example - it uses the delegate field to define the layout for a listview. I think it creates a pool of them an recycles them as you scroll https://wiki.qt.io/How_To_Use_QML_ListView )

(I don't think this one is implemented using comctl ListView, but I could be wrong).

Not anymore, though the code should be somewhat similar... That being said, that's all text and pictures, which are trivial to do using a custom layout and fixed-height items without wasting resources. Adding arbitrary controls is another story (one Explorer has an edge on in that it can access the Windows source code =P ). I guess listview could do it, maybe...

I think my first choice would be to hold back the parts API and just limit uiTable to textual/numeric data only - in line with the lowest-common-denominator win32 implementation. But that's just me being selfish :- )

That could work for now. Something for custom column formats would be ideal later. Especially since I want to do trees eventually too... (And in fact, I wonder if we should be doing this to the tree view instead; see also this.)

b) document that the parts API doesn't work on windows (I'd be fine with this too).

This could work too, but holding back anything beyond just the parts API and editing would be a situation I would not merge into master (so text-only tables should still work on Windows).

(out of interest, here's a QML code example - it uses the delegate field to define the layout for a listview. I think it creates a pool of them an recycles them as you scroll https://wiki.qt.io/How_To_Use_QML_ListView )

Is there a screenshot or runnable example of this?

Some notes to self:

• Drawing animated comtcl32-style progress bar controls: https://stackoverflow.com/questions/296857/animated-glow-in-owner-drawn-progress-bar-listview-datagridview
• List view custom draw and owner draw: https://stackoverflow.com/questions/36442894/does-owner-draw-only-work-in-report-mode-for-listview/36443401#36443401
• Maybe WTL does it too?

holding back anything beyond just the parts API and editing would be a situation I would not merge into master (so text-only tables should still work on Windows).

Yep, that sounds good to me. I don't mind either way about the current parts API (other than not supporting a lot of it under windows for now). I just wanted to make sure it'd been thought through.

re editing: I think the comctl listview handles string editing for you, without any messing about with creating your own controls... I'll check, and see if I can add editing support across the three platforms.

re: the QML example. Sorry, no screenshots, but If you've got Qt/QML installed there's a standalone qml executable which can run QML files. Anyway, I just posted it because it's a simple(ish) example of how it delegates the item display to a separate (and full-blown) widget layout. I like the concept, although when I tried to use it for a non-trivial app I found it slow and cumbersome (I only needed plain text cells!).

comctl32 listview only supports editing the first column (LVITEM.iSubItem == 0).

Cocoa does something similar to what QML does; I think you need to use nib files for it, but I'm not sure. It's a design that could work for libui's list box view thing, in any case.

comctl32 listview only supports editing the first column (LVITEM.iSubItem == 0).

Ha! Of course it does. Thanks Microsoft.
Sigh.
Windows is just positively riddled with stuff that edges tantalizingly close to competence but ends up having some fundamental flaw which renders it useless except in very specific, limited circumstances...

I think I'm recanting my position on uiTable not being able to handle sorting itself : -)

While I think it'd be ideal if the model could be kept nice and abstract and the listview hit the model as sparingly as possible, I think most assumptions are that the model is more-or-less kept in RAM. Fair enough.

So I think it probably makes sense for sorting to be an entirely view-oriented thing, and independent of the model. That is, sorting affects only the view of the model, not the underlying model itself.
I've had a peek at GTK, and it's default assumption is sorting the view actually sorts the model too (which isn't what we want, as multiple views can share a model). But it has a handy adaptor, GtkTreeModelSort which handles the sorting while leaving the underlying model unmolested. Looks nice and simple.

I've not yet looked at listview sorting in Cocoa or windows, but that'll be next on my todo list, so I'll report back here then.

Yes, GTK+ assumes sorting the view means sorting the model, but for GTK+ you don't actually alter the model. Instead, there's a type called GtkTreeModelSort that implements sorting for you — you hand it your regular tree model, and it wraps around your tree model, effectively mapping row numbers between your model and whatever sort is currently being used. Then you pass that to the GtkTreeView. Of course, this means that with selections and other view accesses, you'll need to go through the GtkTreeModelSort to get your underlying model's actual row number.

There is a similar facility for filtering out rows provided by GtkTreeModelFilter.

Ideally libui should provide similar abstractions to avoid needing to explicitly sort and filter data yourself; I'm not sure how to provide sort or filter predicates on Windows or macOS (unless I did it with ohv and forgot).

Mainly for my own reference - a Raymond Chen blog post about some of the underlying ad-hocness in the windows treeview: Tree view check boxes: A sordid history

@andlabs: by the way, I've been a bit busy on other stuff, but I'd like to do a push on getting the table stuff merged in over the next couple of months. Do you have any specific things that you regard as "must-haves" that are still missing from the stuff I've been doing? Or things you'd definitely want changed?
Or maybe it'd be better to keep the table stuff as a separate library for now? (I don't think there are any changes to the core libui. It all seems to apply fine on top of your utflib-and-attrstr branch).

That's why I was using branches to begin with =P I want to review everything you've done up to the point I'm ready to push before I push anything, since I'm not sure what you have done since the last time I commented.

And yes, I'm following the treeview checkbox articles closely.

Cool - let me know if there's anything I can do to make reviewing easier.

The last couple of commits I made (a couple of months back) were:

11a98bf (add OSX support for iterating over currently-selected items)
0e56638 (add OSX support for the onSelectionChanged callback)

This brings all three platforms up to par (barring the windows lack of support for non-text sub-widget parts).

The selection iteration stuff is probably the major thing you'd want to review - it's the biggest API addition. I've documented the functions in the header file with godoc-style comments.

The other API change is that uiNewTable now takes style flags. The only flag currently supported is MultiSelect, but there are a few others which would make sense (and which are required at creation time for some platforms).

Hope this is useful!

Hi @andlabs do you have any update on this being merged into master? I'd love to use it in the go bindings (I'm working on a linux music player app in golang and the table would be for showing the song list/browser)

No, but soon (next few days, perhaps?) I'll be done with utflib-and-attrstr as it currently stands and I'll start merging PRs afterward. This particular patch set will need to be reviewed, of course (just as all PRs do).

@jrgp: I did hack up some golang support for my table branch: https://github.com/bcampbell/ui/tree/table

If you decide to try it out, I'd love any feedback you have. It needs more people hammering on it to get the core API properly rounded out. And using the golang bindings will help exercise the underlying C api, so go nuts ;- )

My own use case for tables is pretty basic - I want to support columns of plain text (although I'd love hyperlink support), with good performance on datasets up to millions of rows. I don't need any other fancy stuff, so my work has been focused on those core features - and luckily, the crappy windows commctl table supports that. I figured it gets us up and running, and fancier things can be added once a better windows table control is available. Extra features would extend the libui API rather than change what's there already.

Replaced with #310.