The Tunes Interfaces Subproject

Automatic Interface Generation


In Fare's words, we must build, for text, graphical, audio, ... UIs,
some automatic (but programmable) interface generators for structures (that is, if I define a structure, automatically generate the behavior of the structured editor for it). Let's take basic constructors lists, vectors, meta-constructors
First, in order to get some feel for what sort of machine intelligence will be necesary for this task, it would be helpful to consider what sort of structures one might want to get automatically-generated behavior for. Probably anything you could put a sort of type on would be a valid candidate. And this would then lead to question the sort of type system Tunes will have.... Perhaps, if this type system will be "cool" enough, then an object's type alone will be enough information to figure out how to intelligently display it to the user. The exact definition of type and object, and if this will be feasible, is yet to be seen.

Once this is done, we must first develop, for each UI "mode" that we wish to support, some basic UI elements that can be relied on at higher levels. Such elements include, as Fare points out, lists and vectors, but also probably some more basic things, such as those pertaining to text and graphics. (In fact, such basic things are very necesary. One can't wish to edit a list of nothing-in-particulars!)

...

Fare's draft of a note:

To me, automatic interface generation would be done as a particular case of dynamic reflective programming with "epsilon" constructs: at some point in your program you tell the system "Here I want an integer", and the system must somehow provide you an integer. Of course, you can be more precise about it: you can add constraints about what objects needn't or shouldn't be in the scope of the system to find the integer; you could add a real-time constraint as for how much time you can wait for the integer, etc. You can also give the system hints and meta-hints about how it should find the integer. For instance, if you tell it "find a large prime integer to make a new RSA key", or "give me the age of the shipmaster", or "give me the position of the text in the scrolling window", this might lead to completely different behaviors.

How the system finds the integer from your specifications is up to it; but of course, the more precise your idea about what should be done, the more precise your hints. The idea is that you should never have to put more into it than you want, and should never have to put less either. You can say exactly what you want.

  • For instance, if you want a scrollbar, ask a scrollbar; but don't draw the rectangle yourself, let the system do it; you don't care if the scrollbar looks like Athena, or Motif or Windows 3.x or Win95, or MacOS, or another widget set, or any combination thereof, whatever; some users might care, so don't waste your time at working against them.
  • Also, you might want not to be even as precise as to ask for a scrollbar. After all, what you want is that the (large) text be interfaced. The blind, as well as many other people, might want a voice-based interface, that has its own "scrolling" mechanism to interface a large text. Why prevent your fine program from being used without a screen? Is a screen essential to it? Perhaps you do have good ideas to interface your program specifically to a screen; but then, Tunes allows you to specify these ideas a part from your program itself, by (partially) specifying the epsilon-filling tactics for it.

    One way to understand programming with epsilons (Epsilon stands Hilbert's (ain't it he?) epsilon symbol for a witness for an existential predicate) is to view them as goals in the classical paradigm of higher-order logic programming (see languages like Prolog, Mercury, Lambda Prolog, Goedel, etc); an epsilon is a hole in the well-founded construction of a computable object; the system thus uses tactics to fill the hole; these tactics are made of rules that may include holes themselves, hence generating filling of the subholes as subgoals.

    The main differences of Tunes epsilon calculus with classical logic programming is that tactics can be dynamically defined from existing tactic constructs and libraries, and they are first-class objects, themselves subject to manipulation, etc. Also, there is a guaranteed constraint that should they succeed, they should return an object that rigorously matches the requested specification; hence, whereas Prolog programs only have a global semantics that can only be deduced from the totality of rules, Tunes tactics can have a nice local semantics with local definitions.

    If O is an object with a hole e, then O[E/e] where E is a compliant tactic to fill hole e is a valid object and specialization for O. E may dynamically generate holes itself, etc.

    the tactics are constrained to return a result that respect the specification for the requested object, instead of being pure arbitrary tactics in a program with global semantics only.

    As for interfaces proper, here is how they could be formalized:

    You can add criteria on the above objects for the computability/acceptability/complexity/whatever of displays and interfaces.
    By having informational and energetical models of the above objects, the relative utility of interfaces could be evaluated: a probability distribution for expected objects in T would allow to evaluate the information contained in the output, the complexity of the output-decoding function g and of the interaction driving functions ()->s , (s1,s2)->t1 relatively to the admitted cultural background would allow to evaluate the ease-of-use of the interface; space cost and time delay for all the human and computer functions involved could be taken into account; etc.
    Not that all these should actually be systematically evaluated (this might be a good master thesis for a research student), but this shows that computers CAN express a lot more than people would generally have them do. As for the large indetermination in the above utility-related concepts, of course it shouldn't be neglected. Epsilon constructs are precisely there to help handle such indetermination...
    Up to the Interfaces Subproject
    Up to the Tunes Home Page