The Tunes Interfaces Subproject
Automatic Interface Generation
In Fare's words, we must build, for text, graphical, audio, ... UIs,
some automatic (but programmable) interface generators for structures (that
is, if I define a structure, automatically generate the behavior of the
structured editor for it). Let's take basic constructors lists, vectors,
meta-constructors
First, in order to get some feel for what sort of machine intelligence will be
necesary for this task, it would be helpful to consider what sort of
structures one might want to get automatically-generated behavior for.
Probably anything you could put a sort of type on would be a valid candidate.
And this would then lead to question the sort of type system Tunes will
have.... Perhaps, if this type system will be "cool" enough, then an object's
type alone will be enough information to figure out how to intelligently
display it to the user. The exact definition of type and object, and if this
will be feasible, is yet to be seen.
Once this is done, we must first develop, for each UI "mode" that we wish
to support, some basic UI elements that can be relied on at higher levels.
Such elements include, as Fare points out, lists and vectors, but also
probably some more basic things, such as those pertaining to text and graphics.
(In fact, such basic things are very necesary. One can't wish to edit a list
of nothing-in-particulars!)
...
Fare's draft of a note:
To me, automatic interface generation would be done
as a particular case of dynamic reflective programming
with "epsilon" constructs:
at some point in your program you tell the system
"Here I want an integer",
and the system must somehow provide you an integer.
Of course, you can be more precise about it:
you can add constraints about what objects needn't
or shouldn't be in the scope of the system to find the integer;
you could add a real-time constraint as for how much time
you can wait for the integer, etc.
You can also give the system hints and meta-hints about
how it should find the integer.
For instance, if you tell it
"find a large prime integer to make a new RSA key",
or "give me the age of the shipmaster",
or "give me the position of the text in the scrolling window",
this might lead to completely different behaviors.
How the system finds the integer from your specifications is up to it;
but of course, the more precise your idea about what should be done,
the more precise your hints.
The idea is that you should never have to put more into it than you want,
and should never have to put less either.
You can say exactly what you want.
For instance, if you want a scrollbar, ask a scrollbar;
but don't draw the rectangle yourself, let the system do it;
you don't care if the scrollbar looks like
Athena, or Motif or Windows 3.x or Win95, or MacOS,
or another widget set, or any combination thereof, whatever;
some users might care, so don't waste your time at working against them.
Also, you might want not to be even as precise as to ask for a scrollbar.
After all, what you want is that the (large) text be interfaced.
The blind, as well as many other people,
might want a voice-based interface,
that has its own "scrolling" mechanism to interface a large text.
Why prevent your fine program from being used without a screen?
Is a screen essential to it?
Perhaps you do have good ideas to interface your program
specifically to a screen;
but then, Tunes allows you to specify these ideas
a part from your program itself,
by (partially) specifying the epsilon-filling tactics for it.
One way to understand programming with epsilons
(Epsilon stands Hilbert's (ain't it he?) epsilon symbol
for a witness for an existential predicate)
is to view them as goals in the classical paradigm
of higher-order logic programming
(see languages like Prolog, Mercury, Lambda Prolog, Goedel, etc);
an epsilon is a hole in the well-founded construction
of a computable object;
the system thus uses tactics to fill the hole;
these tactics are made of rules that may include holes themselves,
hence generating filling of the subholes as subgoals.
The main differences of Tunes epsilon calculus
with classical logic programming
is that tactics can be dynamically defined
from existing tactic constructs and libraries,
and they are first-class objects,
themselves subject to manipulation, etc.
Also, there is a guaranteed constraint that
should they succeed,
they should return an object that rigorously matches
the requested specification;
hence, whereas Prolog programs only have a global semantics
that can only be deduced from the totality of rules,
Tunes tactics can have a nice local semantics with local definitions.
If O is an object with a hole e,
then O[E/e] where E is a compliant tactic to fill hole e
is a valid object and specialization for O.
E may dynamically generate holes itself, etc.
the tactics are constrained to return a result that
respect the specification for the requested object,
instead of being pure arbitrary tactics
in a program with global semantics only.
As for interfaces proper, here is how they could be formalized:
- an channel is a structure with which an objects can send data to another;
it needs not be a stream of raw binary data.
For the object sending the data, the channel is output;
for the object receiving data, the channel is input.
- a display function for type T on output O
is the data of function f from T to O,
and a reconstruction function g from O to T,
such that someone observing the output
from f applied to an object of type T
can reconstruct the object thanks to g.
Example: the sequence of characters
123896 is an integer being displayed;
you can reconstruct the meaning of the integer from this string output.
Example: outputing a 1000-line text on a tty without scrolling
is displaying the text. Because of the time needed to read things,
one may consider that an acceptable text display should
pause before scrolling forward.
- for any input I and output O connected to a same external object,
the couple (I,O) is named terminal I/O.
Examples: I=keyboard, O=text array display;
or I=keyboard&mouse, O=graphic display;
or I=a few buttons, O=loud-speakers;
or I=braille input, O=braille output;
or I=voice analyzer, O=voice synthetizer.
- a reversibly interaction on a terminal I/O
is a function from I to O with internal state S,
such that for any two states s1,s2,
there exist input sequences t1 and t2 to go from
state s1 to s2 and from state s2 to state s1 respectively.
Note: an I/O-interface is not a particular O-display.
- An interface function for type T on terminal I/O
is a display function from type T
to reversibly interacting objects on terminal I/O.
Example: piping a 1000-line text into /usr/bin/less that allows
to scroll up and down is interfacing the text;
piping it to /bin/more that only scrolls forward isn't.
Example: broadcasting a song on radio is displaying the song,
but not interfacing it;
putting it on a tape that one can rewind is interfacing it.
You can add criteria on the above objects
for the computability/acceptability/complexity/whatever
of displays and interfaces.
By having informational and energetical models of the above objects,
the relative utility of interfaces could be evaluated:
a probability distribution for expected objects in T
would allow to evaluate the information contained in the output,
the complexity of the output-decoding function g
and of the interaction driving functions ()->s , (s1,s2)->t1
relatively to the admitted cultural background
would allow to evaluate the ease-of-use of the interface;
space cost and time delay for all the human and computer functions
involved could be taken into account;
etc.
Not that all these should actually be systematically evaluated
(this might be a good master thesis for a research student),
but this shows that computers CAN express a lot more
than people would generally have them do.
As for the large indetermination in the above utility-related concepts,
of course it shouldn't be neglected.
Epsilon constructs are precisely there to help handle such indetermination...
Up to the Interfaces Subproject
Up to the Tunes
Home Page