8 Principles of Qi
After many patient hours meticulously crafting Qi flows, you may find that you seek a deeper understanding; insight into guiding principles and inner workings, so that you can hone your skills on firmer ground.
Welcome. Your wanderings have brought you to the right place. In this section, we will cover various topics that will help you have a fuller understanding and a sound conceptual model of how Qi works. This kind of facility with the fundamentals will be useful as you employ Qi for more complex tasks, enabling you to engage in higher level reasoning about the task at hand rather than be mired in conceptual building blocks.
8.1 What is a Flow?
A flow is either made up of flows, or is a native (e.g. Racket) function. Flows may be composed using a number of combinators that could yield either linear or nonlinear composite flows.
A flow in general accepts m inputs and yields n outputs, for arbitrary non-negative integers m and n. We say that such a flow is m × n.
The semantics of a flow is function invocation – simply invoke a flow with inputs (i.e. ordinary arguments) to obtain the outputs.
The Qi language allows you to describe and use flows in your code.
8.2 Values, Paths and Flows
Flows accept inputs and produce outputs – they are functions. The things that flow – the inputs and outputs – are values. Yet, values do not actually "move" through a flow, since a flow does not mutate them. The flow simply produces new values that are related to the inputs by a computation.
Every flow is made up of components that are themselves flows. Thus, each of these components is a relationship between an input set of values and an output set of values, so that at every level, flows produce sequences of sets of values beginning with the inputs and ending with the outputs, with each set related to the preceding one by a computation, and again, no real "motion" of values at all. There may be many such distinct paths over flow components that could be traced (borrowing the term "path" as used in graph theory in this sense), and we may imagine values to flow along these paths.
So indeed, when we say that values "flow," there is nothing in fact that truly flows, and it is merely a convenient metaphor.
8.3 Values are Not Collections
The things that flow are values. Individual values may happen to be collections such as lists, but the values that are flowing are not, together, a collection of any kind.
To understand this with an example: when we employ a tee junction in a flow, colloquially, we might say that the junction "divides the flow into two," which might suggest that there are now two flows. But in fact, there is just one flow that divides values down two separate flows which are part of its makeup. More precisely, -< composes two flows to yield a single composite flow. Like any flow, this composite flow accepts values and produces values, not collections of values. There is no way to differentiate, at the output end, which values came from the first channel of the junction and which ones came from the second, since downstream flows have no idea about the structure of upstream flows and only see the values they receive.
The way to group values, if we need grouping, is to collect them into a data structure (e.g. a list) using a collection prism, ▽. In the case of a tee junction, the way to differentiate between values coming from each channel of the junction is for the channels to individually collect their values at the end. That way, the values that are the output of the composite flow are lists generated individually by the various channels of the flow.
8.4 Counting Flows
Everything in Qi is a function. Programs are functions, they are made up of functions. Even literals are interpreted as functions generating them.
Consider this example:
(~> sqr (-< add1 5) *)
There are six flows here, in all: the entire one, each component of the thread, and each component of the tee junction.
8.5 Flowy Logic
Qi’s design is inspired by buddhist śūnyatā logic. To understand it holistically would require a history lesson to put the sunyata development in context, and that would be quite a digression. But in essence, sunyata is about transcension of context or viewpoint. A viewpoint is identifiable with a logical span of possibilities (catuṣkoṭi) in terms of which assertions may be made. Sunyata is the rejection of all of the available logical possibilities, thus transcending the very framing of the problem (this is signified by the word mu in Zen). This kind of transcension could suggest alternative points of view, but more precisely, does not indicate a point of view (which isn’t the same as being ambivalent or even agnostic). This idea has implications not just for formal logical systems but also for everyday experience and profound metaphysical questions alike.
But for the purposes of Qi, what it means is that the existence of a value is a logical span within which it takes on specific forms. Sunyata is the difference between a value taking on a form indicating tangible output (e.g. 5 or "hello") or indicating absence (e.g. (void) or "") or failure (e.g. #f), or provisionality (e.g. 'suspended) or certainty (e.g. #t) – it’s the difference between these, and not existing at all.
The same considerations extend the other way as well, from nonexistence to existence to existence of more than one. As each value corresponds to an independent logical span of possibilities, sunyata in the context of Qi translates into the core paradigm being the existence/non-existence and consequently also the number of values at each point in the flow.
In practice, this means that Qi will often opt to either return or not return a value rather than return a value signifying absence or raise an error. This principle even suggests considerations for the design of ordinary functions and the evaluator itself, which from a Qi/sunyata perspective, could model absence and number in positions that typically expect values.
For example, the following would seem to be in accord with these principles:
Variadic functions, on receiving no input, produce a nullary value (e.g. a monoid identity) if applicable, or no output otherwise, instead of raising an error. For instance, (max) currently raises an error, and under this guideline would produce no output, instead.
Upon receiving multiple values in positions where a single value is expected, the evaluator "forks" the continuation so that each possibility is independently (combinatorially) evaluated. The results from these parallel computations could be "flattened" as multiple output values, but it may be more correct to evaluate them instead as completely independent computations. These could be manipulated distinctly – for instance, each forked continuation could be fulfilled by a distinct process, and the evaluator could provide a means to refer to the output of these processes from within the language or in a metalanguage, while keeping the processes themselves abstracted. (cons (values 1 2) (list 3)) independently produces (list 1 3) and (list 2 3).
Upon receiving no values in positions where a single value is expected, there are a few cases to consider. If no values were received in a position corresponding to the "object" of the function, in the grammatical sense, then the function produces no values. If no values were received in another position, then the function produces the input object(s) (again, objects in the grammatical sense) – which would be consistent with the expected output type. If there is more than one object position, then the result of the function is empty if any of the object positions is. If no values were received in a position serving another grammatical role (e.g. a modifier or adverb such as a function-valued argument), the function generally produces nothing. In none of these cases does the function raise an error. (cons (values) (list 1 2 3)) produces (list 1 2 3), while (cons 1 (values)), (add-two-numbers (values) 5), and (sort (values) (list 2 3 1)) (where the first argument position indicates a comparator function to be used in sorting) produce nothing.
If the function receives valid inputs but is unable to produce a valid result (for instance, if the inputs fail some runtime requirement), it produces nothing.
If an invalid input (such as one of an unexpected type) is received, the function raises an error.
See Write Yourself a Maybe Monad for Great Good for an example that applies some of these ideas to implement the Maybe monad commonly used in many functional languages. Although, note that if the above design were adopted by the underlying language and interpreter, Maybe would be unnecessary in most cases.
8.6 Phrases
When reading languages like English, we understand what we read in terms of words and then phrases and the relationship of these phrases to one another as clauses of a sentence, and then the relationship of sentences to one another, and then paragraphs to one another. The resourceful speed readers among us even do this in the reverse order at first, discerning high level structure before parsing the low level component meanings. Just like human languages, Qi expressions exhibit phrase structure that we can leverage in similar ways. Here are some common phrases in the language to get you started thinking about the language in this way.
(~> △ (>< f) ...) – "sep-amp". A standard mapping over values extracted from a list.
(~> (-< _ f ...) ...) – "augment". The tee junction may augment the flow in any order – the signature of this phrase is the presence of a _ in the tee junction.
(~> (-< f ...) g) – "diamond composition". This is one way to adapt a flow f of arity k to a flow g of arity m, that is, by branching the k inputs into m copies of f (assuming f produces one output). It is the same as the "composition operator" used in defining primitive recursive functions.
(group 1 car-flo cdr-flo) – "pare". This is analogous to "car and cdr" style destructuring with lists, but for segregating values instead. Note that while it is analogous, this isn’t "destructuring," since the values taken together do not form a data structure.
Some of these phrases may someday make it into the language as forms themselves, and there may be higher-level phrases still, made up of such phrases.
8.7 Identities
Here are some useful identities for the core routing forms. They can be used to simplify your code or say things in different ways.
8.8 Flows and Arrows
[The connection between flows and arrows was pointed out by Sergiu Ivanov (Scolobb on Discourse).]
It turns out that the core routing forms of Qi fulfill the definition of arrows in category theory and Haskell (in an unfortunate conflation of terminology, these "arrows" are an entirely different notion than morphisms, which are also sometimes referred to as arrows). The specific correspondence is as follows:
Qi’s "base case" of treating any function as a flow corresponds to the arr method of arrows.
~> is equivalent to the (>>>) of arrows.
The relay, and specifically, (== a _), (== _ b), and (== a b), respectively correspond to first, second, and their composite (***) in arrows.
(-< f g) corresponds to what is referred to as "fanout composition", (&&&), in arrows.
So evidently, flows are just monoids in suitable subcategories of bifunctors (what’s the problem?), or, in another way of looking at it, enriched Freyd categories.
Therefore, any theoretical results about arrows should generally apply to Qi as well (but not necessarily, since Qi is not just arrows).