(Updated Sat 2022-02-26) (Updated Mon 2022-02-21) (Updated Wed 2021-12-01)

Functional Components

In the process of re-writing my old Perl accounting software in Fexl, I am stumbling onto a neat technique for writing software "integrated circuits," aka "functional components," aka "parts."

The interesting bit that's allowing me to progress rapidly is to write isolated functions which do interesting things, but not bother too much with naming them. I actually roll a random number and call the function something like "_8214". Not a very illuminating name I know, but a block comment above it explains roughly what it does. The name itself is like a standard "part number" of a hardware IC.

That sounds weird, but it liberates me from the "tyranny of naming."

As I go, I relentlessly refactor functions to minimize the number of such components. At an upper level, I might end up with such gems as:

# Return formatted data for the "Change in Net Asset Value" section.
\_2997=(_2479 _1006)

# Return formatted data for the "Balance" section.
\_1622=(_2479 _8214)

Yeah that's right: the _2997 component is just the result of plugging a _1006 into a _2479.

What could be clearer? 😉 OK, I could rename "_2479" as "format_numeric_data_as_monetary_values" and "_1006" as "get_numeric_data_for_the_change_in_net_asset_value_section", but why?

By the way, that's not a great example because it's not exactly how I do that particular task. I concocted the example when experimenting with how far I could take this combinatorial approach.

Sometimes when developing a new capability I am tempted to build that into a part that's already stable and minimal. But now I can say no, I'll just make a new part that provides exactly the new thing I need without disturbing the old part. If I find that the old and new parts have a lot of structure in common, I can abstract the similarities into a new common part — passing in paramters which can include data or even entire parts (functions) themselves. I don't always have to do that, because sometimes it is possible to extend the old part without disturbing anything. But I like making relentless forward progress and never having to look back. Plus, I don't degrade the efficiency and independence of the old part.

Names versus Topology

It's not so much the names that matter, but the topology of the interconnected functions. The whole thing could be laid out using a sort of computer-aided circuit design tool -- instantiating components, drawing traces, etc. Physically, I don't have to worry about voltage and capacitance, but the functions do have propagation delays and memory footprints, so it does involve actual physics.

I don't carry this approach all the way down of course: there are still functions with names such as "map" and "append" and "format_money". But the neat thing is that when I discover a common functional pattern between two otherwise distinct functions, I can abstract that pattern instantly into a new function with a "part number" without having to obsess over giving it a long descriptive name. Plus half the time I'm not even sure that will turn out to be the best design, so why waste time with the name? I just proceed and see if I like it.

I can imagine a CAD tool popping up a block comment when I hover over a component in the functional circuit, and providing a search function for locating components based on these comments.

Cleaning up

I can always clean up names after the fact. Often a name becomes clear in hindsight, upon examining the inner structure of a function. I am not discounting the importance of names. I am describing a process of developing a conceptual structure of functions first, and then assigning names to those concepts after the fact. Sometimes premature naming can interfere with the process.

Elaborating on the process

A colleague writes:

Sometimes the act of working out what it does gives you the name, and then the problem goes away. If it's an abstracted function without a clear semantic, maybe it shouldn't be abstracted (yet).

Precisely: the act of working out what it does gives me the name.

For example, I may yet rename _5822 as def_capital, though I haven't bothered yet. Initially I created _5822 quickly without thinking of a proper name, simply because I opportunistically observed a common pattern of computation in two places that went something like this:

(\obj
...
with "beg_nav" beg_nav;
with "start_nav" start_nav;
with "deposit" deposit;
with "withdrawal" withdrawal;
obj
)

Now in that case it certainly does have a clear semantic, but I just wanted to factor it out quickly without considering a proper name. For one thing, I wasn't sure the code was going to stay that way, so I didn't want to waste time naming it up front until I was sure.

In another place I observed a common pattern that went like this:

(\obj
...
\total_fee==(list_sum; map obj ["admin_fee" "mgt_fee" "incent_fee"])
\income=(+ (obj "gross_income") total_fee)

\start_nav=(obj "start_nav")
\end_nav=(+ start_nav income)

\factor=(/ end_nav start_nav)
\ror=(- factor 1)

with "income" income;
with "end_nav" end_nav;
with "factor" factor;
with "ror" ror;
obj
)

So I abstracted that into a common _2308 function at first. Later I discovered I was always applying it at the end of two distinct cases (entire fund versus individual partner), so at that point there was only one call to _2308 and I simply expanded it inline because why not.

In contrast, in the case of _5822 (def_capital), it is logically impossible to call that in only one place, so it must remain a named function. (Or rather, the only way to reduce it to a single call would be through weird combinatorial tricks to avoid the lambda symbol.)

One reason I haven't bothered to rename _5822, although I could do it right now in 5 seconds if I cared, is that it is only called in one prescribed area of code. It is not "exported" in some kind of re-usable library. If it were, it would be cruel to the user to keep it as _5822.

Lately I've even done the random-name trick on files. I may evolve a function _9416, and I'm tired of seeing its definition in a parent file, so I quickly move it out to a "_9416.fxl" file and replace its definition with:

\_9416==
    (
    value;
    std;
    ... plus any other definitions I wish to pass along ...
    use "_9416.fxl"
    )

The decision to use "==" or "=" to define it depends on whether I am certain it will be called, or whether it might be called multiple times. I don't want to load it if I don't need it, and I don't want to load it twice if I use it twice. I might even use "\=" (once) if I'm not certain of anything and just want to guarantee that it will be loaded either once or not at all.

So then I have an entire file with a meaningless name. In this case I am almost certain I won't keep that structure anyway, I'm just moving it aside mechanically for now. At this point I'm refactoring existing code into a cleaner structure, and I'm pretty sure _9416.fxl will not remain as a permanent entity. All that stuff is going into a configuration object that gets passed as a parameter.

Objects

When I use the word "object" in Fexl, I mean exactly one specific thing. An object is a function that maps a string to an arbitrary value. Period. It's not mutable, but it can be extended with further definitions using the "with" function.

For example, here is a definition of an object that maps a few strings (keys) to numeric values:

\obj=
    (
    with "a" 1530;
    with "b" 2515;
    with "c" 0621;
    void
    )

To show the value of "b" I can say:

say (obj "b")

Now if I want to extend that with a definition for "d", I can say:

\obj=
    (
    with "d" 4207;
    obj
    )

Although it is possible to shadow an existing definition, e.g. for "b", I tend to avoid that because I should not have given the wrong definition for "b" in the first place. That would be lying. 😇

Yes, you could adopt a style of code where "b" is redefined along the way, but that devolves into simulating a mutable object, and practically speaking the stack of re-definitions will pile up in memory. I could conceivably define "with" in such a way that it avoids that, but my inclination is to avoid such things altogether. Don't simulate mutability, just define "b" to its correct value once and be done with it.

As it turns out, the "with" function is crucial enough to where I've defined it directly in C code so it can be very fast. It reduces to a quick linked list search. I could potentially make it faster using a contiguous vector, though I'd need to avoid any extra copying that might negate the gains. I can readily imagine some neat adaptive techniques for that, but I see no need to bother yet.

Of course, the "with" function can be defined in plain Fexl as:

\with=(\key\val\obj \obj=obj \x eq x key val; obj x)

Or to spread it out a bit:

\with=
    (\key\val\obj

    # Explicitly evaluate the tail object to avoid any repeated
    # evaluation on multiple calls.

    \obj=obj

    # Now return a function that is equivalent to the tail object,
    # except that the new key is defined as val.

    \x
    eq x key val;
    obj x
    )