All posts by Josh

Minds: @JoshFuhs Twitter: @JoshFuhs
Computing

Language Musing

Leave a comment

If you’ve been in the software development world, you may have noticed some larger themes while going about your day-to-day. As you create, you (or I) tend to start with relatively large, unruly blobs of code. Then in several passes, those large, unruly blobs of code are broken down into smaller, more structured, and more generalized bits that are used by more specialized bits. This is the essence of functional decomposition, but it can happen along interesting axes. Most languages allow you to break down main routines into subroutines, but can you break down types? Some languages let you break down types, but can you parameterize them? Some languages let you parameterize your types, but can you classify and manipulate your language elements based on their properties?

We can watch computer language development go from simple to abstract, and this mirrors the path of mathematics. The power of the abstractions cannot be understated. As each new abstraction layer appears, our ability to compactly express ideas grows by some exponent. The cumulative effect will have a total power difficult to predict.

Mathematics is a language of “what is true.” One of its peculiarities is that it can be completely removed from the physical world and still work. Mathematics of some form acts as the core of most software languages, but that’s not their totality. We can call the myriad ways to express truth as Truth Language.

Programming languages take some way of describing “what is true” (i.e. mathematics and logic) and layer on top the ability to talk with a computer’s inputs and outputs. This makes software languages capable of interacting with the world.

Thus, programming languages are languages of sense and action. Every piece of software in its most generalized form is Input -> Computation -> Output [ -> Repeat ]. The inputs are some form of sensory data — key presses, camera data, mouse movement, network packets, etc. — that inform the rest of the process. The Computation is a series of transformations based on known truths, and the Outputs manifest the results of this computation in the world. These processes can range from simply putting characters on a screen to the full complexity of running a manufacturing plant and beyond. Whatever the goal, it’s manifested in the real world. We can call these Action Languages.

Not only do we use programming languages to direct computers, but we use them to communicate with each other about what computers will do or have done. The emphasis on human readable code is strong in the software engineering discipline(s). One might also notice that the core of general human language is Action Language — we use our language largely to describe things to do.

However, general language includes more layers atop Action Language: a semantics of history and future, past actions and intent. Computer systems generally express past as log of some form while intent is expressed as an event handler configuration or schedule. These things tend to be described within software languages by how they are configured and/or formatted rather than as first-class elements of the software languages themselves, but there may be a slow shift toward more formality.

By advancing the state of Action Language, computer science is essentially advancing the state of general language. Though we don’t yet completely use software languages this way, it’s clear that we’re educating millions at a time in their use to communicate both with our machines and each other. The long-term effect of this can only be more power and precision in our ability to express ideas and intent.

More than communicating amongst ourselves, software language is rapidly allowing us to express intent to and command the universe itself. Computers were an impressive step, giving each individual a capacity for logical and mathematical expression never before possible by even the whole of humanity. As technology advances toward increasingly more advanced creation machines like CNC machines and 3D printers, our words, expressed as software, achieve increasing creative potency.

To date, software represents the pinnacle of the power of the written word, and we still seem to have a lot of improvement ahead of us. If you don’t yet know how, pick up a programming book or tutorial and get started. Incomputency in the 21st century will be looked on as illiteracy in the 20th.

Quality

Why I Don’t Use Debuggers

Leave a comment

Other notable authors have already written good posts about this topic, but I was recently encouraged to do so as well.

First, the title isn’t entirely true. I’ll break out gdb when I need to get the backtrace of a native application crash. I’ll do the equivalent for any runtime that doesn’t provide information about the method or function that produced the exception. However, I otherwise avoid them.

Debuggers make it possible to make sense of larger sets of code than you might otherwise be able to. This is helpful, but it can lead you into believing you can deal with more complexity than you can. Debuggers are a crutch that get you past some of your limitations, but when the limitations of the debuggers are reached, you may find yourself in a briar patch.

Complications

Threading and multiple processes

Stepping through multiple threads can be a bear. Threading and multi-processing in general can be dangerous to your health. I prefer concurrency models that isolate to an extreme degree any concurrency primitives. I’ve not tried but I’ve heard good things about Threading Building Blocks.

Runtime Data Models

Investigating data in a debugger may require some familiarity with runtime representations of data structures rather than interacting with your data structures via familiar interfaces.

Additionally, the runtime data structures typically don’t have standard implementations, so they can change from version to version without warning. In fact, depending on the goals of the implementation, the underlying form of the data could change run-to-run or intra-run. I tend to prefer to rely on the documented interfaces.

On the other hand, debuggers provide a good way to get familiar with runtime data structures.

Complex Failure Conditions

When using debuggers in my more youthful days, I found that complicated issues were easier to track down if some code changes were made to provide breakpoints under odd conditions. This seemed antithetical to the purpose of a debugger. Maybe I was doing it wrong…

Not Recorded

I’ve never seen anyone record a debugging session such that they could return to a particular debugger state. I’m sure it could be done, but I don’t know how valid that technique would be if small modifications were made to the original software and retried.

Debugger Unavailability

In some rare circumstances, debuggers aren’t available for the environment you’re using. If you haven’t developed any other techniques for finding problems, you may be stuck.

Preferred Techniques

Unit Testing

I cannot stress this enough: unit testing, done well, forces you to break your code into smaller, functional units. Small functional units are key to solid design. Think about the standard libraries in the language(s) you use: the standalone functionality, the complete independence of the operation of your code, the ease with which one could verify their correct functioning. That’s how you should be designing.

Note that once you know the method/function in which the fault happened, you’ve narrowed the problem down significantly. If you did a good job at functional decomposition, you’re typically only a few steps from the source of the problem. If not, judicious application of the other techniques will tease it out.

If you find that the behavior of your module is too complicated to easily write unit tests, that may be a sign that your module is too big. On rare occassion the input domain is so rich with varying behaviors that directly testing the interesting input combinations is impractical. Those can be managed with more advanced testing techniques, see: QuickCheck.

Invariant Checks

When code separation proves to be difficult because of a lack of regression testing around the parts that you’re changing, invariant checks are a technique that can be used as a temporary shim. These are tests that run inline with the code of interest to check conditions usually before and after a method/function call or a block of code.

The invariant code can form some of the foundational functions when you do eventually get around to creating the unit tests for your new module.

Print/Log Debugging

The dreaded printf() debugging! This isn’t ideal, but it can give some quick info about a problem without much fuss. If you find you’re adding hundreds of printf()’s to track something down, I might suggest that you’re using some of the other techniques inadequately. If this is the position you’re in, a debugger might actually be a benefit.

Note that, again, all code written to try to weed out your problem may be valuable for one or more tests.

I’ve also used ring buffers embedded in devices that store the last 1000 or so interesting events so that when an obscure failure happens, there exists some record of what the software was doing at the time.

Where Debuggers Shine

Compiler debugging. This is an absolute pain without debuggers, and I wouldn’t recommend it.

Heisenbugs: Those bugs that disappear with any change to the code. Then a debugger is about the only way to attempt to get a look at it. Those bugs usually warrant pulling out all the stops. Fortunately, many of the newer languages have entirely eliminated these bugs. Good riddance.

Other Tools

I do appreciate other tools like code checkers and profilers. They usually work without much input and communicate their results in terms of the language they’re checking. I’m a fan of this model.

A seemingly close relative of debuggers, the REPL tools, look promising. I’ve never used them, but they look like they operate almost entirely in terms of the language that they’re debugging. I’m a fan of this model.

Summary

I prefer debugging techniques that produce additional useful work and provide support for refactoring if the situation warrants it. Every bug potentially reveals a need to re-implement or re-design. Debuggers feel more like patchwork.

Ergonomics

Basics: Revision Control

Leave a comment

Engineering disciplines have been using revision control techniques to manage changes to their documents for decades if not centuries.

With computers we get new, automated, and more comprehensive techniques for doing this.

Advantages

You have a history of changes. Where did that file go? Look through the history. What changed between this revision and the last? Some revision control tools will show you exactly what changed.

They can coincide with and support your backups, some completely passive. Depending on the frequency with which you do backups, you may already effectively have a form of revision control, though it might be difficult to get some of the related features.

For engineers and other professions, it’s sometimes incredibly important to know what you were seeing at a particular point in time. Some revision control tools give you exactly that.

Disadvantages

Space. Recording revisions usually requires extra disk space. Disk space is relatively cheap, and most revision control systems are far more effective than recording multiple full copies of the same file(s).

Complexity. Controlling revisions isn’t as simple as just writing contiguous bytes to disk. Unfortunately, filesystems aren’t generally as simple as just writing contiguous bytes to disk either. Computing capabilities are solid enough that this is usually a minor concern.

Classification

There are probably two broad categories into which revision control tools can be divided, passive and active. Each has its own advantages and disadvantages.

Passive

As the heading suggests, these revision control systems require minimal interaction to make them do their job.

Some passive revision control systems include:

  1. Dropbox – Look at the web UI. You can easily find older revisions of files that you’ve stored.
  2. ownCloud – This is an open source, self-hosted web tool similar to DropBox. It’s supported on all major operating systems and has apps for every major mobile OS. I use this at home.
  3. Apple Time Machine – Apple provides a way to periodically backup to a secondary drive and provides the revision history of those files. There are similar tools for Windows.
  4. Copy-on-Write Filesystems (CoW) – Several filesystems offer revisioning as a core capability based on their underlying data model.

Most of these will not record every change but will instead catch changes that fall below some roughly-defined frequency. Revision recording throughput is affected by the number of files that have changed and the total amount of extra data that would need to be stored. Because these snapshots are taken without user intervention, there’s really no chance to augment the changes with additional information or to group related changes in meaningful ways.

However, with just a little setup — pointing at a server or drive, entering account credentials, choosing a directory to sync — you can rest assured that changes made to your files will be recorded and available in the event of emergency or curiosity.

I believe every computer shipped should come with some form of passive revision controlling backup system out-of-the-box.

Active

Active revision control systems offer much more capability at the expense of learning curve. However, there’s simply no better way of working on digital (is there any other?) projects.

Some active revision control systems include:

  1. Subversion – an old favorite, but slowing deferring to
  2. git – distributed tool that’s consuming the world
  3. cvs – “ancient” predecessor to subversion

This list is far from exhaustive. I can think of at least five or six others off the top of my head, but I don’t think any others have nearly the significance today.

The features that the various active revision control tools offer are vastly varied, but they all provide a core set of functionality that distinguish them from the passive revision control tools.

  • Explicit commits. Every bit of work committed to the revision control system is done so explicitly. Specific changes can usually be grouped. Comments and other meta-data can be included with the commits to provide extra context for the change.
  • Change diffs. Every modification made can be compared to the previous version and changes between versions can be viewed.
  • View history. Every commit and meta-data ever made can be listed.
  • Checkout previous revisions. It can often be helpful to look back in time to find out why a problem didn’t seem to exist in the past or to determine when it was introduced. In rarer circumstances, you might want to know why a problem seemed to disappear.
  • Revert commits or to a previous revision. Sometimes changes were committed that were ultimately detrimental and should be removed.
  • Multi-user commits. Virtually all active revision control systems support accepting work from multiple users with techniques for merging changes that can’t be trivially combined.

Like the passive revision control systems, not all active revision control systems are also backups. In most cases you would need to take extra steps to backup the revision control system. Pairing an active revision control system with a passive revision control system could be a way to do this.

Few, if any, of the active revision control systems handle binary data well. They can usually be handled, but the efficiency of storage might be lacking and the diff capability is usually absent. This might be their single largest weakness.

No significant (or insignificant) project should be started without one of these revision control tools, and project tools should be structured in a way that allows independent, verifiable revision control.

Visualization

Most of these tools don’t seem to provide much at-a-glance functionality, and I think it’s really useful to have. The things I’m most interested in seeing:

  1. Have any files been modified (active)?
  2. Are there files that could be lost (active)?
  3. Are there any upstream changes that aren’t synced (active, especially useful for multi-user projects)?
  4. Are there any local files that haven’t been recorded (passive)?

For the active revision control questions, tools like Github Desktop for Mac and Windows, TortoiseGit and TortoiseSVN for Windows, and RabbitVCS integration for Nautilus on Linux might do the trick. Some active revision control systems provide these features out-of-the-box, but they tend to be pricey.

On (4), I’ve not seen a passive system that provides this information. It seems like it might be useful to know if all local files have synced before shutting down for a while. I’ll keep my eye out for this.

For those with a bash habit, I have a version of ls that provides (1) and (2) above. I plan to make this available shortly.

Computing

Disadvantages of the Cloud

Leave a comment

As the world moves toward AWS, Azure, and Google Cloud, I would like to take a moment to reflect on areas where the cloud services maybe aren’t so strong.

Connectivity/Availability

While the popular cloud services are the best in the world at availability and reliability, if your application requires those qualities at a particular location, your service will only be as reliable as your Internet connection. For businesses, high-quality, redundant connections are readily available. For IoT-in-the-home, you’re relying on the typical home broadband connection and hardware. I won’t be connecting my door locks to that.

When disaster strikes, every layer of your application is a possible crippling fault line. This is especially the case when your application relies on services spread across the Internet. We saw an example of this recently when AWS experienced a hiccup that was felt around the world. The more tightly we integrate with services the world over, the more we rely on everything in the world maintaining stability.

Latency and Throughput Performance

If latency is important, locality is key. Physics can be difficult that way. Many attempts at cloud gaming have fallen apart because of this simple fact. Other applications can be similarly sensitive.

Likewise with bandwidth, even the best commercial connections are unlikely to achieve 10Gbps to the Internet. If your requirements stress the limitations of modern technology, the typical cloud services may not work for you.

Security

While the cloud providers themselves are at the cutting edge in security practices, the innumerable cloud services and IoT vendors built atop them often aren’t. Because of the ubiquity that the cloud model presents, it’s getting more difficult to find vendors that are willing to provide on-premise alternatives — alternatives that could be more secure by virtue of never shuffling your data through the Internet.

There’s also the simple fact that by using cloud services, you’re involving another party in the storage and manipulation of your data. The DoD types will certainly give this due consideration, but events like CelebGate (1 and 2) are indications that maybe more of us should. Every new online account we create is another door into our lives, and the user/password protected doors aren’t the most solid.

Another concern along these lines is legal access to your data. If you’ve shared data with a cloud provider, can it be legally disclosed without your knowledge via warrant? Can foreign governments request access to any data stored within their borders? These issues are still being worked out with somewhat varying results around the globe. This might be an even smaller concern to the typical user, especially for those of us in the US. However, I feel I would be remiss if I didn’t note that politics of late have gotten…interesting.

Precision Metrics

I haven’t heard about this problem recently, but it has been noted that it’s difficult to get good concrete metrics out of at least some of the large scale cloud providers. Software failures can occur for reasons that are invisible to the virtualization layer: poor node performance due to throttling of nodes in particular locations, odd behaviors manifesting between two VMs using resources in unexpected ways, etc. Good luck getting the hardware metrics needed to debug these from the cloud providers.

Cost

This is fast becoming a non-issue as prices race toward zero. However, at the time of writing, those AWS bills could still add up to a significant chunk of change. Delivering the kind of reliability that the cloud titans provide isn’t easy, and the cost can often be worth it.

Custom Hardware

This is a fairly extreme case, but customization with the big services is rough unless you’ve got significant cash to throw at the problem.

Scale

If you’re already the size of Amazon or larger (Facebook), you’re probably better off doing something that works for you. I don’t imagine many will be in this position.

 

There you have it. The best set of reasons I have for avoiding the cloud.

If none of those reasons apply to you, and high-availability, high-reliability, available-everywhere is what you need, nothing beats the cloud computing providers. They’re ubiquitous, generally easy to use and automate, and are getting cheaper every day. Sometimes you can even get better deals with specialized services like Bluehost or SquareSpace, but beware the drawbacks.

However, if you have a concern along any of the lines above, it’s at least worthwhile to take a second look.

Howto

Ubuntu on Asus Zenbook Flip UX360C – Power Performance

Leave a comment

I spent a few weeks watching a Zenbook Flip drain its battery doing different things via Ubuntu 16.10, and here I present the results.

Charging while idling (screen on):

Trial NumberInitial ChargeFinal ChargeTime (hours)Total Charge Time (hours)
Average2.565
10.1201.0002.4092.737
20.0910.9902.0002.225
30.0410.9902.5342.671
40.0131.0002.5682.602

Admittedly, this has less to do with the OS than the hardware, but it was easy info to collect while running the other tests.

Discharging with video playing and full screen brightness:

Trail NumberInitial ChargeFinal ChargeTime (hours)Total Discharge Time (hours)
Average4.268
11.000.0294.0504.171
21.000.0284.2424.364

I was pulling video from YouTube via Firefox, specifically https://www.youtube.com/watch?v=SrlHtqTL_8o&list=PLU0V6ITiWqjhk5k-7hAasE5oAQspuTYgp. Gaming playlists make a good source of non-stop video.

Discharge passively with screen at 40%:

Trial NumberInitial ChargeFinal ChargeTime (hours)Total Discharge Time (hours)
Average9.733
10.9750.02910.38110.974
21.0000.02910.63010.947
31.0000.0297.0687.279

I’m not sure what happened with the last trial. Nothing should have changed between them. Regardless, I think the picture is clear enough.

I had started to do passive discharge times at full brightness, but later decided against it. The data I gathered is below:

TrialInitial ChargeFinal ChargeTime (hours)Total Discharge Time (hours)
11.0000.0307.4627.693

This looks similar to the odd trial from the previous table, so that might offer an explanation.

The last test I ran was battery rundown while sleeping. This test should be completely independent of Ubuntu, but it’s still useful.

Rundown time: 35 days from 100% to 13%

I only have the patience for one trial, but I’m more than satisfied with the result. Presumably it could go 40+ days before losing state.

It looks like Ubuntu 16.10 will give you anywhere between 4 to 10 hours depending on usage, where 10 hours is barely any usage. I’m inclined to believe Windows performs better, but I haven’t run the tests myself.

Computing

Benefits of Functional

Leave a comment

In a couple of previous posts I’ve provided what I believe to be a good definition of a functional language (Functional Programming Definition) and why a functional model is ultimately required due to the constraints of reality (Everything is Functional).

However, I don’t know that I’ve been clear about exactly why functional programming is so beneficial. I think the benefits can be summed up in one word: predictability.

When I’m working with functional code, I know that I can lift any part from any where and get identical behavior given the same inputs wherever I may need it. When I know the language enforces these rules, I can be supremely confident in this quality of the code, even unfamiliar code.

When working in languages like C++, Java, and Python, I go to some lengths to try to maintain a functional quality to my code. It can be done with most languages. However, when veering into unknown code, my confidence in my ability to modify without odd unexpected side-effects drops dramatically, and it makes those changes much more of a slog. As bad a practice as it is generally recognized to be, I still bump into the occasional global mutable variable or static class member. It doesn’t take many of those to wreck your day.

I think a second closely-related benefit of the functional style is composability. When you have supreme confidence in your components, putting them together can be a snap. This benefit has a little more dependence on good architecture and design practices, but outright avoidance of the features that make code-reasoning difficult will certainly have an impact.

In reference to my earlier post, everything is functional if you’re willing to zoom out far enough. You could start up another process, container, VM, or cluster to restore your faith in the predictability of your code. Note that this approach has costs that could be fairly easily avoided by starting functionally from the start.

Disclaimer: it’s always possible to write bad code in any language or style. However, it can be argued that it’s much more difficult to reach the spaghetti-horizon where the functional style is enforced.

Howto

Ubuntu on Asus Zenbook Flip UX360C

Leave a comment

Notes

  • Model: Asus Zenbook Flip UX360C bought from Microsoft Store – link
  • Software: Ubuntu 16.04.2 LTS Desktop via DVD ISO – link

Getting Started

  1. Power up the Zenbook Flip
  2. On the boot-up screen, push ESC
  3. Select “Enter Setup”
  4. Go to Boot menu
  5. Go to “Add New Boot Option”
  6. “Add Boot Option” -> <pick a name> (I used “USB DVD”)
  7. “Path for boot option” ->
    1. Select the option that isn’t the HD (Mine is “PCI(14|0)\USB(0,0)\CDROM(Entry1)”)
    2. “<efi>”
    3. “<boot>”
    4. “bootx64.efi”
  8. Create
  9. OK
  10. ESC key
  11. “Boot Option #1” -> <the name you chose earlier> (I used “USB DVD”)
  12. Go to Save & Exit menu
  13. “Save Changes and Exit”
  14. Yes
  15. From the boot options, choose “Run Ubuntu” (I could not get a direct install to work)
  16. Run “Install Ubuntu 16.04 LTS” found on the desktop
  17. Choose the installation parameters you prefer. I like:
    1. Pull updates while installing
    2. Leave Secure Boot ON
    3. Encrypt the disk
    4. Encrypt the home drive

Out of Box Problems

Big Ones

  • Some lid-close/open suspend-resume cycles bypass the lock screen entirely. This is a security problem. When this happens, the mouse click doesn’t function. However, pressing the Windows key a couple times seems to restore that.
    Try it for yourself: Open a Terminal, System Settings, and Firefox and go through some suspend-resume cycles.
    They say this is the problem: https://bugs.launchpad.net/ubuntu/+source/gnome-screensaver/+bug/49579
    I’m not entirely convinced.
  • Reboots are a gamble. After the main decryption passphrase, Ubuntu (kernel 4.8.0-41) can stop at an arbitrary point and never continue. Using the advanced Ubuntu options to boot with a different kernel (4.8.0-36) seems to get past this. Unfortunately this isn’t the first time I’ve seen this kind of instability with Ubuntu + Zenbook.
  • The touchscreen can become useless after several suspend-resume cycles. It registers a move of the pointer, but no clicks. I couldn’t find a way to restore this short of a reboot.

Annoyances

    • Lots of complaints of problems on startup: 
      System program problem detected
    • Suspend-resume is generally fragile. After many attempts, I find that I can’t connect to wifi. It claims to see one network, not mine. Cycling the Ubuntu networking off and on doesn’t help. Rebooting worked, but this may also help: 
      $ sudo service network-manager restart

      If I see the problem again, I’ll give it a try.

    • The lock screen after suspend-resume also seems to flake out every once in a while with an odd flicker and/or focus change. The password field should always be focused. One odd manifestation of this was auto-starting a guest session on resume. Another was the absence of a password box, leaving me unable to log back in. 
      $ sudo service lightdm restart

      or a machine restart will get you running again, but you will lose your desktop session.

    • Lid-close suspend-resume can result in windows resizing. This might be related to the above problem.
    • Sequences of opening and closing the lid can leave the laptop active while closed. This isn’t new to this model laptop, but it would be nice if it didn’t happen. This generally results in returning to a dead or near dead battery sometime later.
    • The Windows sticker on the back doesn’t come off clean. It leaves behind a sticky residue that seems to prefer the chassis material. Some soft cotton cloth, rubbing alcohol, and elbow grease will clear it up. Be careful to avoid the Serial Number sticker. That may be helpful to have in the future.
    • The brightness function keys don’t function. This is a pretty common Zenbook issue, but it looks like it may be fixed with the 4.10 kernel (17.04?): https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1348890
      Some workarounds are listed with the bug, but you can use the built-in UI controls, xbacklight, or xdotool.

Preferences

  • Power button asks what to do — on this model it should just suspend. This page has the answer 
    $ gsettings set org.gnome.settings-daemon.plugins.power button-power 'suspend'
  • Suspend when inactive for some time (30 min works for me). I hate returning to a dead battery after getting side-tracked for a few hours. 
    $ gsettings set org.gnome.settings-daemon.plugins.power sleep-inactive-battery-type 'suspend'
$ gsettings set org.gnome.settings-daemon.plugins.power sleep-inactive-battery-timeout 1800
  • Also, shut down when the battery gets critically low (defaults to 10% battery). A clean shutdown is usually preferable: 
    $ gsettings set org.gnome.settings-daemon.plugins.power critical-battery-action 'shutdown'
  • Lock screen after short period. This is a small usability feature that I like in the event that I see the screensaver kicking on but can’t get to the machine in time to avoid entering a password. It doesn’t need to be long (30 s): 
    $ gsettings set org.gnome.desktop.screensaver lock-delay 'int32 30'

Other Thoughts

Ubuntu on Zenbook hasn’t been the most robust usability combination. Most of the applications work fine. It’s just the edges that could use some polish. The lock screen specifically has been a persistent problem. It looks like it may have gotten worse with this notebook model.

With that said, I prefer having an Open Source option, and Ubuntu seems to be among the best.

Computing

Functional Programming Definition

1 Comment

A dozen years ago, when I was first investigating functional languages, the distinctions between functional and not were much more apparent. Functional languages had functions as values and parameters, closures — an inline-defined function that captures (closed) context for use in future invocations, and if not outright prohibiting variable updates most of them discouraged programming styles that did variable updates.

Today, most new languages are a thorough mix of imperative, object-oriented, and functional styles. Functions-as-parameters, and closures (or something roughly similar) are as common as the procedural paradigm in the 90’s. However, the one key point that has been largely missed is the discouragement of modifiable variables. The former two styles — imperative and object-oriented — basically require it.

As a result, we’re left with programmers today making a lot of the same non-functional mistakes of years past, and the software is little better for it.

Therefore, my favorite definition of “functional language” is what is more commonly referred to as “pure functional” — languages that do not permit side-effects (state changes). This idea breaks down around the edges of software design, and that’s where a lot of diversity among pure-functional languages can be found, but it is the property that gives the functional style the majority of its benefits.

When we imagine the intricacies around implicitly changing state as part of a computation in systems that are large and separated, I think we should begin to understand why Everything is Functional.

Computing

Everything is Functional

1 Comment

At a high enough level, all computing is functional. If your language isn’t functional (state-change and side-effect free), your process is isolating it. If your process is modifying global state, then your hypervisor or individual machine is isolating it. If your machine is working with a cluster of machines to maintain and modify global state, then all other machines not in the cluster work in isolation.

The model of global state is a difficult one to maintain as scale increases. The time to synchronize data grows and the number of complications due to stale data gets worse. At the largest scales, the distinction between database and communication layer (think pub-sub) breaks down, and they effectively become one. This is the model of tools like RethinkDB where the query becomes an asynchronous request for updates to a particular subset of the data model.

The latest software modeling paradigms make a point of restricting state to a locale via microservices. Each microservice is allowed its own state, but communicates with all other parts of the system via a common message bus. Tools like Storm and Spark specialize in connecting these services together into a larger dataflow model.

It makes sense. Physical locality is guaranteed to eventually constrain software, regardless of the programming models we’ve gotten away with in the past. I think we would do well to recognize that, when stretched to the largest scales, software is relatively small units of computation and the data flowing between them (just like hardware). Aligning our heads, tools, and techniques around this model is likely to yield the best results long-term.

Pick up and learn a functional language today!

Haskell
OCaml
Scala

Ergonomics

User Interface Smorgasbord

Leave a comment

I’m tired of user interfaces. Most of them are clunky, poorly implemented re-incarnations of better interfaces past. Some tools do a really good job, but they aren’t readily re-usable. Some apps have great data models but no way to effectively use them. Way too often I find that I want to use one UI with another app’s data, but alas.

I should say that given the choice, I typically prefer the command line. That’s probably atypical of most users, but most users probably don’t realize what they’re missing. The power of a bash one liner is tough to beat — offering the composition of numerous single-purpose apps in ways that regular GUI users might never imagine. I often think it would be helpful to be able to have standard ways to interact with GUIs programmatically, and I know there are probably techniques and tools for doing so, but it seems like a path fraught with peril.

Standardizing Between Applications

The web browser did a tremendous thing in terms of partially separating the UI from the application data model. Every app developer is allowed to take advantage of or is otherwise forced to consider:

  • Standard navigation: forward, back
  • Anchors/bookmarks: Copy-paste URLs
  • Page searching: single-page only, but ubiquitous
  • Standard page structure, styling, and activity

This has made for an improvement in UI consistency that might have been difficult to envision otherwise.

Windows 8 and 10 made a move toward more consistently applied UI features via their “charms” interface. I thought this might be a good idea, but I never got around to trying it.

More Multi-Purpose UIs

I believe this move should continue and to a drastic degree. When I think about the types of data structures that I regularly deal with, I usually can’t come up with many more than ten (though I may not be all that imaginitive):

  • Plain text
  • Tables
  • Trees
  • Graphs
  • 2D maps
  • 3D maps
  • Audio
  • Video
  • Arbitrary data
  • Collections of the above

There are certainly a large number of variations on each of these, but I think this group captures the general categories — “arbitrary data” being the catch-all, of course. Step one: find a set of adapters that can represent most of the data formats that I care about as one of the above.

What kinds of things do I tend to do with my data?

  • Inspect/view
  • Search
  • Navigate
  • Edit
  • Diff
  • Transform

“Transform” being the bucket for “all the ways in which I could change my data”. These are innumerable, but also not necessarily interesting enough to warrant consideration here. Step two: find a UI tool for each of the above data categories that performs each of my standard actions. Granted, worst case this is roughly 50 tools, but I tend to think that some of the functions could overlap and in some cases, like audio diffing, they may not be all that interesting.

This is an approach I’ve been slowly pursuing for a couple years. I plan to make some of my work available in the near future.