Integer Division With Recurring Decimals

I've been doing some programming tests and puzzles while job hunting lately. One quick challenge was quite nice, reminding me a bit of Project Euler questions, and I nerd sniped myself into doing a 2nd pass at it here.

Question

Produce a Python function which takes two integers, numerator and denominator, and returns the result of their division as a decimal fraction in a string. E.g:

divide(1, 4) -> "0.25"

If the decimal places contain an infinite recurring pattern of digits, then enclose the recurring digits in parentheses. E.g:

divide(1, 3) -> "0.(3)"
divide(1, 7) -> "0.(142857)"

Wrong approaches

Evaluating the division using normal floats is going to trip you up in several ways with the limited precision.

For one, a large enough denominator might have a recurring sequence which is longer than the number of decimal places you have available (more on this later), which makes it impossible to detect recurring sequences by examining the division's decimal digits.

Worse, the inherent imprecision of floating point, e.g. if a simple division like 10/3 comes back as 3.3333333333333335, then examining the trailing digits of this looking for recurring digits is going to be problematic.

Using the decimal module does markedly improve precision and control. But infinitely repeating sequences are still going to return results like Decimal(20) / Decimal(3) -> Decimal('6.666666666666666666666666667'), which is going to trip us up.

We can sidestep all these complexities if we see that the question is asking us to perform this division ourselves, longhand. We are going back to elementary school! Wheee!

Better

Let's just do basic division first, ignoring infinite or recurring digits:

def divide(numerator: int, denominator:int) -> str:
    # Accumulate parts of our result here
    results = []
    while True:
        int_part = str(numerator // denominator)
        remainder = numerator % denominator
        numerator = remainder * 10
        results.append(int_part)

        # If there is no remainder, we are done
        if remainder == 0:
            break

        # Add a decimal point after our first integer part
        if len(results) == 1:
            results.append(".")

    return ''.join(results)

The only confusing parts of this are that int_part might contain several digits on the first iteration, but is only ever one decimal digit thereafter. Plus we have to be careful to get the ordering right for our checks to exit the loop, versus appending the decimal point to the output, to avoid weird looking outputs like divide(6, 2) -> "3.".

Trying this out:

>>> divide(1, 4)
'0.25'

It works! But we haven't yet handled infinite decimals, they result in an infinite loop:

>>> divide(1, 3) # Hangs!

Recurring digits

Because we're dividing integers, we cannot get infinitely varying decimal places. If we have an infinite number of decimal places, it must be because of a cycle of one or more recurring digits. To detect such a cycle we have to notice a couple of things.

First, simply seeing a digit in the output which we have seen before is not enough. Looking at the three assignments at the start of the above while-loop, which capture our state:

int_part = str(numerator // denominator)
remainder = numerator % denominator
numerator = remainder * 10

Here, int_part gets the value of each successive decimal digit. However if it takes on the same value as in a previous iteration, the accompanying remainder might be different, and it is the remainder which is used to generate the numerator for the next iteration, and hence generate the sequence of digits after this.

So, as we already knew from common sense, two iterations with the same int_part may go on to produce different sequences of subsequent digits. However, The value of remainder is the only thing which determines the inputs to our next iteration:

  • int_part depends on numerator and on denominator (which is constant)
  • numerator depends on remainder.

Hence, two iterations might produce different digits, but produce the same remainder, and from that point onwards, they will be in lockstep. If we find two such iterations, then we have detected an infinite recurring cycle of digits.

So, before the loop begins, initialize a dict:

# Remainders seen to date, mapped to their position in the result
remainders = {}

Then inside the loop, after everything else, use our new dict to detect if we have seen the current remainder before:

# If we have seen this remainder before, we are now in exactly the
# same state as then, so we have found a recurring digit sequence.
if remainder in remainders:
    # We have found a cycle of decimal digits! Insert parens into our results,
    # from the last seen position of this remainder, up to the current digit.
    last_pos = remainders[remainder]
    results = (
        results[:last_pos] +
        ["("] +
        results[last_pos:] +
        [")"]
    )
    break
# Remember the position at which we saw this remainder
remainders[remainder] = len(results)

Trying this out:

>>> divide(1, 3)
0.(3)
>>> divide(1, 7)
0.(142857)

OMG it works!

Defensive coding

We're putatively done, but the grumpy old dev in me is uncomfortable leaving that while True in there. By deduction, we always must eventually hit the if <condition>: break to escape from it, so ostensibly it's fine. But if I have a bug in the code or my reasoning, then it might lead to an infinite loop, in some scenario I'm not thinking of. Can we limit the number of iterations in some other, foolproof way? Turns out we can.

We've seen already that a repeated value of remainder means we can break from the loop. Also, notice that remainder, given by:

remainder = numerator % denominator

can only take values from 0 to denominator - 1. So it can have denominator possible values, and this is the maximum number of iterations we will ever need.

Hence, we can safely replace the while(True) with:

for _ in range(denominator):
    ...

Splendid! Much less anxiety-inducing

The source is on github.

SVG Trees using recursive Python functions

Inspired by a woodland hike under the first blue skies we've seen this year, I got home and showed the kiddo how to draw an SVG tree with recursive functions in Python.

At first the generated shape looked kinda lumpy and uninspiring, but it did demonstrate the principle. We were thinking of calling it a day, but I did a little bit of tweaking on parameters to control how each branch differs in length and direction from its parent. Suddenly, the generated shape really came alive, and started to look a lot more like the trees we'd seen on our hike that afternoon.

Silhouette of tree against a blue sky, drawn by a Python program

This image uses a recursion depth of 18, yielding 2^18 twigs, i.e. 250,000, which generates a 100MB SVG file. This takes about ten seconds to generate, and another ten to display in an SVG viewer. Alternatively, I can convert the SVG to a lossy webp, as displayed here, which is only 280kB and displays instantly.

Pushing the generation to greater recursion depth makes my SVG viewer and conversion tools start to stutter and barf. Presumably I could be smarter about the SVG I generate -- maybe generating the outline of the tree as points on fewer, more complex polygons, instead of a polygon for each branch segment? No matter, the artifact is the thing here, and it's done now.

Source is at https://github.com/tartley/tree-art.

The Black Parade: Level 04: Death's Dominion

So. Looking Glass's seminal 1998 PC game Thief: The Dark Project spawned an active and long-lived modding community, who created hundreds of fan-made extra levels, many of which are extremely artful and creative.

One group of particularly obsessed loons spent seven years crafting an extraordinary set of such levels, forming an entirely new single-player campaign for the game, named The Black Parade. This was released last year and I only just became aware of it. I'm four missions in, absolutely loving it, and completely lost in the catacombs beneath the pseudo-medieval city.

Hence, my lovingly hand-drawn map of mission 4, Death's Dominion:

spoilers

Map of mission 4, Death's Dominion



That Which Gave Chase

Released in 2023, played on Linux in 2024.

spoilers

Mush your dog sled across cruel Arctic wastelands, driven onwards by a brisk and intense companion, who hired you to take him back to some remote spot, where it becomes apparent he had some sort of revelation, or maybe a breakdown.

The low-res, dithered presentation conveys the harsh, blinding conditions, as you struggle to make out details through the relentless wind and ice. The days and nights of the journey blur into one another, leaving you only fragmentary, disjointed memories: sledding across the ice; arriving at crude wooden huts for the night; mounting the sled before dawn; collapsing into rough bunks; righting the sled while your companion curses you for a fool; silent moments alone.

Smash cuts amongst snowy wastes echo the discontinuities in Alan Moore's "Nemo: Heart of Ice", albeit this is a far more understated tale. The sense is of a protracted, exhausting time spent covering the distance, through punishing conditions, and it's surprisingly evocative.

The narrative leans into the disorientation, making nothing clear. Your companion becomes increasingly cryptic. He urges you onward, never pausing more than absolutely necessary. The deer behave increasingly strangely. Your companion regales you with sickening tales of the investigative mistreatment he subjected them to on his previous visit. By the time the strange mushrooms come into play it is very obvious that you are in a place to which you should never have come, very far from anywhere or anyone, with mounting dread, alone with with a madman. What happened the last time he took this route? What did he leave behind here? What awaits at your journey's end?

It's hard to know whether the difficulty of interpretation, or the non-literal aspects of your journey, are intended as the result of your character's mushroom-induced fever, or the pretensions of intrusively figurative allusions. Most likely, it seems to be both. The deliberate ambiguity runs deep.

Doesn't outstay its welcome, all done in an hour. But the memories remain.

Overhauled Manual for Epomaker Galaxy80 Tri-Mode Keyboard

Loving the new keyboard, an Epomaker Galaxy80 with Feker Marble White switches.

My requirements are pretty much the same as last time I bough a keyboard:

  • Tenkeyless layout, or TKL as it's known, i.e. without a numpad. The kiddo and I fit two side-by-side gaming stations at this desk, and the extra mouse-swiping space is precious, as is the ergonomics of putting the mouse just a few inches closer.
  • Standard ANSI layout, to match the other keyboards I commonly use.
  • Mechanical, although I'm not experienced enough to know a good one from a bad one.
  • At least two connections which are easy to switch between, for work and personal computers. This one has five, three of which are Bluetooth.
  • At least one of those connections should be reasonably low latency, i.e. <5ms, which means wired or a dedicated 2.4GHz dongle, not Bluetooth. The Galaxy80 has both. I'm a long way away from being a pro gamer, but even down here in the GamerDad leagues, I seem to be more aware of annoying latency than most people are.
  • Backlit. I don't especially care about per-key RGB, but that seems to be extremely common. Shine-through keycaps would be nice, but these seem to be increasingly rare outside of garish gamer-boi cyber-monstrosities, so not a big deal.
  • Hot-swappable switches. This is the requirement I compromised on last time I bought a keyboard, settling for the Logitech G915, which was great, but got old after switches started failing. I'm tired of desoldering them and am noping out to buy something else, a mere 16 months later.
  • Not egregiously incompatible with Linux. It would be hard to find a keyboard which doesn't actually work with Linux, but maybe some manufacturer has buried some vital configuration detail in badly written Windows-only configuration software that doesn't play nice with Wine, etc.
  • Without expensive features I don't need, like configurable activation height, or OLED screens.

The switches are described as sounding "like marbles clacking", which worried me that they might be too loud and piercing. But now it's arrived, they are actually quieter than any other mechanical switch I've had. The sound is deeper than I expected. Recognizably like marbles, but merged with the sound of pebbles, and a hint of a wooden xylophone.

I really like it! Although since pulling the trigger I've seen that Reddit doesn't like Epomaker. I'm just not going to read those posts for now.

As seems traditional with all keyboards, the manual is quite hard to read. Here's my overhauled manual, for future reference.

Update: The incantations needed to get function keys working the way you want them to on Ubuntu :eyeroll: etc. Despite the mention of 'apple' in here, this still assumes you have the keyboard switched into 'Win' mode. (via Reddit):

echo "options hid_apple fnmode=2" | sudo tee /etc/modprobe.d/hid_apple.conf
sudo update-initramfs -u

and reboot.


TIL: Constructing a PDF from .jpg image files

I have some folders of .jpg images that make up a comic. I want to convert them into a PDF to read on my tab and other devices, and import into my Calibre bookshelf.

1. Install some prerequisites

sudo apt install imagemagick pdftk

2. Do the conversion

The versatile ImageMagick has a 'convert' command that seems to handle it:

convert *.jpg output.pdf

But this has some issues:

2.1. Failure due to security policy

'convert' currently refuses to generate PDFs: 'attempt to perform an operation not allowed by the security policy'. Apply the fix described on StackOverflow. :eyeroll:

2.2. Failure due to cache space

You might not need this fix if you generate smaller documents, or generate chapter-by-chapter as described below, but here it is in case.

Don't close that editor! In the same policy.xml you were just editing are resource size declarations for memory and disk. If 'convert' barfs with an error about running out of cache space, then bump up the disk resource size. I set mine to 8GB. StackOverflow again for details. :eyeroll: again.

3. Include a table of contents

I want to add bookmarks to the generated PDF marking each chapter.

Put the .jpgs into subdirectories by chapter, eg:

src/
|--chapter01/
|  |--0001.jpg
|  |--0002.jpg
|  |  ...
|--chapter02/
|  |--0001.jpg
|  |--0002.jpg
|  |  ...
|
...

Pad the chapter numbers with preceding zeros so that they sort into the correct order. I added an artificial 'chapter00' containing the front cover, separate from individual chapters.

Now we need to generate individual PDFs for each chapter. We can then use 'pdftk' to count the number of pages in each chapter, and use those counts to place bookmarks on the correct pages when pfdtk combines the chapters into one final output PDF.

I ended up regenerating each chapter a bunch while I tweaked the content, such as deleting adverts from the images. So I put these commands into a Makefile:

help: ## Show this help.
    @grep -E '^[^_][a-zA-Z_\/\.%-]+:.*?## .*$$' $(MAKEFILE_LIST) | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-12s\033[0m %s\n", $$1, $$2}'
.PHONY: help

chapter_dirs=$(wildcard src/*)
chapters=$(chapter_dirs:src/%=%)
chapter_pdfs=$(chapters:%=%.pdf)
bookmarks=bookmarks.txt
output=output.pdf

clean: ## Delete all generated PDFs
    rm -f $(chapter_pdfs) $(output)
.PHONY: clean

chapter%.pdf: src/chapter%/*.jpg ## Each individual chapter, use 2 digits
    convert src/chapter$*/*.jpg $@

$(bookmarks): $(chapter_pdfs)
    ./make-bookmarks >$(bookmarks)

$(output): $(chapter_pdfs) $(bookmarks)
    pdftk $(chapter_pdfs) cat output - | \
    pdftk - update_info "$(bookmarks)" output "$(output)"

all: $(output) ## Build final output PDF
.PHONY: all

Where 'make-bookmarks' is a bash script that generates the intermediate 'bookmarks.txt' file:

#!/usr/bin/env bash

set -e # exit on error
set -u # treat unset vars as errors
# set -x # debugging output
set -o pipefail

# Generate a bookmarks file for all the matching PDF files

fmt="BookmarkBegin
BookmarkTitle: %s
BookmarkLevel: 1
BookmarkPageNumber: %d
"

declare -a files=(chapter*.pdf)
page=1
for file in "${files[@]}"; do
    title="${file%.*}"
    printf "$fmt" "$title" "$page"
    num_pages="$(pdftk "$file" dump_data | grep NumberOfPages | awk '{print $2}')"
    page=$((page + num_pages))
done

Now make all will produce the final output.pdf. You might want to open up the generated bookmarks.txt and edit the placeholder "chapter01" names. Then run make all again to regenerate the final output PDF with your fixed chapter names.

Rorschach II meets Adrian

TIL: Shell environment variable tricks

envsubst is an executable you likely already have on your PATH (part of the gettext package, often installed with dev packages), which is a convenient way to replace $VAR or ${VAR} style environment variables with their values. This allows rendering templates without heavyweight tools like Ansible, Jinja, or embedding with heredocs. Usage is:

envsubst <template >rendered

For example:

$ envsubst <<<'Hello $USER'
Hello jonathan

(Note the use of single quotes so that $USER isn't expanded by our shell, as it wouldn't be in the file which <<< is emulating for us.)

If you'd like to use KEY=value declarations from a dotenv-style .env file, you can auto-export them by setting the -a Bash option:

set -a; source .env; set +a

Something I've managed to avoid ever realizing for 30 years, but now that I've seen it I can't imagine a week going by without using it. The kind of thing that should be part of everyone's "Week 1 in a terminal" training that formal education courses never include.

Ferris Bueller's Day Off

Ben Stein in Ferris Bueller's Day Off

Directed and written by John Hughes, 1986. IMDB

So, way back in my teen years, we had a VHS tape of this, which friends and I played and played and played, probably racking up more rewatches than any other movie in my life. So it was a pleasure to break it out for our 11 year-old, some some 37 years later (!), to see whether it still holds up, and find that it really does.

By chance this was the week after we'd just watched The Blues Brothers, so we got to compare and contrast two movies set in Chicago - a privileged white story, and a poverty stricken, largely colored one, which even share scenes filmed in the very same restaurant.

Back then, I had no idea who Ben Stein was, so it was amusing to see him now and suddenly join the dots. Apparently his infamous "voodoo economics" speech had no script and was ad-libbed.

Reviewing Rooney's comical attempts to break into the Buellers' house made me realize for the first time that this was Hughes' dry-run at what would become Home Alone.

I had always been frustrated that I'd never been able to lay my hands on the "You're not dying" song that Cameron plays while sick in his bedroom (i.e. here's the few seconds of it on Youtube, exactly as it appears in the movie.)

Now we have the Internet, I can see that this failure wasn't exactly my fault - there is no such song. The few bars we hear were whipped up by Ira Newborn specially for the film, based on an old Louis Armstrong song, Let My People Go. Fortunately for us, one man was obsessed about it enough to actually recreate a full length song based on the snippets from the movie. Here is Daniel Simone's Let My Cameron Go, full of a lush Pink Floyd sound, and ripe with the sort of ecstatic anticipation that even Roger Waters would be proud of.

Duly added to my rotation for next time I'm sick.

Update: Dany Boyd's CinemaStix posted a lovely YouTube about how Ferris isn't the main character of this movie.



Resolution and The Endless

Resolution

Resolution (2012)

The Endless

The Endless (2017)

I only had a hazy awareness of Justin Benson and Aaron Moorhead, the writer and directors, before watching these movies. But having discovered them, I now realize that they are doing just about my favorite thing in film: Quirky, intense, psychological drama wound around some high concept science fiction.

Going in, I hadn't realized the two films are related. But then they contain the same scene, viewed from two different angles (pictured above), and it starts to become clearer. As it happens, I watched them out of order - my enthusiasm for The Endless caused me to look up their earlier Resolution. But with hindsight, I think this is actually the best order to view them. If Resolution has a weakness, it's that the science fictional elements seem a bit arbitrary. Why should this supernatural entity focus its narrative-obsessed attentions on these two men, here in this cabin, out in the middle of nowhere? But in The Endless, this particular brand of supernatural outlandishness is revealed to be just part of a wider pattern, affecting many people in this geographical area. Although this is the bigger, weirder story, it is more fully fleshed out and becomes more believable, creating a setting which recontextualizes and improves the earlier film.

Rating: 10/10 if you like mindbending SF horror, 0/10 if you prefer something a bit more polished and comfortable.


Aurora

Aurora cover

by Kim Stanley Robinson, 2015

It has long been held by fans of science fiction that fantasy is a lowly subset of science-fiction, or perhaps a disreputable cousin, one for whom the normal rules of discernment do not apply. If such unlikely and unrealistic things as dragons and magic are allowed, the reasoning goes, then the book cannot be relied upon to deliver any kind of coherent narrative experience, since the lapsed rule-set now allows for any old ex machina plot twists to save the day. A magical "defeat the evil" spell? No problem. A new mythical creature capable of defeating the previously unassailable one? Why not? All reason is gone.

It's more useful though, is to invert the hierarchy of this received wisdom, and consider science fiction as a subset of fantasy. Mentioning this in fandom circles blows mental fuses. Does not compute. But the speculative flights of science fiction are also fantasies. Just fantasies that a particular type of person finds especially beguiling, compelling, and believable. To some extent, I concede that on occasion they are believable because they seem to be a reasonable extrapolation of our current situation. But no matter how reasonable your extrapolation seems to be, it's always possible that reality will zig instead of zag, and even the most humdrum tale of a rocket man's life will find itself at odds with the unexpected reality of suspended human spaceflight in the face of spiraling real-world costs. The vision that one is selling is always, to a greater or lesser extent, a wishful one - a fantasy.

This becomes immediately apparent once we stray beyond the confines of low Earth orbit, to take in the wider scope of science fiction, the vast majority of which encompasses tales across the galaxy, nay, the universe, including time travel, teleportation booths, aliens of every color, quantum reality displacement, and multiversal escapades in which literally everything is possible. These are very clearly fantasies, and it is intensely curious to me why this sort of fantasy is considered more "realistic" or "believable" than, say, flying lizards with fiery breath. Even though the narrative hand-waving that explains away the former - "It's an alternate universe, where different rules apply" - is abundantly adequate to more than completely explain anything in the fantasy realm.

I once asked my guru science fiction critic Damien Walter what makes people consider some stories believable, while other are not. He replied with a statement that has stuck with me ever since: People are willing to invest the effort to provide the conceptual scaffolding around an idea to make it seem believable (e.g. to speculate on the mechanism that might allow for a faster-than-light hyper-drive) when the story fulfills some deeper psychological need for them.

Hence, a story on the same topic as Aurora, of a generation ship sent to colonize an Earth-like planet orbiting the nearby star of Tau Ceti, is (usually) a story about the triumph of modernism. Such stories leverage the sources of strength in the modern world, science and technology and colonialism, and a reader who is invested in a modern world-view will feel validated and empowered by this type of fantasy. They will be will be willing to exercise whatever extracurricular creative effort is required on the part of the reader to make the story believable. Doing so will inspire them with the feeling that their world all makes sense, is leading to something, so that their daily grind is a part of the heroic story of how humanity transcends its planetary origins. This is much more fulfilling than investing any effort getting on board with the waning powers of superstition that are represented by the fantasy genre.

spoilers

In keeping with this, Aurora's colonists are granted every conceivable boon that science and industry can supply. A ship fully ten kilometers across, enclosing twenty four massive biomes, each stuffed full of hills and lakes, soil and forests, microbes and wildlife. A population of well over a thousand human beings. Miraculous nanotech fabricators, and megatons of elemental feedstock to run them. A miraculous acceleration laser, fired from Titan for decades after departure, allowing the ship to coast up to 0.1c, making the journey in only seven generations, while retaining enough fuel to decelerate for arrival. A miraculous magnetic shield protects the ship from catastrophic collisions with stray particles along the way. A benign AI runs the ship, amusingly pressed into service as the narrator of the tale, and grows visibly more sentient, emotionally robust and capable as the years pass.

And hot damn, they are going to need all these things, because in this story, human interstellar colonization is revealed for the fantasy it really is. Nothing works out, and the problems encountered are far bigger than anything the ship's designers planned for. Although the ship does limp into orbit around the destination planet, soon after that people start dying, major disagreements emerge which descend into catastrophic riots, and the ship's society falls apart - when have humans ever invented a reliable form of governance?

There is a rip-roaring final act, that stretched my credulity, but revives the stakes and entertainment value in what might otherwise be a relentless downer of a read.

The book, and interviews with the author, caused quite a stir in science fiction circles. People were extremely angry. The book was attacking their deeply held beliefs that the future of humanity is as a successfull space-faring species. They had invested their identity in this world-view, because of how it serviced their psychological needs for fulfillment and meaning. They had developed a religious conviction around this particular kind of fantasy.

The point of Aurora is to highlight the idea of human interstellar colonization as a dangerous distraction from the very real project of taking care of the long term health of our planet and our society right here on Earth. It is going to take beyond miraculous levels of technology and resources to start thinking about interstellar travel. If, by some miracle, we make it to a year 10,000 utopia, with infinite resources and the wisdom to manage them, then sure, we can worry about interstellar travel. But for now, can we just focus on some of the very basic problems of existence here on Earth, like how to make everyone fed and liberated, educated and fulfilled, without killing our planet to do it? Maybe invent some sort of government that is reliably able to do that? That would be nice.