Mike Moran, software engineer, dad and baldie

The Diagonal Method vs. The Rule of Thirds

Sun, 26 Dec 2021 00:00:00 GMT

I am an occasional photographer and I became interested in how compositional techniques could help me produce better pictures. I've been using the rule of thirds, not least because my camera has an easy to use overlay for it. However, what method should I be using?

I happened upon the Diagonal Method method website. The discussion references a few examples of paintings, adverts, and images which have key elements which overlap a diagonal. However, the validation of this technique is still very unscientific. It's easy to bias your selection towards choosing examples that support your method. Is there a less biased way to test this?

Stepping back a bit, can we answer the question: what are the areas of pictures that people find interesting? If these generally overlapped with those recommended by a particular method then that would support its use.

This is where flickr comes in. They some have idea of 'interestingness'; they don't define it, but we can use it as a proxy for how good the picture is. You can leave notes on areas of photos; let's use this as an indication of where the interesting areas of a picture are. You can follow the history of my investigation on flickr itself, but here's the end result:

What I did:

I found the most interesting image that had been uploaded on each day between 2004 and 2010 (inclusive); to be exact I restricted to images that allowed re-use, just in case I wanted to do something else with them later.
I then restricted this set to only those which have some notes. Here I ignore any note that was square, on the basis that if you've changed the size away from the default aspect ratio, then you've put more thought into the note.
Each note in an image then contributes according to its specificity; the larger the note the smaller the contribution. I assumed that more specific notes were better indicators of points of interest.
The note areas are normalised to a 500x500 pixel square.
Each pixel in a note area then contributes something towards a pixel in the heatmap. If there are multiple notes in an image that overlap then I average the contribution from each note at each overlapping pixel. It is these combined contributions that then get added to the heatmap. In the image on the left, you can see an example heatmap after 10 images have been added.
The heatmap is then simply the sum of all contributions.
I create an image out of a heatmap using a simple linear scale which is visualised on a (Blue -> Green -> Red) scale.

The final result is on the right. The movie is the result of iterating over the 707 images, and shows the heatmap accumulating over time, sampled and shown every 10 images.

My conclusion is that there is a simple bias towards the centre of the image. This data doesn't give any support for either the diagonal method, or the rule of thirds. For both of these, I would have expected more activity away from the centre.

There are some caveats here. The grid for the rule of thirds is unaffected by my normalisation to a square heatmap. However, the diagonal method grid changes depending on the aspect ratio. This means that any bias towards the cross-over points in the rule of thirds grid would show up clearly, whereas a similar bias from Diagonal Method focal points would be smeared out. However, I see no bias away from the centre at all, which I think removes the need to investigate and undo this deficiency. Aside: I can fix this, but it involves a reasonable amount of work to translate the collecting of heatmap hits out of image space and into something more like a parameter space. There are some nasty details to be careful about, like not overcounting grid hits because of the differences in areas of the two grids.

What next? Flickr notes are used for many things, and I've just assumed they always indicate interesting areas. This may not be true often enough to bias the results. If so, I could look more at other information about notes which indicates what purpose they are being used for.

I could also look for other sources of data.

However, for now, I am sticking with the Rule of Thirds.

L Systems

Sun, 26 Dec 2021 00:00:00 GMT

I first encountered LSystems during university; this was in the mid-90s when fractals were becoming mainstream. However, it was only in 2009/2010, when I needed a toy problem to learn Scala and JAX-RS with, that I started writing renderers.

The current version lives at lsystems.houseofmoran.com and I encourage you to play with it! The URLs are designed to be easy to hack. For example, the path /F:F-%3EFF+(+F-F-F)-(-F+F+F):F/1.0-25.0-3.200x200.png breaks down into

/<rules>/<stepsize>-<angle>-<iterations>.<width>x<height>.<format>

Alternatively, just slap "/edit" on the end, to get an editable form. Here a few examples of what you can get:

History

See the following for a history its development: 2009-06-27, 2010-02-28, 2010-03-08, LSystem Explorations on Flickr and a sort-of screencast of a previous version, using Java2D, on YouTube.

Project 365

Sun, 26 Dec 2021 00:00:00 GMT

Back in June 18, 2010, I started taking a photo a day, as part of the Project365 flickr group. I did manage to almost always take one photo per day, but it wasn't until 2013 that I actually fiddled with and uploaded them all.

I wondered if I was alone, in being so lax.

I grabbed data for all submissions to the group from around 1800 members of the group over around the past two years. The plots below show the results of some initial investigations.

The top graph shows the result of grabbing the maximum day reached by each person, and plotting how many people reached that date. There is a strange peak around 140 (the dotted line) followed by a thinning of submissions, and then a last gasp run to 365 (the peak on the right).

The bottom graph shows that most people actually submitted very few pictures; the dotted line on the right marks 365 uploads. Some people submitted more than 365 pictures; this is either a bug in my scripts or some over-enthusiasm on their part.

Weekend Hack: Degrading QR Codes

Sun, 15 Nov 2020 00:00:00 GMT

My friends (Siwei and Hugo) and I were recently discussing some of the possible limitations of QR Codes. This re-animated an idea I'd had many years ago which relied on deliberately degrading QR Codes. One of the aspects was varying the bg/fg colors.

So, I thought I'd challenge myself to get something working in a weekend which allowed me to test this out. However, I started this on a Friday evening and wanted something ready by Sunday, with plenty other stuff to do amongst that.

So, I ended up with a fairly simple front-end implementation (Github):

uses create-react-app
uses a color scale from d3-scale-chromatic twice to get bg/fg color pairs
randomly chooses a pair and renders it using an embedded Rust QR Code library
allows people to say "Not Scannable"/"Scannable" and send this as a custom event to Plausible.io where it is recorded as Goals
the Goal summaries can then be manually downloaded from Plausible.io as JSON, and sorted and rendered again in the app as QR Codes.

Learnings

Some QR Codes that are really easy to distinguish by eye are unreadable by pretty much all of the QR Code Apps I tried (on iOS). This likely means I'm going to have to be a lot more subtle in my degradations. Update: turns out if you use the same qr code repeatedly, which I was, then the iOS Camera App will not re-identify it. So, I updated the site to show a unique code each time, and now have more successes.

I really need to be more understanding of context when asking people for help. I'd asked people on a local Edinburgh Slack to help me collect data, but got frustrated when they pointed out a button wasn't obviously working; I'd deliberately went for the simplest possible thing (for me) of making the button do something behind the scenes, even if it didn't indicate it in the UI. Of course, that was the first thing people noticed!

My reaction was quite negative on this. I was strongly forcing myself to not be a perfectionist as that would risk not completing something in a weekend. However, nobody else knew that, because I hadn't told them. As it turns out, making the button do something to indicate it worked wasn't too hard, so I did get something better out of this.

I'd been doing a lot of Rust/WASM stuff for my Garibaldi project about 6 months ago, and I found the Rust/WASM parts a lot more buggy than expected. This time I encountered problems in publishing and in building. It didn't take me long to find workarounds, but this indicates that Rust/WASM is still not plain sailing.

Btw, a Rust version of the QR Encoder is not strictly necessary. However, I am trying to follow the pattern of doing the UI parts in React and anything that could benefit from stronger-typing or performance in Rust. This also allows me to get more practice in Rust on real problems.

Next Steps

I will most likely return to this Baton project again, but probably not for a while. This was just a weekend hack as a bit of a "palette cleanser" before I return to my Playout work.

Finally, if you have a few minutes, please go to https://baton.houseofmoran.io/ and give it a try yourself. All Data welcome, and I'll even accept criticism of the UI :-)

Playout 1.0, a Game for COVID times

Sun, 01 Nov 2020 00:00:00 GMT

TLDR: I've made a game: Playout. Please DM any feedback or feature requests!

The germ of this idea came from teams moving around at Skyscanner. Eva is an office manager there and often had to manually work out where to place teams when there was an office rearrangement. She had to take into account many preferences and constraints. Iva (a colleague) and I both thought this was something where a computer could be a helper, perhaps representing the task as a Constraint Satisfaction (CS) problem.

There wasn't much spare time to take it further. However, this Summer I went down to four days a week at Skyscanner as part of a temporary COVID money saving plan. So, I thought I'd focus some of my new spare time on seeing if this was do-able.

Iva had given me some pointers on how to represent this as a CS problem, but I was still far from familiar with the practicalities of CS. So, I split it: learn about the domain (make a game) and learn about CS (take a course on it).

Playout is my "toy" version of the problem which I made into a standalone game, because that sounded fun :-)

The core look-and-feel hasn't changed all that much over time, but the internals and explanations repeatedly changed based on feedback from friends and family. There are still tonnes of things to add, but you have to draw a line somewhere, and 33 releases later, 1.0 is it!

Next steps are to integrate what I've learned about CS back into the game, and also gather ideas/bug-fixes/suggestions from people playing the game. So, please have a go at it and let me know!.

Map Maker: Ups and Downs

Sat, 25 Apr 2020 00:00:00 GMT

Ups and downs this week, ups:

The talented King Carmen of Carmenland created a small Garibaldi Biscuit logo for me. This is even though she despises anything associated with squidgyness. Thank you for suffering through this Carmen.

Downs:

This here shows the same browser, same city, same area, but on different screen resolutions. Some areas are connected in one screen that are not in others. This is a known possible issue I have been putting off tackling for some time because I had some more important things to prove first. Thanks to Matt and Laurence for helping me find me this, and show now is becoming the time to fix it.

The problem occurs because I draw the streets on the HTML Canvas and then pass over an image of it (see below) to the Rust part to find connected regions and then return this as polygons.

For simplicity, I use the same canvas to do this as I do for display, which means it has the same resolution as the display area. However, I think this means that on smaller resolutions the image ends up with some blurring which causes some areas to become connected.

The solution will likely be to have two canvases, one for doing crisp rendering at highest resolution (by making canvas bigger) and another for actual display. I've already thought about doing this for different reasons so have some idea of how it will work but will require a bit of refactoring.

I'd really like to fix this bug, but I think I'll do the refactoring first.

Map Maker: Spiking on Plus Codes and Mapbox Turf

Sun, 19 Apr 2020 00:00:00 GMT

In the current version of Garibaldi I have a thing called the "Reticule" which is the central area of the Map that is split apart into streets. This is defined as being some percentage reduction of the browser viewport size, so it is entirely relative to the current display. This was more than enough for me to get this far.

However, to be able to mix and match segments of different Maps I'll need to register the segments as real physical objects with size and position. My planned way to do this is to make the Reticule correspond to a real physical area by having a defined center + a width and height in kilometres, and then use this as a reference area for the embedded segments. I also want something that can be used as a good canonical address for any interesting area.

For this spike I've been looking at Open Location Codes (aka Plus Codes) for the center and Turf for defining the width / height.

I've got some good prior experience in using geohashes, but wanted something with a bit more awareness; Plus Codes having Google Maps support is a biggy for this. The description of OLC in the official repo is also quite good at explaining why they invented it. If I do find any problems, I can always switch over to geohashes as they work very similarly.

Turf is another library from Mapbox and is as an easy way to use physical areas and distances. In this Spike I am using it in a slightly hacky way to get what I want:

get lat, lon point center from Plus Code
for each of width and height, buffer this point by the corresponding km radius (so that it turns into a circle)
get the bounding box for each of these circles
construct a MapBox LngLatBounds by taking the latitudes of the corners from the buffered width circle and the longitudes from the buffered height circle

This is super inefficient but gives me what I need.

I then combine these together into a summary text on top of the displayed Reticule, which updates as a I scroll around. Job done! (this Spike may still be previewable, but if not see video at start).

Likely next step: before I can incorporate this, I'll need to do a bit of refactoring in the main App.

The Journey to Map Maker (aka The Coronavirus Diaries)

Sat, 11 Apr 2020 00:00:00 GMT

My current WIP in the Biscuits Project (Garibaldi) is now at parity with my previous iteration on the things I care about: being able to segment a map based on the structure of its streets. Except that it is all now being done client-side.

It's not obvious it's doing anything special as I deliberately just put them back in their original locations. Being able to take something to bits and put it back together again is a key ability! However, it's not very interesting to look at.

Hence Map Mixer. The core idea is to see what a Map of a city looks like if I swap over similar-size blocks from another city. I don't know what this will look like, so time to explore.

My general strategy for personal projects is similar to the Double Diamond approach. The above is me doing "building the right thing" i.e. what am I gonna build next? What comes next is actually building it.

Doing this seems to break down into these steps:

take two pre-segmented Maps
analyse segments by similarity in each Map
take destination Map and lay it out
take source Map and find segments in destination Map that most closely match destination
replace segments in destination with source segments (do repeated fill if more segments in destination than source)
allow it to re-layout
solidify and re-draw i.e. turn it back into a Map with streets

Next, I will try to see what I already have that can fit these needs, perhaps doing some re-factoring / clean-up to get there. I will also do various spikes on different technologies or approaches that fill the gaps. Then I will incrementally start re-integrating these learnings back into the main app, trying out examples as I go. Finally, I will apply iteratative optimisations on the code as I use it, where it seems sluggish.

First spike: Layout. I am assuming that laying out the Map will not be as simple as drawing a Map because bits from each Map won't fit together cleanly. So, some "jiggling around" will be required. To make it look like a real Map, some constraints need to be followed, such as not having overlapping segments. An idea here is to use some sort of physics engine that can apply these constraints.

I've been doing a lot of Rust recently; the small segmenting core of https://garibaldi.houseofmoran.io/ is built in it. So, my first choice for layout it out was seeing what was available in Rust. Specifically, nphysics2d for doing the layout.

Unfortunately, I had some issues getting nphysics to compile under wasm, using wasm-binding. I don't think it's a dead-end, but I realised for the sake of experimentation, it may be simple to use an old-favourite of mine: d3-force layout. So, stay tuned on that.

Edinburgh Film Festival 2019 (Shortlist of Films, Updated for availability)

Sun, 16 Jun 2019 00:00:00 GMT

Definitely want to see

~~Memory: The Origins of Alien~~
~~The Art of Self-defense~~
~~The Dead Don't Die~~
~~Never Look Away (Werk ohne Autor)~~
Extreme Job (Geukhan jikeop)
- date: Thu 27th
Sakawa
- date: Sat 29th
Ever After (Endzeit)
- date: Fri 28th
~~Abracadabra~~

Maybe

~~Aniara~~
~~2040~~
~~In a Foreign Land (En tierra extraña)~~
Modest Heroes — Ponoc Short Films Theatre
- date: Thu 27th
~~any of the Mclaren Animation events~~
~~Children of the Universe~~

Not sure, but could

~~Strange But True~~
~~The Vast of Night~~
The Hidden City (La ciudad oculta)
- time: Fri 28th
~~I See You~~
Schemers
- date: Sun 30th

A look back at FOSDEM 2019

Sat, 09 Feb 2019 00:00:00 GMT

This year I am continuing my plan to do a mix of conferences. Last year it was GraphQL Europe and PyData Berlin. I'm starting 2019 with FOSDEM 2019. It's now just over a week since I was there, so it's time to reflect before I forget it all.

First impression: "My god that's a lot of talks". I sought out who'd been before and that impression was confirmed along with a subtle warning that it is a little chaotic.

The whole conference is unticketed so in principle there is no limit to how many can arrive. This means it becomes a little bit of a race to get into talks. My first choice of the day was an introduction to Quantum Computing. I soon realised that the people I had squeezed past weren't just coalesced around the hallway demonstrations, but were also in the queue for my talk. Not only that, it was people waiting to get into an already-full seminar room. Thankfully, next door was a somewhat less busy introduction to Ada (outcome: I'm still not gonna use Ada, but I now have a first-hand summary from an expert).

This ecleticism of subjects is something you struggle to find at other conferences. My strategy became to try to go for a mix of talks on topics I know and others I was far less knowledgeable about, and look for clusters of talks in the same room, back-to-back. I won't go over all the talks, but here's some that stand out for me.

"Java Checkpoint/Restore" from Christine H Flood.

The gist of this is to bring to Java fast startup times based on taking a snapshot of a fully warmed JVM using CRIU, applying bespoke Java minimisations to this snapshot, and re-using this as the starting image of separate JVM's. The presentation is effectively: "I've got a prototype that shows this approach works for Java using CRIU, what do you think?".

Given the audience are also experts into these technologies the Q/A part then becomes a lot more useful e.g. one of the audience members implemented this same thing for Smalltalk so could give direct advice and support for the approach. Combine this with the general egoless stance of the audience and the presenter and you get a short but illuminating talk out of it.
"DNS over HTTPS - the good, the bad and the ugly".

This was weird in that I got more out of the chat about it on the way there with Barney than the talk itself. This talk really suffered from another pattern I noticed at FOSDEM: late arrivals. In minute 25 of this 50 minute talk there were still people arriving through the very clunky fire doors, leading to "scrape, scutter, clunk" noises every 30 seconds. At the end the Q/A was ruined by the noise of people leaving as soon as the talk ended; I have to admit I became one of them as soon as it became obvious it was pointless to remain. This is a shame as this topic is controversial and very impactful to the future of the internet.
Saturday: Open Source .NET / Sunday: Open Source C#, .NET, and Blazor - everywhere PLUS WebAssembly.

Hearing Hanselman speak on Sunday was a nice bookend to my .NET experience which started around the time I last saw him during his visit to Scotland in 2012. I'd newly joined Skyscanner, which was largely a .NET / Python / SQL Server shop. The expectations of my colleagues were in stark constrast to what I was recently used to i.e. Microsoft leads the way with language change and useful libraries might need to be paid for, as opposed to a largely stagnant Java language with most meaningful features coming from open-source libraries or new JVM languages. Fast-forward to 2019: Microsoft have 884 open-source projects and they like Github so much that they bought the company.

Hanselman's talk was entertaining, as always, but the real surprise was the Panel on Sunday. I'd went into it slightly blind and it was only when the chat started that I realised it was Miguel De Icaza doing most of the talking; it was technically a Panel, but probably should have been titled "a conversation with Miguel" ☺.

He's obviously achieved somewhat more than me ☺ over his life but as Miguel and I are around the same age, I personally found it very relevant hearing his history and reflections and tying them back to what I was doing.

It was particularly sad to hear how he ended up turning away from some of his projects due to "GamerGate" style death-threats. I recommend listening to all of this Panel talk, even if you're not in your 40's and have never used .NET.

Full list of talks I went to, or attempted to

Saturday

Sunday

About time for a (site) change

Fri, 04 Jan 2019 00:00:00 GMT

Now that Tumblr is showing signs of being on it's way to death I am wary of leaving my content on there. This angst is not reduced by the behaviour of other companies like Flickr, deciding maybe it wouldn't be keeping all those photos for you after all.

The world has also moved on since I last seriously touched this setup years ago, with HTTPS now expected and lots more choices. So, I decided to bring it all back under my control again as a static site.

I'd been eyeing Gatsby for the past year as something I wanted to learn. I finally got round to going through the tutorials over xmas and liked the simplicity.

The first pass port of Tumblr content went well. I then ported over the remainder of the content that was reachable via the old front-page. More details available under "about this site".

The final thing to do was add https support for things I link to as embedded images, like LSystems and Sham Tartans.

I wanted to improve my understanding of how Kubernetes fits together, whilst not spending too much, so took advantage of the new Kubernetes support on Digital Ocean. I've opted for a single cluster which provides ingress and https to services embedded in it e.g. LSystems.

Rather than beefing up the cluster with lots of monitoring and observability from the start I am instead working from the outside-in by starting with a cheap external monitor, updown.io, and will add more detailed monitoring as and when I need it. This is sufficient for hobby projects like this where I am trying to minimise costs whilst learning incrementally, as opposed to going for 100% availability. This is WIP, but so far, after a week or so, I am finding it good.

When backend is frontend, and back again

Tue, 08 Aug 2017 00:00:00 GMT

If you're building a mostly-frontend dynamic site, but want to make it share-able on Facebook/Twitter with a nice preview image, then here's one new (slightly weird) way to do it.

I was playing around with a Sham Tartan generator because, well, why not? I started off rendering this as overlapping divs, but this is no good as an og:image or twitter:image, as Facebook/Twitter won't pick these up as images.

I changed the React front-end to instead render these on canvas with the intent that I then re-use this JS code in the back-end. However, after reading about what the recent headless mode of Chrome gives you, I realised I had another option.

The core of the idea is to run an instance of Chrome alongside your backend. When a request comes in for a preview image you ask it to take a screenshot of the page you want to use as a preview which you then proxy back to the browser.

For example, on https://shartan.houseofmoran.com/ I set the og:image in the meta tag to https://shartan.houseofmoran.com/preview.png. A request for https://shartan.houseofmoran.com/preview.png then causes a screenshot of https://shartan.houseofmoran.com/ to be taken as a PNG which I then return.

The page being used for the preview image here is the same as the homepage, but could be a different url.

My implementation of this is on github, but there are bound to be better versions out there.

You'll notice you still need a backend, so what's the benefit of this? The main benefit is that any technology that runs quickly in Chrome can be used to make a preview image. If I wanted to, I could go back to my div-based Tartan rendering method and it would all still work.

There are sure to be limitations to this, not least the stability of chrome headless itself. But, in summary, if you want to write most of your code on the front-end and want a generic method to create a preview image, then consider this pattern.

Tautona: Visualising similarities between Slack channels

Thu, 18 Aug 2016 00:00:00 GMT

I hadn't done any lo-fi text analysis for ages, and since we've been using Slack at my job for a little while now, it was time to get munging!

My MVP was really simplistic, but actually surprisingly good. I didn't record the output of the implementation at the time, but in summary it used the Jaccard Index between channel memberships as a measure of similarity.

I then did a fairly standard thing of inverting this into a distance measure, and visualising this with a D3 Force Layout. So far, so good, without even touching any messages!

Enter: Gensim. This is a really neat little Python library that I discovered via a colleague at Skyscanner.

It focusses on doing a small number of jobs, but well. I've been playing about with its word2vec implementation in another project, but here I used TF-IDF + Latent Semantic Indexing to produce channel similarities:

The procedure is similar to my MVP, except I treat each channel as a document, consisting of a big bag of words from up to the last 1000 messages. TF-IDF is applied to boost up effect of unusual words and LSI extracts topics. I then compute all channel to channel similarities, convert to distances, and visualise as a force layout.

You can see the video of the output above, but you can also play with a visualisation of Skyscanner, lightly obfuscated.

To run this, you literally only need Python and a Slack API_TOKEN. I would love to see visualisations of other people's Slack channels. Please share them, or ping me on Twitter, and I can host them on Github!

The (lack of) relationship between price and value

Sun, 20 Sep 2015 00:00:00 GMT

In Vienna I had an espresso-fueled discussion with Dimo and Georgi about how the price of goods relates to their value. We got through quite a few different ways in which they are indirectly connected, so I thought I’d write them all down, even if only for my future reference.

Choice paradoxes. For example, if you want people to tend to choose a product at a particular price, you can present it in order with two others: one which is far more expensive, and another which is far cheaper.
Price as independent signal of value. For example, vases, which are sometimes valued because of uniqueness, do not have that aspect as a directly visible property. If one is initially advertised at a “bargain” low price then people may not buy it because this is interpreted as a signal that the seller does not think it is rare. The seller can then raise the price to remove this effect. As long as it is still cheap enough that the buyer does not do an extensive check before purchase, then they may be able to sell a vase which is actually quite common.
Price as future resell value. Similarly to above, prices for extensive pieces of Art are hard to determine. Many collectors buy and sell on the basis of what a group of experts think a piece will sell for at a future auction. To some extent this price is then self-fulfilling, particularly since collectors are often independently rich and can wait to sell when they think the price they want can be reached.
Artificial scarcity. This is the classic "available only today" scam, where the price is reduced for a limited time. A vendor can easily do this across a large catalogue of products where any given product is given a synthetic limited availability.
Misleading discounting. Similar to above, except a product is sold at a high price for some minimum duration, before being "discounted" and advertised at the sellers desired price.
Multi-item bundles, or volume discounts, irrespective of need. This is things like "2 for the price of 1", where only 1 item is really needed. For example, you may only want a small coffee, but the per-volume cost is noticeably more than the medium size, so you are pushed into choosing it instead. You could argue this is also a form of item 1, since you often see small/medium/large as options.
Artificial correlation. For the airline industry alone, I suspect you could write a book on these. For example, typically a seller will have to pay a fee to the credit card issuer on each purchase. However this is a per-transaction cost and does not have any relation to, for example, the number of people travelling, but some airlines use that as a multiplier of the credit card charge.
Brand uplift. For two similar items in two separate catalogues, the company with the better brand is able to charge a higher price. To some extent this is reasonable, because there is value in the expectation of higher quality, reliability, better servicing and possible future resell value. However, these extras come at a cost to the seller, so there is an incentive to smear out the benefits of the brand whilst reducing costs. Brands are built on reputation, but if you can skimp on quality, whilst keeping it hard to notice it has been reduced, or get disproportionate gain in positive publicity from a loss-leading set of products, then you get the benefits of a good brand whilst reducing costs.
Brand as the product itself. There are extreme examples, like trainer brands, where wearing the same brand as your peers is more important than the shoe itself.
Negotiable prices. For example, tipping. There is not a preset price, but you must balance the shame of a too low price against not wanting to spend too much. There may be a generally accepted service charge percentage, which the the buyer just has to know. In some ways, this is a purer price because the buyer, within some constraints, can use the charge as a signal of the true value for them of the service.

Phew. There are almost certainly more, but 10 is enough for now. Suffice to say that advertised prices alone are not a valid direct indicator of true value for all the reasons highlighted above. Technically, yes, "a price is what a buyer is willing to pay", but in the context of all the above, this is just a weightless tautology which does not tell you much about value.

The Global Minotaur: a review

Mon, 07 Sep 2015 00:00:00 GMT

I occasionally read books on economics, and they uniformly scare me.

As a parent, you look back to your own chlld-hood and realise your parents did not know what they were doing. Not because they did something bad, but because you know now that you don’t have all the answers, so they can’t have had them either. The veil of authority is removed but you can still have some degree of respect, even if not 100% agreement.

Not so with economics.

The typical economist, commentator, or politician on the Today programme will ooze confidence in their own reasoning. However, read almost anything on the fundamentals of economics, and you soon realise most of it is founded on large-scale bets, fuzzily-defended confidences and ridiculously dodgy assumptions.

I can respect practitioners, like Entrepreneurs, and to a certain extent, Bankers; more specifically, anyone who has “skin in the game”, as Taleb would say. They have to work with the horrific uncertainties of buying, selling, loaning and building, and make appropriate accommodations to get by. They support no illusions.

Go back now to the voices of Radio 4 and hear the timbre of authority, unchanged. Recent events, such as the Scottish Independence vote, and the Euro crisis, have revealed this to be nothing but bluff and bravado. As pointed out by the book, Yanis Varoufakis is disliked by the establishment because he is erudite, capable, and more than willing to point out the many flaws in the mechanisms of the modern world.

Surprisingly, this is not really a book about Greece. Against the themes of this book, Greece is at the level of roadkill. This is not a book about blame either. Many people are discussed as potent and deliberate actors but there is no desire for retribution.

This is mostly a book about balance at the macro-economic level, with a side-order of theoretical sleight-of-hand thrown in as horrific entertainment on the side.

His main thrust is that the required mechanisms of balance between exchange rates, surpluses and trade that were set up in the post-war period, based on sound fears of a repeat of the Great Depression, were deliberately manipulated in the 70s by America in a way which inevitably led to over-exploitation in the '90s, '00s, and ultimately, to the Crash of ’08.

In my reading of this, the madness of instruments like Collateralised Debt Obligations and Credit Default Swaps was just one possible way an inherently unstable system (The Global Minotaur) could have fallen.

Overall, the book is a great compacted history lesson. My knowledge of the inter-country economic history of the past 100 years is limited, so this is welcome, but this also means I can’t say what he is leaving out. For what my opinion is worth, it was enjoyable and consistent. It also makes me want to go read up on Keynes (for his part in post-war negotiations) and the '70s (for when the madness really started). Whether I’ll have the time is yet to be seen, but it has definitely provided the motivation.

The writing style, pre-minotaur period, is quite strong and easy to read. I read most of this book on the bus and/or on the walk from school to work, so needed to pick it up again repeatedly. However, I found this hard to do when he got into his explanation of The Global Minotaur, for two reasons.

If you’ve heard Yanis speak, you know he can be clear, but sometimes get distracted by florid descriptions, and over-dramatic framing. The same goes here, with repeated entwining of the real with the simile. Regardless of whether you already knew the Greek myth, he makes it work as an explanatory tool, initially. However, he returns to it too often, stretching the isomorphism to make a point when the facts themselves, once you’ve understood the framing, can stand alone.

Over and above the re-use of the simile, he generally repeats himself too often. It’s as if the later chapters are a series of blog posts which have been concatenated together. In that context, where the reader may not have seen all previous instances, it makes sense to re-state. However, here it made me wonder if I’d accidentally skipped back a page, after re-engaging with the book.

Once you get to the end, does he offer any hope, any solution?

Early in the book, he suggests that from great turmoil comes great change. These Crises (with a capital C) are expected, and are almost a built-in razing to the ground in which new seeds can grow. He offers no new ideas, at least not in this book. In a way, he just documents the large scale needs that the global economy has now, and always has had, effectively reminding us that there is no free lunch.

He also repeatedly points out that most of what is happening now, whether it is Austerity or the European Financial Stability Facility, is just more fiction; distractions and abstractions over impotence.

The latest commentary from Yanis in the book is in the preface from 2013. In it he recommends that we resist, that we say “No!” to the mechanisms that he demonstrates in this book as illogical, and that make no accommodation for the The Global Minotaur that has so recently fallen. As we’ve seen, this strategy has not done him, nor Greece, a whole lot of good, but I honestly can’t see what else would have been better.

Looking for Bobby but found Paris instead

Thu, 06 Aug 2015 00:00:00 GMT

Go to any local attraction and you'll see people taking selfies with statues, towers, all sorts of sights. Once, walking past tourists taking pictures of Greyfriars Bobby in Edinburgh, I reckoned that some of those will be posted on Twitter. I should be able to find these, and similar pictures, automatically. How difficult could it be?

Problem: Porn. If you do a search for tweets containing "selfie" in the raw firehose, then you'll find about it's about 60% Porn. I suspect the equivalent live search on twitter is doing some sort of filtering.

Even apart from the problem of Porn, there is no gaurantee they'll even use the term "selfie", so we need a different kind of filter.

My hypothesis was that the selfies I was after would have one person in the foreground on the bottom left or right, with the object of interest in the background. I coded this up, and left it running for a bit.

Umm, ok.

ขำสัสส pic.twitter.com/aEJcf5iYjR
— ฐานิ (@Purepareal) August 2, 2015

ไม่ตอบไม่ได้แปลว่าหยิ่ง แต่มึงตั้งชื่อเฟสงี้เองนะ pic.twitter.com/DHBqWxAhS7
— งัวพีช (Official) (@PeacHZ_YYY) June 8, 2014

I seem to have written a chat app detector. What else fits my description?

Tv shows?

1 yıl önce bugün İŞiD Şengale saldırdı Binlerce Ezidi katledildi binlercesi esir alındı... http://t.co/kuIMIw8465 pic.twitter.com/ybijb5dvEV
— Nurcan Baysal (@baysal_nurcan) August 3, 2015

pic.twitter.com/9jwY1lQsZu
— خالد العوسي (@ousika) August 3, 2015

Framed quotes, inspirational or otherwise.

「村山談話」実態明らかに！内閣を無視だまし討ちのように発表された！談話の中には中国共産党「人民日報」がよく使用するフレーズが使われている。事前に中国と調整した疑いがある＝和田議員 http://t.co/E6DdpoLt99 pic.twitter.com/yOl4Rah2WY
— ななみ9019 (@superneoblack) August 3, 2015

Privatisation is nothing more than shifting wealth from everybody to a privileged few. @Corbyn4Leader will stop it. pic.twitter.com/sI0xK175Wk
— George Aylett (@GeorgeAylett) August 2, 2015

And headlines :-/

Larries: pic.twitter.com/zU3BUbTnHW
— liv (@drunkpaIs) August 4, 2015

With the above, it's easy to see how it matched, but some others are a bit more obscure. For each of the following, have a look at the tweet image for a bit before clicking through to the highlighted face.

ฟินเละเทะ ฟินพังพินาศ เอะอะลากเข้ามุม เอะอะกอด #ริวจิ #AF12 pic.twitter.com/gUjExo6Tn2
— หนุงหนิง (@niing_jaoka) August 3, 2015

face

It's 2 A Day Monday.Good luck to Harlandale Lady Indians Volleyball.U had a Great Camp last week. #E3P #Winning pic.twitter.com/OC3UOjuMxz
— E3Ptraining (@E3Ptraining) August 3, 2015

face

【ピヨ剣乱舞】ついに全ピヨそろいました！！！りつふぁぼ応援してくださった方々ありがとうございます！！！！！ピヨコはとってもかわいいです！！！！！次に誰かが実装されたらまたよろしくお願いします（画像のピヨの順番は愛用している一覧から） pic.twitter.com/0vkwJ7UbAP
— ミラで原稿しよう (@sauri_uto) July 29, 2015

face

You can see more of these examples and other types in my github repo.

But, did I actually find anything? It took me a about an hour of trawling through around a thousand tweets it had found, but here's a perfect example of what I wanted:

Sin lugar a dudas una de las ciudades más hermosas y mágicas ! @RestoDelMundo13 #IvanEnParís pic.twitter.com/B32iQUbFas
— Facundo Gambandé (@facugambande) August 3, 2015

A one in a thousand hit-rate is not great, but given this was barely a days worth of implementation and tweaking, it's not bad. It's also obviously improvable. For example:

I'm using the most basic face detector available in OpenIMAJ, and I'm not even filtering that by confidence.
I can easily filter by relative size as well e.g. only include those faces which take up around 25% to 20% of the image.
There is probably a text recogniser I can use to exclude images with mostly text outside the face.
I can use a more focussed source than the firehose. For example, look for geotagged tweets nearby tourist destinations, or use a larger set of keywords than just "selfie". However, the wide-open nature of the firehose is appealing, so I'd like to see how far I can get with only image analysis.
It's really a single-process app, even though I'm using Spark. If I had a bigger sample, and more limiting filters, then perhaps I could find more examples. I'm limited by the built-in firehose throttle, but I could probably scale out the image lookup, as long as that isn't then throttled.

This is a work in progress, and I'm quite happy with what I have so far, so might leave this for a bit. Always other projects on the go!

Finally, I'll end with one example I found which doesn't quite fit my original intent but which I like nonetheless:

Soon. pic.twitter.com/fuPfEfsRqh
— Arsenal Related (@ArsenalsRelated) August 2, 2015

Drone Futures

Sun, 31 May 2015 00:00:00 GMT

According to BusinessWeek, long-haul trucks in America could be the first beneficiaries of self-driving technology. As far as safety and productivity goes, this should be a win.

However: self-driving cars are already arguably safer than people in urban areas, driving between states is a comparatively easy problem, and the occupant of a truck is not a necessary part of the cargo. So, it’ll be hard to argue to keep a driver behind the wheel.

Around 3% of people in the US are directly employed by the trucking industry. A large fraction of these jobs will disappear, and related jobs will also be at risk. For example, there are many small towns whose livelyhoods are dependent on truck stops for trickle-down income. When truck drivers stop less, or are removed completely, then these towns will disappear, like the towns who were decimated when the highways came along (see: Cars :-) ).

Even so, there may still be a net increase in overall happiness if more companies (e.g. Uber) get their wish of self-driving vehicles. Not least because it would stop Uber applying their apparent policy of treating their human employees like this centuries systematised labor.

http://t.co/Xu8Yw4WGbu : I would like to see some counterpoint, because I worry that Uber is just this centuries systematised slave labour
— mike_moran (@mike_moran) May 17, 2015

The changes coming from self-driving vehicles will be big, and may even be positive. However, vehicles are something we already have and mostly understand. The infrastructure we have now limits where they can go and how fast they can travel. They still have to interact with pedestrians, even if they do it more efficiently than we can. Ultimately, they will be limited by having to follow 2D constraints. Drones, however, have no such limitations.

There is a lot of hyperbole said about drones, but a lot of it is just slightly ahead of real events. Easy availability of cheap drones that can act individually or collectively in a clever way is the sort of change that affects system-level assumptions, because what used to be hard will now be easy.

On the walk to school, my son and I take turns making up imaginary machines. There is a rough theme, and one day it was construction. Inspired by youtube, I came up with a “carpet of drones”. If you wanted a new build of something you'd hire a rolled up carpet of drones for a day, and perhaps a mini fabrication machine. You’d unroll the carpet, lay it flat, attach power, start up the fab, and upload your plan from your phone. The drones would auto-organise by role, some carrying bricks, some small dollops of binder, some straightening out the bricks to cover small mistakes. Come back in a few hours, pack them all up, and you’re done.

Drones are numerous, adaptable, and already cheap enough to almost be disposable. The drone future is combinatorial and far less bounded than self-driving cars. Any prediction I make is going to be wrong or wildly unambitious, so I won’t make any right now. I’ll just leave you with something which, as a fan of Lego, I can't believe I've only just discovered now:

The Microservices Elephant

Sun, 08 Feb 2015 00:00:00 GMT

The term “microservices” is going through a phase where everybody has their own conceptions of it, so as soon as you start talking about it in any detail you quickly realise you all mean something slightly different, and you start questioning the value of language and why you bother talking at all. However, given there is now a book out, and a conference on it next week, it feels like it’s about to become a lot more definite. I thought I’d brain-dump my own thoughts on a few of the conceptions that are out there, before my own stance gets hopelessly clouded by many others.

modules + network = microservices

This is from an article by @tef, whose opinion I generally respect and, whilst I agree that “spreading bits of state across a network is a ... world of pain and suffering” and that “keeping data in sync across a network while tolerating failure is an incredibly hard engineering problem”, I really don’t think that’s the problem that microservices are meant to solve. Note that there are plenty of other things I do agree with in that article, so please go read it.

As in so many things, if you’re faced with a very hard problem, the first thing to seriously ask yourself is whether there is actually a far easier problem lurking within, that is the one you should be solving. In this case, the problem is not keeping state in sync. Bits of state, encapsulated behind a network interface maybe, but separate bits of state, which don’t even need to always be consistent. For example, I wouldn’t build a networked filesystem using microservices techniques, but I would build an advertising system using them.

microservices = contractors for hire

Some of my contractor friends might be offended by the following, but here goes :-) Microservices collaborating within a microservices architecture don’t fully trust each other. They make reasonable demands of each other, and expect responses, based on an agreed contract, but there is always a need to be ready for that contract to be cut short, or expectations to not be met. Microservices are about wrapping unreliable actors, and making them reliable in aggregate and/or ensuring unreliability at the micro scale does not become a macro problem.

This analogy also captures the aspect of taking advantage of heterogeneity: microservices allow you to select the right tool/library/language for the job, as long as it presents itself as a semi-standard service.

A microservices architecture should treat some demands as contractually required and others as nice-to-haves. For example, all services should guarantee a low-latency response, but the quality of response can be varied, or be a partial response, so that the latency remains guaranteed. High quality should be optimised for, but low quality allowed; high latency should be survivable, but low latency expected.

microservices = a portfolio of risks

This is analogy doesn’t fit all microservice contexts, but does fit the ones I like working with. Here, you actively hedge one microservice against another, so that when one fails, or gives a low quality answer, you can fall back on another. This is actually a tactic which works within a service as well as across them. For example, within service: using an in-memory out-of-date cached copy when an external service fails; across service: use recommendations from a less-personalised service when the personalised one times out.

microservices = agent programming for the web

@robstewartuk @russmiles @boicy @russmiles no, no, and three times: no. AOP was a pie-in-the-sky dream, MS is derived from implementations.
— mike_moran (@mike_moran) February 7, 2015

(Gack; I wasted a lot of time in the 90’s thinking about agents).

As you’ll have read above, my own take on Microservices very much treats co-operating services as a network of partially trusted actors, each with their own behaviours and preferences. It’s fine for myself and others to think about them that away, and to design libraries and tools that work well together given these assumptions, but there is absolutely no need to embed this understanding in the systems themselves. I’m not saying you can't build robust, flexible, and apparently clever systems with agents, but rather that they are not necessary.

In conclusion, based on the above, my own idea of what microservices are is very much tied up with what I want to do with them, as opposed to the more wide-ranging meanings of others. No surprise there really, but now I’ve got that brain-dump out of the way, I look forward to finding out more and/or being usefully proved wrong in my assumptions about how to build systems well!

Cost of Delay vs Cost of Failure: a balance of forces

Sun, 21 Dec 2014 00:00:00 GMT

This article presents my understanding of how the Cost of Delay model, in combination with Cost of Failure, can be used to choose between and decompose alternatives.

TLDR: A DevOps team can get long-term rewards of minimising Cost of Delay by balancing it against the stabilising force of Cost of Failure.

I went to a presentation on Cost of Delay many months ago; thanks to Ozlem Yuce for presenting it, and to Seb Rose and BCS for organising it.

There is a lack of clarity below over whether we are measuring possible cost or actual benefit, but you’ll just have to live with this until I integrate/understand this further myself! Think of this as a progress report on my understanding. Feedback appreciated, as always.

An aside on numerical interpretations (can be skipped):

I’m currently interested in using Cost of Delay as a qualitative method, partly because I don’t yet fully understand the implications of the quantitative side. However it is mostly because I am wary of numerical interpretations of methods that affect people (where “people" includes both “employees” and “customers”). As soon as a qualitative measure is represented as a number there is a strong tendency to start doing number-like things with it e.g. summing, averaging, comparing. This is often done very naively and can lead to pain for both the producer of the number and ultimately the consumer. It’s not that it is impossible to do correctly, or even necessarily hard, but it requires a bit of nuance to keep the number attached to it’s history e.g. some measure of uncertainty, like error bounds. To some this may come across as elitist or an over complication, but to me it’s about having respect for the numbers and the people they ultimately effect.

Cost of Delay

The basic idea behind Cost of Delay is to consider the loss you make as a result of not making an investment. The accumulated cost over time is some function of time. This is a very bland way of putting it, so I think it deserves a sketch:

The shapes on the right illustrate some of the different ways that the cost function can behave. To pick one example (green rectangle): say you currently have an account on a cloud provider. They have a deal where you can use their services for free for 12 months, but thereafter you have to pay a yearly fee. This highlighted graph shows the cost over the next 12 months (it would look like a step function if extended indefinitely). Another example (blue ellipse), the classic compound interest on debt: you incur some debt, and then the interest on the debt becomes part of the debt itself, cost increases exponentially.

I hope you get the gist, but let’s try an example closer to home. You have a great new idea you are working on, but you know it could take months to make it perfect, and you are not totally sure of it’s eventual worth. You think there may be a simpler idea you can ship pretty-much immediately, but it’s a bit crap in comparison. Let’s call these choices A ("definitely ok but available now”) vs B ("probably great, but not available for a while”):

Maybe I just think more in terms of functions, but to me the diagram on the left is already helpful in forcing us to think about choices. So, what are some of the possible futures, based on this formulation? Note that now we are talking about possible futures, so these costs are turned into predicted benefits. From left to right:

B2 is a tweak on B which has higher value than A (larger gradient), not as much as B, but which is shippable now. We never come back and do B, because B2 was good enough.
We go ahead and ship A, and let it make some money in production, whilst we work on B. B replaces A when it is ready, and it turns out to be great.
This is like 2, except another more important priority comes up and we don’t do B. Regardless, we’ll have made money on A in the mean time.
We decide to immediately do an experiment on B, but on a cut-down version which will only work for two weeks. It turns out it wasn’t so great (perhaps customers don’t like it). We end up going with A after-all.

There are many more choices or combinations than these, but I’ll leave you to come up with those yourself. My point is that thinking this way, even just diagrammatically, allows us to compare these various choices against each other, and provides a means to communicate these choices to others.

Great, is this all we need to consider? No, but before we go further, let’s call out one aspect I am explicitly not emphasising here: waste. One of the things that can hold you back from shipping an A followed by a B (future 2 above) is that option A is work that you plan to discard later. There is no way round this; it’s definitely the plan. However, we are not optimising to minimise total work done. If anything, we are optimising for expected total value accrued over time. It’s not that efficiency is unimportant, but more that it should not be considered first. I could write more on this but not here; perhaps another blog post :-)

Ok, back to Cost of Delay: there is some risk in these choices which we are not considering, which brings us to Cost of Failure.

Cost of Failure

A very simple model of the Cost of Failure consists of:

“probability”, or 'time till failure': how long from now until it fails
“impact”: when it does fail, how bad is it?
“time to recover”: once you’ve identified it has failed, how quickly can it be rectified or limited in cost?

Variation in each of these produces different models of cost, which dictates the accrued cost over time.

Here, cost may be measured in units of ‘downtime’, in actual money lost, or some other measure. These formulations are still qualitative and the ‘cost' in ‘Cost of Delay’ is not exactly the same as the ‘cost’ above.

The relevance of this comes when you see this as the flip-side of a choice that you want to make for Cost of Delay reasons. For example, consider option 4 previously, where we want to do an experiment over two weeks on B to measure value. If the failure curve looks like an escalation curve (red rectangle above, for example a small increase in site latency suddenly turns into an outage) then we won’t take the risk. However, if we can make it auto-recover (green rectangle above, for example an auto-restart of a service to clear memory leaks) then we may be happy with that for the duration of the experiment. The key aspect here is that the same team, or group of decision makers, must be willing and able to take both types of cost into account.

The great thing about DevOps teams, if you’ve set them up properly, is that you have awareness of both sides, and can choose the best compromise at all times; you see the pain and the gain. If you only see one side, then then there is an imbalance.

In summary: A DevOps team can reap the long-term rewards of minimising Cost of Delay by balancing it against the stabilising force of Cost of Failure.

Sharing applied to Healthcare

Fri, 23 May 2014 00:00:00 GMT

Can we ever bring the sharing economy to health care coverage? If you are not using your insurance, give it to someone else. @yeohsiewhoon
— Filip Filipov (@Filipov) May 23, 2014

The idea of sharing unused insurance has a lot of assumptions baked in, and to be frank, I am no expert on emerging economies (referenced in a later tweet). However, let's unpack this.

Let's start with our goal. Can we assume, for the sake of argument, that it is to provide both protection of the individual against ailments and collective protection of the 'herd' against transmissible diseases ? To keep this simple, let's assume this is provided in the scope of a single country i.e. we don't attempt any cross-border solution.

The choices for a solution come down largely to what degree collective versus individual responsibility is assumed. At one end of the scale we have the USA, where the individual largely expects to have to pay a company for insurance, and then claim on this per usage. The government covers some collective responsibility in the form of disease control, and Obamacare is changing things, but the assumption is largely that you have to look after yourself.

At the other end is the UK where we still pay insurance, but it is a largely flat payment and we don't tend to pay a significant amount per usage. It is a hybrid in that private insurance also exists but that is typically used as an add-on rather than for primary care. Disease control is solidly in the domain of the government.

The notion of "sharing" what you individually have only arises on the USA end of the scale. The implication is that since you've paid for something but not used it, why not enable altruism and allow it to be passed on? In this light, maybe it does make sense to enable this in some way.

At the UK end, individual sharing makes no sense, since there is nothing for you to pass on. In fact, you're already sharing it with others by giving money to the government who does it on your behalf, except you have no control over who it's shared with (unless indirectly via elections).

However does re-sharing even make sense at the USA end of the scale? To be exact, do you actually own your lack of a claim? I suspect insurance companies would say no. The insurance has been sold to you at a particular price on the basis of actuarial data about likelihood of a claim. You have been sold an option to claim healthcare at some future point. That option is not transferable because it would upset the model of expected claims.

Now there is nothing stopping an insurance company from changing the way they sell it at the beginning so that it is transferable in some way. However, I suspect you'd then pay more for that and there would still be restrictions (e.g. only shareable amongst family members). An insurance company would not allow a very open interpretation of sharing because they make a lot of their money on people who pay but don't claim.

I'm reading between the lines here a little and assuming you mean "sharing economy" in a very open sense. Given this interpretation, we're already doing it in the UK, just not under our individual control, in the form of the NHS. At the USA end of the scale, it would only be possible in a very circumscribed form. This implies that more sharing is possible in a Healthcare system like the UK.

When it comes to how this applies to emerging economies, I really don't know. I suspect the level of sharing of unallocated resource depends on where it falls on the spectrum between USA and the UK.

Naked only please

Fri, 03 Jan 2014 00:00:00 GMT

Navigating a newly-discovered German swimming baths is a bit like an episode of the Crystal Maze. For those younger than me: this involved a series of rooms with dastardly puzzles which you needed to solve to get to the prize. Except the 'prize' here is men's bare arses.

If you're used to the slightly prudish British swimming baths, where full nakedness is explicitly called out as something you must not do, then it comes as a bit of a shock. It's only really made acceptable because of stereotypical German efficiency.

In a typical Edinburgh swimming bath, showers and locker areas are generally unisex with small changing cubicles along the sides in which you can hide your private parts from view. This flexibility makes the building very open, architecturally.

Unfortunately, this seems to coincide with greater responsibility put on the bather. In Portobello baths you must remember to bring a pound to put in the locker as deposit; you can choose the locker yourself, which is good, because about one in ten are broken. In contrast, today at Hofwiesenbad Gera I paid and received a key for my locker. How did I know which one is mine? Well, I read the number on the key and used that one, of course.

German baths are like a factory line, where everything has been worked out for you in advance, if only you can go the right way. Once I did, I found a very well looked after private men's shower area. Unfortunately, it came with some less well-kempt buttocks on display.

I've yet to decide if the efficiency is worth the prize.

Looking back on my 5:2 diet

Thu, 02 Jan 2014 00:00:00 GMT

So, a year on, how's my 5:2 diet going? The good news is that, looking at the data, I've lost a whole, umm, 2kg over the past year ... hmm, wait. Ok, maybe not so good. The full graph tells a bigger story:

I reached my lowest weight of 86kg first on the 28th June 2013. If you compare that to my maximum of 97kg on the 5th of January then a reduction of 11kg over 6 months isn't too bad. However, as you can see from the graph, I started trending up again soon after.

The second axis, of total_calories expended through exercise, probably explains a large part of why. It shows that the extent and frequency of my exercise reduced soon after mid-year. Also, If I remember correctly, I kept bingeing a bit too much on off-days, as I was afraid of.

So, conclusions? All I can really say based on this is that a combination of exercise and 5:2 fasting worked for me. Now I just need to back to doing it properly again!

Spikes vs Tests: Brothers not Enemies

Thu, 26 Dec 2013 00:00:00 GMT

I write this post with a certain amount of guilt. If you look at my public coding persona (https://github.com/mikemoraned) it’s not unusual to see code with minimal tests, and a wide variety of styles and libraries. Compare that to my ‘at work’ style and, you’ll have to take my word on this, you’ll find a lot more tests and consistency. Why the difference? Most of my non-work code is the result of a Spike.

In a similar vein to my other article, it’s important to know why you are doing something, and what ‘done’ looks like.

I do a Spike for various reasons, like wanting to explore a new language, or I’ve got an idea about how something might work, but I’m not totally sure it’s feasible. A Spike could possibly go on forever, so you have to choose a semi-arbitrary limit on when to stop. If I’m on holiday then I’ve got more free time, but that also means it’s easy to get distracted into Yak Shaving territory and I also usually have a stack of ideas waiting to try out. In practice, I’m usually ‘done’ either when I’ve learned what I needed to, or a couple of days have passed.

Analogy hat on. Spikes serve the purpose that mockups do in other disciplines. It’s easy to have an idea in the shower, a lala-land where things always work, and think you are done. However, when you try to make it you find the bits are too heavy and fall apart when put together. Let’s consider a couple of recent examples.

My Playing music when someone is typing idea came when I was on a plane. After breaking it down I identified what I thought was the most difficult technical part and I did a Spike on that. I almost ran out of time, because I’d made a stupid mistake interpreting radians as degrees. However, I got to the point where I thought it was proven feasible, so I stopped and wrote it up.

Should you use tests during Spikes? Maybe. Should it be test-driven? Definitely not. If I’d done this Spike that way, then I probably would have found the radians bug, but the whole process would have taken a lot longer. Test-driven development is good for situations where you need to go from no software, to working software, delivering small units of working, valuable software along the way. With Spikes you’re taking on the risk of either delivering nothing, or, at-best, ending up with something which basically proves the point but which is very fragile. This is ok, because this is balanced against the risk of not finding out anything at all.

The reason I suspect most people who do Spikes still advocate doing them in a test-driven way is because they suspect the Spike-produced code will be taken from their hands and used for realz. You can make this less likely by using technical hints, like doing it on a branch. Ultimately, the problem of “shipping the prototype” is a social problem, not a technical one, and not one I have any easy solution to. However, for me, personally, on my own projects, I can just not be stupid.

This first example was about proving if it’s technically possible to detect fingers moving a keyboard. Any final product would look nothing like this. My second example is different, and is more akin to a working mock-up. It started out as this (excuse my ridiculously bad sketching):

As explained in the post, I have some ideas about using GeoHashes to show geocoded images. This formed in my head as way of exploring a series of pictures, and I did the sketch above around what it might look like. Over a day or so, I found a source of images, and started exploring how to show them. It was important to use real images, because I wanted to get a feel of whether exploring a real, but pseudo-geocoded, space made sense. I soon dropped a lot of what I’d originally envisaged, such as sliding your finger over images to browse them, because although it would be fun to implement, it wouldn’t help me with my larger mission. In the end, I ran out of time, but I did get as far as seeing what it felt like to explore the space, and what the problems were (“where the bits were too heavy"). I haven’t learnt as much as I’d liked but it’s important to not just blindly bludgeon your way forward, because that way leads to eventual pain.

So, in summary, use a Spike when you want to explore, don’t be ashamed to not use tests, limit your time, and keep the learning, but not the code.

Playing music when someone is typing

Sun, 22 Dec 2013 00:00:00 GMT

It all started when Carmen asked me (something like): "couldn't you play that music from Murder She Wrote at work whenever you're typing?" Given I was on a plane at the time, with nothing much else to do, why not think about it?

Requirement: Play music when someone is typing

Basic idea: play that random annoying music from Murder She Wrote in the office whenever someone is typing

Parts: detect typing + interpret as instructions of when to play music + play music

The hard part is detecting the typing, not because you couldn’t write a program to listen to key presses, but more because people wouldn’t want to install it.

All we actually care about is a ‘key stream’ of activity, which indicates someone is typing, so we don’t care about the individual key presses, or even what the keys were. We can also accept a high false positive rate on individual key press, as long as we smooth out any variability over time.

Some ideas that don't involve having direct access to a key-press stream:

Listen to the noise of people typing. Probably technically possible, but hard to get right. Would vary a lot over keyboards, and need calibrated.
Detect when people’s hands are over the keyboard and moving.

This last one has legs. We could actually look for moving flesh i.e. we look at the keyboard area, detect the pixels which are probably from someones hands, and look to see if the number of them change rapidly over time. This is a very low-level detector which would ‘detect’ lots of other things, if it wasn’t looking at a keyboard, but is probably good enough for this problem.

We can validate this perhaps by using an existing flesh-detector and feeding it pictures taken from a camera looking at a keyboard and see if it sees changes. The sampling rate would probably need to be quite high because peoples fingers flit over a a keyboard very fast.

To smooth out any noise, we could interpret the %-age of fleshy pixels that are moving as ‘energy’. This could be used with a simple threshold, or used as energy of a 1D ball being thrown up in the air, which we let decay, so that small gaps in energy don’t show up.

We then just need to turn this into either a 'stop playing' or 'start playing' signal. And hire an orchestra. Job done. (Carmen adds: "Also, somebody needs to make a hairdresser’s appointment for everyone typing so that they can all have blue rinse perms done.")

Having a go

Since I'm on holiday, and I've done something similar before, I decided to have a go at the 'flesh-detector' part. Overall, it basically works. The video below shows me using a mirror to get my laptops camera seeing both the keyboard and my hands:

The output is very noisy, and show's a reasonable amount of non-skin pixels, but given I've skipped over quite a few of the steps of the Fleck and Forsyth (described in "Naked People Skin Filter") algorithm, I'm sure it could be cleaned up. For example, I don't try to filter skin-pixels not surrounded by other skin-pixels.

Next steps?

I'm not gonna take this any further, even though it was fun, because I've got plenty of other projects to make progress on. If I was I would most likely look for an existing library and see if that works better. If I couldn't find any better library, the minimum to turn this into a 'typing detector' is probably to do diffs between successive skin-pixel snapshots, and look for motion; something very similar to the method in the article I used as a template.

So, in summary, no need to book the orchestra quite yet.

Pair Programming, the Good, the Bad and the Ugly

Sat, 21 Dec 2013 00:00:00 GMT

Reading http://phinze.github.io/2013/12/08/pairing-vs-code-review.html has made me consider again what I think of as "pair programming". It’s a phrase that is misinterpreted, so let me clarify what it means to me.

I’ve worked on different teams where, none, some and constant pair programming was done.

Not doing any is dangerous, and leads to the classic rock-star/lone/hero programmer situation where, by the time you realise you have a codebase consisting of a series of opaque but clever hacks, it’s too late to change anything.

Pair programming all-day, every-day can be liberating to begin with, but degenerates into absurdity e.g. pair reading of documentation. It’s not as bad as none at all, but it covers up the benefits, and provides plenty of ammunition to those who are against it entirely.

So, I definitely come down in the favour of ‘some’ pair programming, but if not all of the time, then when? Rather than prescribe a schedule, it’s important to realise why you might be doing it, and apply it as a tool.

During pairing for knowledge transfer, between an expert and a learner, then you can expect the task to take longer, because the primary goal is learning, not fast completion of the task. These sorts of sessions are best triggered as a result of a planning session conversation, where it’s obvious only one person on the team can do the task. The pairing session is ‘done’ when it becomes clear the learner can now do the task on their own. It’s important to realise that pairing for knowledge transfer is not always the best way to pass on knowledge. Repeated pairing is effectively a very slow ‘flood fill’ algorithm. Sometimes, it’s good to spend time doing a short one-to-many presentation on what you know.

If we are instead pairing to solve a problem, then the expectation is that one half of the pair is stuck in some way, or a larger problem needs to be discussed. Here, ‘done’ is when the person becomes unstuck, or a plan has been reached that looks feasible. These sessions can last seconds, minutes, or hours. Some of my most useful pairing sessions have lasted a few minutes, ending as soon as it becomes clear I’ve missed something obvious.

That last example is one reason why I mis-trust code-review, since it can too easily come at the end of a unit of work. The right feedback at the start of a task can totally change how it’s done. Once you have a functional bunch of code that’s ready to check-in or to deploy, it’s hard to push back on a bad design. A lot of the differences are down to social context. Code-reviews typically happen in a context of challenge, where one programmer presents a completed artefact for others to find fault with, whereas pair programming is typically collaborative, where you work together to share knowledge or solve a problem.

In conclusion, "pair programming” for me is a tool, not a way of life, and should be applied based on a clear statement of why you are doing it.

Playing around with Verlet JS

Sun, 08 Dec 2013 00:00:00 GMT

I'm considering using Verlet.JS as a minimal Physics engine for another project of mine. I tried it out by porting over my little LSystems Crochet project to use it. You can see the results here: http://mikemoraned.github.io/lSystemsCrochet/verlet.html

It mostly works, apart from one crucial aspect: repulsion. Without it, there is nothing forcing the nodes to bunch up and form interesting patterns. The D3-based version (http://mikemoraned.github.io/lSystemsCrochet/) does this by default.

This is not really a fault of Verlet.JS, as it could probably be implemented as a constraint, but it is interesting how such a seemingly small difference can lead to such a boring outcome!

I'm gonna leave this here for now, as I most likely don't need repulsion for my other project. Give Verlet.JS a try, it's pretty easy to get going.

Web Performance Days, New York: "Load Testing" Open Spaces Session

Mon, 21 Oct 2013 00:00:00 GMT

One of the best hours I spent in New York was in an open spaces section of web performance days, which followed on after Velocity NY. It was on "Load Testing" and was led by Alexander Podelko and had a few other contributors, including myself.

I'm writing this partly for my own benefit because it was one of the few sessions where I didn't take any notes straight after, and I want to brain dump whilst it's still relatively fresh. Below, what I remember us saying is in normal text whilst my follow-on thoughts and commentary are in [italics].

Load Testing can be broken down into three kinds: Smoke Testing, Regression Testing and Scalability Testing [aka: 'Did I just break the build?', 'Will this affect production?', and 'Oh frack, we're on Reddit, what now?']

[We didn't talk much about Smoke Testing, but ...] Smoke Testing is all about doing enough to sanity check that you haven't done something incredibly stupid. [In the context of Load Testing this means hitting the system under test with enough load and with enough variety of requests to trigger obviously bad behaviour. For example, for a search service this could be submitting a mix of query difficulties over a short period of time, and asserting that the response time is always below some threshold. The most simplistic way to derive a synthetic set of tests is to run your integration tests repeatedly.]

How you go about Regression Testing depends a lot on how you do releases. If you have big-bang releases where the time between them is large, e.g. weeks, then Regression Testing is more about modelling and/or testing your system under expected inputs, and seeing how it responds.

If you have releases that take hours or days, then you have the luxury of up-to-date samples of system behaviour, but very little time to understand the model that underlies it all. [It wasn't stated explicitly, but I think there was also general skepticism that straight-forward models even exist]. Given this context it's possible to think of your systems performance as one of the many measures in an A/B test, and to skip Regression Testing. As long as you have a way to quickly roll back your changes and have bounded your possible losses as part of normal A/B test procedure then this can be a way to test in production safely. If full A/B test rigour is not required, then an even simpler version is to have a canary (aka staging) box in production, which is examined for performance degradations over time.

Another technique blurs the Smoke/Regression test distinction: replaying a subset of your real traffic against a pre-production system and checking for performance regressions against what production did with the same input. [This can be automated to an extent by comparing summary statistics e.g. differences in standard deviations.] If your system under test can't be fully isolated from outside influence, which is typical, then you'll have to allow for some skew between production and test. [For example, if you're storing information about a customer then there will typically be more of it later on so performance of anything based on it will be different. ]

[If you want to minimise the effect of the skew, then you could replay the traffic against two non-prod systems, one running production code and the other running the new code. However, this has a lot of setup cost and isn't too dissimilar from an A/B test, or canary in production, so why not just do that?] Additionally, each call replayed may have a real monetary cost, over and above the hardware required, which could become significant if calls are replayed in bulk.

[Replay testing may be justified if what you're testing isn't ready for production for non-performance reasons. For example, if you're re-implementing an API using a different back-end, but it isn't functionally complete yet. In these cases, it's desirable to treat the performance of the existing system as a feature that has to be maintained in parallel with any features added.]

[We didn't talk much about Scalability Testing, which is what people normally associate with Load Testing, unless I missed something?]

[Here endeth the braindump. If you were at the meet-up and any of the above is incomplete, misleading, or just plain wrong, then please let me know and I'll update it.]

min(weight) = 90.0kg! (gotta work on avg(weight))

Sun, 24 Mar 2013 00:00:00 GMT

I've been doing the 5:2 diet now for around three months now, and just this week my min weight hit 90.0kg! As you can see from above there is a lot of volatility in my measurements; I measure myself every weekday at around the same time, but there is still easily a range of 2kg in the measurements. My fast days are Monday and Thursday, and my weight, unsurprisingly, is lowest after those days.

If you look at the data since my records began, you see the effect on the weekly average:

It'll still be a while before I average below 90kg, but it's encouraging to see it going down.

Overall, the fasting days are a lot easier to do than I'd thought. You have 600 calories to spend, and I usually split these between 300 calories for coffee/lattes in the morning and during the day, followed by 300 calories for a meal in the evening. I don't have much variety in meals; typically it's some pitta bread with sliced meat. When I do vary, I stick to what I can easily scan into http://www.myfitnesspal.com/

I chose Mondays and Thursdays as fast days because I don't do much in the evenings on those days, which makes it easier to stick to. I've found I can do exercise on these days too; I'm up to 6 mile runs with no obvious ill-effect.

On off-days I've been taking advantage of "eat what you like" a little too much. The freedom is nice, but to be honest, I do sometimes binge. I don't think this will ultimately effect whether I will reach my goal, just when. I've done around 3kg in 3months, and being down to 87kg by June would be nice, but 85kg would be even nicer!

BrowserStack: thumbs up on first minimal try-out.

Tue, 22 Jan 2013 00:00:00 GMT

I just tried out http://www.browserstack.com/ for testing my minimal IE8/IE9 changes for Nice to have. It's actually really quite handy. Take my review below with a pinch of salt though, as I only used it for 30 mins. I did a free registration around xmas and forgot about it, so that's all I had left today! Silly mike.

All the standard debugging tools are set up (e.g. firebug lite), and you get a single window to play in. You set up a tunnel through a signed app so that it sees your server, choose your OS/browse combo, and your off. You do your edits locally, as normal, using whatever libraries and editor you fancy. It is a bit laggy, but usable, and that could have been my connection to blame.

It's around £12 ($19) dollars a month. Given I can set up a VM locally for IE testing if I really need to, that's not in the range where I'd immediately upgrade to paid, but I'll definitely consider it next time I'm doing a bunch of IE testing.

Brickman Experiment 1

Mon, 21 Jan 2013 00:00:00 GMT

I had some fun making snow bricks at the weekend. Technically, Nils was present but it would be unfair to say I was playing with him; "directing", perhaps.

So, anyway, obviously you should try to use snow bricks to make a Lego man. Obviously.

It turns out that moo milk cartons are quite good brick molds if you cut off one side and leave the flap on.

This one got to the point where I was just about to make a head, but then thought I should defer until later to give me a chance to get Nils interested. Unfortunately, headlessness was the least of its problems: it fell over a minute after I came in. Oh well. Looks like balance is gonna be a bigger problem than structural strength.

Here's hoping it snows again next weekend!

Reading the Tea Leaves

Sun, 30 Dec 2012 00:00:00 GMT

When I was at Amazon I looked at a lot of graphs. Graphs of hits, graphs of cpu, graphs of all sorts of shit. Graphs are great, I loves them. However, the problem is that if you aren't in a habit of looking at the relevant graphs every day, and then something happens, you're faced with the problem of deciding what's normal. It is so easy to find patterns, especially if you have a good reason to think something important is happening (either for a good or bad reason).

I'm reminded of all this when I happen on a GeoCities-tastic page like this: http://www.trading-naked.com/headandshouldersREVERSE.htm I am no stock expert but I'm willing to bet the patterns he is using are tenuous at best. Then again, maybe the market wouldn't work without people like him?

"Nice to have" v0.0000001

Sat, 29 Dec 2012 00:00:00 GMT

Ugh. Enough holiday hacking for now. I had an idea for an app a few weeks ago, and finally had some time between xmas celebrations to hack up a first version. Here it is: http://nicetohave.houseofmoran.com/ (source: https://github.com/mikemoraned/nicetohave-app)

I'm not gonna talk much more about it until I've dog-fooded it a bit myself for my own personal backlog. Feel free to use it though, I always appreciate feedback.

For now, I'll just leave some links which helped me to write it.

Trello:

https://trello.com/docs/gettingstarted/clientjs.html http://jsfiddle.net/E4rLn/ https://trello.com/docs/gettingstarted/index.html https://trello.com/docs/api/card/index.html#put-1-cards-card-id-or-shortlink-pos

D3:

Scales: http://www.jeromecukier.net/blog/2011/08/11/d3-scales-and-color/

Ordinals aren't great to work with: https://github.com/mbostock/d3/pull/598

Brushes/selections: http://people.ucsc.edu/~pmerritt/d3/examples/brush/brush.html http://bl.ocks.org/1667367#index.html

Knockout custom bindings:

http://knockoutjs.com/documentation/custom-bindings.html

Knockout Event bindings:

http://knockoutjs.com/documentation/event-binding.html

Twitter bootstrap form examples:

http://www.w3resource.com/twitter-bootstrap/forms-tutorial.php

Ejs/express:

http://robdodson.me/blog/2012/05/31/how-to-use-ejs-in-express/

Sammy + Knockout example:

http://learn.knockoutjs.com/WebmailExampleStandalone.html

"Less" middleware:

http://www.andyjarrett.co.uk/blog/index.cfm/2012/4/11/Express-Less-and-lessmiddleware

Annoyances:

"top" and "left" css attributes don't work in FF unless you specify a unit i.e. "px". Why do I always forget this?

Resources for a designer transitioning from print to the web?

Sat, 29 Dec 2012 00:00:00 GMT

A friend of mine who has a background in print design (copy, layout, some photography) is wanting to move his skills over into web design. I am most definitely on the other-side of this divide, so I asked another web designer friend what resources (website/book/whatever) they'd recommend:

Most things by Jakob Nielsen

About face 3

Steve Krugs Don't Make Me Think!

Seductive Interaction Design

The UX Book

Sketching User Experiences

Any book by Bill Buxton

I think that's a good start I've purposely left out any core design as he should have a solid foundation in that already

Any more recommendations?

Heed!

Tue, 31 Jul 2012 00:00:00 GMT

And so, the saga continues, baldie once again.

Interestingly, this time, I was very conscious of when it started happening. It may just be my imagination, but soon after I got a few midgy (or Mücken) bites in Germany, I noticed my head was a bit itchy. In the following days, the hair loss seemed to accelerate rapidly.

My hypothesis is that given that aa is an auto-immune disease a reaction to a bite could somehow start my immune system off, but I obviously can't prove it. However, it's about as good as any hypothesis I ever heard from my doctor since she basically had no idea. To be fair, she was very apologetic about it.

Anyway, c'est la vie. Now I can go back to disturbing people by absent-mindedly rubbing my head in meetings ;-)

Truth is more depressing than fiction?

Fri, 20 Jul 2012 00:00:00 GMT

This holiday, I finally finished "The City & The City" by China Miéville.

My paper version of the Miéville book was a gift. Its hugeness makes it hard to carry, so I never got round to reading it. I bought it again for my Kindle (I wonder how often that happens?). Alongside his newly coined words there are many that lie just outside my vocabulary, so the built-in Kindle dictionary was very handy. Some of his invented words, e.g. "grosstopically" and "toppelganger", make so much sense. I almost wish I lived in his cleaved world of seeing and unseeing, just so I could use them.

I'm reading another book of his, Embassytown, right now. It is more outright Science Fiction, and is spattered with new words. A main subject of the book is language, so this richness makes sense in this context, but I imagine I would grow sick of it if the usage wasn't so relevant.

I seem to be into a 'divided city' shtick this holiday, as I also read Jerusalem by Guy Delisle. A lot of it is just day-to-day errands and looking after his kids; not surprising, given it's basically a diary. However, it is leavened by the many WTF moments when he encounters another bizarre behaviour of the locals. The strange but rigid rules of movement in The City & The City seem pedestrian in comparison to the complex enforcements in place in Israel and Palestine.

Delisle isn't primarily there for journalistic purposes; he mostly just happens to be there with his wife, who works for MSF. This make Delisle a lot softer than other authors, like Sacco. I find it hard to read more than one of Sacco's books in quick succession. They are lively, in-depth, but simply too depressing.

However, I haven't read one of Sacco's in a while, so it may be time to spend more holiday money on Amazon ... recommendations?

Boomeranging returns

Sun, 13 May 2012 00:00:00 GMT

Yesterday, I went out with Nils and threw my boomerangs for the first time in 10 years. It was really cool, not least because Nils decided to go get them for me, unbidden! On a windy day like that it makes a big difference having someone fetching them.

None of my boomerangs are suitable for Nils, so I've ordered some online. Apart from boomerangs.com most of the sites seem to have been last updated when I stopped throwing!

I'd be interested to hear of anyone else throwing boomerangs in Edinburgh, not least because I want to try out their 'rangs (see what I did there?).

LEDs in the rain

Mon, 30 Apr 2012 00:00:00 GMT

The NVA Speed of Light practice session was the best time I've spent in the rain for a while! However, I am sooooo glad I bought myself a proper thermal top and rain jacket. Not cheap, but worth it.

I really enjoyed myself tonight. I would love to have seen the faces of any passers-by stumbling upon the lines of led-encrusted runners, all trotting along like some sort of Alien Congo dance.

It's a pity the weather meant we aborted practicing more of the moves on the crags for real, but hopefully there will be some free spaces in future training slots. Regardless, I can't think of a better reason to stand around wearing LEDs in the rain.

The hair, eet ees back ...

Thu, 12 Apr 2012 00:00:00 GMT

... sort-of. As I had expected might happen, my Alopecia has receded and the patches that were bald before are now white. I'm gonna give it a go, as I'm happy having a bald head but to be honest I'd be happier not having to shave it every other day. However, I'm not gonna change my Gravatar just yet.

I took the photo above during our visit to the Newcastle Centre for Life. Carmen did the (excellent, if I may say so) illustrations for the "Naughty Monsters Explore the Planets" planetarium show that just started recently.

La Grand Weekend, Gromit

Sun, 08 Apr 2012 00:00:00 GMT

On Friday, I had the opportunity to see La Grand Illusion at the Filmhouse,

I can totally see why this film was so hated by the Nazis, because it portrays war as ultimately pointless. There is no great 'enemy' within these prison camps, on either side. There are only world-weary men waiting, and hoping, for the end to come.

My only criticism is that the characters seem almost too humane; but perhaps this is my cynicism speaking. Regardless, it's a wonderful film that well lives up to expectation.

If you like this film, then you may be interested in others from 'Breaking Out and Breaking In: A Distributed Film Fest of Prison Breaks and Bank Heists'; don't worry too much about the timetable, I'm already three months behind.

All change on Saturday for "The Pirates! In an Adventure with Scientists!". It's been a while since I've seen an Aardman film, but thanks to cheap tickets via Hugh (thanks Hugh!) and a resounding answer of "YES" when I asked Nils, we went along.

I'm beginning to think I'm very conservative when it comes to violence in kids films. Pirates isn't a particularly scary film, compared to some others, but it still has plenty of knives, killing (off-screen) and impending doom. Nils survived, and there were no nightmares, but he was keen to point out that he had to hide next to me three times during the film (perhaps this number should be a new indicator of scaryness?).

I think I lolled three times and at one point Nils made a noise I've never heard before which was 20% anguish and 80% joy, so overall, a hit.

I'm going to see Aliens on Tuesday, so this month is turning into a proper film-fest after months of nothing. Don't worry, I won't be taking Nils along to that one ;-)

Finally, the weekend was rounded off on Sunday by searching for Easter eggs in the garden. The chocolate was very nice, but the "Easter Bread" made by Carmen was even better (it's like Mr Kiplings Exceedlingly Good cakes, but with the marsipan top spread liberally throughout the bread). Carmen claims she made it a bit too sweet, by accident. I can live with these mistakes.

You've never had it so good

Sat, 24 Mar 2012 00:00:00 GMT

What's the opposite of Nostalgia? Nowadays, I can think of an idea, code it up and have it on heroku in minutes. Working at a start-up in 2001--2004 was somewhat different.

Back then, the first task was to decide which of the many (over-priced) Cisco routers you should stick in your closet. The people at our service provider were all friendly and helpful, but the idea of personally visiting your boxes now seems mildly archaic.

Going with a cloud-based provider is still not a trivial choice. Little sysadmin-level expertise within-company means you are more beholden to AWS/Azure/Who-ever to keep your service running. Security and trust are issues to be settled, but it's a welcome choice to be able to spend on your unique selling points.

In summary: you've never had it so good.

p.s. We also had to sweep the lake.

Beach Facts

Sat, 17 Mar 2012 00:00:00 GMT

Some things established today on the beach:

The statutory countdown sequence for throwing stones is "3, 2, 1, <throw>", and not "10, 9, ... 1, 0" which is reserved for Rockets.
It is surprisingly hard to guide water pools to the sea.
Nils didn't do anything interesting, and wanted just to pretend he was pushing a car (rock) through snow (sand), but he enjoyed himself anyway.

Normal service will resume ...

Mon, 12 Mar 2012 00:00:00 GMT

You may have noticed some blog posts are missing. This is because I am migrating them across gradually from my old blogging system to Tumblr.

Also, the RSS feed for tumblr (which I use on the main site) takes a super-long time to update; may have to work round that.

S'all

Note to self: do not invent time machine

Thu, 16 Feb 2012 00:00:00 GMT

If I did have a time machine, I'd be spending a lot of my life getting slapped in the face by my future self.

Last week I spent far too long debugging why Heroku was refusing to accept my rails app when I did a "git push heroku master". I had all sorts of theories and learned about the various reasons Heroku has for saying "Heroku push rejected, no Cedar-supported app detected". The actual reason? I was on a local git branch and Heroku was being pushed to from my master branch, which was not a rails app.

SLAP!

Incidentally, future Mike, if you do want to push from a non-master branch (e.g. "stuff"), then do this:

git push heroku stuff:master

Glasgow Science Centre Visit

Wed, 15 Feb 2012 00:00:00 GMT

Once again, simple physics wins against ambitious simulations on computer displays.

For example, we briefly played with an illustration of a red/cyan Anaglyph. It allowed some very simple computer models (e.g. a box, a molecule) to be rotated on-screen and viewed through glasses. However the refresh rate was abysmal and overall the interface was not responsive. Nils' attention was lost in seconds.

In this high-distraction environment, you need to immediately show or do something, even if it's not the main feature.

My favourite display of the whole day was a rotating disk on which you could place plastic wheels. If you let them spin up in the right way, and then let go, they'll stay in the same place for a surprisingly long time. Even when they start drifting off, they can orbit a few times in interesting ways, before eventually flying off. And when they fly off, you just have to have another go.

In these physics-only displays, if you do come up with a cunning workaround or variation, reality is ready to simulate it perfectly, and reward your ingenuity.

Getting in the holiday mood

Mon, 26 Dec 2011 00:00:00 GMT

Got in the holiday mood last night by watching the film of The Road. This is right up there with "Grave of the Fireflies" on the brilliant-but-sad scale (see 2008-03-22).

That's it, all gone!

Sun, 20 Nov 2011 00:00:00 GMT

Those of you who've seen me around recently will have noticed a gradual disappearance of hair on my left side. This was accompanied by a gradual increase in attempts to make a comb-over look like a real haircut. No more.

About a year ago I started developing a small bald patch on each side of my head. This developed into a large patch on one side. I got this diagnosed as Alopecia Areata. See the BAD website for details, but in summary, it's an Auto-Immune disease that attacks my hair follicles, causing them to fall out. It's not uncommon, but it is still annoying, particularly because it's possible, maybe, cross your fingers, that it'll grow back. And it has. In patches. S l o w l y.

Today I noticed a large area missing on the other side. So, enough is enough, a stereo comb-over is definitely not sustainable. I declare 20/11/2011 an auspicious day to shave all my hair off!

Let's not forget, there are many advantages to having a shaved head:

I can realise massive savings on shampoo.
I have a web-comic all about me.
I can be mistaken for Zed Shaw; ok, maybe not a benefit.
It is impossible to catch head-lice (disclaimer: only advantageous if you have a child)

... and so on, the list is endless!

Here's what I look like now (I took this just after I shaved it all off):

Frazzled

Sun, 28 Aug 2011 00:00:00 GMT

Totally frazzled. Between work, festival, and home-life, I've been burning the candle at three ends. It's been good though. Hopefully, when I regain some energy, I'll write-up what I saw that's worth shouting about. However, for now, I hereby declare my festival: OVER.

EIFF top picks

Sun, 10 Jul 2011 00:00:00 GMT

Despite overly expensive films and the usual EIFF website broken-ness, I managed to see a few films this year.

There were five shorts in Mapping the Extraordinary. All were good, but these three make me we want to see more from the directors:

Pentecost by Peter McDonald

I suspect this will mean more to you if you had any sort of Catholic upbringing. The end is a righteous, well deserved, FU to the local church establishment. The football analogy was a little overplayed at times, but it didn't spoil it.

Animal Control by Kire Papput

Link:

The trailer doesn't do this justice. The very spare use of dialogue and the repetition of the same shots, with subtle but significant changes, build up nicely to illustrate the change in the inner life of the main character.

Six Strands by Chaitanya Tamhane

Six Strands Trailer from Chaitanya Tamhane on Vimeo.

Though just a short, the story is well-developed and deep. It leaves you satisfied and wanting to return and watch it again.

The only main feature I saw was Almanya, Welcome to Germany:

This had me laughing for half of it and with a lump in my throat the rest of the time. If you've ever stayed in Germany for any length of time then I'm sure you'll recognise some of their reactions (particularly to the food). Not only is this a great story with emotional depth, it's also cleverly told. I'm definitely buying this on DVD.

Taking R for a drive

Mon, 04 Jul 2011 00:00:00 GMT

I've been playing with data from the Project365 group I am a member of. I would usually do this analysis using some combination of perl, ruby, and unix shell (sort, uniq etc). This time I thought I'd stretch myself a bit.

These are the two books I've been using to learn "R". Why R? Apart from being 'interesting' to google for, "R" is distinguished by often popping up on the visualisation blogs I subscribe to. It also offers another way to slice and dice data. Although I can achieve some of these things in a combination of Excel and shell, it always feels very brittle and throwaway. R offers the hope of some re-usability, or at least, repeatability.

Taking the Project365 data as a driving example, I plunged into learning R.

From a first pass, "R" is definitely 'new Lego': lots of large pre-formed parts that look great, when they fit. However, it doesn't really feel like a cohesive language. For example, what do you think as.vector() does? It sounds like it should always return a vector, yes? Nope, it tries its best, and if it doesn't work, well, then you get back what you put in. There may be a perfectly valid reason for this, but it still feels 'perly'. I like my languages to tell me when something hasn't worked.

If you follow along with the documentation, then you can get a lot done. However, I found myself often falling flat whenever I tried to generalise; it did not follow the principal of least surprise.

There is power there but it is often frustrating getting to it. I'm going to leave it for a bit before I dive back in.

No Java, no Eclipse

Fri, 07 Jan 2011 00:00:00 GMT

This is the first time I've been on holiday in Germany and not launched Eclipse. Also: not a single line of Java. It's refreshing.

I now know I can write incredibly slow Ruby, albeit for a good cause ("The Diagonal Method vs. The Rule of Thirds"). I rediscovered a love for Unix. I finally admit: maybe this Rails thing is quite good, and: git is better than svn.

Oh ye, and I got loads of presents ;-)

Aim Low

Sat, 01 Jan 2011 00:00:00 GMT

The key to achieving a New Years resolution seems to be to aim low and then forget about it.

Wired magazine has always been high on style and low on content. This didn't stop me from buying it once every few months. Not only that, but reading about geeks doing cool stuff was actually consuming the time I could be doing, well, something. My one and only resolution was to give up buying Wired magazine for a year.

Standing in the airport looking at another puff-piece cover story powered by garish American enthusiasm, I realised: I'd made it.

Waterstones is a Charity

Sun, 17 Oct 2010 00:00:00 GMT

Waterstones: £18.99. Amazon: £11.89, free delivery. This is one of the books I flipped through whilst sitting in the Starbucks. I may never buy this book, but if I do, I'm not paying an extra 7 quid out of the goodness of my heart. I feel a little bit guilty for writing this, as they let me flip through it without first purchasing it. But hey, maybe that's a quid max of guilt money I'd pay.

I don't feel too sorry for Waterstones, as they've always had a large amount of crappy books. However, I really value Blackwells, particularly the one on North Bridge. Being in a physical bookshop gives you a chance to peruse through the whole book, which you still can't get online. Maybe I'm a little old-fashioned, but I like being surrounded by books. Blackwells gives me that, and I'm willing to pay a little, but not a lot, over the minimum to prop up the institution.

We are surely in a transitionary period. At least two other tables in Waterstones had someone reading from an iPad. Next time, I'll look out for kindles. How long before bookshops like this can only afford to carry mainstream pap that punters off the street will buy who can't or won't look up the price online?

Today, I accidentally ran 10k

Sat, 09 Oct 2010 00:00:00 GMT

It started as a 5k run for burma in Inverleith park. After a frantic ride on my bike to make registration, I started at 11. We looped round a couple of times and I passed the steward, who said "go right if you've seen me twice". I couldn't remember seeing her in particular, but it felt about right. I ran on and, curious about my time, checked my phones SportyPal app. Unfortunately, it said I had only ran 1.86km. Shit.

I dutifully turned around, passed by the steward again, and trudged off for another few laps, wondering how the hell I'd managed to do 5k in half an hour only a few weeks previously. I finally finished about thirty minutes later, feeling quite embarrassed as I ran past people already heading home.

What I didn't know, is that my app had stopped at 1.86km. One time around Inverleith park is just under 2km. I had done closer to four or five times around.

What's the lesson from this story? That I can run 10k without realising it, or that I put far too much trust in electronic devices?

Who was in Tron and Babylon 5?

Mon, 04 Oct 2010 00:00:00 GMT

More specifically, how can I find this out using freebase? Turns out the steps you should take are:

Find the film you want to use as a constraint.
Slap the film id and type as constraints into the query editor, using "*": null to get all properties of that type.
Expand your query until you find the actor somewhere within the tree (keep on using "*": null at each level down to get more detail)
Select the actor leaf in the query and use "Turn inside out" to arrange the query so that it will return actors constrained by film.
Repeat the above for the tv series
Combined the film and tv series constraints into one query.
Voila, you have all actors who appeared in Tron and Babylon 5.

You can easily extend this to do things like finding all Science Fiction films that anyone in Babylon 5 has starred in.

What you definitely shouldn't do is:

Find the MQL documentation
Read it
Wonder why you need to know so much about type theory

Now, that was a wasted hour. Honestly, just go straight to the Query Editor. You know it makes sense.

"How can I tell which of these damn tabs is making a sound?"

Wed, 18 Aug 2010 00:00:00 GMT

Hmm, I'm out two for two on random summer holiday projects. First, the hadoop quickstart example doesn't reduce. Now, my idea to list what tabs in Chrome are making a noise is also a non-starter. Seems I'm not alone in wanting this. It's pretty trivial to make a Chrome extension, quite nice in fact, but there seems to be no api to help find what tabs contain something which is making a sound.

In general, anything other than YouTube which makes a noise in my browser pisses me off. Even then, you can have the problem of re-opening a bunch of restored tabs and having to hunt down the one playing that video you watched two weeks ago and forgot to close the tab of. It doesn't happen often, but when it does, it's pretty damn annoying.

I was hoping for one central point that everything went through to play sound, but the answer to that may be 'the operating system'. That's a little too gross a level. I want those tabs, dammit.

Segway tour of Berlin

Mon, 09 Aug 2010 00:00:00 GMT

The Segway has always intrigued me. Even after the overblown hype of 2001 died away.

Given I was in Berlin, I took the chance to look like a dork and try out a Segway tour of the area.

How do you they drive them? Simple answer, to go forward: lean on your toes, backwards: stick out your ass. Friends I talked to after were surprised that there really are no controls. There is some steering via an upright which you can move to turn. Even then, it is more of an indication; if you're stationary then a bend left means "turn on the spot", if moving forward, it produces a smooth turn, with the centre of the circle outside the machine.

I've never ridden a horse but riding a Segway feels like I'd imagine it to be. You give it guidance, and it responds appropriately. Another example: the feedback to speeding is a slight push back on the upright so that it presses lightly against your stomach. It's subtle, but hard to ignore and the meaning is obvious. If you continue to speed (by leaning straight forward) then it'll refuse and go slower and slower until it stops. Again, a nice organic reaction: it is in a huff.

Beware though, it can also bolt; the tour guide was at pains to point out that it will continue forward if you leave it leaning a little when you get off. Here the animal illusion breaks down; even the stupidest horse would surely be better than this.

Would I use this for real, for commuting? My Segway was fun, but I was basically standing up, forever. Fat people would hate it. Here's a telling (translated) quote for the "mobile rescue" version on the German site:

Comfort mats reduce fatigue occurring after standing for long

The Segway doesn't easily fit into any category of existing vehicle. In the middle of Berlin we didn't look too out of place because of the smorgasborg of various transport options present. However, when on the road, we interfered with bikes. Whilst on the paths, we surprised pedestrians.

This versatility is felt as a positive by the Segwayer (?), but just means everyone else is pissed off. In the better bike towns there is a clear demarcation between bike and non-bike areas; Berlin is a good example. The Segway breaks these legal (and social) rules. Once the shiny newness of it all wears off, this will become more obvious. Outside of the feel-good TED talk, I don't see this as much more than a footnote in transport history.

Also, given that it costs ~~£WeWontTellYouHowMuchWithoutAnAppointment~~ (Update: you can of course find out the huge price on amazon) , I don't see myself buying one anytime soon.

Would I do it again? Hell yes! Though, next time, I'll bring a lighter bag.

Festival goers + Bike = ...

Sat, 07 Aug 2010 00:00:00 GMT

Day six of Carmen and Nils being way and I've started talking to myself. The day after she left I went to work and realised, "shit, I've not talked to anyone for over twelve hours".

In other time-based news, in one day I have gone from avoiding cycling into tourists to actively aiming for them. If you visited Edinburgh this year and had some arsehole on a bike go 'ping ping ping' on his bell and almost run you over, that was me. Or maybe, it was any other cyclist in Edinburgh at the moment. This is why I love the Festival.

Back on the road again ...

Sat, 24 Apr 2010 00:00:00 GMT

And so, another german holiday ends. As always, there are regrets; I didn't get to wander around Berlin for long, I missed some nice german bread by staying in bed too long, etc. On the plus side, I got a regular (wonderful) lie-in, and made some solid progress on a programming project (not as interesting to link to as LSystems).

What have I been reading?

What would you do if you were the last man on earth? Get beat up by women seems to be one answer. Flashbacks back and forth are cute but a little annoying, though used to good effect to set it all up; very cinematic. The story is so-so, but this simplicity means that I'm going to buy the german version to help me get some reading practise.

Which brings me to this one. It is way too much for my current german reading level. It took me about 10 minutes to get through the first couple of pages with a dictionary next to me. Admittedly, there is a lot of setup text before we get into the main story, but even so, I need that setup! Currently on loan to a family friend in Germany. Will try again later!

Liebe Schaut Weg (Love looks away) is more my level; it has little text and only half of it is in german, mostly uttered by children! It's a simple biography of the authors American and German roots. As a story, there isn't much drama, and so you aren't really drawn in. The drawing style is subtle and the textures are just very nice. Overall, a relaxing nice book, but wouldn't have been too disappointed to miss it.

This is my pick of the holiday, and not just because it's all in english! There are touches of M.C.Escher here in style and recursiveness of the story. It's an investigation into the depths of The Louvre, told from the viewpoint of a newly-arrived cataloguer of artifacts. No garish colours here, just challenging perspectives, I recommend a look inside to see what I mean.

I've got a fuller review of this but in summary: interesting but I prefer The Human Story.

LSystem Sunday

Mon, 08 Mar 2010 00:00:00 GMT

It's LSystem Sunday! I was curious about investigating an assertion that I'd made the other day: that the dragon curve loops back and traces over itself. Turns out, I was talking nonsense:

X+YF+,Y->-FX-Y:FXY/1.0-90.0-7.150x150.png" />

X+YF+,Y->-FX-Y:FXY/1.0-90.0-7.150x150.animOver70s.gif" />

The above shows the dragon curve at it's 7th expansion, with an animation of how this is drawn on the right. As you can see, it has overlapping points, but not overlapping line segments. I was wrong.

Btw, yes, I know, but today is still logical Sunday for me.

Note: animated gif rendering supported by http://elliot.kroo.net/software/java/GifSequenceWriter/

Btw, doing conditional GET in Jersey using ETags is piss-easy. Combine this with apache server-side caching and you have a very nice persistent caching solution. I can now do some pretty inefficient LSystem drawing without worrying about my server melting ;-) For example:

LSystem Sunday: Does Dragon Curve trace over itself?

Mon, 08 Mar 2010 00:00:00 GMT

It's LSystem Sunday! I was curious about investigating an assertion that I'd made the other day: that the dragon curve loops back and traces over itself. Turns out, I was talking nonsense:

X+YF+,Y->-FX-Y:FXY/1.0-90.0-7.400x400.jpg" />

X+YF+,Y->-FX-Y:FXY/1.0-90.0-7.400x400.animOver70s.gif" />

The above shows the dragon curve at it's 7th expansion, with an animation of how this is drawn on the right. As you can see, it has overlapping points, but not overlapping line segments. I was wrong.

Btw, yes, I know, but today is still logical Sunday for me.

Note: animated gif rendering supported by http://elliot.kroo.net/software/java/GifSequenceWriter/

LSystem Distractions

Sun, 28 Feb 2010 00:00:00 GMT

LSystems have once-again distracted me, this time from German homework:

LSystems distractions

Sun, 28 Feb 2010 00:00:00 GMT

LSystems have once-again distracted me, this time from German homework:

More arsing around with Scala

Tue, 07 Jul 2009 00:00:00 GMT

More arsing around with Scala. This time, trying to reimplement lesscss. This is about two hours worth of hacking:

object LessParser extends RegexParsers {
  def parse(spec: String) : Sequence = {
    parseAll(sequence, spec) match {
      case Success(l, _) => l
      case Failure(m, _) => throw new IllegalArgumentException(m)
      case Error(m, _)   => throw new IllegalArgumentException(m)
    }
  }
  def sequence = rep(namedNested | atom) ^^ {
    Sequence(_)
  }
  def namedNested = nestedName~"{"~rep(atom)~"}" ^^ {
    case name~"{"~contents~"}" => NamedNested(name, contents)
  }
  def nestedName = """[\.\#][a-zA-Z]+""".r
  def atom = variableDef | variableUse | text
  def text = "[^@{}]+".r ^^ { Text(_) }
  def variableDef = "@"~variableName~":"~variableValue~";" ^^ {
    case "@"~name~":"~value~";" => VariableDef(name, value)
  }
  def variableUse = "@"~variableName ^^ {
    case "@"~name => VariableUse(name)
  }
  def variableName = "[a-z-]+".r
  def variableValue = "[^;]+".r
}

abstract class Tree {
  def eval(defs: Map[String, String], out: StringBuffer) : Map[String, String]
}
case class Sequence(parts: List[Tree]) extends Tree {
  def eval(defs: Map[String, String], out: StringBuffer) : Map[String, String] = {
    parts.foldLeft (defs) { (accumDefs : Map[String, String], part : Tree) =>
      part.eval(accumDefs, out)
    }
  }
}
case class NamedNested(name: String, contents: List[Atom]) extends Tree {
  def eval(defs: Map[String, String], out: StringBuffer) : Map[String, String] = {
    out.append(name)
    out.append(" {")
    contents.foldLeft (defs) { (accumDefs : Map[String, String], part : Tree) =>
      part.eval(accumDefs, out)
    }
    out.append("}")
    defs
  }
}
abstract case class Atom() extends Tree
case class VariableDef(name: String, value: String) extends Atom {
  def eval(defs: Map[String, String], out: StringBuffer) : Map[String, String] = {
    defs + (name -> value)
  }
}
case class VariableUse(name: String) extends Atom {
  def eval(defs: Map[String, String], out: StringBuffer) : Map[String, String] = {
    out.append(defs(name))
    defs
  }
}
case class Text(value: String) extends Atom {
  def eval(defs: Map[String, String], out: StringBuffer) : Map[String, String] = {
    out.append(value)
    defs
  }
}

object Less {
  def main(args : Array[String]) : Unit = {
    println(LessParser.parse("@nice-blue: #5B83AD;"))
    println(LessParser.parse("@nice-blue"))
    println(LessParser.parse("@nice-blue: #5B83AD; @nice-blue"))
    println(LessParser.parse("@nice-blue: #5B83AD; #header { background-color: @nice-blue; }"))

    var fullExample = """
    @nice-blue: #5B83AD;
    #header { background-color: @nice-blue; }
    """

    var buf = new StringBuffer
    LessParser.parse(fullExample).eval(Map(), buf)
    println(buf)
  }
}

Output:

Sequence(List(VariableDef(nice-blue,#5B83AD)))
Sequence(List(VariableUse(nice-blue)))
Sequence(List(VariableDef(nice-blue,#5B83AD), VariableUse(nice-blue)))
Sequence(List(VariableDef(nice-blue,#5B83AD), NamedNested(#header,List(Text(background-color: ),
VariableUse(nice-blue), Text(; )))))
#header {background-color: #5B83AD; }

It does variables and the beginnings of mixins. It's actually quite lame e.g. it throws away whitespace. Hohum, time to learn more about parsers in Scala...

LSystem Explorations

Sat, 27 Jun 2009 00:00:00 GMT

So, another summer holiday is at an end, and another programming interval. This time I was using my LSystem Explorations as a way to get more practice with Scala and Android.

Random aside: I find it hard to get excited about non-JVM languages nowadays. I develop on a Mac and deploy to Linux, Windows (sometimes) and now Android. I really don't want to worry about whether the language or platform I'm using has native components hidden somewhere inside. I've spent enough time doing "./configure" in the past to be seriously bored with non-portability. That was even going between two Unixes (Solaris and Linux)! I know Java is really "write once, debug everywhere" (e.g. Graphics2D problems on the Mac), but at least these are problems in particular library features, not a total inability to run.

Anyway, what have I learned? Scala in Eclipse has become usable enough for me with the latest Eclipse plugin for me to use Scala as my main Programming Language. It's still the case that the Scala builder gets confused too often and you have to do a Project->Clean. This even happens with pure Scala builds, and not just with mixed Java/Scala projects. The Editor integration is also flaky e.g. copy and paste between files can cause errors, and you can forget about consistently completing a Rename Refactor. However, even given all that, the ability to mix Scala and Java in one project is what really does it for me. Overall, it's still basic compared to the JDT. I don't yet want to push it for my day job, but for personal projects, where you can absorb the extra time fixing things if you break them, it's fine.

Scala as a language is really growing on me. As previously mentioned, I was working through the "Programming in Scala" book. Even though Scala fundamentally has a small core, it has the appearance of a much larger language. I really needed to read most of this book to get past quite a few roadblocks, but I did manage it.

For example, I'd looked at parser combinators before but found the examples posted at the time and the scaladoc impenetrable. This is not too surprising as operators in Scala appear as methods on the class, so class-level docs can't really tell you much about how to use them in context. The JSON example in the book was large enough to give me sufficient transferrable knowledge to write a simple parser for LSystem descriptions:

import scala.util.parsing.combinator._



object LSystemLanguage extends RegexParsers {  

  // This parses "B; F -> FF, B -> F[+B]F[-B]+B; FB" into an LSystem

  def parse(spec: String) : LSystem = {

    parseAll(lSystem, spec) match {

      case Success(l, _) => l

      case Failure(m, _) => throw new IllegalArgumentException(m) 

      case Error(m, _)   => throw new IllegalArgumentException(m) 

    }

  }

  def lSystem = axioms~";"~rules~";"~visible ^^

    { case axioms~";"~rules~";"~visible => LSystem(axioms, rules, Set() ++ visible) }

  def rules = repsep ( rule, "," )

  def rule = variable~"->"~rep( variable | constant ) ^^

    { case lhs~"->"~rhs => new Rule(lhs, rhs) }

  def axioms = rep(element)

  def visible = rep(element)

  def element = variable | constant

  def variable = 

    "[A-Z]".r ^^ ( (s : String) => { Var(s.charAt(0)) } )

  def constant = 

    ("[a-z]".r | literal("+") | literal("-") | literal("[") | literal("]") ) ^^ 

    ( (s : String) => Constant(s.charAt(0)) )

}

Apologies for the probably non-idiomatic Scala (and lack of syntax-highlighting on this site, I'll upgrade one day, honest), but it has the important property that it works, as opposed to most of my previous attempts. From further reading elsewhere, I get the impression that they are papering over some details (the !RegexParser combines both tokenisation and parsing, which they don't cover separately from what I read of the book). However, who cares, it works!

I'm now seriously considering supporting the full Fractint LSystem syntax, which I wouldn't have considered doing in Java. Obviously, you can do all this with JLex etc, but that means going back and forward between languages and probably, gack, writing an ant script. Seriously, I spent more time during my holiday getting a simple ant task to work than I did writing this parser.

Modulo one instance where Eclipse crashed and I had to rebuild my workspace, programming in Scala using Eclipse has been fun. The combination of an object-functional language, a half-decent IDE and rich platform libraries is very powerful. It's important to emphasize what this can give you. For me it means that if I have a spare hour or so in which to add a new feature or tweak an existing one (like extending the parser) I'll consider doing it and expect to complete it, and enjoy it.

Technically, I'm capable of getting the same feature implemented in Java, in combination with other tools. However, I'd start out with a big "harumph" as I picture the fields of verbosity laid out before me. It would definitely not be fun.

This brings me to Android. The Scala compiler spits out JVM bytecodes. The Android platform, though not strictly a Java VM, does claim to support Java class files. Theoretically, I should be able to write a library in Scala and use it on Android.

I am explictly not trying to write an Android app in Scala. Instead, I created an LSystem library, written in Scala, which exposes itself solely through these interfaces:

public interface LSystemViewFactory {

  public LSystemView evaluate(String spec, double angle, int rounds);

}



public interface LSystemView {

  public void renderTo(Renderer renderer);

}



public interface Renderer {

  interface Point {

    double x();

    double y();

  }



  public void addPath(Iterable<Point> path, int length);

}

In this way, I was trying to avoid relying on any special help that could come from using one Scala project with another. It does work, in a way.

Warning: what follows is a history of my stubborn depth-first method of getting something showing on my phone. There may a better method.

In Eclipse 3.4, you need only make an Android project depend on a Scala project and everything will compile. However, when you run it on the phone, it will be missing the required Scala class definitions. You can download the scala-library.jar and add it as a required library. The Dex compiler then starts churning away on this file, heating up your lap as a side-effect, and then fails with a compiler error. Schäde.

There is some joojoo in "scala-library.jar" that the Dex compiler just doesn't like. I think it is an unsupported annotation of some kind, probably somewhere in scala.xml.transform, but since it takes a minute or so for dex to do its thang to this library, I decided to adopt a slash and burn approach, by quickly removing anything I knew I didn't need on the phone. So, goodbye to any of scala.actors, scala.reflect, scala.testing or scala.xml.transform.

Random Aside: it was at this point I had the afore-mentioned elbow injury^H^H^H^H...problem with ant. It is frustratingly hard to get ant to behave in any sort of sensible and consistent manner. In this case, I had the temerity to require more than one exclude pattern (for the multiple packages). The docs claim that you can put them in one string, comma-separated or space-separated; obviously multiple elements are for wimps and no-one will ever need a pattern that contains both a literal space and a comma. I tried this, and it didn't seem to work. There is, of course, no "verbose" option on the "jar" task so I couldn't confirm whether my patterns were getting picked up. I ended up using a separate "excludesfile". A muted "hurrah" and I'm back to work. It's at times like this that I briefly consider moving back to Make. But only briefly.

This did the trick and dex now compiled the scala-library.jar. My Android Activities and Views (which depend, indirectly, on Scala classes) successfully resolved all references and I could run my app.

All is not satisfactory, however. As previously-mentioned, the dex compilation phase now takes a minute. This compilation happens in the background whenever you make a change to any file, whatsoever. That's right: tappitty-tap, muscle-memory says "auto-save", and bzzt you've got a minute penalty. On top of this, the dex loader on the phone takes about 10 to 15 seconds to parse the app description. I haven't yet packaged the app and used it outside of the debugger, so I can't tell if this 15 second penalty is incurred ever time you launch it.

Both the IDE and run-time problems should be solved by running proguard over it. If I choose the set of interfaces above as the ones to preserve, then I should be left with a much smaller working set of classes. I didn't try this, partly because I ran out of time, but mostly because it would probably involve writing an ant build file. Can you tell I don't like ant?

Phew, so after all that I have a (slightly buggy) drawing of an LSystem on my phone:

For reference, here's the same LSystem rendered using the same library, but with a Java2D backend:

Is Scala mature enough as a main programming language?

Sat, 27 Jun 2009 00:00:00 GMT

So, another summer holiday is at an end, and another programming interval. This time I was using my LSystem Explorations as a way to get more practice with Scala and Android.

import scala.util.parsing.combinator._

object LSystemLanguage extends RegexParsers {
  // This parses "B; F -> FF, B -> F[+B]F[-B]+B; FB" into an LSystem
  def parse(spec: String) : LSystem = {
    parseAll(lSystem, spec) match {
      case Success(l, _) => l
      case Failure(m, _) => throw new IllegalArgumentException(m)
      case Error(m, _)   => throw new IllegalArgumentException(m)
    }
  }
  def lSystem = axioms~";"~rules~";"~visible ^^
    { case axioms~";"~rules~";"~visible => LSystem(axioms, rules, Set() ++ visible) }
  def rules = repsep ( rule, "," )
  def rule = variable~"->"~rep( variable | constant ) ^^
    { case lhs~"->"~rhs => new Rule(lhs, rhs) }
  def axioms = rep(element)
  def visible = rep(element)
  def element = variable | constant
  def variable =
    "[A-Z]".r ^^ ( (s : String) => { Var(s.charAt(0)) } )
  def constant =
    ("[a-z]".r | literal("+") | literal("-") | literal("[") | literal("]") ) ^^
    ( (s : String) => Constant(s.charAt(0)) )
}

I am explictly not trying to write an Android app in Scala. Instead, I created an LSystem library, written in Scala, which exposes itself solely through these interfaces:

public interface LSystemViewFactory {
  public LSystemView evaluate(String spec, double angle, int rounds);
}

public interface LSystemView {
  public void renderTo(Renderer renderer);
}

public interface Renderer {
  interface Point {
    double x();
    double y();
  }

  public void addPath(Iterable<Point> path, int length);
}

In this way, I was trying to avoid relying on any special help that could come from using one Scala project with another. It does work, in a way.

Warning: what follows is a history of my stubborn depth-first method of getting something showing on my phone. There may a better method.

Phew, so after all that I have a (slightly buggy) drawing of an LSystem on my phone:

For reference, here's the same LSystem rendered using the same library, but with a Java2D backend:

How the hell do I do this basic thing?

Wed, 17 Jun 2009 00:00:00 GMT

I've been programming Scala here and there and every so often get stuck in a "how the hell do I do this basic thing?" moment. So, I'm gradually working my way through "Programming in Scala", as I thought I should finally get more of an idea of this language works.

I definitely like the application of the "everything is a library" strategy for implementation of what would be core features in other languages. However, one example leaves me wary, "::". A List may be constructed (val l = a :: b :: Nil) and deconstructed ((a :: b :: Nil) = l) using the same syntax. However, this is only possible because "::" is both a method on List and a case class. Along with the right-associativity of any operator ending in ":" these conspire to give the illusion of the "::" operator that is seen in other languages, like ML.

This all seems a little too "just-so" for me. It's if they really wanted to use Lists like in ML and engineered the defaults (like right-associativity) to make it easy. This default doesn't really help make other, non-List operators, very obvious, like "/:" and ":" (for foldLeft and foldRight, if I remember correctly).

There is quite a lot of pragmatism in the Scala language and libraries, which I like. However, some of it seems a little too skewed towards particular features and may lack generality.

Grave of the Fireflies

Sat, 22 Mar 2008 00:00:00 GMT

Grave of the Fireflies is not a film I'd recommend be seen by anyone with children.

After we had Nils, I imagined myself being affected by all the schmaltzy Speilberg movies featuring children in trouble. Thankfully, that never happened. However, this one caught me by surprise. It was one of the few movies in a Studio Giblhi collection that we didn't recognise. We put in on, naively expecting another piece of Totoro-style magic. I was instead left devastated.

"Star rating: 5, Would you recommend this to anyone: never"