HHhH

I’ve just finished reading HHhH, by Laurent Binet. An excellent novel based on an thrilling, and true story.

It tells the story of the assassination of Nazi leader Reinhard Heydrich, Operation Anthropoid, in Prague during the second world war. Not a person I had heard of before reading the book, he was responsible for many terrible crimes and one of the main architects of the Holocaust.

Much of the book discusses his rise to power within Nazi Germany and how he ended up becoming the “Protector” of annexed Czechoslovakia. It was whilst in this post he became the target of an assassination attempt organised by Czechoslovak government-in-exile in London, with the help of the British.

The book then introduces the assassins, the heroes, Jozef Gabčík and Jan Kubiš, as well as the many others who risked their lives for the mission.

It’s a great story that needed to be told1. Understandably, it’s very dark and tragic in places, but the bravery of the resistance fighters and ordinary Czechs really shines through.

The way the book is written is also very interesting. It’s full of his own thoughts, questions and humour, inserted alongside the story. There may have been a couple of times when it was a bit much, but overall I feel it made the story more personal, more lifelike, and reminded you that this is a real story of incredible courage and monstrous crimes.

Definitely worth reading.


  1. Of course, the story has been told before. But never like this.

GopherCon 2014

Some of my notes, thoughts, and observations after attending the excellent first GopherCon in Denver, Colorado, US.

Recurring Themes

Interfaces

A number of talks highlighted Gos Interfaces as something you should be using throughout your app. By separating data from behaviour, they help you write composable code. They are how you structure programs in Go.

12 Factor App

It’s not Go specific, but the Twelve-Factor App methodology came up a few times as they way people should be structuring and writing their apps. Go makes this easier, as you are always building single binarys, which is one of the principles of a twelve-factor app.

This is something I have also been looking at and would recommend you read the book (it’s a short read).

It’s also worth thinking about how you can use Docker (written in Go) to help create and deploy your apps.

Microservices

Again, this isn’t Go specific, but I noticed both SoundCloud and Poptip have/or are moving to a microservices architecture.. This was also a theme from QCon London.

There was a talk about Go Circuit, which allows processes (services) to talk to each other. I’m not completely certain on the details, but it sounds like services communicate by creating special files on the file system (under /circuit). This provides language independence - each implementation just needs to understand the files. A separate process is then used to allow different services to communicate over the network.

Error Handling

One of the things that can catch people out when learning Go is the lack of exceptions. But there are two arguments made by the Go team as to why this is the case in Go:

  • Error values should be just like any other value
  • No hidden control flow (throw, try, catch, finally) improves readability

I don’t have a strong opinion either way. In Go, you might end up with more if/else statements, where as in Java you can have many catches. But by only having one way to deal with exceptions, you know exactly how a library will handle them, no matter who wrote the code, which is not the case in some other languages.

Highlights

Some of the talks I would recommend watching when they are available:

  • Hello Gophers - Opening day keynote. Looking back on why Go was successful. Interesting snippets on some of the design decisions that went in to Go. - slides | video
  • Go Circuit - I need to watch again to try to understand it… - slides | video
  • A Channel Compendium - Some clever ways to use channels. - slides
  • Go for Sysadmins - Deploying Go apps. - slides | video
  • Heka - Brief introduction to Heka, then explaining why they chose Go. - slides | video
  • Go, from C to Go - The Go team are writing a tool to convert the Go compiler from C to Go. Interesting discussion on the challenges of rewriting source code and how they are addressing them. - slides
  • NSQ - A message platform written in Go. Similar to AMQP, but with some interesting features. - slides
  • Best Practices for Production Environments - Deploying Go at SoundCloud. - slides | article
  • database/sql - Working with databases in Go. - slides
  • Build Your Developer Tools in Go - Writing command line apps in Go. Covers solving some interesting problems including bundling asset files, terminal UI, auto updating.
  • Building Web Services in Go - Also talks about interfaces and reflections in Go. - slides
  • Go for gophers - Closing keynote. Using Gos features to write effective code. - slides

Interesting Projects

A list of some interesting (to me) projects that people discussed during the conference (in no order):

  • Heka - Used to collect and process metrics. Something I have been looking at recently and will likely be using.
  • Docker - The way we all of us will deploy apps in the near future.
  • go-update - Automatically update your Go apps.
  • go-qml - Cross platform applications written in Go, using Qt.
  • Gobot - Robots (and more). Programmed with Go. Also wins the award for best live demo!
  • Camlistore - Store all your data, privately.
  • C-to-Go - A tool written by the Go team to convert the Go compiler from C to Go.
  • Hugo - Fast and flexible static site generator. Looking at using it to render this site.
  • Ramsgate - A multiplayer game server demoed by @ernestmicklei in the lightning talks.
  • go-metrics
  • go-tigertonic - Framework for building JSON services.
  • gh - GitHub client, which also drops down to Git for it’s commands.
  • eventsource - Server-sent events.
  • Bazil - File system that intelligently moves your data to where you want it.
  • beego - Go webapp framework.
  • termbox-go - CLI UI.
  • mousetrap - Help Windows users with their command line apps.
  • roshi - time-series event store.
  • Martini - web framework.
  • fsnotify - file system notifications.
  • Deck - Scalable presentations.
  • Disco - An alternative to Hadoop, built by Nokia1.

Final Thoughts

I was very impressed with the first GopherCon. It didn’t feel like the first; such was the organisation and professionalism. The talks were of a really high quality; including the lightning talks on the last day. I learnt a lot, and it’s going to take a while to sink in. Thanks again to everyone involved!

  1. This smells like NIH syndrome, but it’s worth looking through in case there are some interesting ideas worth borrowing. Also, as the presenter said, competition is good, so I don’t want to judge it without first looking through it.

Copy Data as JIRA Table From the SQuirreL SQL Client

Quick tip. I use the SQuirreL SQL Client when working with databases. Sometimes, I want to paste the results of an SQL statement into a JIRA issue, but the default text copy doesn’t format well, and you can’t use HTML in a JIRA issue.

Since version 3.3.0, Squirrel allows you to copy as wiki text, once you have defined how the wiki text should be formatted. Here is the configuration that works for JIRA:

To use it, run some SQL, then select the data you want to copy and right-click. From the context menu, you can now copy as JIRA.

I’m sure my colleagues will be appreciating the nicer formatting of data in my JIRA issues :-)

Vikings: Life and Legend at the British Museum

The Sea-wolf

Yesterday afternoon I visited the Vikings exhibitionn currently on display at the British Museum. Unfortunately you’re not allowed to take photos, though there are some on Flickr.

The first half of the exhibition looks at how the Vikings set up an impressive network of trade and contacts, from the North Atlantic through Europe and to the Middle East, using their famous longboats to navigate the seas and rivers.

Then second half is more about their military might1 and their raids on Europe and the British Isles. This part of the exhibition is dominated by the remains of a Viking warship, Roskilde 6, which is really breathtaking.

The exhibition finishes off showing how the Viking period ended with the emergence of the Scandinavian states and Monarchies and the adoption of Christianity.

Overall I really enjoyed the exhibition. It flowed nicely, and contains some interesting and beautiful items. My only criticism is that it could have shown us more about how the Vikings lived their day-to-day lives, which although we caught a glimpse of through some of the items worn by the Vikings, it wasn’t really explained. Their religion was also largely missing from the exhibition.

I would recommend visiting if you can. Best to go off peak, as when I went it was very busy.

  1. We think of Vikings as great warriors, but in reality the evidence suggests they lost at least as many battles as they won.

Agile Is Dead (Long Live Agility)

A great post that for me sums up what developing using agile (or with agility) means.

Let’s develop with agility

  • You aren’t an agile programmer - you’re a programmer who programs with agility.
  • You don’t work on an agile team - your team exhibits agility.
  • You don’t use agile tools - you use tools that enhance your agility.

Agile isn’t about scrums, user stories, story points, consultants, etc. It’s about being agile, and developing with agility.

QCon London - Day Three

Notes from the final day of QCon London 2014.

“The World after Cloud Computing & Big Data” by Prof. Dr. Gunter Dueck

Discussion on how as more is automated, all professions are at risk of being replaced. I think this has been clear since the Industrial Revolution.

Not everyone was keen on the keynote:

That was utter bollocks #qconlondon

Another good tweet:

Feel like this keynote is from 10 years ago. All developers are not autistic and it's a bad excuse for IBMs 20% women tech staff #qconlondon

Have to agree.

“Next Gen Hadoop: Gather around the campfire and I will tell you a good YARN” by Akmal B. Chaudhri

A good introduction to Hadoop and related technologies, including YARN. Sets the foundation for the later Hadoop focussed talks.

“What can Hadoop do for you?” by Eva Andreasson

A futher introduction to Hadoop. Very similar to the previous talk, though made a couple of different points.

“High Performance Reactive Applications with Vert.x” by Tim Fox

Hadn’t heard of Vert.x before, so found this talk really interesting. Has some good ideas and features, helping you create a microservice achitecture.

Will be keeping an eye on Vert.x and watching how it develops.

“A Call for Sanity in NoSQL” by Nathan Marz

I’ve been reading the eary access edition of Nathan’s Big Data book and this talk helped reinforce some of the ideas behind the lambda architecture.

If you are dealing with large volumes of data you really need to be aware of the lambda architecture. I highly recommed getting the book.

“How Shutl delivers even faster using the Neo4J, the Graph Database” by Sam Phillips and Volker Pacher

A look at how Shutl uses Neo4J. Didn’t go into enough depth for me - was like they wansn’t sure how technical to make the talk. However, graph databases do look really useful for certain problems.

“How Elasticsearch Powers the Guardian’s Newsroom” by Graham Tackley and Shay Banon

A great talk on how the Guardian use Elasticsearch to produce real-time analytics on what people were reading on their website.

As I mentioned after the “Revealing the Uncommonly Common with ElasticSearch” talk on day one, Elasticsearch looks like a great tool for quickly analysing strcutured or semi-structured data on the fly.

Conclusions

So, that’s it for QCon this year. One of the best conferences I have been too (and certainly the biggest). Looking back on it and trying to think about what I have learned, there were a number of obvious trends over the 3 days.

1. Microservices

It’s clear any big application these days ought to be split into microservices. Netflix and Groupon were two of the companies who spoke about this here, while Vert.x gives you a microservice architecture for free.

It doesn’t look like there are any emerging standard tools/technologies to help with your microservice architecture. Netflix has developed their own tools, and open sourced them. Sounded like Groupon have done the same. It would be interesting to know of other companies building microservice architecture and what they are using.

2. The lambda architecture

Another trend was the discussion of the lambda architecture as a way to build your Big Data applications. Many of the tools to build it have already emerged and are becoming mature (Hadoop, Flume, Storm, Spark, etc), we just need to use them together in the right way.

Once again I recommend you check out the Big Data book by Nathan Marz and James Warren.

3. Open Source

All the Big Data technologies over the last 10 years are open source (with the exception of Splunk), which is great! No upfront costs; allowing you to start small, experiment and iterate, and no vendor lock in.

QCon London - Day Two

Following on from yesterdays post, some notes on the second day of QCon London 2014.

“Forty Years of Teams” by Tim Lister

A nice, inspirational keynote drawing from Tims 40+ years of experience.

“A Big Data Arsenal for the 21st Century” by Matt Asay

A very good talk discussing how to choose the tools we use when building our Big Data infrastructure.

One of the key points was to build on open source tools. This is important for many reasons - one of which is to avoid large upfront costs, as that increases the cost of failures.

Almost all of the new data technologies in the past 10 years are open source (the only exception is Splunk). Open source has moved from imitator to innovator.

You also want something flexible, as data is messy, and getting messier. NoSql technologies are a much better fit for this than traditional databases.

“From Development to Production with a Free (and Fun) Cloud Stack” by Baruch Sadogursky

A demonstration of using Bintray to host your open source software tools. I was hoping part of the solution would be applicable to internal software, but unfortunately this wasn’t the case.

“New Opportunities for Connected Data” by Ian Robinson

A nice introduction to Neo4j with examples. They also gave away copies of the Graph Databases book at their exhibition stand.

Like the ElasticSearch talk yesterday, I would like to try using this on some real data - maybe event data.

“Migrating to Microservices” by Adrian Cockcroft

Adrian spoke about the microservices architecture used at Netflix and the benefits it has brought them. A contrast to the Etsy talk from yesterday. Looks like a great way to work when creating an application the size of Netflix.

“I-Tier: Breaking Up the Monolith” by Brian McCallister

Following on from Adrian’s talk, Brian talked us through how Groupon are moving from one monolithic Rails application to a microservices architecture.

“Lambdas & Streams: Taking the Hard Work Out of Bulk Operations in Java SE 8” by Simon Ritter

A run-through on the new Lambdas and Streams features coming up in Java SE 8. Looks like a good implementation that I look forward to using.

A side effect of implementing Lambdas is that Java now has something that looks a lot like Roles in other languages. Will be interesting to see how this feature gets used by Java developers, and if it will cause more problems that it solves.

QCon London - Day One

Some notes and opinions on the talks I attended on the first day of QCon London 2014. A very good first day; looking forward to tomorrow.

“Life, The Universe, and Everything” by Damian Conway

I always enjoy seeing Damian talk. Interesting, funny - a great way to start the conference.

“Development, Deployment & Collaboration at Etsy” by Daniel Schauenberg

An interesting overview of how Etsy manage the development and deployment of their website. The main takeaways for me were:

  • Monitor and graph everything that changes
  • Create dashboards - lots of them
  • Stream all your logs and make them available in a web page

Curious that all of Etsy, including their admin views, is just one big PHP application. This tweet says what the rest of us were thinking:

Etsy at #qconlondon make it clear to me why monolithic apps are a dead end. Use microservices for continuous scalable deployments

It’s obviously working out for them, so not quite a dead end, but it’s surely not the future.

“Real Data Science at NASA” by Chris Mattmann

Interesting talk on some of the challanges faced at NASA and some of the tools they are using to solve them. Was a bit light on technical detail for me - well, in terms of tools and architecture… There was plenty of detail about the projects, and a lot of acronyms, as summed up nicely in another tweet:

A lot of hairy terms flying around at the NASA talk at #qconlondon. I feel like a #java programmer who wandered into a #haskell talk.

“Revealing the Uncommonly Common with ElasticSearch” by Mark Harwood

I had already seen a version of this talk at the recent ElasticSearch User Group meetup, but was worth seeing again. A great talk on how you can use ElasticSearch to detect anomalies in your data. Uses the upcoming significant_terms aggregationn which should be available in 1.1.

I can think of so many use cases for this - even if it is just for categorising/tagging log files on the fly. Can’t wait to try it out on some real data.

Design Patterns for Large-Scale Real-Time Learning by Sean Owen

Sean talked about Data Science and Machine Learning in general and what it means, then gave an introduction to the Lambda architecturee and Oryx.

Oryx can be seen as a reference architecture on how to build a simple lambda-like architecture, but going forward it is likely there will be better systems to help achieve this.

Once again, I was hoping for more technical detail in the talk. This was more of an introduction to a topic I’m started to gain an understanding of.

Instrumenting Your Business For Success with DevOps by Robert Benefield

I thought this talk was going to be about the metrics you should consider collecting to ensure your application/business succeeds, but instead it was more about how to show off DevOps to people in business (managers, etc). So this wasn’t really my cup of tea. Maybe I read the description wrong.

A Brief History of Data by Damien Dallimore

The solutions track contains talks put on by the sponsors. Some of them sound like they are just sales pitches, but this one looked more interesting than that so I went along.

And it was quite interesting, looking at how data might be changing in the near future (more of it, more variety) and what changes we need to make (for example, the schema needs to be defined at read time, not write).

Then the second half was the sales pitch for Splunk. They do seem to have a good product - similar in some ways to what we are building at work1. Good to know we are looking at data in the same way.

  1. It’s an internal system for collecting generic data, based on open source software. I plan to write a bit more about it soon.