QCon London - Day Three
Notes from the final day of QCon London 2014.
“The World after Cloud Computing & Big Data” by Prof. Dr. Gunter Dueck #
Discussion on how as more is automated, all professions are at risk of being replaced. I think this has been clear since the Industrial Revolution.
Not everyone was keen on the keynote:
That was utter bollocks #qconlondon
Another good tweet:
Feel like this keynote is from 10 years ago. All developers are not autistic and it's a bad excuse for IBMs 20% women tech staff #qconlondon}}
Have to agree.
“Next Gen Hadoop: Gather around the campfire and I will tell you a good YARN” by Akmal B. Chaudhri #
A good introduction to Hadoop and related technologies, including YARN. Sets the foundation for the later Hadoop focussed talks.
“What can Hadoop do for you?” by Eva Andreasson #
A futher introduction to Hadoop. Very similar to the previous talk, though made a couple of different points.
“High Performance Reactive Applications with Vert.x” by Tim Fox #
Hadn’t heard of Vert.x before, so found this talk really interesting. Has some good ideas and features, helping you create a microservice achitecture.
Will be keeping an eye on Vert.x and watching how it develops.
“A Call for Sanity in NoSQL” by Nathan Marz #
I’ve been reading the eary access edition of Nathan’s Big Data book and this talk helped reinforce some of the ideas behind the lambda architecture.
If you are dealing with large volumes of data you really need to be aware of the lambda architecture. I highly recommed getting the book.
“How Shutl delivers even faster using the Neo4J, the Graph Database” by Sam Phillips and Volker Pacher #
A look at how Shutl uses Neo4J. Didn’t go into enough depth for me - was like they wansn’t sure how technical to make the talk. However, graph databases do look really useful for certain problems.
“How Elasticsearch Powers the Guardian’s Newsroom” by Graham Tackley and Shay Banon #
A great talk on how the Guardian use Elasticsearch to produce real-time analytics on what people were reading on their website.
As I mentioned after the “Revealing the Uncommonly Common with ElasticSearch” talk on day one, Elasticsearch looks like a great tool for quickly analysing strcutured or semi-structured data on the fly.
Conclusions #
So, that’s it for QCon this year. One of the best conferences I have been too (and certainly the biggest). Looking back on it and trying to think about what I have learned, there were a number of obvious trends over the 3 days.
1. Microservices #
It’s clear any big application these days ought to be split into microservices. Netflix and Groupon were two of the companies who spoke about this here, while Vert.x gives you a microservice architecture for free.
It doesn’t look like there are any emerging standard tools/technologies to help with your microservice architecture. Netflix has developed their own tools, and open sourced them. Sounded like Groupon have done the same. It would be interesting to know of other companies building microservice architecture and what they are using.
2. The lambda architecture #
Another trend was the discussion of the lambda architecture as a way to build your Big Data applications. Many of the tools to build it have already emerged and are becoming mature (Hadoop, Flume, Storm, Spark, etc), we just need to use them together in the right way.
Once again I recommend you check out the Big Data book by Nathan Marz and James Warren.
3. Open Source #
All the Big Data technologies over the last 10 years are open source (with the exception of Splunk), which is great! No upfront costs; allowing you to start small, experiment and iterate, and no vendor lock in.