Twice today I’ve seen references to the benefits of iterative improvement, and I found that the topic resonated with me. At the moment I’m halfway though a pretty lengthy software project in which it’s sometimes hard to see the wood for the trees. The backlog is substantial, resulting in a hefty set of tasks for our team. In these situations, where the to-do list doesn’t seem to change day to day, it’s easy to feel crippled by inertia. However, these two references illustrate that we can succeed provided we ensure we’re making healthy, gradual progress.

One might expect that the references I’m alluding to originate from a tome of software development, like “The Pragmatic Programmer“, or from one of the many celebrated development gurus of Twitter, but in fact they are not directly concerned with software at all. The first comes from a talk discussing the pros and cons of iterative improvement , linked to by my colleague (and all-round good chap) Owen Phelps, by Tim Harford at last year’s Wired UK conference:

If you put together enough marginal improvements, in enough areas, you get something that’s truly outstanding.

Give it a watch – it’s entertaining and informative.

The second reference came from an altogether different source – an interview with renowned mix engineer Alan Moulder in this month’s issue of Sound on Sound magazine, in which he discusses his approach when applying VST plugins to his music projects:

All the things I use do something to make the sound a tiny bit better, and if  you add everything together, the end result will be a lot better… It’s simple: better is better, whether it’s a tiny bit better or a lot better.

Through never-ending sprints, features and tasks, iterative improvement is something all development teams should strive to achieve.

I was having a nice, relaxing Saturday morning until I read through my Twitter timeline and was directed to Jeff Atwood’s latest post on Coding Horror, “The PHP Singularity“. After reading through it, and the comments from other readers, I got annoyed. Really annoyed.

I won’t repeat the article, but the gist of it is:

• PHP is a horrible language
• Something must be done
• Jeff is the person to do it

I quite enjoy blogging but I don’t do it very often. I try to make sure anything I talk about is well researched, because a baseless opinion is worthless, but I don’t have a lot of free time – so if I don’t have something I consider worth saying, I don’t say it.

Jeff doesn’t have the same concern. You may wonder why I read his blog, given that I clearly take exception to some of its content, but the funny thing is that when talking about things he is knowledgable of, he has some great advice.

Like when he talks about his children, because he obviously knows them, or routers, where he’s clearly done some investigation. Now, I’m about to become a father, so I really enjoyed the former post, and if I needed to buy a router, I’d probably just be lazy and pick one from the latter because he knows a lot more about them than I do.

But here’s the thing. Jeff doesn’t seem to know much about PHP. I’d guess he’s read some posts about it, seen a few Q&As on Stack Overflow about it (a site he deserves massive kudos for), read through the PHP Manual, whatever – but he’s not a PHP developer. When he says PHP is ‘terrible’ or ‘deeply flawed’, I wonder how he’s come to that conclusion? Has he ever developed anything substantial in PHP? What is he basing this opinion on? He’s played with his children, he’s bought and installed a router. But when it comes to PHP, is the extent of his knowledge that which he has gleaned from others? Because if that’s the case, he really should think twice about offering his opinion on it.

The most glaring statement in the entire article is this one:

Therefore, I’d like to submit a humble suggestion to my fellow programmers. The next time you feel the urge to write Yet Another Epic Critique of PHP, consider that:

1. We get it already. PHP is horrible, but it’s used everywhere. Guess what? It was just as horrible in 2008. And 2005. And 2002. There’s a pattern here, but it’s subtle. You have to look very closely to see it. On second thought, never mind. You’re probably not smart enough to figure it out.
2. The best way to combat something as pervasively and institutionally awful as PHP is not to point out all its (many, many, many) faults, but to build compelling alternatives and make sure these alternatives are equally pervasive, as easy to set up and use as possible.

How anyone can offer a ‘humble’ suggestion and then go on to question the reader’s intelligence is besides the point (though it does nicely illustrate Jeff’s technique of drawing the reader in with deference only to hit them with a massive dose of self-righteousness) – but there’s two things I really take exception to here. Firstly, as anyone who has spent any real time with PHP knows, the language is far, far different now to what it was in 2002. For heaven’s sake, PHP5 (with a brand new object model that made proper OOP possible) didn’t make its debut ’til 2004, and support for register_globals (one of the worst features of PHP, ever) was removed in the most recent version, 5.4. Those facts are available under PHP’s entry on Wikipedia, but perhaps Jeff’s not smart enough to Google the history of PHP before commenting on it?

Secondly; assuming for a second that PHP is as horrible as he deems it to be – the best way to ‘combat’ it isn’t to ‘build compelling alternatives’ and ‘make sure these alternatives are equally pervasive, as easy to set up and use as possible’ – no, the ‘best’ thing to do (considering the amount of developers and sites worldwide who depend on the language and are getting things done with it every single day) is get involved with and actually understand PHP – read the internals mailing list, make requests for comments, code contributions, etc. It’s a silly person who decides to try something new before attempting to fix what’s broken, when there is so much invested in it. Perhaps if Jeff wasn’t trying to drum up interest in this new project of his, he’d have been a little less inflammatory?

Look, as someone who has used PHP successfully for the last decade, obviously I’m going to disagree with Jeff’s opinion. But the language has not only helped me get a great job, a nice house, and will shortly be helping me pay for one of those baby things, it’s also made Facebook the biggest website in the world, and enabled Yahoo! to build (ok, and throw away, but it’s not PHP’s fault) an empire. When Jeff’s got off his soap box and tried to build something equally successful in PHP and failed, I’ll start taking his opinion seriously.

As I said, I don’t like making statements unless I’ve researched them fully. I’ve read Coding Horror for a long time, and I feel qualified to say that Jeff Atwood is a muppet.

If you think app development and football have nothing in common, read on.

There was a discussion on our IRC server at work this morning which centred on how to best mitigate the risk involved with using third party tools in web applications. Not just at the code level, as in a JavaScript library like jQuery, or a PHP framework like Symfony2, but right through the stack – from the OS, to the web server, to the app codebase and beyond.

Much of the discussion was along the lines of what you’d expect when devops (or sysadmins) and developers collide – those responsible for managing the stack want things that can be easily automated, are verifiable (ideally through something like checksums) and reliable (i.e. not placed directly on a production server from a Pentium 3 machine in some teenager’s bedroom half way round the world), while those responsible for writing the code want to be able to use the best tool for the job, whether it be a tried and trusted web server or the newest, shiniest Symfony2 bundle.

The general consensus was that both have equal importance. Personally, I think that depending on the nature of the app, it’s wise to proportion risk accordingly; an app with a USP centred around some novel, interactive UX may want to incorporate a new innovative JavaScript toolkit even if it’s yet to reach version 1.0, whereas an app with paramount speed and scalability requirements might want to try PHP-FPM with nginx in place of a more familiar solution like mod-php with Apache.

In isolation, either of these decisions probably wouldn’t pose a problem. Combining many such scenarios, though, may eventually load the project with enough risk as to eventually make it untenable. The holy grail is to arrive at a point where we have an app that delivers its required functionality while making use of as many trusted, reliable tools as possible, and can be managed in such a way as to make deployment and maintenance safe and simple.

Now, to the title of this post. While this discussion was raging in IRC, there was much furore over the selection of the provisional England squad for the Euro 2012 tournament. It occurred to me that the new England manager, Roy Hodgson, is facing many of the same challenges myself and my coworkers face when trying to decide on a strategy for selecting the players he thinks will combine to great effect in a few month’s time. He wants a team that blends reliable, experienced, mature players with younger, modern ones – with maybe a sprinkling of unpredictable, yet highly creative, firebrands that can make the difference between a good squad and a great one. This is essentially what I was trying to articulate in the conversation, but swapping players for libraries, or tools, written by myself, or by others.

So when I was scanning the list of players that are going to Poland and the Ukraine this summer, I couldn’t help but think of them in a technical fashion. For example:

• John Terry (Defender) – Very experienced, but can be unreliable and prone to catastrophic behaviour. (Windows OS – ever tried hosting a PHP app on Windows?).
• Alex Oxlade-Chamberlain (Midfielder) – New kid on the block, has a speciality that offers a new dimension (see: nginx and content distribution networks)
• Stuart Downing (Midfielder) – Does what he does and does it well enough, but isn’t going to make a massive difference overall. Comes with baggage and divides opinion. (CodeIgniter/CakePHP/any other technically limited PHP framework)
• Wayne Rooney – Best of breed, unrivalled, links up with others very well, difficult to replicate or replace (jQuery)

It’s a popular argument at the moment that more people should be aware of, and be able to, code. But perhaps we’re missing a trick – perhaps a few games of Football Manager is all it takes to be able to understand how to compose a fabulous web app these days.

Step 1: Buy a Mac

On the twentieth of November, 2011, the company for whom I work (Box UK) held a hackday in Cardiff at the Student’s Union. It was the first time that we’d organised such an event, but it seemed to go really well – feedback from the participants was good, it was well attended, and personally I had a really good (if stressful!) time.

The theme of the hackday was ‘For the social good’ – applications built during the day had to have some sort of social theme, ideally to improve the lives of the people that would use them. That’s quite a broad topic, and it resulted in an array of different apps – not just in the sense of intended purpose, but in style, and also in platform. While teams typically used scripting languages to build their apps within those used (PHP, JavaScript, Python) were a lot of varied frameworks (Symfony2, Lithium, Django, jQuery Mobile, Node.js, PhoneGap and more), as well as SQL and No-SQL solutions alike. I initially suggested creating an iOS app, until it was pointed out to me that the verbose nature of Cocoa meant we’d probably only get one class done in the 8.5 hours we had!

My team comprised of myself; Paul, a former colleague, and Tom, a 3rd Year student at Cardiff University. Paul and I had decided to try to create a mobile app that used geolocation to provide the user with trivia and quizzes for the places of interest around them; we were only going to concern ourselves with Cardiff, so it would be a tourist aid for the city, or perhaps a learning tool for schoolchildren. Tom then joined us, and had originally wanted to work on a Corkboard-style app – after a discussion we modified that into a Q&A app which would be incorporated into our exploration tool idea, so the user could additionally ask questions and receive answers in realtime. That was the idea, anyway. We had 8.5 hours to implement it!

Photo courtesy of dangreenphotography.com

We split our responsibilities up such that Paul would build the mobile app (he plumped for jQuery mobile, version 1 of which having been released just days earlier), Tom would build an app to aggregate questions from users and allow a pool of authors to submit replies (he wanted to use Cappuccino), and I would build the webservice/content management system both apps would run on (I thought Symfony2 would be a good framework). It sounded good in theory, but then it always does.

I’ve dabbled with Symfony2 since it was in beta so I knew what it could do, but not necessarily how to make it do it. So the first hour of the day was spent scanning the documentation and looking through the available ‘bundles’ (Symfony nomenclature for packages of redistributable code) to see what community efforts I could re-use. I came across the SonataAdminBundle relatively quickly and it seemed a good fit – its purpose is to generate a CRUD interface for the models within the application’s domain. This would give me a quick way to add and manipulate the data in the app, so I plumped for it. I guess it’s a bit risky using an unknown in a hackday since if it turns out to be unsuitable, there’s not a lot of time to try something else – but I thankfully had no such problem here.

The SonataAdminBundle is impressive, in that with relatively little configuration and code I had a very functional app. The project documentation assumes a fair bit of Symfony2 knowledge from the developer, which isn’t ideal in a high pressure scenario where I didn’t really have time to read the source code, but I’d worked it out by lunchtime. I’d say within a couple of hours I had an app with which I could create models, specify forms which related to them, and manage data.

Most of the rest of my day was spent working with Doctrine2, a PHP ORM that is tightly integrated with Symfony2 and the SonataAdminBundle. Doctrine2 I found to be the perfect tool for a hackday – create a class, put some annotations on properties to define the type of the field, and any relationship to other classes, then run a command line script which generates all the accessor code, and even updates the database schema – saving a substantial bit of work. There were some hiccups along the way, but I did manage to get a webservice together which could supply Paul and Tom’s apps with data.

By this point we were running quite low on time (I think I spent too long chowing down on the copious amounts of free pizza!), and so rather guiltily I left the other members of the team with only an hour to integrate the webservice into their apps – both were true heroes though, quickly turning everything around so that we could demo something fully featured to the other teams.

Each of the other teams not only produced an app (I’ve been to hackdays where people don’t even get something together to demo, which is a little disappointing) but it was also gratifying to see that each app was functional, innovative and different. My personal favourite was Tomazs’ mobile app for the Welsh Premier League (it doesn’t get enough love!), but the grit.ly team (who produced a mashup of several data sources to plot on a UK map areas at risk of accidents due to poor weather and lack of gritting), who won first prize, and a team of several 1st year Cardiff Uni students with another former colleague at Box UK, Warren Seymour, who took second with a cool Corkboard app for university students, thoroughly deserved their awards.

It was also great to see Tom take the best individual developer prize, as he demonstrated some serious coding chops despite having to wait until the very last for the data his app depended on. He also ran into some issues with Cappuccino during the day, and rather than abandon what he was doing, he just switched to Prototype and got on with it in a very pragmatic fashion.

As a bonus, after the prizes were handed out we decamped for a drink in the Tafarn, something I haven’t done since leaving Cardiff University 10 years ago. It was almost worth attending the event solely for the excuse to walk down memory lane!

The code for the CMS app I built can be found on my Github page. Time permitting, I’d like to take the idea a bit further; though it’ll require a bit more planning than the 15 mins it got on Sunday to be a proper success!

Tomorrow night (09/11) I’m going to be giving a 20 minute talk on how I’ve found the transition from PHP to Objective-C, which I’ve been doing over the last year or so (transition is probably the wrong word, since I still regularly develop in PHP). The talk will be a slimmer version of the one I’ve submitted to a couple of conferences, but the proposal will still apply so I thought I’d post it here in case it encourages anyone to attend!

Last year, after 8 years of developing almost exclusively in PHP, I decided to learn something new and begin developing for the iOS platform in Objective-C. I’d noticed that the typical modern PHP application, like an iOS application, is written using a framework and cutting-edge language features like closures and runtime dynamic binding, and thought that would serve to make the transfer easier. I also wanted to expand my skill set in an increasingly mobile, recession-hit world.

In this talk I will share my experiences of going from PHP to Objective-C and Cocoa; how I took the paradigms that I was accustomed to in PHP, like syntax, memory management, unit testing and design patterns, and mapped them to the iOS ecosystem. I’ll also describe some techniques used in Objective-C that can be applied in PHP to enhance our code. Finally, I’ll discuss what I consider to be the pros and cons of each platform, and how I believe each has its place in our future careers.

PHPSW is a monthly PHP meetup held in Bristol, at the Golden Guinea, on the second Wednesday of every month. Everything kicks off at 7pm!

This issue has bitten me so many times I’m going to document it here so hopefully I’ll remember it in future. Now and then, when deleting files using Windows Explorer, random files would fail an access check even though I’m the system administrator. Unlocker reported no locks on the file, but it wouldn’t let me delete it either. The only way I could get around it was a restart.

This got annoying so the next time it happened I started shutting down all my running programmes one by one and retrying the delete, until I discovered that it was in fact IIS that was causing the lock – even though the files weren’t web accessible. Turning off the webserver was enough to allow the files to be deleted.

It’s long overdue but here’s a quick recap of the Hacks and Hackers Hackday I attended on the 11th March 2011 at the Atrium, Cardiff. It was my first hackday – though I’d been itching to try one for a while it seems they don’t tend to happen in Cardiff that often. I wasn’t disappointed, the day was really enjoyable.

The hackday was sponsored and organised by Scraperwiki, an open-source effort which aims to make webscraper tools, and the data they harvest, easily available and modifiable to the general public. In particular, the event attempted to bring journalists (hacks) and developers (hackers) together so that by combining each other’s strengths, we might teach each one another how to obtain and interpret the masses of public data that exists on the internet.

I went along with Carey and Warren from Box UK, and we formed a team with three ‘hacks’ – Steve Fossey, Eva Tallaksen from Intrafish and Gareth Morlais from BBC Cymru. We named ourselves Co-Ordnance, though I can’t remember why – think it was decided while I was busy hacking!

Inspired by some ideas Eva had we initially decided to try to write an application to model stock market data, specifically a tool to collate stock market updates from floated companies. With a little more brainstorming we decided to make this a sub-feature of a larger tool that would plot companies on a UK map according to their registered addresses. The user would be able to filter by the type of business and the region in which it was registered, as well as letting them choose a specific point in time. Thus one could make observations about the behaviour of businesses, such as the popularity of certain sectors of business in certain areas, the impact the recession had on business growth or the collapse of businesses, which regions are ripe for investment, and so on.

While Warren and Eva focused on writing a Scraperwiki script to collect the stock market data the rest of us planned the application. The first thing we considered was how we could obtain geolocation data for the companies that we were to be referencing, and that led us into trying to scrape data from the Companies House website.

This proved to be difficult, as URLs within the site feature hashed components – possibly in an attempt to prevent spiders from navigating the site. This meant we’d never come up with a scraper solution in the time that we had available, but thankfully a member of the Scraperwiki team told us about OpenCorporates, which aims to “have a URL for every company in the world”. Many of those URLs provide addresses for companies, which we could geo-encode (using Google’s Geoencoder) to obtain UK co-ordinates for that company. So we were able to plot companies per region on a map.

Carey wrote a scraper tool so that we could download data for each company into a local MySQL database for indexing. This would allow us to retrieve businesses for a particular region. A nice feature of the OpenCorporate dataset is that one is able to access the data in JSON, which made saving and accessing the data really easy, especially as the application was mainly to be written in JavaScript using the ExtJS framework. Incidentally, I took the opportunity to try the beta version of ExtJS 4, but since the app ended up being pretty much just a map and a form to manipulate the plotted data I didn’t get to play with any of the stuff added in the new release.

So once we’d downloaded the data of a few thousand companies (enough for a sample dataset) and indexed the data by location, I needed to write the frontend to plot the companies on a Google Map. By this point we had a little under three hours left, so it was a bit of a push – especially since I hadn’t done any Google Maps development in about three years, and the API had completely changed during that period. Thankfully a generous supply of sweets, Coke and beer kept me energetic, and we just about made it. We didn’t get chance to integrate Warren’s work into the app, which was a shame, but we still managed to achieve two solutions, which I thought impressive considering the time we had available to us (about 7 hours, once we’d got started).

In the reckoning we came third, which we were chuffed with. In addition to this Warren also won an individual prize for making the best use of Scraperwiki, which was totally deserved. It would perhaps have been nice if the judges had explained why they voted as they had, as I, and others I spoke to, were confused as to what the marking criteria was. Nevertheless it was a really good day, which challenged me in ways I’d never been challenged before, and forced me to work within a totally new team, under pressure, on a topic that I knew nothing about. I’d recommend the Scraperwiki tools, and the next time I need to harvest some data I’ll definitely be making use of them.

The code I wrote on the day is available at my Github page, and I’m hoping to get the app into a more usable state and hosted on this site in the near future.

[This is a blog post related to the PHP on Azure competition.]

I’ve spent the last few days reading about Windows Azure, mostly just trying to get a feel for what features it has and how I could use them in an application. The competition rules suggest that entrants “can earn extra points by using other parts of the Windows Azure platform” and the more I get to learn, the better, so I’m going to try to make use of as much of Azure’s functionality as possible.

Though I haven’t yet decided what application I’ll be writing it’ll probably be some sort of a CMS – it’s the genre of web app I’m most familiar and its likely I’ll get more done in the short time frame I have if I stick to what I know best. As a result I’ve been looking at Azure’s feature list and thinking “how I can map this platform feature to a feature of a CMS?”

I found plenty of resources which helped me get an overview of Azure, and which I’m sure I’ll continue to use as I progress. The most helpful were:

• Maarten Balliauw’s exhaustive Sitepoint article
• Josh Holmes’ blog (plenty of posts concerning PHP on Azure)
• Brian Swan’s blog (loads of good Microsoft/PHP content, including lots of Azure-related posts)

The Sitepoint article does a great job of summarising what Azure offers a PHP developer. As I was reading it I kept thinking “Ooh, that’d come in handy for this” or “That’d make so-and-so easy…” – here’s my reaction to some of the topics covered in the article.

Platform as a Service

The article starts by explaining that unlike, for example, Amazon’s EC2, which is an example of Infrastucture-as-a-Service, Windows Azure is a Platform-as-a-service. This essentially means that a developer using Azure doesn’t need to worry about maintaining the server, and just has to focus on deploying and configuring the applications that run on it. This sounds good to me; while my experiences of EC2 have been mostly good, there is a lot of work involved with getting servers into a state where they are usable. This can be mitigated somewhat by creating build scripts, but even then these have to be maintained, while managing server security is another headache that I’d be glad to be rid of. I’m optimistic that Azure will prove itself to be very hospitable, though I suppose there’s a chance that it’ll be too inflexible for more demanding applications that require atypical server configurations.

Blobs, Tables and Queues (oh my…)

Judging by how often they’re mentioned in the articles I’ve read, these three features seem to be what Microsoft deem the most valuable. All three will play their part in a highly scalable application, which is one of the advantages of the platform. Blobs can be leveraged to provide resources (images, downloads and the like) through a Content Delivery Network, accessed via a REST API – this is very relevant to a CMS, so I’ll definitely be taking advantage of this feature.

Table storage is a document-oriented non-relational database in the same vein as MongoDB or CouchDB. While I imagine the application will use relational databases, hopefully there will be an opportunity to use Table storage; perhaps to hold the content itself. I don’t have a lot of experience with this type of database so it’ll be a good chance for me to learn something.

Queues are very exciting; a built-in way to pass processor-heavy tasks off to worker machines. I can immediately think of a few ways a CMS could make use of this feature: generating PDFs of pages; creating ZIP files of multiple downloads; analysing uploaded files for automated tagging and classification; computing internal analytics… there are plenty of scenarios where queuing could be invaluable. This feature interests me the most.

SQL Azure

Microsoft have provided SQL Server support, albeit in a modified fashion, through SQL Azure. I’ve used SQL Server frequently over the last 4 years but have always preferred to use MySQL since until recently it’s been better supported by PHP through both highly maintained PDO drivers, and the mysqli client library. That has changed in the last year or so with Microsoft’s SQL Server Driver for PHP, and this will be my first chance to make use of that driver. It’s likely that I’ll employ Doctrine 2′s database abstraction layer since it has support for the driver, and it’ll make my application more portable in future – plus I’ve been looking for an excuse to put Doctrine 2 through its paces, having used Doctrine 1 extensively.

Windows Azure Platform Appfabric

Appfabric is a substantial element, comprising many components including Access Control, Caching, and Service Bus, which allows applications and services to communicate even through restrictive firewalls. Access control is a way to perform user authentication through federated identity services such as Active Directory, Microsoft Live, Yahoo!, Facebook and Google, which will be useful to web 2.0 style social applications that often delegate user identification. If it’s straightforward enough to do (and Microsoft say that it is) I may manage user authentication in this way; the web certainly doesn’t need another authentication device.

I’m unclear as to whether the caching system can currently be used; this article mentions only .NET applications, and suggests that it is only available for preview, which is a shame. If it isn’t, then there are other options available for caching – simple file-based caching, which will probably be too slow (and isn’t very scalable); the Wincache extension, which uses shared memory, which will be faster; or perhaps I’ll be able to implement a Memcached solution.

While I think Service Bus is a neat idea, I’m not sure whether my application will require communication with other network devices – it seems to be more relevant to applications concerned with restrictive networks.

Conclusion

I’ve read plenty about Windows Azure this week, and am quite excited about what it can offer a PHP application; some of the features make it almost too easy to dream up application functionality! Next I need to design my application, and get my development environment set up to work with Azure and its tools.

I’m a fan of Swiftmailer for sending emails through PHP – it’s a much more elegant alternative to the mail() function and recently was absorbed into the Symfony project, so its future seems certain. A big plus is its support for sending mail using the SMTP protocol, which makes it great for more complex scenarios where authentication or performance are concerns.

Up until this week I’d only used Swiftmailer on Windows, with an SMTP server on our network. This changed when I needed to do a demonstration using my Macbook, and as OSX comes installed with Postfix I thought everything should just work if I configured Swiftmailer to use an SMTP server on localhost at the standard port (25). This was nearly the case – as I read here while Postfix is installed, it’s not enabled by default. After following the steps outlined there email started sending as expected.