Craig Marvelley

Software and such

It’s a Kind of ImageMagick – Installing the PHP Extension for Windows 7.

I had a bit of a nightmare recently trying to get the ImageMagick PHP extension working on Windows. Builds of the extension as a DLL are provided by Mikko, which is awesome, but I had some trouble getting it to work.

The workstation I was using was running 64-bit Windows 7N, and the issue I was seeing was that the extension was being registered (if I ran php -m from the command line it seemed to be loaded) but when inspecting phpinfo() output the module was conspicuous by its absence.

Windows 7N is an EU version of Windows that comes without the Media Player framework – presumably an anti-trust related provision. Upon inspecting the extension DLL with Dependency Walker I discovered that it was linked to several missing DLLs that are supplied by the Media Player framework, and an additional DLL, IEShims.dll, which is an Internet Explorer library.

I installed the Media Feature Pack which resolved the majority of the missing DLLs. I also copied IEShims.dll from Program Files\Internet Explorer to Windows\SystemWOW64 (this probably wasn’t a good idea as the architectures of the DLLs will be fundamentally different, SystemWOW64 expects 32-bit DLLs – but I haven’t encountered any adverse effects as yet). After doing this the extension began to work as expected, and was visible in phpinfo() output.

There was a crucial comment on Mikko’s blog that I’ll repeat here – if the workstation is running PHP under IIS/FastCGI, install version 6.6.2-10 or earlier of ImageMagick (available from http://image_magick.veidrodis.com/image_magick/). Don’t install any versions later than that – they were built with VC10 and will not be compatible with the extension, which was built with VC9.

Finally, when I attempted to use the extension (I wanted to convert some images from SVG to PNG) I ran into errors because ImageMagick defaults to using the Windows temp directory as its conversion directory, and IUSR doesn’t have rights to write there. Rather than add it to the permitted users list (not a very secure approach) I set the IMAGICK_TMP environment system variable to a less prominent location; this required a restart to take effect.

For the Social Good – Box UK Hackday #1 Recap

On the twentieth of November, 2011, the company for whom I work (Box UK) held a hackday in Cardiff at the Student’s Union. It was the first time that we’d organised such an event, but it seemed to go really well – feedback from the participants was good, it was well attended, and personally I had a really good (if stressful!) time.

The theme of the hackday was ‘For the social good’ – applications built during the day had to have some sort of social theme, ideally to improve the lives of the people that would use them. That’s quite a broad topic, and it resulted in an array of different apps – not just in the sense of intended purpose, but in style, and also in platform. While teams typically used scripting languages to build their apps within those used (PHP, JavaScript, Python) were a lot of varied frameworks (Symfony2, Lithium, Django, jQuery Mobile, Node.js, PhoneGap and more), as well as SQL and No-SQL solutions alike. I initially suggested creating an iOS app, until it was pointed out to me that the verbose nature of Cocoa meant we’d probably only get one class done in the 8.5 hours we had!

My team comprised of myself; Paul, a former colleague, and Tom, a 3rd Year student at Cardiff University. Paul and I had decided to try to create a mobile app that used geolocation to provide the user with trivia and quizzes for the places of interest around them; we were only going to concern ourselves with Cardiff, so it would be a tourist aid for the city, or perhaps a learning tool for schoolchildren. Tom then joined us, and had originally wanted to work on a Corkboard-style app – after a discussion we modified that into a Q&A app which would be incorporated into our exploration tool idea, so the user could additionally ask questions and receive answers in realtime. That was the idea, anyway. We had 8.5 hours to implement it!

Photo courtesy of dangreenphotography.com

We split our responsibilities up such that Paul would build the mobile app (he plumped for jQuery mobile, version 1 of which having been released just days earlier), Tom would build an app to aggregate questions from users and allow a pool of authors to submit replies (he wanted to use Cappuccino), and I would build the webservice/content management system both apps would run on (I thought Symfony2 would be a good framework). It sounded good in theory, but then it always does.

I’ve dabbled with Symfony2 since it was in beta so I knew what it could do, but not necessarily how to make it do it. So the first hour of the day was spent scanning the documentation and looking through the available ‘bundles’ (Symfony nomenclature for packages of redistributable code) to see what community efforts I could re-use. I came across the SonataAdminBundle relatively quickly and it seemed a good fit – its purpose is to generate a CRUD interface for the models within the application’s domain. This would give me a quick way to add and manipulate the data in the app, so I plumped for it. I guess it’s a bit risky using an unknown in a hackday since if it turns out to be unsuitable, there’s not a lot of time to try something else – but I thankfully had no such problem here.

The SonataAdminBundle is impressive, in that with relatively little configuration and code I had a very functional app. The project documentation assumes a fair bit of Symfony2 knowledge from the developer, which isn’t ideal in a high pressure scenario where I didn’t really have time to read the source code, but I’d worked it out by lunchtime. I’d say within a couple of hours I had an app with which I could create models, specify forms which related to them, and manage data.

Most of the rest of my day was spent working with Doctrine2, a PHP ORM that is tightly integrated with Symfony2 and the SonataAdminBundle. Doctrine2 I found to be the perfect tool for a hackday – create a class, put some annotations on properties to define the type of the field, and any relationship to other classes, then run a command line script which generates all the accessor code, and even updates the database schema – saving a substantial bit of work. There were some hiccups along the way, but I did manage to get a webservice together which could supply Paul and Tom’s apps with data.

By this point we were running quite low on time (I think I spent too long chowing down on the copious amounts of free pizza!), and so rather guiltily I left the other members of the team with only an hour to integrate the webservice into their apps – both were true heroes though, quickly turning everything around so that we could demo something fully featured to the other teams.

Each of the other teams not only produced an app (I’ve been to hackdays where people don’t even get something together to demo, which is a little disappointing) but it was also gratifying to see that each app was functional, innovative and different. My personal favourite was Tomazs’ mobile app for the Welsh Premier League (it doesn’t get enough love!), but the grit.ly team (who produced a mashup of several data sources to plot on a UK map areas at risk of accidents due to poor weather and lack of gritting), who won first prize, and a team of several 1st year Cardiff Uni students with another former colleague at Box UK, Warren Seymour, who took second with a cool Corkboard app for university students, thoroughly deserved their awards.

It was also great to see Tom take the best individual developer prize, as he demonstrated some serious coding chops despite having to wait until the very last for the data his app depended on. He also ran into some issues with Cappuccino during the day, and rather than abandon what he was doing, he just switched to Prototype and got on with it in a very pragmatic fashion.

As a bonus, after the prizes were handed out we decamped for a drink in the Tafarn, something I haven’t done since leaving Cardiff University 10 years ago. It was almost worth attending the event solely for the excuse to walk down memory lane!

The code for the CMS app I built can be found on my Github page. Time permitting, I’d like to take the idea a bit further; though it’ll require a bit more planning than the 15 mins it got on Sunday to be a proper success!

Speaking at PHPSW: From PHP to Objective-C

Tomorrow night (09/11) I’m going to be giving a 20 minute talk on how I’ve found the transition from PHP to Objective-C, which I’ve been doing over the last year or so (transition is probably the wrong word, since I still regularly develop in PHP). The talk will be a slimmer version of the one I’ve submitted to a couple of conferences, but the proposal will still apply so I thought I’d post it here in case it encourages anyone to attend!

Last year, after 8 years of developing almost exclusively in PHP, I decided to learn something new and begin developing for the iOS platform in Objective-C. I’d noticed that the typical modern PHP application, like an iOS application, is written using a framework and cutting-edge language features like closures and runtime dynamic binding, and thought that would serve to make the transfer easier. I also wanted to expand my skill set in an increasingly mobile, recession-hit world.

In this talk I will share my experiences of going from PHP to Objective-C and Cocoa; how I took the paradigms that I was accustomed to in PHP, like syntax, memory management, unit testing and design patterns, and mapped them to the iOS ecosystem. I’ll also describe some techniques used in Objective-C that can be applied in PHP to enhance our code. Finally, I’ll discuss what I consider to be the pros and cons of each platform, and how I believe each has its place in our future careers.

PHPSW is a monthly PHP meetup held in Bristol, at the Golden Guinea, on the second Wednesday of every month. Everything kicks off at 7pm!

Three20, XCode 4 and the Case of the Missing Header Files.

After a good hour of hitting my head against a brick wall, I finally managed to get Facebook’s three20 iOS library integrated with my XCode 4 based project, documented here for posterity and the hope that it may save someone else from the raw, savage agony I’m currently feeling.

I followed the instructions on the three20 website to the letter, using the python installation script to add the library to my project, but whenever I tried to include the main Three20.h header file within my application code I got a ‘file not found’ warning which failed the build. After much trial and error / Stack Overflow searching I came across this post which seemed to suggest trying every header search path under the sun. Indeed, one of these finally worked – I narrowed it down to the $(BUILT_PRODUCTS_DIR)/../three20 entry, but YMMV.

As much as I like iOS, and XCode, Apple’s solution for integrating libraries is extremely error prone – and this was with a custom installation script. Hopefully once XCode 4 adoption has grown the process can be simplified, as newer libraries like RestKit seem to be less complicated to install.

IIS File Locking

This issue has bitten me so many times I’m going to document it here so hopefully I’ll remember it in future. Now and then, when deleting files using Windows Explorer, random files would fail an access check even though I’m the system administrator. Unlocker reported no locks on the file, but it wouldn’t let me delete it either. The only way I could get around it was a restart.

This got annoying so the next time it happened I started shutting down all my running programmes one by one and retrying the delete, until I discovered that it was in fact IIS that was causing the lock – even though the files weren’t web accessible. Turning off the webserver was enough to allow the files to be deleted.

Using a PHP Framework With Windows Azure

[This is a blog post related to the PHP on Azure competition.]

It’s been a while since my last PHP on Azure contest post, mostly due to commitments @work that have reduced my free time considerably. There’s not long left ’til the competition closes though (I have a week), so I need to get my skates on!

Casting my mind back to my last post, I was able to finally deploy to the local development fabric. In the interim I’ve set up an Azure account and successfully deployed a simple script to the cloud, so everything’s fine on that front. It’s time to start developing my application.

Since my time is limited my application will be quite straightforward – a CMS to manage media files. This should allow me to use all the features of Azure (queuing, table storage, blobs) I’m interested in.

Bittersweet Symfony

The first step in building the application is to decide on  framework to speed up development. The main reason I got into the contest in the first place was because I wanted to have more experience with Zend Framework – while I’d used its components in isolation before I’d never built an application with it. However in the months since I entered the competition I’ve started using Symfony 2 heavily. That, and the realisation that ZF2 is around the corner, have made me think that ZF isn’t the best choice for this contest, so I decided to use Symfony 2 instead.

That was easier said than done. I dropped the Symfony framework into my Azure WebRole project, and put together a ‘hello world’ script, only to immediately run into this issue – apparently Azure has issues when building projects when paths to files or directories exceed 260 characters. Of course, in a framework of Symfony’s size, with many nested directories, this limit is unrealistic.The solutions recommended on Jim’s blog include changing the project paths to be shorter (impossible in this case) and setting a temporary environment variable which moves the location at which Azure builds projects, thus reducing the prefix of any paths. As Jim says, this isn’t a guaranteed fix and indeed, it didn’t work for me.

So including Symfony within the WebRole didn’t seem to be possible. I experimented with including in within my PHP include path, in the hope that it would save some characters, but that didn’t work at all, Azure didn’t even pick it up. I need to look around the net to see how Azure manages include paths, since it’s something I use often in PHP. How do PEAR modules work, I wonder? Can you manage PEAR within Azure as you would with a local PHP installation, or do you have to place the components within each WebRole? Topics for future posts, maybe.

Anyway, it didn’t look like Symfony was going to work any time soon, so I needed an alternative, and since Ben Waine had successfully deployed using Zend Framework I thought briefly of using that after all. Then, I came across Silex.

Along came a Silex

Silex is a micro-framework based on Symfony 2 components, inspired by Sinatra, the Ruby on Rails framework which presents an alternative to typical MVC web applications. It’s succinct and straightforward, and, crucially, is deployed as a PHAR archive. This means I only need to include a single file in the root of my project to be able to use the library – which means no long paths that Azure can’t handle.

I downloaded the Silex archive from its website and dropped it into the root of my project, then copied the ‘hello world’ example provided within the Silex documentation into my index.php file:

require_once __DIR__.'/silex.phar'; 

$app = new Silex\Application(); 

$app->get('/hello/{name}', function($name) {
    return "Hello $name";
}); 

$app->run();

Then I deployed my project to dev fabric, and navigated to /hello/Craig. Of course, this didn’t work immediately – nothing ever does for me – because I didn’t have a web.config file set up to send all requests through the front controller, index.php. I got an IIS 404 page rather than the ‘Hello Craig’ I was expecting. So I used IIS’s Apache config importer to convert the sample .htaccess file provided by Silex into a web.config file, then copied the relevant rewrite rules into my WebRole’s config file. Rebuilding the application resulted in a working application. Good to go!

Here’s what I needed to add to my web.config file in case it’s of any use to anyone:

Compiling Silex on Windows

Because Silex is pretty new I’m expecting to find issues here and there, plus when working on Windows I’m accustomed to having to make little changes here and there to get PHP code that’s been written on *nix to run properly – so I wanted to build the Silex PHAR from source so I could make changes if necessary and properly understand how everything fits together.

This is quite straightfoward. First, I checked out Silex from its github page:

git clone https://github.com/fabpot/Silex.git silex

I merged in a pending fix which addresses compilation issues on Windows – there’s probably a fancy way to do this in git but I just applied the fix manually.

Then I needed to get Silex’s dependencies:

git submodule init
git submodule update --recursive

Silex provides a PHP script, named ‘compile’, which puts the archive together. I had to make some changes to my php.ini file before I could run it successfully:

phar.readonly = Off
phar.require_hash = Off

Then creating the archive was as simple as running:

php compile

Preventing Aggressive Caching of JavaScript Files With IIS7

I was playing with the new beta of Ext JS 4 the other day and noticed that sometimes the scripts were being cached, so my changes weren’t taking effect. Initially I thought this was because I’d upgraded to Firefox 4 – I use the Web Developer add-on which allows me to disable browser caching, and I’d assumed the cache setting had reverted when I upgraded. However the cache was indeed disabled, so I had to dig a little deeper.

It turns out that IIS 7 will automatically cache static resources, such as JavaScript files. While this is probably what you’d want for a default production environment it almost certainly isn’t from a development point of view, when changes are being made by the minute. It wasn’t too hard to turn this feature off, however. To do so, just do the following:

  • Open up IIS Manager
  • Optionally select the site you’re working with
  • Open the ‘Output Caching’ module
  • Select ‘Add…’ from the right hand menu
  • Enter a file extension of ‘.js’ and disable both user-mode and kernel-mode caching, click OK.

Select the ‘Output Caching’ Module

Disabling JavaScript caching

Disabling JavaScript caching


According to the IIS blog
IIS should flush the cache when files are modified; however this wasn’t the behaviour that I was seeing. I had to completely disable caching to guarantee file modifications made it to the browser.

Hacks and Hackers Hackday Cardiff

It’s long overdue but here’s a quick recap of the Hacks and Hackers Hackday I attended on the 11th March 2011 at the Atrium, Cardiff. It was my first hackday – though I’d been itching to try one for a while it seems they don’t tend to happen in Cardiff that often. I wasn’t disappointed, the day was really enjoyable.

The hackday was sponsored and organised by Scraperwiki, an open-source effort which aims to make webscraper tools, and the data they harvest, easily available and modifiable to the general public. In particular, the event attempted to bring journalists (hacks) and developers (hackers) together so that by combining each other’s strengths, we might teach each one another how to obtain and interpret the masses of public data that exists on the internet.

I went along with Carey and Warren from Box UK, and we formed a team with three ‘hacks’ – Steve Fossey, Eva Tallaksen from Intrafish and Gareth Morlais from BBC Cymru. We named ourselves Co-Ordnance, though I can’t remember why – think it was decided while I was busy hacking!

Co-Ordnance (courtesy Scraperwiki blog)

Inspired by some ideas Eva had we initially decided to try to write an application to model stock market data, specifically a tool to collate stock market updates from floated companies. With a little more brainstorming we decided to make this a sub-feature of a larger tool that would plot companies on a UK map according to their registered addresses. The user would be able to filter by the type of business and the region in which it was registered, as well as letting them choose a specific point in time. Thus one could make observations about the behaviour of businesses, such as the popularity of certain sectors of business in certain areas, the impact the recession had on business growth or the collapse of businesses, which regions are ripe for investment, and so on.

While Warren and Eva focused on writing a Scraperwiki script to collect the stock market data the rest of us planned the application. The first thing we considered was how we could obtain geolocation data for the companies that we were to be referencing, and that led us into trying to scrape data from the Companies House website.

This proved to be difficult, as URLs within the site feature hashed components – possibly in an attempt to prevent spiders from navigating the site. This meant we’d never come up with a scraper solution in the time that we had available, but thankfully a member of the Scraperwiki team told us about OpenCorporates, which aims to “have a URL for every company in the world”. Many of those URLs provide addresses for companies, which we could geo-encode (using Google’s Geoencoder) to obtain UK co-ordinates for that company. So we were able to plot companies per region on a map.

Carey presents our app (courtesy Scraperwiki blog)

Carey wrote a scraper tool so that we could download data for each company into a local MySQL database for indexing. This would allow us to retrieve businesses for a particular region. A nice feature of the OpenCorporate dataset is that one is able to access the data in JSON, which made saving and accessing the data really easy, especially as the application was mainly to be written in JavaScript using the ExtJS framework. Incidentally, I took the opportunity to try the beta version of ExtJS 4, but since the app ended up being pretty much just a map and a form to manipulate the plotted data I didn’t get to play with any of the stuff added in the new release.

So once we’d downloaded the data of a few thousand companies (enough for a sample dataset) and indexed the data by location, I needed to write the frontend to plot the companies on a Google Map. By this point we had a little under three hours left, so it was a bit of a push – especially since I hadn’t done any Google Maps development in about three years, and the API had completely changed during that period. Thankfully a generous supply of sweets, Coke and beer kept me energetic, and we just about made it. We didn’t get chance to integrate Warren’s work into the app, which was a shame, but we still managed to achieve two solutions, which I thought impressive considering the time we had available to us (about 7 hours, once we’d got started).

In the reckoning we came third, which we were chuffed with. In addition to this Warren also won an individual prize for making the best use of Scraperwiki, which was totally deserved. It would perhaps have been nice if the judges had explained why they voted as they had, as I, and others I spoke to, were confused as to what the marking criteria was. Nevertheless it was a really good day, which challenged me in ways I’d never been challenged before, and forced me to work within a totally new team, under pressure, on a topic that I knew nothing about. I’d recommend the Scraperwiki tools, and the next time I need to harvest some data I’ll definitely be making use of them.

The code I wrote on the day is available at my Github page, and I’m hoping to get the app into a more usable state and hosted on this site in the near future.

Using Eclipse to Run PHP in Azure DevFabric

[This is a blog post related to the PHP on Azure competition.]

As detailed in my last blog post I had some… problems getting a PHP/Azure development environment set up after creating a Windows 7 partition. Well, I scrapped the partition, said goodbye to my old XP OS and started afresh, and that looks to have done the trick – I’m finally able to deploy PHP applications to development storage on my local machine. I’m none the wiser as to why it didn’t work the first time, which is annoying, but at least it’s working now.

After I reinstalled Windows 7 I took care to install all the automatic updates before I went near any other programmes. After a lot of rebooting I eventually installed IIS and, through the Web Platform Installer, the Azure SDK and the Azure command line tools for PHP, along with their dependencies. Then, I installed Eclipse PDT and, following this excellent article, the Azure plugin for eclipse that Brian Swan wrote a great tutorial on.

I came across a few issues while doing all this that I should detail:

  • I had to set Eclipse to run as an administrator (right click eclipse.exe-> properties -> Compatibility -> Run as administrator) before it would install the plugin and allow me to use it within the application
  • I signed up for the free introductory trial of Azure so that I could follow the tutorial properly and configure a database on Azure

It’s a bit annoying that even when developing an app one has to run against SQL Azure in the cloud (which will eventually incur costs). I’d assumed I’d be able to use SQL Server locally but that doesn’t seem to be the case. Some sort of free quota for use in the development stage would make sense to me.

The really good thing about using the Eclipse plugin is that it brings together the many disparate projects that one needs to leverage to work with PHP on Azure. When I was trying to install all this stuff on my own I got a bit muddled, especially since some of the libraries are installed through the Web Platform Installer. The other advantage of the plugin is one can deploy to the cloud from within Eclipse. I’m not a fan of the Eclipse platform but I have to say in this case it’s a good experience (especially since the alternative is the command line!).

If I’ve one criticism of the documentation Microsoft provide (which is generally well-written and thorough) it’s that there’s no ‘official’ site for PHP on Azure which gives a simple one-page overview of everything – after reading about this stuff for over two weeks I still don’t feel like I’ve grokked it, and without blogs like Brian’s and Maarten’s I’d be struggling. As a PHP developer I’m used to the awesome PHP manual and its direct approach to documentation and perhaps Azure (as far as PHP is concerned) could do with something similar. For developers like me, who are new to Azure and trying to get up to speed quickly in a short space of time (and with little time to read around) that sort of resource would be invaluable.

For the moment though I’m just happy that I can finally start developing my application. It’s been a frustrating week battling to get to the point where I can just write a line of code, but from here on in hopefully it’ll be worth it.